Monday, 17 September 2012

Auto Scaling your AWS fleet with Cloud Watch and SQS


This blog post shows you how to setup Auto Scaling in AWS using Cloud Watch to dynamically expand and collapse your Amazon EC2 fleet in line with SQS expansion.

Before we get our hands dirty, I am making a few assumptions here:
  1. You are already familiar with the basics of AWS
  2. You have an AWS account - if not, sign up here.
Also, this was done on a Windows box so your mileage on a Mac or a Linux box may vary.

The first step is to download  the Auto Scaling tools from Amazon and configure the command line interface (CLI). If you do not have Java v1.5 or newer you'll need to update here.

I like to pop the following commands in a batch file that I can reuse as needed and makes the CLI tools more portable between workstations.


set AWS_AUTO_SCALING_HOME=C:\aws\AutoScaling-1.0.61.0
set PATH=%PATH%;%AWS_AUTO_SCALING_HOME%\bin
set AWS_CREDENTIAL_FILE=C:\aws\AutoScaling-1.0.61.0\credential-file-path.template
set JAVA_HOME=C:\Program Files\Java\jre6
set PATH=%PATH%;%JAVA_HOME%\bin



Once that is done, you can confirm the setup was done correctly by using the as-cmd command.


C:\aws\AutoScaling-1.0.61.0\bin>as-cmd

If successful, you will see all of the available Auto Scaling commands.

Command Name                                Description

------------                                -----------

as-create-auto-scaling-group                Create a new Auto Scaling group.

as-create-launch-config                     Creates a new launch configuration.

as-create-or-update-tags                    Create or update tags.

<snip />


as-update-auto-scaling-group                Updates the specified Auto Scaling group.

help
version                                     Prints the version of the CLI tool and the API.

    For help on a specific command, type '<commandname> --help'


The next step is to create a Launch Configuration for the SQS processor instances. The launch configuration specifies the type of Amazon EC2 instance that Auto Scaling creates for you. I chose to use a micro instance using the Amazon Linux AMI however you can use any AMI you like, including your own.

You create your launch config via the as-create-launch-config command:

as-create-launch-config DFQPconfig --image-id ami-a0cd60c9 --instance-type t1.micro

If successful, you will see: OK-Created launch config

Now, create an AutoScaling group for the SQS processors:

as-create-auto-scaling-group DFQPgroup --launch-
configuration DFQPconfig --availability-zones us-east-1e --min-size 1 --max-size
 5

If successful you will see: OK-Created AutoScalingGroup

The Auto Scaling group existence can be verified with the following command:

    as-describe-auto-scaling-groups --headers

The results will look similar to this:

AUTO-SCALING-GROUP  GROUP-NAME  LAUNCH-CONFIG  AVAILABILITY-ZONES  MIN-SIZE  MAX
-SIZE  DESIRED-CAPACITY
AUTO-SCALING-GROUP  DFQPgroup   DFQPconfig     us-east-1e          1         5
       1
INSTANCE  INSTANCE-ID  AVAILABILITY-ZONE  STATE      STATUS   LAUNCH-CONFIG
INSTANCE  i-8c93d9f6   us-east-1e         InService  Healthy  DFQPconfig

Now, let's configure Auto Scaling policies that with both add and remove 1 node to and from the group as demand specifies.  They will be invoked when the number of messages on the queue increases to a predefined level or decreases to an acceptable level:

as-put-scaling-policy DFQPscaleup -g DFQPgroup --adjustment=1 --type ChangeInCapacity

Be sure to make note of your ARN you will need it later to create CloudWatch alarms.

arn:aws:autoscaling:us-east-1:512617028781:scalingPolicy:78218513-4cae-4907-a882
-a47ee9cbc576:autoScalingGroupName/DFQPgroup:policyName/DFQPscaleup

**note**
The amount to scale the capacity of the associated group. Use negative values to decrease capacity. For negative numeric values, specify this option such as --adjustment=-1 for Unix, and  "--adjustment=-1" (quotes required) for Windows.

as-put-scaling-policy DFQPscaledown -g DFQPgroup "--adjustment=-1" --type ChangeInCapacity

Be sure to make note of your ARN you will need it later to create CloudWatch alarms.


arn:aws:autoscaling:us-east-1:512617028781:scalingPolicy:1611a772-cea0-492c-b245
-f1f24c59d490:autoScalingGroupName/DFQPgroup:policyName/DFQPscaledown

Next, we need the Amazon CloudWatch tools. Download the command line tools here.

Similar to the Auto Scaling config, this is my setup for Cloud Watch - yours might vary a little:

set AWS_CLOUDWATCH_HOME=C:\aws\CloudWatch-1.0.12.1
set PATH=%PATH%;%AWS_AUTO_SCALING_HOME%\bin
set AWS_CREDENTIAL_FILE=C:\aws\CloudWatch-1.0.12.1\credential-file-path.template
set JAVA_HOME=C:\Program Files\Java\jre6
set PATH=%PATH%;%JAVA_HOME%\bin

Once that is done, we will build out two alarms, one to trigger when there is a high number of ApproximateNumberOfMessagesVisible and another when there is a low number of ApproximateNumberOfMessagesVisible.

mon-put-metric-alarm --alarm-name DFQPHighAlarm --alarm-description "Scale up when ApproximateNumberOfMessagesVisible is high" --metric-name ApproximateNumberOfMessagesVisible --namespace AWS/SQS --statistic Average --period 60 --threshold 100 --comparison-operator GreaterThanThreshold --dimensions="QueueName=sqs_populator" --evaluation-periods 5 --alarm-actions arn:aws:autoscaling:us-east-1:512617028781:scalingPolicy:61432c71-0679-4988-928f-4a58a867d71f:autoScalingGroupName/DFQPgroup:policyName/DFQPscaleup

If successful, you will see: OK-Created Alarm

mon-put-metric-alarm --alarm-name DFQPLowAlarm --alarm-description "Scale down when ApproximateNumberOfMessagesVisible is low" --metric-name ApproximateNumberOfMessagesVisible --namespace AWS/SQS --statistic Average --period 60 --threshold 100 --comparison-operator LessThanThreshold --dimensions="QueueName=sqs_populator" --evaluation-periods 5 --alarm-actions arn:aws:autoscaling:us-east-1:512617028781:scalingPolicy:1611a772-cea0-492c-b245
-f1f24c59d490:autoScalingGroupName/DFQPgroup:policyName/DFQPscaledown

If successful, you will see: OK-Created Alarm

This example the Alarms cause the EC2 to scale up by adding instances when the number of visible messages on the queue remains above 100 for 5 minutes. It will also scale down when the number of visible messages falls below 100 for 5 minutes.


No comments:

Post a Comment