In the previous article on the subject of cloud computing using AWS, we setup a simple LAMP application that used a single web server to present data that was queried from a single RDS instance. In this guide we will see how to save the changes we made to the EC2 instance, create more EC2 instances, and setup load balancing across our web servers.
As of the end of the previous article, here is how our cloud infrastructure looks:
Before we add more web servers to that picture, we need to create our own Amazon Machine Image, or AMI, that will allow us to rapidly clone our customized EC2 instance, and not have to repeat all of the work we've already done.
When we create an AMI from our EC2 instance, the resulting AMI will be stored within Amazon's S3 service. In order to make sure that someone other than you cannot take and use your AMI, the AMI is encrypted using your X.509 certificate. Remember those two *.pem files you stored within your ~/.ec2 directory in the last article? Well, we need to copy those to our EC2 instance:
scp -i ~/.ec2/gsg-keypair ~/.ec2/*.pem root@ec2-174-129-154-209.compute-1.amazonaws.com:/mnt
You want to put them in /mnt as that location is excluded when we create the AMI; there is no good reason to have your certificate bundled inside the AMI.
Since we will need to store our AMI in S3, let's create an S3 bucket to hold the AMI. Since this is our first time running s3cmd, we first need to configure our S3 account access. You do this by running:
s3cmd --configureThe s3cmd command will ask you for your Access Key ID, your Secret Access Key, and a password to use in encrypting the files it stores in S3. Don't forget the password you use as you won't be able to recover your data from S3 if you both forget it and lose the ~/.s3cfg file that is created by s3cmd.
Now that s3cmd is configured, let's create a new S3 bucket to hold our AMI files:
s3cmd mb s3://test-ami-bucketThe mb stands for Make Bucket. This will create a near-infinite space within S3 for you to put your files. S3 is essentially what is called a CRUD interface; that is, you communicate with it over HTTP (it rather acts like a WebDAV share) and tell it to either Create, Read, Update, or Delete items.
Now that we have a bucket ready and waiting for us, let's now login to our one EC2 instance via SSH:
ssh -i ~/.ec2/gsg-keypair root@ec2-174-129-154-209.compute-1.amazonaws.com
We will now bundle the root file system into an AMI by running:
ec2-bundle-vol -k /mnt/pk-NUMBER.pem -c /mnt/cert-NUMBER.pem -u ID -d /mnt
Replace NUMBER with the string from your *.pem files, and replace ID with your Amazon account ID (the one that looks something like 1234-5678-9012) but without the hyphens.
By default, the ec2-bundle-vol command will exclude the following locations from being bundled inside the AMI:
- /sys
- /proc
- /dev/pts
- /net
- /dev
- /media
- /mnt
- /proc
- /sys
- /tmp/image
(This is why we uploaded our certificate files to /mnt.)
The bundling process will take a long time, so be patient. Once it completes, you'll have a bunch of image.part.* files in /mnt, along with a file named image.manifest.xml, which is the catalog for all of the other individual files.
Next, let's upload the AMI files into our S3 bucket:
ec2-upload-bundle -b test-ami-bucket -m /mnt/image.manifest.xml -a ACCESS -s SECRET
Replace ACCESS with your Access Key ID, and SECRET with your Secret Access Key.
After the bundle upload completes, log out of the EC2 instance, and from your local system, take a look at your S3 bucket:
s3cmd ls s3://test-ami-bucket
You should see all your AMI files. Now, before we can use our shiny new AMI in the creation of more EC2 instances, we need to register it with Amazon. You can do this by running:
ec2-register test-ami-bucket/image.manifest.xml -n "Test AMI"
Amazon will reply with an AMI identification number, which will look something like this: ami-c8a1b5e3. Make a note of it, as we'll be using it momentarily.
Just for fun, let's check out our budding AMI collection:
ec2-describe-images -o selfPretty nifty, eh? Okay, let's create a second EC2 instance, but this time, it'll be from our own AMI:
ec2-run-instances ami-c8a1b5e3 -k gsg-keypair -g TestServerSecurityGroup
After a moment or two, it should be up and running, and we should now see two EC2 instances:
ec2-describe-instances
Open the new instance name in your favorite web browser (it'll look something like http://ec2-199-234-197-153.compute-1.amazonaws.com) and you'll notice that it is already showing the PHP test page, same as the first instance.
So at this point, our cloud infrastructure now looks like this:
What's lacking, however, is an ability to send traffic to both instances without needing to individually specify them. In other words, what we need is a load balancer. Enter Amazon's Elastic Load Balancing (ELB) service.
Setting up an ELB is really quite straightforward. It's a two-step process: first, we create the load balancer itself; and then we register existing EC2 instances with it.
Let's create a load balancer:
elb-create-lb TestLoadBalancer --availability-zones us-east-1d --listener "protocol=tcp,lb-port=80,instance-port=80"
This will create a load balancer named TestLoadBalancer that is listening on TCP port 80, and passing traffic to TCP port 80 on its EC2 instances. This is pretty convenient if you're running a service that listens on a different port, for example, Tomcat on port 8080; you could in that case say --listener "protocol=tcp,lb-port=80,instance-port=8080".
When the command completes, you'll be given a hostname, something like TestLoadBalancer-601146347.us-east-1.elb.amazonaws.com. This will be the new address you visit in your browser. At the moment, it doesn't do much, as we haven't added any EC2 instances to it. Let's do that now:
elb-register-instances-with-lb TestLoadBalancer --instances i-f72d739f i-737f211bNow our load balancer has two instances associated with it. To try it out, open the load balancer's hostname in your favorite browser. You can also see what instances are in the load balancer by running:
elb-describe-lbs --show-longWith the addition of the load balancer, our cloud infrastructure is beginning to look a lot more complete:
So far, so good. But what happens if one of those EC2 instances were to fail? Let's find out. SSH into one of the instances, and run:
service httpd stop
Then back on your local system, wait a moment, and then check:
elb-describe-instance-health TestLoadBalancer
After a minute or two the above command should report that the instance you stopped Apache on is OutOfService. Start Apache back up on the instance, and the load balancer should return it to service after a minute or so.
Note that this doesn't account for the total loss of an instance, for example, if you were to run ec2-terminate-instances. In that case, your load balancer pool would permanently shrink to a single instance.
This also doesn't help us with handling traffic spikes and surges. If two instances can handle 100 concurrent users, for example, what happens when there are 1,000 concurrent users? This is where Auto Scaling (AS) joins the scene.
Auto Scaling is simply a method for monitoring some health aspect of your running EC2 instances (for example, CPU utilization) and then dynamically starting (running) new instances as demand grows, and later stopping (terminating) instances as demand shrinks. It accomplishes this both by monitoring your instances and in communicating with your load balancer.
The first thing we need to setup is something called a launch configuration. This spells out what kind of EC2 instance we want started when the AS group detects a condition that calls for starting more instances. Since we've previously created our own AMI, let's use that:
as-create-launch-config TestLaunchConfig --image-id ami-c8a1b5e3 --instance-type m1.small
Next, we will create the AS group:
as-create-auto-scaling-group TestAutoScalingGroup --availability-zones us-east-1d --launch-configuration TestLaunchConfig --max-size 10 --min-size 2 --load-balancers TestLoadBalancer
This creates an AS group named TestAutoScalingGroup which will keep a minimum of two instances that conform to our TestLaunchConfig configuration up and running in the TestLoadBalancer pool at all times. But it doesn't yet know when to grow the pool above two; let's set that up now:
as-create-or-update-trigger TestTrigger --auto-scaling-group TestAutoScalingGroup --namespace "AWS/EC2" --measure CPUUtilization --statistic Average --dimensions "AutoScalingGroupName=TestAutoScalingGroup" --period 60 --lower-threshold 40 --upper-threshold 80 --lower-breach-increment"=-1" --upper-breach-increment "1" --breach -duration 300
What this does is to create a new trigger, called TestTrigger, that when it sees the CPU utilization exceed 80% for over 300 seconds it starts one new instance (that's the --upper-breach-increment parameter.) When the CPU utilization falls below 40% for 300 seconds, it starts negative one new instance, or in other words, terminates one instance (until the minimum defined is reached, which in our case, is two instances.)
We can now see this in action by terminating one of our EC2 instances, and watching AS start a replacement one:
ec2-terminate-instances i-737f211b
Give it a moment, and the load balancer should realize that the instance is now OutOfService:
elb-describe-instance-health TestLoadBalancer
If you wait another moment or two, the AS group will automatically start a replacement instance and add it to your load balancer. You'll end up seeing something like this:
INSTANCE-ID i-737f211b OutOfService Instance has failed at least the UnhealthyThreshold number of health checks consecutively.
INSTANCE-ID i-59722c31 InService N/A N/A
INSTANCE-ID i-43732d2b InService N/A N/A
The very first instances you registered with the load balancer will stick around forever, even after you've terminated them. To clean their cruft out of the load balancer, run this with their instance IDs:
elb-deregister-instances-from-lb TestLoadBalancer --instances i-f72d739fAh, much better. Here is how our cloud infrastructure now looks:
