Programming Amazon Web Services

Automated Management of EC2 Instance Pools with Lifeguard

In Distributed Application Services with BOTO,” we created an AMI that can be used to launch EC2 instances that run a boto service, such as the MultiCommmandService. This AMI serves as the central component in a service-oriented architecture, which allows you to increase the processing power available to a distributed application by starting more EC2 instances to act as service components. It also allows you to minimize the expense incurred by the application, by running no more instances than are necessary to complete the work at hand.

Although the ability to resize your pool of EC2 instances gives you great flexibility, it also imposes a management burden, because you must manually monitor the performance of your application and launch or terminate instances in response to changes in demand. In this application, we will use the Lifeguard project (http://code.google.com/p/lifeguard/) to automate the management of a distributed application running on instances in EC2.

The Lifeguard project has a number of capabilities, but the functionality we will focus on is its ability to monitor the load experienced by a distributed application and to respond to changes in load by ramping up or down the number of EC2 instances running the application. Lifeguard allows application owners to treat EC2 instances as a virtually unlimited pool of service components, and it provides the intelligence to add or remove instances from the pool as ...