{"id":14478,"date":"2016-08-24T12:15:45","date_gmt":"2016-08-24T09:15:45","guid":{"rendered":"https:\/\/www.webcodegeeks.com\/?p=14478"},"modified":"2016-08-21T23:01:25","modified_gmt":"2016-08-21T20:01:25","slug":"autoscaling-purpose-strategies","status":"publish","type":"post","link":"https:\/\/www.webcodegeeks.com\/devops\/autoscaling-purpose-strategies\/","title":{"rendered":"Autoscaling: Its Purpose and Strategies"},"content":{"rendered":"

In my previous article, \u201cAutoscaling on Complex Telemetry\u201d<\/a>, we discussed a method for determining effective autoscaled cluster size from internal application metrics. That article assumed you wanted to autoscale and so did not discuss under what conditions you might choose to, let alone what your choices for scaling are. Let\u2019s go ahead and delve into all that here.<\/p>\n

This article will use the autoscaling terminology of Amazon Web Services, but the discussion will be broadly applicable to other cloud providers.<\/p><\/blockquote>\n

What Is Autoscaling?<\/h2>\n

Autoscaling is a cluster-scaling technique to ensure that a compute cluster has just enough resources to meet demand, plus some use-case-determined safety margin.<\/p>\n

Consider how things were before convenient cloud hosting. Let\u2019s say you needed to host a website for what will be a very popular movie<\/a>. Close to the time of the film\u2019s release, load on your servers will be high but will tend to fall off over time. Servers take a long time to get provisioned \u2014 plugged into racks, software installed and configured \u2014 so you have to provision for peak consumption. This is frustrating. Time and money gets spent to decommission machines as traffic falls naturally over time, and even worse, the method is error prone. How do you determine traffic for something that doesn\u2019t yet exist? It\u2019s inefficient even during the period of peak traffic \u2014 most of these fancy machines will sit idle during the night.<\/p>\n

Autoscaling is also a capacity-planning technique that reduces the error from unknown traffic demands. It removes inefficiency by requisitioning computers when needed and decommissioning them when not.<\/p>\n

In AWS, this is done by one-hour allotments from a pool of existing machines that may be rapidly flashed to a new machine image. AWS uses the term \u201cgroup\u201d to describe a cluster of machines. Each \u201cgroup\u201d is set with a template for machine creation and a definition of the minimum, maximum, and desired number of machines to allow in the group. The desired number of machines will be maintained unless adjusted or<\/em> the minimum\/maximum bounds are violated.<\/p>\n

Implicit in this approach is the notion that each computer in the group can be treated as interchangeable and replaceable. This is a significant complication which might well offset the advantage to be gained otherwise from autoscaling. Recent work toward building non-trivial immutable infrastructures<\/a> points the way to reducing this complication, but it\u2019s an active research problem.<\/p>\n

A group\u2019s desired value is managed by a \u201cscaling plan.\u201d At its simplest, the plan is a fixed schedule: \u201c+10 computers at 10AM, -10 computers at 10PM.\u201d If a fixed schedule will not do \u201cdynamic scaling,\u201d via system telemetry, this might get more complicated.<\/p>\n

Scaling Plan Strategies<\/h2>\n

Being aware of and choosing an effective scaling plan is the trickiest bit of the autoscaling technique. Let\u2019s explore some scaling plan approaches and their tradeoffs.<\/p>\n

Fixed size<\/h3>\n

A fixed-size scaling plan is the null plan. No matter what happens, the scaling plan will keep the desired capacity of the autoscale group fixed.<\/p>\n

This approach can be useful in the prototype phase of a project when you\u2019re unsure of what the practical behavior of your system will be. A fixed-size autoscale group will let you determine per-machine performance characteristics. It\u2019s also much simpler to operate in a time when the prototype system likely has many unknowns associated with it.<\/p>\n

A production system that rolls out with a null scaling plan is perfectly acceptable. If you are aware that the system\u2019s demand will be relatively steady, it\u2019s extremely sensible to reduce the operational complexity of the deployed system by avoiding autoscaling.<\/p>\n

Similarly, if you believe that the demand on your system will grow only gradually and can tolerate potential under-capacity situations, manually adjusting desired targets is perfectly acceptable.<\/p>\n

Autoscaling is both an expense optimization and a safeguard against the unknown. If you don\u2019t have concerns in either regard, avoiding the additional complexity is well worth it.<\/p>\n

Scheduled adjustments<\/h3>\n

A conceptual step up from fixed-sized scaling is scheduled adjustments in fixed-scale sizes.<\/p>\n

Say you\u2019re processing sensor telemetry from a factory floor that only runs first and second shifts and the system can keep ahead of telemetry in online operation. This would mean, in a fixed-scaling approach, you\u2019d strictly have more computers than necessary from roughly midnight to 8 a.m. If the number of machines needed to support this factory during its operation are small, then this is not a concern. However, if the number of machines is large, you suddenly have an expense to optimize.<\/p>\n

In this case, setting a well-known time for computers to come on and offline is great. Scheduled adjustments suffer the same problem with regard to demand as fixed-size scaling.<\/p>\n

Fixed Increment adjustments (simple dynamic scaling)<\/h2>\n

\u201cSimple dynamic scaling\u201d adjusts the desired target of the autoscale group by some pre-configured amount. This is a break from fixed-size and schedule scaling in that the autoscale group is now a complex component of the overall system design with behavior that must be understood.<\/p>\n

Effectively choosing a criteria to drive dynamic scaling is a complex topic<\/a>, but a simple proxy for system health, CPU load, is often enough to get started.<\/p>\n

Say your cluster target is for no more than 50-percent CPU utilization to be used on any machine in the cluster in a five-minute period. This is easy to set up with Amazon\u2019s tools. But by how many computers should your cluster increase, or by what percentage should you cluster size increase?<\/p>\n

This is where discretion and experimentation comes in. Increase too much, and you\u2019ll spend more money than you might have otherwise and increased the cooperation burden of your machines more than you might have otherwise. Increase too little, and your system will fail to meet demand.<\/p>\n

It\u2019s a challenge when experimenting to avoid over-fitting to a certain pattern of traffic: a 20-percent increase in system demand each day at lunch is much, much different than a 100-percent increase at 8 p.m. because of a promotional deal.<\/p>\n

Fixed increment adjustments can be a challenge to get right and must be treated as a control system part and parcel of the developed system. Unfortunately, its limitations with regard to dealing with unusual traffic patterns and lack of fine-grained feedback (configuration is done via a set of threshold rules) means that this approach will never be entirely \u201cset and forget.\u201d<\/p>\n

This used to be Amazon\u2019s sole dynamic scaling option. With the introduction of step scaling, it is now superseded.<\/p>\n

Targeted increment adjustments (step scaling)<\/h3>\n

\u201cStep dynamic scaling\u201d is a superset of simple scaling. With this method, it is possible to set a target criteria for the cluster, measure the difference between the cluster\u2019s output and the target criteria, and fire one of a set of rules based on this difference.<\/p>\n

An example will be useful. Say, as before, we\u2019re scaling on maximum CPU load in the cluster, and we have an autoscale group configured with the following rules:<\/p>\n