Autoscaling Explained: Horizontal, Vertical and Scheduled

Elasticity Is a Design Choice, Not a Setting

Autoscaling is sold as the cloud's headline feature: capacity that grows when demand arrives and shrinks when it leaves, so you stop paying for a peak that happens twice a year. The mechanism is real. The catch is that it only works on workloads built to be replaced at any moment.

Plenty of teams enable a scaling group, watch it add instances during an incident that had nothing to do with load, and conclude the feature is broken. It was doing exactly what it was told. The instructions were wrong.

This article separates the three kinds of scaling, describes what each platform actually provides, and is honest about when a fixed size fleet is the better engineering decision.

Three Ways to Add Capacity

Horizontal: more instances

Add or remove identical machines behind a load balancer. This is what people usually mean by autoscaling. It has no ceiling short of provider quotas, it improves fault tolerance as a side effect, and it demands that any instance can disappear without taking state with it.

Vertical: a bigger instance

Resize the machine to more processors or memory. It suits workloads that genuinely cannot be split, most often a primary database. On most platforms a resize requires a stop and start, so it is a planned change rather than an automatic reaction, and it stops at the largest instance type available.

Scheduled: capacity on a clock

Change the fleet size at known times. Deeply unfashionable and frequently the correct answer, because a lot of demand is predictable: business hours, a nightly batch window, a weekly report run. Scheduled scaling reacts instantly because it does not have to detect anything first.

These combine. A common production pattern is a scheduled floor that raises the minimum before the morning rush, with a reactive rule on top to absorb whatever the schedule did not anticipate.

What the Application Has to Be First

Horizontal scaling assumes instances are interchangeable. Four properties make that true, and skipping any one of them produces intermittent bugs that are miserable to diagnose.

• Sessions live in a shared store or a signed token, never in process memory.
• Uploads and generated files go to object storage, not the local disk.
• Configuration and secrets are fetched at start up rather than baked in by hand.
• Start up is fast, because a machine that needs six minutes to become useful cannot respond to a surge.

That last point is where most of the engineering effort goes. Build a prepared image, or bootstrap with a short first boot script, rather than installing packages and pulling dependencies while users wait.

The load balancer in front needs to cooperate too. It must add new instances only once their health check passes, and drain connections from instances being removed. The distinction between connection level and request level balancing is covered in cloud load balancers explained.

What Each Platform Calls It

The virtual machine constructs share a shape: a template describing the instance, a minimum and maximum count, a health check, and policies that adjust the desired count. The EC2 Auto Scaling user guide is a thorough reference for the model even if you deploy elsewhere.

In Kubernetes the same job splits in two. The horizontal pod autoscaler changes replica counts based on metrics, while the cluster autoscaler adds nodes when pods cannot be placed. Both are documented in the Kubernetes autoscaling documentation, and running them on a small team is discussed in managed Kubernetes for small teams.

Smaller providers generally give you the primitives rather than the product: an API to create and destroy servers, plus a load balancer you can attach members to. Teams there typically write a small controller against the provider API, which is more work but keeps the logic visible.

Picking the Signal to Scale On

Processor utilisation is the default and often a poor proxy. An application waiting on a database is barely using its processors while queueing badly, so utilisation stays flat as latency climbs.

Better signals describe work rather than resource use. Requests per instance, queue depth for a worker fleet, or active connections all correlate with what needs doing. Target tracking policies, where you name a value to hold steady and let the platform compute the count, are easier to reason about than a pile of threshold rules.

Whatever the signal, the metric pipeline has to be trustworthy first. Scaling rules are automation built on top of your monitoring, so the collection and alerting foundations in monitoring cloud servers come before any policy.

Set the maximum count with deliberate thought. It is your blast radius limit when a bug or a scraper generates artificial load, and it is also bounded by the compute quotas attached to your account. A scaling group that cannot reach its maximum because of a quota will fail quietly at the worst moment.

The Ways It Goes Wrong

Flapping. Aggressive rules in both directions add instances, see the metric drop, remove them, and repeat. Cooldown periods and asymmetric behaviour fix it: scale out quickly, scale in slowly and conservatively.

Scaling the wrong tier. If the bottleneck is a single database, adding web servers increases pressure on it and makes the outage worse. Confirm where the queue is before automating a response to it.

Amplifying an incident. A dependency failure causing retries looks exactly like a traffic surge. Without circuit breakers, autoscaling responds by pointing more clients at a struggling backend.

Termination without warning. Scale in events remove instances, sometimes mid request. Handle the termination signal, finish in flight work, then exit. The same graceful shutdown behaviour that deployments need serves double duty here, as covered in blue-green deployments.

Simpler Answers That Are Often Right

Autoscaling earns its complexity when demand varies by a large factor and is hard to predict. Below that, several alternatives cost less to operate.

Over provisioning slightly and leaving it alone is a legitimate strategy for steady workloads. A cache or a content delivery network in front can flatten a spiky read pattern entirely. A single larger instance, resized during a maintenance window, handles plenty of businesses whose traffic doubles over a year rather than over an hour.

And where the workload is genuinely intermittent, a serverless runtime scales to zero without you operating anything at all.

If you do want the full managed experience, the hyperscalers have the deepest implementations. The Google Cloud account listings describe an account with managed instance groups available, and the Azure versus Google Cloud comparison covers how the two model scaling differently.

Frequently Asked