How an autoscaler actually decides, and why it always lags
An autoscaler is not watching your traffic. It is dividing two numbers every fifteen seconds, and it can only react after the load has already arrived.
The whole of the HPA is this, run on a loop:
desiredReplicas = ceil( currentReplicas × currentMetric / targetMetric )Watch it run against a traffic spike, and every complaint anyone has ever had about autoscaling becomes obvious.
A traffic spike, and an HPA targeting 70% CPU
1 / 6| Point | Average CPU across pods |
|---|---|
| 0s | 42% |
| 15s | 45% |
| 30s | 44% |
Steady state. Three pods averaging about 44% CPU, comfortably under the 70% target, so the formula returns the replica count it already has.
The four things the formula explains
Why scaling always lags. The metric has to be scraped, the loop has to run, the pod has to schedule, pull and pass readiness. That is tens of seconds minimum, and the load arrived first. An HPA cannot absorb a spike: it can only recover from one. If you need to survive the first thirty seconds, that is headroom or a queue, not autoscaling.
Why it overshoots. The formula uses the current metric, which is already past its peak by the time it is read. A short spike can leave you with double the pods you need for a load that has already gone.
Why resources.requests is the most important field in the manifest. CPU
utilisation is measured against the request, not against the node. Set
requests: 100m on a pod that genuinely uses 500m and you are permanently at
500%, and the HPA scales to its ceiling and stays there. A wrong request does not
make the HPA slightly off; it makes it meaningless.
Why scale-down is deliberately slow. Coming down immediately means the next small spike scales you straight back up, and the one after that, forever. The five-minute stabilisation window is a damper, not a bug.
| Horizontal (HPA) | Vertical (VPA) | |
|---|---|---|
| What changes | The number of pods. | The CPU and memory requests of each pod. |
| Disruption | None: new pods are added alongside. | The pod is usually restarted to resize. |
| Ceiling | Cluster capacity. | The largest node you have. |
| Fits | Stateless services behind a load balancer. | Workloads that cannot be split, like a single writer. |
| Speed | Seconds to a minute, plus startup. | A restart, so slower and more disruptive. |
| Together? | Only on different metrics: both on CPU and they fight each other. | Best used to *set* the requests the HPA then measures against. |
Making it less bad
- Set
minReplicasfrom your real floor, not from one. Scaling from one pod means the first spike hits a single pod at 100%. - Get
requestsright first. Everything above depends on it. The VPA in recommendation mode is a good way to find the true number without letting it resize anything. - Tune
scaleUpseparately fromscaleDown. They are different problems: up should be fast and generous, down should be slow and cautious. - Scale on the metric that actually saturates. For a queue consumer that is queue depth, not CPU: a worker blocked on I/O sits at 5% CPU while the backlog grows for an hour.
Remember this
- 1desired = ceil(replicas × current ÷ target). Every autoscaler surprise comes out of that one line.
- 2Utilisation is measured against resources.requests, so a wrong request makes the HPA meaningless rather than merely inaccurate.
- 3An HPA recovers from spikes, it does not absorb them: the load always arrives before the pods do.