Horizontal autoscaling tuned for cost.

Here is how each lever works and how to set them without trading away the latency the autoscaler was protecting.

Why does HPA so rarely cut the bill?

Three defaults conspire to keep cost high. The minimum replica count is usually set for peak comfort, so the workload never drops below a floor that was sized for the busiest hour. The target utilization is set conservatively low, often a CPU target in the forties, so the autoscaler adds pods early and removes them late. And the scale down behaviour is left at defaults that stabilise aggressively against any dip, so a workload that could halve overnight instead coasts at daytime size until morning.

The result is a setup that is excellent at absorbing spikes and useless at recovering money. It looks healthy on a latency dashboard and bleeds on the invoice. Because the cost lands in a shared cluster bill rather than against the service, nobody sees the idle pods unless someone allocates cost back to the workload. The cure is to tune the three controls deliberately rather than accept the safe defaults, and to make the cost visible so the team has a reason to.

What metric should the autoscaler watch?

CPU is the default and the wrong choice for most services. Scale on the signal that genuinely drives the work. For web and API services that is requests per second or in flight concurrency, exposed as a custom metric, so capacity tracks traffic directly instead of guessing from CPU. For queue and stream workers it is queue depth or consumer lag, so pods follow backlog and drain it without overshooting. Only for genuinely compute bound services, such as encoding or model inference on CPU, is processor utilization the metric that maps to real demand.

Matching the metric to the workload does two things. It scales up at the right moment, so you are not adding pods on a CPU blip that does not reflect user load, and it scales down honestly when demand falls, because the metric drops with the traffic. A request based target also lets you reason about cost per request, which is the number an engineering lead can actually defend in a spend review.

How should you set the floor, the ceiling, and the scale down window?

The minimum replica count should reflect the real overnight floor plus enough headroom to survive a single pod failure, not the daytime peak. For many services that is two or three replicas, not the eight that quietly run all night. The maximum should be high enough to absorb the real peak so you never sacrifice latency, since the ceiling only costs money when it is actually reached. The target utilization should sit high enough to pack pods efficiently but below the point where a sudden spike outruns the scale up, which is where stabilisation and the scale up policy earn their keep.

The scale down window is where most of the saving hides. A short, sensible stabilisation period lets the autoscaler return capacity within minutes of demand falling rather than holding peak size for an hour of caution. Pair that with the right metric and a workload that doubles for the business day will genuinely halve overnight, which on a five day pattern removes idle capacity for the majority of every week.

A worked example of tuning the three controls

The change broke nothing because the ceiling stayed high. Capacity was always available when demand asked for it. What changed was that the cluster stopped paying for capacity nobody was using.

How does this fit with node autoscaling and commitments?

Pod autoscaling only saves money if the cluster gives back nodes. HPA changes replica count, the cluster autoscaler changes node count, and the two have to be tuned together so that fewer pods actually translate into fewer nodes rather than emptier ones. Bin packing, pod disruption budgets, and node group sizing all influence whether scaled down pods free a whole node or just leave it half full.

Commitments sit underneath both. Savings Plans and Reserved Instances on AWS, Reservations and the Azure Savings Plan on Azure, Committed Use Discounts on GCP, and Universal Credits on OCI should cover the stable node baseline the cluster never drops below, while the variable tail rides on demand or on spot and preemptible capacity for the interruptible work. Cover the floor with commitments, let autoscaling handle the peak, and you capture the discount on the steady part without carrying utilization risk on the part that moves.

Tune autoscaling and the cluster gives money back

We tune Kubernetes autoscaling, requests, and commitment coverage together across AWS, Azure, GCP, and OCI, with zero provider commissions and engineering reality respected: optimization that breaks production is not optimization. Our guarantee: we reduce your cloud spend or we reimburse our service fee. Pricing is either a Fixed Fee scoped up front or Gainshare, a share of verified savings with no retainer and no risk. Book a strategy call and we will show you where idle pods are costing you this week.