Autoscaling Next.js on Kubernetes: HPA and KEDA

Short answer: for a Next.js application to scale properly on Kubernetes, the HPA is only the last piece. You first need measured CPU requests (typically 100 to 250 m per pod for light server-side rendering), a memory limit with no CPU limit, separate readiness/startup probes, a terminationGracePeriodSeconds consistent with the ingress drain, and a PodDisruptionBudget. Only then a HorizontalPodAutoscaler on CPU at 60-70%, complemented by KEDA when the load is a queue or scheduled traffic. Without that preliminary work, the HPA oscillates, doubles the bill and fixes nothing.

We regularly take over clusters where the HPA has existed for months without ever having served a purpose. Either it shows <unknown>/70% because the pods have no requests, or it climbs to maxReplicas at the slightest image compilation and comes back down twenty minutes later, or it scales while users get 502s because the old pods are killed before being removed from the ingress backend. This article covers the complete tuning process, in the order it should be done.

Why the HPA doesn't work as long as the requests are wrong

The HorizontalPodAutoscaler in Resource mode doesn't reason in absolute CPU terms. It computes a ratio: observed consumption divided by the container's declared requests. The Kubernetes documentation on the HPA is explicit — without resources.requests.cpu, the utilization metric is unavailable and the controller decides nothing.

The practical consequences break down into three cases you find everywhere:

So the first step is a measurement, not an estimate. On an existing deployment, with metrics-server installed:

# instantaneous consumption per pod
kubectl top pod -n prod -l app=web --containers

# over a period, if you have Prometheus
# rate(container_cpu_usage_seconds_total{namespace="prod",container="web"}[5m])

For a Next.js application in output: 'standalone' serving mostly prerendered pages and a few dynamic routes, we commonly measure 40 to 120 m of CPU per pod at rest under moderate traffic, with brief spikes at 400-600 m on server-side rendering or revalidation. Hence the configuration we apply by default:

resources:
  requests:
    cpu: 150m
    memory: 256Mi
  limits:
    memory: 512Mi

Two choices deserve justification.

No limits.cpu. CPU is a compressible resource: beyond the requests, the pod takes whatever is free on the node. Setting a limit activates CFS throttling (cpu.max in cgroup v2), and on a single-threaded Node.js runtime with short spikes, throttling translates directly into p95 latency. The phenomenon has been documented for years and the CFS burst fix isn't enabled everywhere. We leave CPU uncapped and protect the node through the requests of the other workloads.

A limits.memory, on the other hand, is necessary. Memory is incompressible: with no cap, a leak in one pod pushes the node into memory pressure and triggers evictions elsewhere. With Node.js 22, remember to align the heap: NODE_OPTIONS=--max-old-space-size=384 in a container limited to 512 Mi avoids a brutal OOMKill in favour of a readable JavaScript error.

Checking the QoS class

kubectl get pod -n prod -l app=web \
  -o custom-columns=NAME:.metadata.name,QOS:.status.qosClass

With requests.memory equal to limits.memory but no limits.cpu, you get Burstable. That's the right compromise: the pod isn't the first candidate for eviction, without reserving a whole core.

Probes: the prerequisite for scaling not to break anything

An HPA that creates three pods while the ingress sends traffic to pods that aren't ready yet improves nothing. The three probes to distinguish:

startupProbe:
  httpGet: { path: /api/health, port: 3000 }
  periodSeconds: 2
  failureThreshold: 30      # 60 s to start, then failure
readinessProbe:
  httpGet: { path: /api/health, port: 3000 }
  periodSeconds: 5
  failureThreshold: 2
livenessProbe:
  httpGet: { path: /api/health, port: 3000 }
  periodSeconds: 10
  failureThreshold: 3

The startupProbe is the one most often forgotten. It suspends liveness and readiness during startup, which avoids the classic vicious circle: a loaded node, a slow start, a livenessProbe that kills the pod, a restart, an even heavier load. A Next.js server in standalone mode generally starts in 1 to 3 seconds, but a pod waiting for its volume or a PostgreSQL connection can exceed 20 seconds.

As for the health route, keep it trivial and free of external dependencies:

// app/api/health/route.ts
export const dynamic = 'force-dynamic'

export function GET() {
  return new Response('ok', {
    status: 200,
    headers: { 'cache-control': 'no-store' },
  })
}

A readinessProbe that queries the database turns a PostgreSQL incident into a complete front-end outage: all the pods go NotReady at the same time and the ingress has no backend left. If you want to check the database, do it in a separate route consumed by monitoring, not by the kubelet. The same reasoning applies to the databases we describe in our comparison of PostgreSQL on Kubernetes versus a managed database.

Graceful shutdown, often the real cause of 502s

When the HPA reduces the replica count, or when a rolling update replaces pods, Kubernetes sends SIGTERM and removes the pod's IP from the Endpoints — both in parallel, not in order. The ingress (NGINX, Traefik, Kong) may keep routing to the pod for a handful of seconds.

terminationGracePeriodSeconds: 40
lifecycle:
  preStop:
    exec:
      command: ["sleep", "10"]

The 10-second preStop leaves time for the Endpoints to propagate. On the application side, you need to close the HTTP server on SIGTERM instead of letting the process die mid-request:

// server.ts (custom server) or a wrapper around standalone server.js
const shutdown = () => {
  server.close(() => process.exit(0))
  setTimeout(() => process.exit(1), 25_000).unref()
}
process.on('SIGTERM', shutdown)
process.on('SIGINT', shutdown)

If you deploy via the pipeline described in our article on GitLab CI/CD for Next.js on Kubernetes, these fields live in the same Helm chart as the HPA: they must evolve together.

The HPA manifest we deploy

The autoscaling/v2 API has been stable since Kubernetes 1.23 and brings the decisive field: behavior.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
  namespace: prod
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30
        - type: Pods
          value: 4
          periodSeconds: 30
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 50
          periodSeconds: 60

The parameters that matter:

• minReplicas: 2 and never 1. With a single replica, any eviction, any node drain, any update takes the service down. Two replicas on two different nodes is the bare minimum for a production site.
• averageUtilization: 65 rather than the default of 80. The full reaction time — metrics collection (15 s by default on the metrics-server side), the HPA loop (15 s), scheduling, pod startup — is measured in tens of seconds. At 80% of the requests, the headroom is consumed before the new pods arrive.
• scaleUp.stabilizationWindowSeconds: 0: scale up fast. scaleDown at 300 s: scale down slowly. The asymmetry is the heart of the tuning; it's what removes the flapping you see on HPAs left at their defaults.

To check that everything is wired up:

kubectl get hpa web -n prod
# NAME  REFERENCE        TARGETS         MINPODS  MAXPODS  REPLICAS
# web   Deployment/web   cpu: 41%/65%    2        10       3

kubectl describe hpa web -n prod | tail -20

If TARGETS shows <unknown>/65%, the problem is upstream: metrics-server missing, or requests.cpu absent. Go back to the first section.

The PodDisruptionBudget, inseparable from the rest

Without a PDB, a kubectl drain during a node update can evacuate all the replicas at once.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web
  namespace: prod
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: web

With minReplicas: 2, minAvailable: 1 is enough. Beyond 4 replicas, we switch to maxUnavailable: 25%. Also add a topologySpreadConstraints on kubernetes.io/hostname with whenUnsatisfiable: ScheduleAnyway, so the replicas don't all end up on the same node.

When CPU is no longer enough: KEDA

CPU is a good proxy for server-side rendering. It is useless for three common workloads.

Queue workers. A worker consuming a Redis or RabbitMQ queue spends its time waiting: its CPU stays at 5% while 8,000 messages pile up. KEDA (a graduated CNCF project since August 2023) solves exactly this case by exposing external scalers.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-emails
  namespace: prod
spec:
  scaleTargetRef:
    name: worker-emails
  minReplicaCount: 0
  maxReplicaCount: 20
  pollingInterval: 15
  cooldownPeriod: 120
  triggers:
    - type: redis
      metadata:
        addressFromEnv: REDIS_ADDRESS
        listName: queue:emails
        listLength: "50"

One pod per batch of 50 pending messages, and minReplicaCount: 0 — KEDA knows how to scale down to zero, which the HPA alone does not. In practice, KEDA creates and drives an HPA under the hood; you only manage the ScaledObject.

Predictable traffic. An SME site whose audience is French and European receives almost nothing between 1 a.m. and 6 a.m. The cron scaler avoids paying for useless replicas:

  triggers:
    - type: cron
      metadata:
        timezone: Europe/Paris
        start: "45 7 * * 1-5"
        end: "30 20 * * 1-5"
        desiredReplicas: "4"

Business metrics. Requests per second, HTTP queue length, p95 latency: the prometheus scaler hooks any PromQL query into autoscaling. It is often more relevant than CPU for Next.js, where an I/O-heavy request (API call, database) blocks a slot without consuming cycles.

    - type: prometheus
      metadata:
        serverAddress: http://prometheus-server.monitoring:80
        query: sum(rate(http_requests_total{service="web"}[2m]))
        threshold: "60"

A rule we apply: one autoscaler per Deployment, and only one. An HPA and a ScaledObject targeting the same object fight each other and produce create/delete cycles. If you move to KEDA, delete the HPA you wrote by hand.

What it really costs

Horizontal autoscaling only reduces the bill if the nodes follow. Otherwise you permanently pay for peak capacity. Two levers:

• Node autoscaling. OVHcloud Managed Kubernetes and Scaleway Kapsule both offer variable-size pools, with a minimum and a maximum per pool. The time to add a node is measured in minutes, not seconds: keep some pod headroom warm.
• Over-provisioning "pause" pods. A Deployment of negative-priority pods that reserve space and are evicted as soon as a real pod arrives. Cost: a fraction of a node, for a gain of several minutes on scale-up.

An order of magnitude on a setup we operate: three 4 vCPU / 8 GB nodes in a fixed pool, plus an elastic pool of 0 to 3 identical nodes. With requests of 150 m, a 4 vCPU node theoretically hosts about twenty web pods, in practice a dozen once the system, the CSI agent and monitoring are taken out. OVHcloud and Scaleway public pricing changes: take the current rates from their managed offer pages, and count storage and outbound bandwidth separately. We detail this approach on our infrastructure services page.

The most frequent billing trap remains the same: a maxReplicas set to 50 "just in case", a revalidation loop running wild, and a night at 40 replicas on an elastic pool. Set maxReplicas to what your budget tolerates, not to what the cluster can absorb, and add an alert on kube_horizontalpodautoscaler_status_current_replicas approaching the maximum.

Frequently asked questions

Should you set a CPU limit on a Next.js pod?

No, in the vast majority of cases. CFS throttling degrades p95 latency on a Node.js runtime with short spikes, and the requests is enough to guarantee fairness between pods. The exception: a cluster shared between several clients where a ResourceQuota or an internal policy mandates a limit. In that case, set it to at least 4 times the requests and monitor container_cpu_cfs_throttled_seconds_total.

Why does my HPA show <unknown>?

Three causes, in order of frequency: metrics-server isn't installed or isn't responding (kubectl top node fails), the container has no resources.requests.cpu, or the scaleTargetRef points to a resource that doesn't exist. kubectl describe hpa gives the exact reason in the ScalingActive and AbleToScale conditions.

HPA or VPA?

The VerticalPodAutoscaler adjusts a pod's requests, the HPA adjusts the number of pods. Both on the same metric conflict with each other. For a stateless web front end, the HPA is the right tool. The VPA is useful in recommender mode only, to measure the correct requests without applying anything automatically — that's how we use it during the first two weeks of a new deployment.

What about the Next.js ISR cache when the pods multiply?

That's the real point of attention. In standalone mode, the incremental revalidation cache is stored on the pod's filesystem: ten replicas, ten caches, ten regenerations of the same page. You need a shared cache through a custom CacheHandler (Redis, S3), or you accept the duplication. Without it, autoscaling multiplies the load on the database instead of spreading it.

How do you test that autoscaling works?

With a load generator, not in production on a Friday. k6, vegeta or hey run from a pod inside the cluster, against a representative dynamic route, for 10 minutes, with a kubectl get hpa -w running alongside. You validate three things: the HPA scales, the new pods reach Ready within the expected time, and the scale-down doesn't generate client-side errors. The same protocol is used to verify restores, as in our guide to Kubernetes backups with Velero.

In summary

• The HPA only works if resources.requests.cpu is measured, not estimated: 100 to 250 m for a typical standalone Next.js pod.
• No limits.cpu (CFS throttling and p95 latency), but a limits.memory aligned with --max-old-space-size.
• A distinct startupProbe, a health route with no database dependency, preStop: sleep 10 and terminationGracePeriodSeconds: 40 to eliminate 502s on scale-down.
• An autoscaling/v2 HPA at 65% utilization, minReplicas: 2, immediate scale-up and a 300 s scale-down via behavior.
• PodDisruptionBudget and topologySpreadConstraints every time, otherwise a node drain cancels out the whole benefit.
• KEDA for queues (scale to zero), scheduled traffic (cron) and Prometheus metrics — but only one autoscaler per Deployment.
• Plan for node autoscaling and a maxReplicas set against your budget, not against the cluster's capacity.
• On the Next.js side, a shared ISR cache is essential as soon as the replica count varies.

Preparing for a traffic peak, or seeing 502s on every deployment? Tell us about your cluster via our contact page: we audit the requests, the probes and the HPA before touching the infrastructure.