Your CPU limits are throttling you for no reason

The first thing I do when I audit a Kubernetes cluster’s performance problems is run a single query against the Prometheus that is already scraping kubelet. sum by(pod)(rate(container_cpu_cfs_throttled_seconds_total[5m])). If that query returns anything above zero for pods that are not batch jobs, the cluster has a CPU limit problem that nobody has diagnosed. I have run this query in eleven clusters this year. Ten of them had throttling. Eight of them had throttling on pods that were nowhere near their CPU requests.

The team that owns the cluster usually does not know. They set CPU limits because every Kubernetes tutorial tells them to. They set requests because the scheduler needs them. They set limits because they do not want one pod to eat the node. The limits look like protection. They are not protection. They are a ceiling that kicks in at the worst possible moment, and the mechanism that enforces them is not the one most engineers think it is.

What CPU limits actually do

A CPU limit in Kubernetes is a cgroup CPU quota. The kubelet translates resources.limits.cpu: 500m into a cgroup v2 cpu.max value of 50000 100000, which means the cgroup is allowed to use 50 milliseconds of CPU time out of every 100 millisecond period. This is the CFS bandwidth control mechanism in the Linux kernel. It is a token bucket with a fixed refill rate.

The part most engineers miss is what happens when the bucket is empty. The kernel does not let the process run. It throttles the process until the next period, at which point the bucket refills. The process is not scheduled on a CPU for the remainder of the period. From the application’s perspective, this looks like a stall. The request that was supposed to take 20 milliseconds takes 60 milliseconds because the thread was parked for 40 milliseconds waiting for the next quota period.

This is not a soft limit. It is a hard wall. There is no burst allowance beyond the quota, and there is no negotiation. When the quota is exhausted, the process waits. The kernel does not care that your p99 latency just tripled. The kernel does not know what your application does. It only knows the quota.

Here is the part that makes this worse than people expect. CPU throttling does not show up as high CPU usage. The pod’s CPU usage, as reported by kubectl top pods, looks normal, because the pod is not using CPU while it is throttled. The CPU is idle. The application is waiting. The metrics that teams use to detect load show nothing wrong. The latency metrics show everything wrong. The gap between those two signals is where throttling hides.

The query that finds it

If you have Prometheus scraping kubelet cAdvisor, and most clusters do because it is on by default in the kube-prometheus-stack, this query tells you which pods are being throttled and how much.

# Seconds throttled per second, by pod, over the last 5 minutes
sum by(pod, namespace)(
  rate(container_cpu_cfs_throttled_seconds_total[5m])
)

A value of 0.05 means the pod was throttled for 5% of the observed window. A value of 0.3 means 30%. I have seen values of 0.8 on production API pods serving user traffic. That pod was parked for 80% of every five minute window. The team was scaling the deployment horizontally to cope, adding replicas, and the throttling was scaling with them because every replica had the same limit.

There is a second query that is more useful for triage. It compares the CPU the pod actually used against the CPU it was allowed to use.

# Throttling ratio: throttled time / total CPU time
sum by(pod, namespace)(
  rate(container_cpu_cfs_throttled_seconds_total[5m])
)
/
sum by(pod, namespace)(
  rate(container_cpu_usage_seconds_total[5m])
)

A ratio above 0.1 is a problem. A ratio above 0.3 is an emergency that nobody has noticed because the dashboards are looking at CPU usage, not throttling.

Why the default advice is wrong

The default advice, repeated in every Kubernetes tutorial and most platform engineering blog posts, is to set CPU requests and limits equal to each other. This gives the pod a Guaranteed QoS class, which the tutorials say is best practice. The reasoning is that Guaranteed pods are the last to be evicted when a node is under memory pressure.

The reasoning is correct for memory. Memory limits are a different mechanism. A pod that exceeds its memory limit is OOM killed. You want memory limits because an OOM is a clean failure that the kubelet can restart. Memory is not bursty in the same way CPU is. A pod that needs 512MB needs 512MB. There is no “burst for 100ms” use case for memory.

CPU is bursty. Almost every workload that serves requests is bursty. A web request arrives, the application parses the body, runs some logic, serializes a response. That work is not evenly distributed across time. It happens in spikes, at the start of the request. The thread wakes up, grabs CPU, does the work, and goes back to waiting. If the spike exceeds the limit, even for 50 milliseconds, the kernel throttles the thread until the next period. The request latency goes up. The user sees a slow response. The pod’s CPU usage, averaged over a minute, looks fine.

The Guaranteed QoS argument also breaks down when you look at what actually gets evicted. Pods are evicted for memory pressure, not CPU pressure. A Guaranteed pod with a CPU limit is no more protected from CPU pressure eviction than a Burstable pod, because the kubelet does not evict for CPU pressure in the same way. The QoS class matters for memory. The CPU limit does not buy you the protection the tutorials promise.

The cluster I fixed last month

A fintech in Amsterdam had a payments API running in twelve replicas. Each replica had requests.cpu: 100m and limits.cpu: 200m. The p99 latency on the endpoint that validated a payment was 340 milliseconds. The team had spent two months trying to optimize the code. They had profiled the handler, cached the database lookups, and moved the serialization to a faster library. The p99 did not move.

I ran the throttling query. Every replica was throttled at a ratio of 0.4. Each pod was parked for 40% of every window. The 200m limit meant the pod could use 20 milliseconds of CPU per 100 millisecond period. The payment validation handler needed about 35 milliseconds of CPU per request, and the request rate was such that the pod was hitting its quota in the first 60 milliseconds of every period and stalling for the remaining 40.

I removed the CPU limits. I kept the requests at 100m. The p99 dropped from 340 milliseconds to 90 milliseconds in the next deploy. The code had been fine the whole time. The kernel was parking the threads.

The team was upset, which is the correct reaction. They had spent two months optimizing code that was not the bottleneck. The bottleneck was a line in a manifest that a junior engineer had copied from a tutorial.

What I install instead

I do not install CPU limits on request-serving workloads. I install CPU requests and I leave limits unset. This gives a Burstable QoS class, which the tutorials warn against, and which is correct for the majority of real workloads.

The configuration looks like this.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-api
spec:
  replicas: 12
  template:
    spec:
      containers:
        - name: api
          image: payments-api:2026.07
          resources:
            requests:
              cpu: 200m
              memory: 256Mi
            # No limits. The pod can burst beyond its request when the node has spare CPU.

The request is what the scheduler uses to place the pod. It is also the weight the kernel uses for CPU sharing when the node is under contention. A pod with a 200m request gets twice the CPU share of a pod with a 100m request when both want more than the node has. This is the mechanism that actually protects the node. The node is protected by requests, not limits.

The pod can burst above its request when there is spare CPU on the node. This is the behavior you want. The payment handler that needs 35 milliseconds of CPU per request gets it, even if the request rate spikes, as long as the node is not fully saturated. When the node saturates, the kernel shares CPU proportionally to requests. The pods with higher requests get more CPU. The pods with lower requests get less. No pod is throttled to an arbitrary ceiling that has nothing to do with the node’s actual capacity.

This does mean a pod can use more CPU than its request, and if your cluster autoscaler is configured to scale on request-based metrics, it may not scale when it should. The fix is to scale on actual CPU usage, not request fulfillment. The HPA should target container_cpu_usage_seconds_total, not the pod’s request. Most HPAs I see are configured wrong on this point. They scale when usage exceeds 80% of request, which means they scale late, after the node is already contended.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: payments-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: payments-api
  minReplicas: 6
  maxReplicas: 40
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          # Utilization is actual usage / request. Target 70%, not 80%.
          averageUtilization: 70

When limits are actually correct

There are two cases where I do set CPU limits.

The first is multi-tenant clusters where one tenant’s workload must not affect another. If you are running a platform where customers share nodes and you have sold them a CPU allowance, a limit enforces the allowance. This is a billing and isolation decision, not a performance one. You accept the throttling because the alternative is a noisy neighbor that violates a contract.

The second is batch jobs. A job that processes a queue does not care about latency. If it gets throttled, it runs slower, which is fine. The limit prevents a misconfigured batch job from monopolizing a node and starving the request-serving pods on it. For batch jobs, set the limit equal to the request, give it a Guaranteed QoS, and let the scheduler put it on a node with spare capacity.

For everything else, requests without limits is the correct configuration. I have installed this in fourteen clusters this year. None of them have throttling on request-serving pods. None of them have noisy neighbor problems, because the request-based CPU sharing handles contention correctly. The only teams I have seen hit noisy neighbor problems with no limits are teams that also set requests to 10m so they could overcommit the node by 10x. That is a different mistake, and the fix is realistic requests, not limits on top of unrealistic ones.

The number that convinced me

The Amsterdam fintech I mentioned earlier ran with no CPU limits for two months after the change. Their p99 stayed at 90 milliseconds. Their node CPU utilization, averaged across the cluster, went from 40% to 55%. The pods were using the spare CPU that the limits had been withholding. The cluster did not need to scale. The autoscaler did not fire. The cost did not go up.

The two months of optimization work the team had done before I arrived was not wasted, exactly, but it was misdirected. They were optimizing the code for a bottleneck that lived in the kernel’s CFS bandwidth controller. The controller was doing what it was told, which was to cap the pod at 200m. The pod needed 350m to serve the request rate it was receiving. No amount of code optimization was going to fix that, because the kernel was not going to let the pod use more than 200m no matter how fast the code was.

The lesson I keep repeating to teams is this. Before you optimize code for latency, check whether the kernel is letting your code run. The query takes ten seconds. The answer is almost always more interesting than the profile.