Context
A container can show modest average CPU usage while short bursts repeatedly hit its CPU quota. The resulting pauses can increase tail latency even when the node has spare capacity. Diagnose quota enforcement, application scheduling, and workload demand together before removing limits or adding replicas.
Separate scheduling requests from enforcement limits
CPU requests influence scheduling and allocation under contention. CPU limits constrain execution through the platform's enforcement mechanism; on Linux this involves cgroups and throttling. These settings answer different questions and should not be interpreted as interchangeable capacity numbers.
Inspect the admitted resources for every container and any relevant Pod-level configuration supported by the deployed version. LimitRange, policy, and platform defaults can supply values absent from the original manifest. Record the effective configuration and cgroup version with the incident evidence.
Correlate throttling with latency
Collect request duration, throughput, runnable work, CPU usage, and supported throttling counters over the same interval. On Linux, the relevant cgroup cpu.stat information can help explain enforcement; the exact fields differ by cgroup version and collector.
If using cAdvisor-style metrics, compare changes in throttled periods with total periods and request behavior. A high ratio of periods containing some throttling is not the same as the fraction of all CPU time lost. Keep the metric definition explicit so a dramatic percentage does not become a misleading capacity claim.
Use short enough observation intervals to see bursts. A process that briefly needs several threads and then waits can remain below a long-interval average while repeatedly exhausting its allowed execution within quota periods.
Test alternative explanations
A busy garbage collector, lock contention, a single-threaded hot path, or an overloaded dependency can also increase latency. Correlate profiles and queueing with the throttling evidence. Do not attribute an incident to one nonzero counter that also appears during healthy operation.
Check node contention separately. Raising a container limit can help a quota-bound workload on a node with headroom, but it may simply transfer pressure to neighboring workloads on a saturated node. Requests, placement, and total allocatable capacity need to remain coherent.
Review autoscaling interaction. An HPA using CPU utilization relative to requests may behave differently after requests change. A capacity correction should include scaling behavior and replica recovery, not only a single-Pod benchmark.
An illustrative low-average latency spike
A JSON-processing API handles brief parallel bursts. Its five-minute CPU graph looks safe, but request p99 rises during each burst and quota counters increase. Adding application threads worsens waiting because more runnable work competes inside the same quota.
The team tests bounded concurrency and a reviewed resource adjustment. It measures the same request mix and checks neighboring services. The accepted configuration meets tail-latency and capacity objectives without assuming that every workload should run without CPU limits.
Quick diagnostic map
| Evidence | Hypothesis | Next check |
|---|---|---|
| Throttle deltas align with p99 spikes | Quota-bound bursts | Effective limit and short-interval demand |
| Throttling absent; one thread saturated | Serial hot path | Application profile |
| Limit raised; neighbors slow | Node contention | Requests and aggregate capacity |
| Request change alters HPA behavior | Scaling denominator changed | Utilization target and replica response |
A focused operator exercise
Run a representative burst pattern in a test environment with the production resource settings. Record the effective Pod specification, cgroup version, throttling definitions, CPU use, request percentiles, and correct responses.
Compare one limit or concurrency change at a time. Include steady traffic and a competing workload so the test does not assume an empty node. Exercise the HPA if requests change. Accept the candidate only when both the target service and neighboring workloads meet their agreed objectives, and retain a configuration rollback.
Turn observations into a resource policy
Define how teams choose CPU requests, when limits are appropriate, and which evidence justifies exceptions. Include admission rules and platform defaults in that policy so the running configuration is predictable.
Alert on meaningful combinations such as sustained latency degradation with relevant resource pressure. A throttling counter alone can describe normal enforcement without requiring an incident. Revisit the profile after major runtime or application changes.
Frequently asked questions
Does low average CPU rule out throttling?
No. Short execution bursts can encounter quota enforcement while a longer averaging interval looks modest.
Should I remove all CPU limits?
Choose a policy based on workload evidence, node isolation, and platform requirements. Test the service and its neighbors rather than applying a universal rule.
Continue the investigation
- Kubernetes HPA Not Scaling: Verify Metrics and Resource Requests Before Changing Targets
- Linux PSI: Read Resource Stalls Alongside Service Latency
Browse the cloud, server and infrastructure guide library for related operational questions.
Explore Clouvenix Server & Website Performance Troubleshooting to discuss implementation, validation and operational handover.