Context
An application can have spare CPU and memory while new outbound connections fail. The bottleneck may sit at the shared egress layer, where many private workloads use a NAT gateway to reach public destinations. Adding more application instances can increase that pressure if each instance opens another large set of short-lived connections.
Imagine an illustrative reporting platform that calls one external API during a scheduled export. Interactive requests work until the export starts. Then connection establishment slows across unrelated workers. The investigation should follow the shared destination and connection lifecycle before assuming that every affected server needs more capacity.
Separate connection establishment from application responses
A timeout while opening a TCP connection is different from an HTTP response saying the external API is overloaded. Capture the phase of failure: DNS lookup, connection establishment, TLS negotiation, waiting for headers, or reading the body. Use the runtime's structured error fields instead of treating every exception as the same network problem.
AWS documents NAT gateway connection limitations and relevant troubleshooting signals, including ErrorPortAllocation when source ports cannot be allocated. Connection pressure is associated with destination address and port combinations, so aggregate traffic volume alone is an incomplete view. Idle connection handling is another documented behaviour to consider when analysing resets. 1
Build a timeline around one destination
Identify the NAT gateway used by the affected subnets, then correlate its metrics with application connection errors and the job schedule. Record destination hostnames and resolved addresses during the incident. Several hostnames may resolve to the same address, while one hostname may spread traffic across several addresses. The observed connection pattern matters more than a logical service label.
Review workers, scheduled jobs, and other applications sharing that egress path. A quiet frontend can suffer because a batch worker consumes shared connection capacity. Compare successful and failed requests within the same minute. If failures occur before TLS and cluster around one public destination, that is stronger evidence than a general report of slow internet access.
Check what the client does with connections
Inspect whether the application reuses a long-lived HTTP client or constructs a new connection pool for every request. Look at idle pool limits, active connection limits, connection lifetime, request cancellation, and whether response bodies are properly consumed or closed. A pool is useful only if its lifecycle matches the process and its resources are released.
Calculate the fleet-wide effect. A setting of a few hundred connections per worker can become tens of thousands after horizontal scaling. Include deployment surge instances and retry workers in the estimate. Keep a separate budget for the external service's own concurrency or rate limits; more egress capacity does not grant more permission to call that service.
Reduce pressure before multiplying it
Temporarily reduce nonessential batch concurrency when the evidence supports shared egress contention. Preserve interactive work by giving it a deliberate allocation rather than letting every producer compete without limits. A queue can smooth arrival rate, but only if dispatch concurrency is bounded and the queue's growing age is monitored.
Use exponential backoff with jitter for eligible transient errors, and set a total retry deadline. Retrying immediately from every worker can turn a small shortage into a sustained storm. Operations should be able to distinguish initial attempts from retries in dashboards. Do not retry operations with external side effects blindly; their business idempotency requirements remain separate.
Evaluate architecture changes with the network owner
Possible changes include additional supported NAT address capacity, distributing egress appropriately, or using private service connectivity for eligible destinations. Validate current service capabilities, quotas, routing, and pricing before choosing a design. A diagram that shows several gateways does not prove traffic actually uses them as intended.
Consider availability and allowlists. If a partner only accepts a known public address, an egress change may cause legitimate traffic to be rejected. Update the partner coordination plan and test both the normal and recovery paths. Avoid moving one workload's problem to another shared gateway that has no capacity budget of its own.
Validate with a controlled burst
Reproduce the workload using a bounded test, with partner permission where necessary. Observe useful completed requests, connection establishment failures, pool wait time, gateway metrics, and downstream response codes. Stop the test if customer traffic degrades or the external service starts rejecting load. A test that only counts attempted requests rewards a broken retry loop.
Compare equivalent work before and after. If throughput improves only because more requests are retried, the system may still be unstable. A successful result combines lower connection failure rates, acceptable request latency, bounded retries, and a queue that returns to its normal age after the burst.
Make the lesson usable next time
Record the egress owner, shared workloads, connection budgets, and the signals that distinguish NAT pressure from DNS or partner-side failure. Include the temporary concurrency reduction that helped and the conditions for reversing it. Keep configuration changes tied to a measured workload rather than a universal connection count.
A focused operator exercise: A concurrency budget example
Assume a test fleet has ten workers, each configured to open up to two hundred outbound connections. That creates a potential two thousand-connection demand before deployment overlap or retries. The numbers are an illustrative budgeting exercise; compare the real destination distribution with current gateway and partner constraints before selecting an operating limit.
Repeat the test with a shared, bounded client pool and lower batch concurrency. Measure successful external operations and interactive latency rather than the number of attempted connections. If the same work completes with fewer connection failures and a queue that drains predictably, the result supports changing the client and admission policy before adding more servers.
Quick diagnostic map
| Evidence | Review | Acceptance boundary |
|---|---|---|
| Connection demand | Correlate destination traffic with connection counts | Bound concurrency and reuse sockets |
| NAT evidence | Inspect allocation errors and traffic patterns | Separate port pressure from routing failure |
| Retry policy | Measure attempts during dependency failure | Prevent retries multiplying connection demand |
Frequently asked questions
Should we add application servers first? Only if application capacity is the constrained resource. Under shared outbound pressure, additional instances can worsen the failure.
Clouvenix can review the client pool, batch schedule, and egress topology together, producing a capacity decision based on completed business work rather than CPU utilisation alone.
For a scoped review of this workflow, discuss the relevant Clouvenix engineering service.