Context
An Application Load Balancer returning 502 and one returning 504 can show the same customer symptom: a request failed at the edge of the application. Their causes can differ substantially. Start by establishing who generated the status and which stage of the connection failed; increasing every timeout usually obscures that distinction.
Separate load-balancer errors from target errors
Inspect HTTPCode_ELB_5XX_Count alongside target error metrics and ALB access logs. A target can itself return a 502 or 504 through a healthy load balancer. The client-facing status alone does not identify the responsible component.
For one request, retain the timestamp, target address, listener and target group, load-balancer status, target status, processing times, and a correlation identifier where available. Avoid copying authentication headers or full sensitive URLs into incident tickets. Check whether a deployment or target replacement occurred in the same interval.
Investigate 502 as a connection or response problem
AWS documents causes including a target reset, an unexpected close while a request is outstanding, a malformed response, and TLS handshake failure to a target. The exact evidence determines which branch applies.
If resets cluster around reused connections, compare the target's keep-alive duration with the ALB's idle timeout and actual traffic gaps. If errors appear during releases, inspect target deregistration, active request duration, application shutdown, and whether the process exits before draining completes.
For malformed responses, examine the application's output path and intermediary server logs. A healthy TCP listener can still emit invalid headers. For HTTPS targets, test protocol and certificate-handshake behavior from the relevant network path. Do not disable verification on unrelated clients as a substitute for diagnosing the ALB-to-target connection.
Investigate 504 as an incomplete connection or response
A 504 can arise when the ALB cannot establish the target connection or when an established target fails to respond before the idle timeout. Network ACL return-path restrictions and a response body shorter than its declared content length are other documented possibilities.
Split target connection time from application execution. A slow database query is a different problem from a SYN that never receives an acceptable response. Trace the target port, security groups, ACLs, listener, and backend dependency timings using a narrow test request.
Align deadlines across the client, ALB, proxy, application, and downstream calls. Give the application a chance to return a useful failure before its caller abandons the request. A longer outer timeout can increase concurrent waiting work and worsen an already saturated database.
An illustrative deployment regression
A service produces 502s only during rolling releases. Its application exits shortly after receiving a termination signal, while several reports are still running. The target group remains otherwise healthy, and ordinary short requests pass every smoke check.
The team reproduces a release with an active report and correlates target shutdown with the reset. It adds a tested drain sequence, bounds long-running work, and verifies the deregistration timing. Increasing the ALB idle timeout alone would not keep a process alive after it exits.
Quick diagnostic map
| Evidence | Hypothesis | Next check |
|---|---|---|
| ELB 502; target resets | Connection closed unexpectedly | Keep-alive and shutdown timing |
| ELB 504; target connection fails | Reachability boundary | Target listener, rules, and return path |
| ELB 504; dependency span is long | Backend execution delay | Query, pool, and request deadline |
| Target returns 504 itself | Target-side proxy or application | Target logs and upstream timing |
A focused operator exercise
In a test environment, create one slow-but-valid request and one target that closes an outstanding connection. Observe the different log fields and customer outcomes. Restore normal behavior before testing a rolling deployment with both short and long requests active.
Capture the accepted maximum request duration and recovery behavior when a client retries. Verify that a timeout after a business write does not create a duplicate operation. Keep the load generator bounded so the diagnostic experiment does not become a capacity incident.
Verify the entire request journey
Confirm the affected operation completes correctly after the change, including a release cycle and an idle connection interval. Compare error rates by target and status origin. Document any deliberately asynchronous replacement for long requests and how callers retrieve completion.
Frequently asked questions
Does a healthy target guarantee no 502 or 504 errors?
No. A lightweight health probe may pass while a particular response, dependency, or shutdown path fails.
Should the ALB timeout match the application timeout exactly?
Design an ordered deadline budget with enough time to return a controlled response. Identical values can create races; the right values depend on the request contract.
Continue the investigation
- AWS ALB Unhealthy Targets: Follow the Health-Check Contract
- ALB Deregistration Delay: Finish Active Requests During a Deployment
Browse the cloud, server and infrastructure guide library for related operational questions.
Explore Clouvenix Network Architecture & Operations to discuss implementation, validation and operational handover.