← All insights

Nginx Upstream DNS: Recover When a Backend Address Changes

Investigate stale Nginx upstream addresses using effective configuration, resolver behaviour, version support, TLS identity, and a controlled backend change.

Nginx Upstream DNS: Recover When a Backend Address ChangesNginx Upstream DNS: Recover When a Backend Address Changes: Selected upstream; Authoritative DNS; Proxy resolver; Backend identity; Live request.. Conceptual teaching illustration; connections show the investigation flow rather than a vendor deployment topology.CLOUVENIX / SERVERSNginx Upstream DNS: Recover When a Backend Address Changes01Selected upstream02Authoritative DNS03Proxy resolver04Backend identity05Live requestConceptual investigation flow. Validate the actual environment and application recovery.
Nginx Upstream DNS: Recover When a Backend Address Changes: Selected upstream; Authoritative DNS; Proxy resolver; Backend identity; Live request.

Context

An application moves to a new private address, but Nginx keeps connecting to the previous one. DNS queries from the server show the correct answer, making the proxy look inexplicably broken. The key is how the configured upstream resolves names and when that resolution is refreshed. A shell lookup and Nginx's active upstream state are different observations.

Review the actual resolution method

Nginx documents upstream server resolution options, resolver configuration, and version-dependent availability. Dynamic resolution support has changed across releases, so check the running version, build, and syntax support before copying a configuration. A hostname in a proxy configuration does not imply every deployment automatically refreshes it at the cadence you expect.

Capture the effective configuration through the supported inspection command, handling sensitive values in output appropriately. Identify whether proxy_pass refers to a named upstream, a literal address, or a variable-based expression. These patterns can have different resolution and URI-handling consequences.

An illustrative private-service replacement

Imagine a reverse proxy pointing to a backend hostname whose address changes during an infrastructure replacement. New connections still target the old address, while an administrator's DNS query returns the new one. Reloading Nginx temporarily resolves the symptom.

Record the old and new addresses, DNS response and TTL, active Nginx configuration, and upstream address reported in access or error evidence. This distinguishes stale resolution from an application that is unavailable at the correct new destination.

Verify the resolver context

Check which resolver the Nginx configuration uses for the relevant dynamic lookup. It may differ from the operating system's ordinary resolver path. Confirm that it can resolve the private zone from the proxy's network environment.

Review address-family behaviour and the backend's actual listeners. A newly returned IPv6 address can create a different failure from stale IPv4 data. Keep the selected address and connection stage visible during testing rather than summarising every failure as DNS.

Keep TLS identity separate from address

For HTTPS upstreams, the correct IP address is only one part of the connection. Review the upstream hostname used for TLS server-name indication and certificate verification, along with the HTTP Host header expected by the application.

Do not disable certificate verification merely because an address change coincided with a TLS error. Inspect the new backend's certificate, trust path, and intended hostname. A DNS correction should not weaken the identity guarantee of the upstream connection.

Choose a supported refresh design

Use the documented dynamic-resolution mechanism available in the deployed Nginx version and configuration model. Review requirements such as shared upstream state where relevant. Validate the configuration with the installed binary before reload.

If using variable-based proxying, review URI construction, resolver requirements, and request-routing behaviour carefully. A workaround that refreshes DNS can also change how paths are forwarded. Test representative application URLs, redirects, and query strings rather than checking only the root endpoint.

Account for existing connections

Name resolution affects destination selection, but existing keepalive or long-lived connections may continue using a previously selected backend. Determine how the application and proxy retire those connections during replacement.

Keep the old backend available for the planned overlap where the architecture requires it. DNS changes alone do not terminate every connection immediately. A migration plan should align address publication, backend readiness, connection lifetime, and old-resource retirement.

Test the address transition

In a controlled environment, start with a known backend and change the hostname's address using the intended DNS process. Observe new connection destinations over time. Verify that the proxy reaches the replacement without an undocumented manual restart.

Repeat with a temporary resolution failure and multiple returned addresses where those cases matter. Confirm the application path, upstream TLS identity, and error behaviour. Record the observed transition time and its configuration context rather than promising a universal DNS refresh interval.

Quick diagnostic map

EvidenceInterpretationNext check
Shell DNS is fresh; proxy is staleResolution mechanism differsInspect selected upstream
TCP works; TLS failsHostname or trust differsVerify upstream identity
Old address remains activeConnection lifetime overlapsInspect existing connections

A focused operator exercise

Capture the running binary version and effective upstream configuration, then identify the resolver used by the relevant lookup path. Record the current authoritative answer, TTL, and actual upstream address selected for labelled requests. Keep shell resolution and proxy-selected destination as separate observations so a successful lookup cannot conceal stale proxy state.

Change a test backend hostname to a prepared replacement and observe new connections through the production-like request path. Verify URI forwarding, query strings, Host header, upstream TLS identity, and application response. Include an existing keepalive or persistent connection so address refresh is not confused with immediate retirement of every old connection.

Repeat with a temporary resolution failure and restore the valid answer. Confirm the proxy recovers through the intended supported mechanism rather than an undocumented manual restart. The acceptance receipt should state the observed transition, relevant configuration, and remaining client or connection overlap. This gives operators a reproducible cutover test whenever the backend platform or Nginx version changes.

Monitor the chosen destination

Include upstream address, connection errors, and relevant timing in operational evidence. Correlate them with DNS and deployment changes. This lets operators distinguish a correct fresh lookup to an unhealthy backend from an old address retained in the proxy.

Clouvenix can help review Nginx resolution and backend cutover together. The result is a proxy configuration whose address refresh and connection behaviour are tested against the service's actual replacement process.

Frequently asked questions

Does a successful dig command prove Nginx uses that answer?

No. It proves the lookup performed by that command. Inspect Nginx's configured resolution mechanism and the address selected for actual upstream connections, using the same time window as the application failure.

Is a reload the best permanent solution?

A controlled reload can be part of an intended deployment workflow, but first understand why resolution is stale. Choose a supported refresh or rollout design and test the complete address transition, including long-lived connections.

Explore Clouvenix nginx engineering for implementation and operational handover.

Technical reference

Official reference 1 Official reference 2