Context
A port remains open after an administrator stops an application service. On a socket-activated system, that can be expected. systemd may own the listening socket and start the service when a connection arrives. The listener and the application have separate unit lifecycles, so examining only the service can give an incomplete picture.
Before changing anything, identify the intended endpoint, the installed systemd version and the package that supplies the units. Arrange a test client and know whether the endpoint carries production traffic. Diagnostic commands below read state; stopping a socket or replacing a listener belongs in a planned change.
Trace the listener to its consumer
- Run
systemctl list-sockets --alland find the relevant address or path. Then inspectsystemctl status example.socketandsystemctl cat example.socket. The socket name need not perfectly resemble the application process name. Record the triggering relationship and compare the configured listen address with what the client actually uses. - Inspect both the associated service and any template unit. The
Accept=setting controls whether systemd passes listening sockets to a service or creates an instance for each accepted connection. A daemon must support the expected activation interface; enabling a socket unit does not teach an arbitrary program how to consume inherited descriptors. - Compare
ss -ltnpfor TCP listeners orss -lxnpfor local UNIX sockets with the unit definitions. Process information may require elevated read access. If a program also attempts to bind the same address itself, inspect its activation mode rather than repeatedly changing port numbers. Establish which component is supposed to create the socket.
An illustrative conflicting startup
Imagine an illustrative metrics collector upgraded from self-managed listening to packaged socket activation. An old local service override still supplies its --listen argument. The socket unit already owns the endpoint, so the program reports an address-in-use error. Killing the process shown as the listener disrupts the activation mechanism and leaves the configuration conflict unresolved.
| Observation | First inspection | Interpretation |
|---|---|---|
| Port open while service is inactive | systemctl list-sockets --all | Listener may be waiting to activate |
| Service starts after a probe | Socket-to-service relationship | On-demand activation is working |
| Address already in use | Inherited socket support and old bind flags | Two components claim listener creation |
| Many similarly named instances | Template service and Accept= | Per-connection activation may be intended |
Correct and verify the activation contract
Choose the package-supported model. Remove an obsolete self-binding option only after confirming the program supports inherited sockets. Alternatively, configure the supported standalone mode and disable its activation path through the distribution's normal mechanism. Preserve unit drop-ins so the change can be reversed without guessing at the vendor defaults.
For a local UNIX socket, examine its filesystem ownership and mode separately from the service account's data access. A socket can permit a client connection while the application later rejects the request. Do not broaden the socket to all users just to simplify a test.
Validate configuration, activate the planned units and connect with a harmless request. Record which unit started, which process handled it and whether the expected response returned. Repeat after a controlled service restart in staging. Also verify the intended idle state, because a service that remains resident may be appropriate even when activation is available.
Keep monitoring activation-aware
Document the socket unit alongside the service in the runbook. Monitor request success and failed activation, rather than alerting on an inactive on-demand service alone. List the endpoint owner, package mode and conditions for stopping both units during maintenance. Clouvenix Linux administration can help make listener ownership explicit before a migration.
Technical reference
systemd socket unit documentation. Newer options may be unavailable on older distribution builds.