Context
When every path to a multipath device fails, the application's behavior depends partly on the configured queueing policy. It may wait for storage to return or receive an I/O error after the supported retry behavior. Decide that contract with the storage and application owners before an outage, because either choice affects recovery.
Identify the logical device and its paths
Record the LUN's stable identifier, multipath map, underlying paths, storage array, transport, filesystem, and application. Use supported read-only status commands such as multipath -ll and the installed multipathd console diagnostics where appropriate.
Confirm that the observed failed paths belong to the intended logical device. Similar device names and changing kernel enumeration can lead to the wrong volume being diagnosed. Keep array-side and host-side identities connected in the runbook.
Distinguish one-path failure from all-path failure
A failed path with another healthy path should be evaluated against the expected failover behavior. All paths unavailable is a different event and can expose retry, timeout, and application assumptions that a routine single-cable test never exercised.
Inspect multipathd status, path checker results, transport errors, fabric connectivity, and storage-array health. A stopped multipath daemon and a genuinely inaccessible array can produce related symptoms but require different corrections.
Record when the application stopped making progress relative to path failure. A database can hold threads and transactions while I/O is queued, causing symptoms elsewhere in the service before a storage error is returned.
Review queueing policy as an application decision
Red Hat documents that queue_if_no_path behavior can leave processes waiting until a path returns. Its guidance also describes bounded retry options such as no_path_retry, with configuration interactions that must match the deployed version and supported array profile.
Changing from queueing to failure can release waiting tasks by returning errors, but the application must recover correctly from those errors. It can also affect filesystems and clustered software. Do not change the policy solely to make a blocked command return without understanding the data and failover contract.
Coordinate transport timeouts, multipath policy, filesystem behavior, database recovery, and cluster fencing. A cluster that declares a node dead while the old node is still queueing writes needs a deliberate ownership design to prevent unsafe rejoin.
An illustrative misleading hang
A database server loses both storage-fabric paths during maintenance. User sessions stall and the operator repeatedly restarts application workers, but the underlying I/O remains queued. The added workers increase the number of waiting operations without restoring access.
The storage team restores a supported path and observes application recovery. The follow-up review defines the intended all-path failure behavior and tests it with the database and cluster owners. The accepted policy is based on recoverable state, not merely on whether a command exits quickly.
Quick diagnostic map
| Evidence | Boundary | Next check |
|---|---|---|
| One path failed; I/O continues | Redundancy operating | Remaining path health and repair |
| All paths failed; tasks wait | Queueing policy active | Effective map policy and transport |
| Paths healthy; daemon unhealthy | Host management issue | multipathd state and logs |
| Paths return; application remains broken | Recovery incomplete | Filesystem, database, and fencing state |
A focused operator exercise
Use an isolated LUN containing disposable application data. Verify backups and record the effective multipath and transport configuration. Rehearse a single-path loss first, then a controlled all-path loss through the storage vendor's supported procedure.
Observe application progress, task states, error delivery, and recovery after paths return. Test the intended cluster fencing and rejoin sequence where relevant. Confirm data correctness with the actual application. Retain exact timeout and policy settings; a successful single-path test does not establish all-path behavior.
Keep recovery access independent
Ensure operators can inspect the host and storage fabric when the affected LUN is unavailable. Store essential runbooks and management tools on an appropriate independent path. A procedure located only on the failed filesystem is not available when needed.
Monitor path-count changes and repeated flapping before redundancy is fully lost. Treat an extended single-path state as reduced protection requiring ownership, even if the application still appears healthy.
Frequently asked questions
Is queueing indefinitely always safer than returning an error?
Neither behavior is universally correct. Match supported storage settings to the application's timeout, recovery, and fencing requirements.
Will restarting the application restore failed storage paths?
Not by itself. Diagnose the fabric, array, host path state, and multipath service before adding more waiting work.
Continue the investigation
- Small requests work but transfers stall: investigate path MTU
- A recovery SOP needs business acceptance, not only a started server
Browse the cloud, server and infrastructure guide library for related operational questions.
Explore Clouvenix NAS, SAN & File Server Infrastructure to discuss implementation, validation and operational handover.