Context
A PostgreSQL primary can accumulate write-ahead log files because a replication consumer has stopped advancing. The database may still serve requests normally until storage headroom becomes small. At that point, an apparently secondary integration problem threatens the primary service. The response must protect the database while preserving a deliberate plan for the stalled consumer.
Consider an illustrative change-data-capture pipeline paused for maintenance. The maintenance lasts longer than expected, but no alert tracks retained WAL growth. Several days later, disk usage accelerates during a busy period. Deleting files from the WAL directory would risk the database; the team needs to identify the retention cause and choose a supported action.
Understand the retention contract
Replication slots retain information needed by consumers. PostgreSQL exposes slot state through pg_replication_slots and provides replication settings that influence retained WAL and slot behaviour. Relevant fields and controls vary by version and slot type. A retention limit can protect capacity while also making a lagging consumer unable to continue without recovery or resynchronization. 123
Record the database version, slot type, consumer owner, and expected recovery procedure before changing settings. An inactive slot may be temporarily paused or genuinely abandoned. Those cases should not receive the same automatic cleanup action.
Inventory slots and their owners
List slot names, active state, relevant restart and confirmed positions, and the associated consumer where known. Match each slot to a replication service, subscriber, backup tool, or application. A slot with no documented owner is an operational gap even if it currently advances normally.
Use supported database functions and views to estimate retained WAL distance, with attention to null values and version-specific semantics. Keep logical and physical slot interpretation separate. The goal is a reliable picture of which consumer holds which retention requirement, not a single unqualified lag number copied into a dashboard.
Estimate time to exhaustion
Measure available storage and the recent WAL generation rate over representative intervals. Account for normal retention, checkpoints, archiving, and other disk consumers. A rough time-to-exhaustion estimate should state its assumptions and be updated as workload changes.
For an illustrative incident, a slot retaining data while the database produces several gigabytes per hour may require action much sooner than a daily percentage threshold suggests. Use actual measured rates for the environment. Do not present a fixed safe retention size that ignores transaction volume and the consumer's ability to catch up.
Diagnose why the consumer stopped
Inspect consumer logs, network access, authentication, resource capacity, and downstream errors. A consumer can remain connected while processing too slowly, so active state alone does not prove progress. Track whether its relevant positions advance and whether lag is shrinking or growing.
Check the downstream destination's capacity before restarting at full speed. A backlog can overwhelm an analytics system or message broker and cause the consumer to fail again. Coordinate catch-up rate with the consumer owner, and preserve checkpoints and operation identities needed to avoid incorrect duplicate effects.
Choose containment with the data owner
Supported options may include restoring consumer progress, adding suitable storage headroom, changing retention controls with understood consequences, or retiring an abandoned slot. The correct choice depends on the system's role and recovery requirements. Avoid dropping a slot simply because its name looks old.
If a slot must be removed or becomes unusable, document how the consumer will reinitialize and how data gaps will be reconciled. A full resnapshot may take time and generate substantial load. The incident plan should include that cost rather than declaring victory as soon as disk usage stops increasing.
Never clean WAL as ordinary log files
The WAL directory is not a general application log directory. Manual deletion can damage recovery and replication. Use PostgreSQL's supported mechanisms and the managed service's procedures where applicable. If the platform controls storage or replication settings, follow that platform's supported operating path.
Preserve enough evidence to explain the retention source before taking action. Record slot state, disk headroom, consumer status, and the chosen decision. Keep credentials and sensitive query content out of the shared incident bundle. The database owner should be able to reconstruct why the action was necessary.
Verify both primary stability and consumer correctness
After recovery, observe free storage, WAL retention, checkpoint behaviour, and database latency. Confirm the consumer advances and that its destination is complete enough for the agreed recovery point. A consumer process being online does not prove that missing or repeated downstream records were reconciled.
Run a safe known change through the source and verify its arrival through the normal downstream path. Check lag over a sustained period so a brief burst of progress is not mistaken for full recovery. Keep any resnapshot or gap-repair work visible until its acceptance criteria pass.
Add lifecycle controls for slots
Create an owner and retirement procedure for every slot. Alert on stalled progress, retained bytes, and projected storage exhaustion. Include planned consumer maintenance in the retention budget and review the limits after workload growth.
A focused operator exercise: A paused-consumer capacity exercise
Pause an authorized test consumer while generating a bounded amount of harmless source activity. Observe retained WAL and the slot's progress fields, then resume at a controlled rate. Record how long the consumer needs to catch up and whether the destination remains healthy during recovery.
Use the measured generation and catch-up rates to estimate an allowed maintenance window with margin. The numbers should come from that workload and be revisited as traffic changes. If a retention control would invalidate the consumer before the window ends, update the recovery plan explicitly rather than assuming every paused integration can always resume from its old checkpoint.
Quick diagnostic map
| Evidence | Review | Acceptance boundary |
|---|---|---|
| Retained WAL | Inspect slot activity and retained bytes | Correlate with consumer progress |
| Disk runway | Compare generation rate with headroom | Define an actionable recovery threshold |
| Consumer recovery | Restore or replace the stalled consumer safely | Understand data-loss consequences before dropping a slot |
Frequently asked questions
Should an inactive slot be deleted automatically? Only under a policy that understands ownership, allowed pause duration, and the consumer's recovery requirements.
Clouvenix can review slot retention and downstream recovery together, helping teams protect the primary database without turning a disk incident into an unexplained data gap.
For a scoped review of this workflow, discuss the relevant Clouvenix engineering service.