Message delivery and a business effect are different events
An event consumer receives a message, performs work and acknowledges delivery. A failure can occur between any two of those steps. If the consumer charges a customer and then loses its connection before acknowledging, another delivery must not automatically produce another charge. Design the business operation to recognize a result that already exists.
Pub/Sub documents an exactly-once delivery feature for supported pull subscriptions, with regional conditions and acknowledgment behavior. It does not remove the need to understand the application's transaction boundary. A duplicate business request can also be published as a new message. A provider message ID and a business operation ID answer different questions.
Design the initial consumer contract
- Define the durable business key, such as an order event plus operation type. Decide where successful completion is recorded and how concurrent attempts are handled.
- Choose the subscription and client behavior deliberately. Document acknowledgment timing, processing timeout, retry handling and where unrecoverable messages are reviewed.
- Rehearse a consumer crash before work, during work and after the side effect. Use a payment or email test adapter so the rehearsal does not create real external effects.
The consumer needs enough observability to link a delivery attempt to one business result. Do not place complete customer payloads or authorization tokens in every retry log.
Illustrative incident: a completed operation is delivered again
A worker saves an export record and then exits before confirming its acknowledgment. The message becomes eligible for another delivery. The next worker finds the existing export using the stable business key and reuses it instead of generating and emailing another document. This is an illustrative design example, not a claimed customer result.
Diagnose what repeated delivery means
Record the message ID, business key, delivery time and acknowledgment outcome. Compare the side-effect record with the attempt history. If a new message carries the same business key, investigate the publisher as well as the consumer.
| Evidence | Question | Why it matters |
|---|---|---|
| Same message ID, unfinished acknowledgment | Did processing exceed the lease or fail? | Delivery recovery may be expected |
| Different message IDs, same business key | Was the event published more than once? | Delivery settings alone cannot deduplicate business intent |
| Result exists, acknowledgment uncertain | Can the consumer recognize completion durably? | Prevents repeating an external effect |
| Backlog grows while workers restart | Is a repeatable failure blocking progress? | Scaling may multiply the same failure |
Avoid relying only on an in-memory 'already seen' set; that state disappears when a worker restarts. An acknowledgment should follow the application's durable acceptance point, with failures represented explicitly rather than silently swallowed.
Recover with controlled replay
Fix the demonstrated processing fault and replay a small reviewed sample. Confirm that a repeated business key produces the intended existing result or a controlled no-op. Review how the application behaves if a dependency accepts an operation but its response is lost. That ambiguity may need a provider-side idempotency facility or reconciliation process, not an unlimited retry loop.
Test parallel deliveries as well as sequential retries. Verify that a worker restart after completion does not lose the deduplication record. Preserve a record of messages that cannot be processed and assign an owner before bulk replay.
Monitoring, reliability and the operator SOP
Measure age of unfinished business work, processing failure rate, acknowledgment failures and duplicate-effect incidents separately. An illustrative internal objective might require all accepted export events to reach a durable completed or reviewed-failure state within 15 minutes during normal operation. It is a sample service target, not a Pub/Sub SLA.
The SOP should identify message source, consumer version, business key, completion boundary, replay permission and reconciliation owner. Escalate uncertain external effects before repeating a sensitive action. Review this contract whenever a consumer adds another downstream operation.
See custom application development for related event-processing design.
Technical reference
Google Cloud: Pub/Sub exactly-once delivery. Reviewed 19 October 2026; validate supported subscription types and regional assumptions before using the feature.