Context
Durable Functions can replay orchestration code to reconstruct its state from execution history. Replay is normal operation. The orchestrator must make the same decisions when presented with the same recorded history, even if the surrounding process restarts or time has advanced.
Prerequisites are an affected orchestration instance identifier, deployed code version, SDK and runtime versions, and sanitized history or diagnostic logs. Identify the durable backend and preserve production state. Do not delete orchestration history to make an unexplained replay error disappear.
1. Locate nondeterministic decisions
Inspect the orchestrator for direct current-time calls, random identifiers, network I/O and ordinary asynchronous tasks. These values can differ on replay. Use context-provided deterministic APIs for supported time and identifier needs, and move external work into activities or supported durable HTTP operations.
Check reads of environment variables and mutable static values. Configuration changed during deployment can alter a branch when an old instance resumes. Pass needed values as orchestration input or retrieve them through an activity so the decision has a recorded basis.
2. Separate orchestration from side effects
The orchestrator should schedule work and combine recorded results. Database writes, email delivery and arbitrary HTTP calls belong in activities. Direct I/O from replayed orchestration code risks both different decisions and repeated external effects.
Activities still require deliberate retry safety. A worker can complete an external operation before its completion is durably recorded. Assign a business idempotency key and check downstream support for repeated requests. Moving a payment call to an activity does not by itself guarantee exactly one charge.
3. Review code changes for running instances
Compare the failing instance's start time with recent deployments. Reordering activities or changing branching logic can conflict with the history of an already running orchestration. Follow the supported versioning approach for the actual Durable SDK and hosting model, rather than assuming every pending instance will resume safely on new code.
Create test histories covering waits, retries and process restarts. Use durable timers for orchestrated delays instead of blocking sleep or ordinary task delay. Keep replay-aware logging so normal replay does not masquerade as duplicate business execution, while preserving enough correlation to investigate genuine repeated activities.
Diagnostic checkpoints
| Evidence | Likely cause | Next review |
|---|---|---|
| Failure after new deployment | History incompatible with code | Scheduled operation changes |
| Branch varies with wall clock | Nondeterministic time | Context API usage |
| Duplicate external side effect | Replay or retry boundary | Activity and idempotency design |
| Repeated logs, one effect | Normal replay logging | Replay-aware correlation |
Illustrative scenario
A hypothetical approval orchestration reads an environment flag to choose its next activity. The flag changes while instances wait for approval. On replay, the new branch no longer agrees with recorded history. The team treats configuration as stable input and reviews versioning for existing instances. This example is illustrative, without a claimed production outcome.
Verification and orchestration SOP
Verify a complete workflow including an approved host restart, a timer and an activity retry. Confirm both orchestration state and business effects by operation identifier. Acceptance requires stable replay and safe repeated activity handling, not merely a completed status on one fresh instance.
Document deterministic coding rules, versioning decisions and active-instance migration procedures in the SOP. Monitor nondeterminism errors separately from ordinary activity exceptions. Review orchestration edits whenever activity order, branching or input contracts change, particularly for workflows that remain open across many deployments.
Explore Clouvenix cloud architecture for implementation and operational handover.