Context
Kafka replication settings define when a producer receives success and when writes are refused. acks=all and min.insync.replicas work together, but the minimum is not a fixed count of acknowledgments used in every healthy state. Understand the in-sync replica set and the failure contract before tuning availability during an incident.
Separate replica count from ISR membership
Replication factor describes the configured number of replicas for a partition. The in-sync replica set, or ISR, identifies replicas currently meeting Kafka's synchronization criteria. A topic can have replication factor three while fewer replicas remain in sync.
Inspect partition-level leader and ISR state, broker health, replication lag, and recent leadership changes. A cluster-wide broker count does not establish that every important partition has the intended protection.
Use documentation matching the broker and client versions. Defaults and additional durability-related settings can change across releases. Record effective topic configuration, producer configuration, and relevant broker policy rather than relying on the values in a sample file.
Interpret the acknowledgment contract
With acks=all, the write requires acknowledgment from the current in-sync replicas under Kafka's documented replication behavior. min.insync.replicas sets a floor required for accepting that kind of write. If the ISR falls below the floor, the write is refused with the relevant error.
For an illustrative topic with replication factor three and minimum ISR two, all three current ISR members participate when all three are in sync. After one leaves ISR, writes can still be accepted with the remaining two. If ISR falls to one, the minimum prevents the same acceptance behavior.
Do not explain the setting as always waiting for exactly two of three replicas. That loses the distinction between the current ISR and its minimum allowed size. Also avoid claiming replication settings alone prove survival of every hardware, power, operator, or regional failure.
Plan the availability tradeoff
A refusal below the minimum can be the intended protection working. Lowering the minimum may restore write acceptance while changing durability expectations. The decision needs an owner and a documented consequence, not an automatic incident script that weakens the setting whenever an error appears.
Investigate why replicas left ISR: broker failures, network issues, disk latency, capacity, or operational changes. Restore the replication path where possible. Review replica placement so a single failure domain does not remove every protected copy simultaneously.
Coordinate producer retry, delivery timeout, and idempotence settings according to the supported client. A timeout can leave an uncertain outcome. Idempotent production helps a specific transport contract, while stable business identity and downstream reconciliation may still be required across application restarts or repeated logical submissions.
An illustrative false recovery
A broker failure shrinks ISR and producers receive insufficient-replica errors. An operator lowers the minimum to stop the alerts. Writes resume, but the service now accepts a protection level the business never approved.
The corrected runbook reports the degraded state, bounds producer behavior, and prioritizes restoring healthy replication. Any temporary policy change is explicit and reviewed. Recovery is complete when accepted traffic and the intended replica protection are both restored.
Quick diagnostic map
| Evidence | Interpretation | Next check |
|---|---|---|
| ISR below minimum; writes refused | Protection floor active | Replica health and recovery |
| Replication factor correct; ISR small | Configured copies not all current | Partition-level replication evidence |
| Producer times out | Outcome may be uncertain | Delivery evidence and business identity |
| Writes resume after lowering minimum | Contract changed | Approved durability and restoration plan |
A focused operator exercise
Use a nonproduction topic with recorded replication factor, minimum ISR, and producer settings. Send uniquely identified records, remove one replica through a supported test procedure, and observe ISR and producer outcomes. Repeat the accepted failure scenario without changing the protection settings silently.
Restore the replica and reconcile the expected record identities from the consumer side. Test producer restart and bounded retry behavior. Record refused, acknowledged, uncertain, and consumed outcomes separately. The result should demonstrate the actual failure contract rather than only an uninterrupted sender counter.
Frequently asked questions
Does minimum ISR two mean acks=all always waits for two replicas?
No. The current ISR matters. The minimum defines the acceptable floor, while all-acks behavior applies to the current in-sync set.
Are insufficient-replica errors always a misconfiguration?
No. They can be the intended refusal when the required protection is unavailable. Investigate the degraded replicas before changing the contract.
Continue the investigation
- Spring Kafka offsets: acknowledgment versus business completion
- Transactional outbox: recover the database-to-message boundary
Browse the cloud, server and infrastructure guide library for related operational questions.
Explore Clouvenix Apache Kafka Streaming Platform Services to discuss implementation, validation and operational handover.