Context
A MongoDB replica set elects a primary so one member can accept writes. If the primary disappears, the remaining members need the required votes and an eligible candidate. The application then needs to discover and reach that candidate. A database election can succeed while clients still report that no suitable server is available.
Prerequisites
Collect the MongoDB server and driver versions, replica-set configuration, member roles, advertised hostnames, network layout, and incident timestamps. Obtain authorized read access to replica-set status and logs. Distinguish self-managed operations from Atlas-managed controls. Keep maintenance and infrastructure changes visible, since restarting members or altering connectivity can explain an election without any database configuration defect.
Follow the availability path
- Capture
rs.status()and the relevant member logs. Record current primary, member health, election events, and replication state. Compare timestamps across members using their configured time sources. Do not repeatedly force elections as a diagnostic shortcut; each intervention can interrupt writes and destroy the quiet interval needed to understand the original event. - Compare reachable members with the voting configuration. A surviving server count is not necessarily the same as a voting majority. Check which members are data-bearing, which can vote, and which are eligible to become primary. Priority-zero members, for example, can have a useful replica role without being election candidates.
- Inspect connectivity between members in both directions, including DNS, firewall rules, certificates, and advertised addresses. A network partition may leave different members with different views of the topology. Determine whether the surviving partition can form the intended majority before considering any reconfiguration. Changing votes to fit a transient outage can create additional recovery risks.
- Once a primary exists, test application discovery using the real driver and connection configuration. Confirm the replica-set name and that clients can resolve and reach advertised member addresses. Review server-selection and retry behavior against the application's request budget. A single direct connection to one member can conceal a broken replica-set discovery configuration.
Diagnostic table
| Observation | Likely boundary | Next evidence |
|---|---|---|
| No reachable voting majority | Topology availability | Member-to-member connectivity |
| Majority reachable, no eligible candidate | Member configuration or state | Priorities and replication state |
| Primary exists, clients cannot find it | Driver discovery path | Advertised names and client routes |
| Elections repeat after recovery | Unstable underlying condition | Network and resource timeline |
Illustrative scenario
A replica set elects a replacement primary in a secondary location. The application can reach the former primary's address but cannot resolve the replacement's advertised hostname. Operators first confirm quorum and a stable database primary, then repair application DNS and routing. Altering election timeouts would not have made the new primary reachable to those clients.
Correct and verify
Repair the demonstrated network, member-state, or client-discovery issue through the approved procedure. Verify one stable primary, healthy replication, and a bounded application write test in staging or the authorized production acceptance workflow. Review write concern separately: a successful election does not by itself establish the durability of every earlier unacknowledged operation. Preserve ambiguous application outcomes for reconciliation.
Operating procedure
Monitor primary changes, member health, replication progress, and application server-selection failures together. Document quorum expectations for each planned failure domain and test maintenance one member at a time. Keep a current advertised-address inventory. After an incident, record whether the failure prevented election, prevented discovery, or affected both, so future alerts lead to the correct operational owner.
Explore Clouvenix database administration for implementation and operational handover.