Context
Kubelet client certificates let a node authenticate to the Kubernetes API. Rotation is a handoff: the kubelet requests a replacement, an authorized controller approves an appropriate request, a signer issues a certificate, and the kubelet begins using it. A certificate nearing expiration does not reveal which part of that handoff has stopped.
Prerequisites are the affected node name, cluster distribution, certificate ownership model, and an administrator-approved way to inspect kubelet configuration and logs. The first API checks are read-only. Managed platforms may own this workflow, so use their documented operational boundary rather than modifying control-plane settings independently.
1. Identify the certificate role
Distinguish the kubelet's client certificate from its serving certificate. The former authenticates the node to the API server; the latter secures incoming connections to the kubelet. They use different signer and approval expectations. Also distinguish these certificates from a user's kubeconfig certificate or control-plane certificates.
Read the effective kubelet configuration through the platform's supported interface. Check whether client rotation is enabled and where its active certificate is obtained. Do not infer the active file from a familiar path on a different distribution, and never collect the private key for diagnosis.
2. Inspect the request lifecycle
kubectl get node worker-a -o yaml
kubectl get csr
kubectl describe csr example-node-request
The names are illustrative. For a relevant CSR, compare username, groups, signerName, requested usages, creation time, and condition history. Confirm it belongs to the expected node and rotation operation. A Pending request, an Approved request without issued certificate data, and a completed request that the kubelet has not adopted point to different boundaries.
3. Correlate configuration and logs
Review sanitized kubelet and signing-controller logs for the same interval. Look for API authentication failures, request creation failures, approval denials, signer errors, and certificate retrieval problems. Check platform time synchronization evidence because certificate validity depends on time, but keep the focus on this Kubernetes credential path.
For an explicitly illustrative example, worker-a creates a client-rotation request that remains Pending. The useful question is why the expected approver did not approve the verified node request. Blindly approving every Pending CSR could also approve an unrelated identity. Investigate approval-controller availability and RBAC before proposing a narrowly reviewed action.
Diagnose the stopped stage
| Observation | Handoff boundary | Evidence to collect |
|---|---|---|
| No replacement CSR appears | Request creation | Kubelet rotation configuration and API errors |
| CSR remains Pending | Approval | Verified requester, signer and approval policy |
| Approved but certificate absent | Signing | Signer configuration and controller logs |
| Issued replacement, old certificate still used | Retrieval or adoption | Kubelet logs and active certificate identity |
Automatic approval is not a universal promise for every signer. In particular, do not assume serving-certificate requests should follow the same approval path as node-client rotation. Validate the identity information and platform policy before approving any request.
Correct through the platform procedure
Repair the responsible rotation setting, controller, permission, or supported node-recovery workflow after reviewing the evidence. Avoid deleting certificate files or rejoining a node as an exploratory step. Those actions can remove working credentials and complicate recovery.
Verify that the intended certificate was issued, that its public identity and validity match the node, and that the kubelet successfully reconnects using it. Check node conditions and a normal workload operation after recovery. A signed CSR alone is incomplete evidence of adoption.
Maintain a certificate SOP listing certificate roles, expected signers, approval owners, and renewal evidence. Monitor remaining validity and abnormal CSR age with enough lead time for investigation. Include a documented escalation path for expired credentials, because recovery may require different authorization from ordinary automatic rotation.
Explore Clouvenix containers kubernetes for implementation and operational handover.