A database StatefulSet losing replicas, and why
A three-replica StatefulSet was running on two. The missing pod wasn't crashing — it was waiting on a volume that would never bind. A deeper check on paid plans connected the two facts and named the cause.
What happened
This finding is deep enough that SreNix hands it to a person rather than guessing.
- Detect: a check that reads StatefulSets together with their PersistentVolumeClaims — part of the Enterprise catalog — found ready replicas below the declared count.
- Cause: the missing replica's PVC had been stuck in Pending for longer than a normal provisioning delay. The pod was healthy; it simply had nothing to mount.
- Decision: restarting the pod would not help — the same PVC would still not bind. Resizing or rescheduling storage touches the data path directly, so it stays off the default closed list.
- Escalate: SreNix sent the storage owner a signed link naming the exact StatefulSet, PVC and storage class involved.
- Verify: once the storage class issue was resolved, the same check ran again and confirmed all replicas were ready.
Why this one waits for a person
A stuck pod is often safe to restart. A stuck volume is not the same problem, and forcing it can put real data at risk. SreNix tells the two apart and only acts on its own where the action is reversible.
The five actions on the default closed list — restart a wedged pod, stop a frozen job, and the like — were chosen because none of them can lose data. Storage capacity decisions aren't on that list for the same reason. See the closed list.
On-call should be quieter every week
Helm install in 5 minutes. No telemetry exfiltration. No per-investigation surprises.