Blog
Is it you or your vendor? A 3am playbook
When checkout fails at 3am, stop guessing. A practical playbook for separating your stack from a dependency using independent reachability and vendor status pages.
Checkout is failing. Your dashboards look fine. The vendor status page still says operational. Support is filling up. The only question that matters in the next five minutes is: is it you, or is it them?
1. Stop deploying and stop rewriting history
Do not push a "fix" until you know whether the failure is inside your network. A flailing deploy at 3am makes a vendor outage look like your regression.
2. Check an independent outside view
Look at something that is not your APM and not the vendor grading itself. StackWitness public truth pages ping the real service host for reachability and separately read the vendor status feed. Those are two different signals on purpose.
Reachable means the host answered our probe. It is not a claim that every feature of their product is healthy. Vendor operational is their self-report. The strong public signal is when we cannot reach the host while they still claim operational.
3. Compare regions, not averages
Partial outages are common. If two regions can reach the host and one cannot, that is the story. Averaging that into a single green light is how teams spend an hour in the wrong codebase.
4. Write the customer update from measurements
If independent probes show the dependency host is unreachable, say that. If your stack is healthy and the vendor is reporting an issue, say that. If you only have your own errors and no outside confirmation, say you are still measuring. Do not invent a green status to calm the room.
5. Keep a timestamp trail
Note when symptoms started, when outside probes failed, and when the vendor status page moved. That trail is what you need later for a postmortem or a credit request. It is also what honest status pages are for.
We measure reachability and read vendor self-reports. We do not certify that a vendor meets an SLA, and we do not issue a compliance verdict. Facts first; narrative second.