Blog
When one zone is down and the region still looks up
Google Cloud's 1 September 2026 incident hit a portion of us-central1-b while the rest of Iowa kept answering. How to name a zone failure without averaging it into a calm region badge.
Packet loss on the nodes in us-central1-b. Cloud Run in that zone is erroring. The rest of Iowa answers. The vendor location field says us-central1. Is Google down? That question is too coarse. You need the zone name, not the region badge.
A recent case: us-central1-b, 1 September 2026
On 1 September 2026 at 14:44 UTC (07:44 US/Pacific), Google Cloud opened an incident for network service degradation affecting a long list of products in us-central1-b, including Compute Engine, GKE, Cloud Run, App Engine, Cloud SQL, Spanner, BigQuery, and VPC. The location on the incident page was Iowa (us-central1). The body said the issue was a portion of us-central1-b, and that the remaining zones in the region were unaffected. Preliminary analysis pointed at routine network fabric path maintenance that triggered unexpected issues in one cluster. The incident ended at 18:52 UTC. Google's own page is incident J5ia5t9p3g9Q5Wi7r8Ev.
Independent day-status writeups, including isinternetup.com's September digest, put the same window at 4 hours and 8 minutes and named the same zone. Cloud Run and App Engine lagged the rest of the product list; Google was still validating those two after GCE, GKE, BigQuery, and the databases had recovered. The lesson below holds for any vendor whose status page talks in regions while your workloads live in a zone.
What a customer of that zone actually did
Expo felt it in public. At 15:51 UTC they opened their own incident: API and website disruption, plus delays processing GitHub events, attributed to their cloud provider. They named Google Compute Engine on the component list. At 17:06 UTC they rescheduled services to run outside the affected availability zone. Their product recovered while Google was still working Cloud Run and App Engine in that zone. Expo marked resolved at 19:13 UTC, after the provider's network was operational in every AZ they use. That timeline is on Expo's incident page.
That is the operator move the region badge will not make for you. Google's own workaround during the incident was the same instruction: if you are severely affected and have capacity in other zones, fail over. Waiting for the last product in the last cluster to recover is how you spend four hours restarting the same failing pods.
Why the region field fools people
A region is a billing and placement label. A zone is where the NIC, the disk, and the packet path live. When the status page location says us-central1 and the body says a portion of us-central1-b, those are different units. Skimming the location field, you will either page as if Iowa is dark, or you will tell customers nothing is wrong because other zones in the region still answer.
A single host probe of a global API hostname would not have told you about packet loss inside us-central1-b. Zone failures live below that instrument. Treat independent reachability of a public host as host answer, not a certificate that every zone is healthy. On StackWitness truth pages (see /status, for example /status/cloudflare), the witnessed column sits next to vendor self-report on purpose, and it is still a host-level signal.
Copy you can adapt
The weak version, which could be pasted onto any incident:
We are aware some customers may be experiencing intermittent issues. A third-party cloud provider is investigating. All other systems remain operational. We apologize for any inconvenience.
That paragraph costs four hours. On-call keeps restarting pods that still land in us-central1-b. Support tells customers to retry. Nobody fails over, because the region is listed as Iowa and Iowa is not dark.
The version that names the unit:
As of 15:51 UTC we are seeing packet loss and API errors on workloads in Google Cloud us-central1-b. Google reports the rest of us-central1 is unaffected. At 17:06 UTC we moved serving out of that zone. Requests that still land on us-central1-b may fail until those instances drain.
A reader who was not on the bridge can check that against the vendor incident and against your own error rates by zone. The weak paragraph cannot be checked, so it cannot be trusted an hour later.
What not to do
- Do not mark the product operational because other zones in us-central1 still answer.
- Do not declare a total regional blackout because one zone is losing packets.
- Do not wait for the vendor's last remaining product (here, Cloud Run and App Engine) before you drain the zone you already know is sick.
- Do not invent independent probe results you did not collect. Attribute vendor status, your own error rates by zone, and any outside host reachability view separately.
Operator checklist
- Name the zone, not only the region. Read the incident body, not only the location field.
- Split your own error rates the same way: by zone, then by product surface (API, website, background jobs).
- If you can fail out of the sick zone, do it on a clock, then say that you did.
- Keep a timestamp trail: when customer symptoms started, when the vendor named the zone, when you drained it, when the remaining products recovered.
- Write the customer update with the split explicit. If only one zone failed, say so.
StackWitness measures host reachability and reads vendor self-reports so teams can attribute faster. A zone outage is still an outage for anyone whose workloads were pinned there. Name the zone you lost.
We measure reachability and read vendor self-reports. We never invent calm green or issue a compliance verdict.