Upstream Alert: How a blank field broke half the Internet

On June 12, a blank field in a new IAM policy triggered a bug in a newly deployed Google Cloud service, causing widespread outages across platforms like Spotify and Snapchat.

RCA: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

What happened?

A blank field triggered a massive incident on GCP on June 12th, 2025. Spotify, Snapchat, Shopify, and more were all impacted. Let’s dive in!

At the end of May, Google shipped a new binary for a backend service called Service Control. This service handles API quota checks and enforces access management. When it crashes, GCP API calls get rejected. It’s like the Cloud version of a nightclub bouncer.

The binary had been deployed and progressively rolled out to all the regions. But the behavior of the new feature had not been validated, and it was not protected by a feature flag either.

On June 12th, a new IAM policy with blank fields was submitted. That’s when things went sideways! The submission was valid and triggered the new feature. The code didn’t appreciate the blank fields and raised a NullPointerException. And it crashed!