Beaver Duty
Subscribe
Sign in
Home
Dam Logs
Logs of Duty
The Control Plane
Upstream Alerts
Duty Swag
About
Upstream Alert
Upstream Alert: Another patching gone wrong
A routine firewall upgrade blocked Australia’s emergency calls for over 13 hours. It’s a chilling reminder that "minor" infrastructure changes can have…
Oct 7
•
Matt
Upstream Alert: A production database has been deleted.
Should we start blaming the AI in postmortems?
Sep 3
•
Matt
Upstream Alert: Your data is not safe anywhere
Not everyone has the same definition of Data sovereignty. Even Cloud companies must follow laws.
Aug 26
•
Matt
Upstream Alert: Amazon Q almost helped to optimise AWS bills in seconds
A malicious pull request slipped into Amazon’s AI coding assistant and nearly turned it into a self-destruct bot. It’s a reminder that prompt injection…
Aug 19
•
Matt
Upstream Alert: We all fall for the same trap
A routine OS update took down Heroku for 23 hours on June 10th, 2025. The incident echoes similar outages at OpenAI and Datadog, reminding us that…
Aug 12
•
Matt
Upstream Alert: Your website can't load!
A quiet config change in June mislabeled Cloudflare’s production DNS (1.1.1.1) as part of a pre-prod test system.
Aug 5
•
Matt
Upstream Alert: where did you store your sensitive data?
5.7 million customer data taken from Qantas through a third-party vendor tool. Are you at risk?
Jul 29
•
Matt
Upstream Alert: One update, no more Internet
A daily OS patch knocked out thousands of OpenAI’s GPU nodes. And with them, a large chunk of ChatGPT and API availability.
Jul 15
•
Matt
Upstream Alert: How a blank field broke half the Internet
On June 12, a blank field in a new IAM policy triggered a bug in a newly deployed Google Cloud service, causing widespread outages across platforms like…
Jul 10
•
Matt
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
