The last mine

I survived Security, Performance, and Release. Only to be taken down by Agile.

With new services, the production deployment is the toughest mile. It’s like being in the middle of a minefield.

I can’t walk back because my product owner would have a heart attack. The product team has been pressuring us to go live with this “game-changing” service for a few weeks already. I am unsure how it is so critical, given that all the nice features were stripped out of the first MVP release due to the impossible timeline they set for themselves. But now that this application resembles a “Hello World”, it won’t accept any further delay.

However, to proceed, I now have to manage the release team, the load testing team, and the security team. Each team is a mine. One wrong word, and it results in an additional two weeks’ delay in clearing their concerns.

First, the security team. Every time I meet them, the production checklist includes five more items. They now request one vendor to perform penetration testing from “outside”. For an “Hello World”. And an internal service. And without any API interface.

I try to explain why it should not be applicable. They should conduct some testing to ensure the internal network is indeed private. But don’t push this to me. Three Principal Engineers parachute into the call. None of them know what this app does, but all of them approve skipping the test. Thank you. Mine avoided.

Next, the performance testing (PT) team. I like them. This is tough, but critical. So I prepared seriously to deal with them, and I have a document with the expected data load that will be consumed and processed. We go through the details together. An issue surfaces. The PT environment can only support up to thirty percent of the production load. Wait. How do other teams do load testing if we can’t simulate more load than expected? The team promises to find a workaround in the next two days. Great. It’s like I’ve stepped on a mine, but I have a demining kit.

Finally, the release team. Most of our recent incidents are linked to production changes. They are now very cautious about any new deployment. I need to justify why I should deploy this. Honestly? I am also wondering why. I'd better involve the product owner for a sales pitch session.

It worked. I don’t know if the team has been convinced or if it just wanted to end this call as fast as possible. But I have the green lights. The only condition is that I must deploy at 1 am, when the traffic is low. That’s frustrating. I hate these late-night activities for low-risk deployments. But am I ready to guarantee 100% I won’t break anything, if the release team itself wouldn’t? Probably not. Never mind, I have the approvals. This release strategy will be a fight for another day.

I did it. I am ready to proceed with the deployment tonight. It feels like I could talk about this as an achievement in my performance review.

I join the team stand-up, excited to share the update. I proudly announce the news. The product owner's reaction is not what I expected. Change of priorities. The MVP scope has changed. In the name of Agile, we will not proceed with the deployment.

I’m furious. It’s like I didn’t see the last mine. I stepped on it, and BOOM.

At least, I will sleep well tonight.