The Rollout Rush

Edition #11: The Rollout Rush

"Production deployments started failing this morning. Turns out security rolled out a new policy last night that blocks our CI/CD service account."

A DevOps team discovered a governance change when their pipeline broke. No announcement, no preview period, no documentation update - just enforcement that started blocking legitimate automation. Three other teams hit similar issues throughout the day, all scrambling to understand what changed and how to fix it.

The intent behind the new policy was good governance. The rollout approach was not.

When governance teams optimize for their own velocity rather than the organization's capacity to absorb change, teams experience governance as something that breaks their workflows instead of supporting them. Emergency exception requests immediately undermine the controls you just deployed.

This is The Rollout Rush - when governance changes deploy too fast without communication, preparation time, or considering whether teams are ready to adapt.

Avoiding Rollout Rush means giving teams time to prepare before enforcement starts. Organizations that adopt the Rollout Practice from the 5 Practices of Cloud Governance coordinate changes instead of surprising people:

→ Use phased rollouts - Draft → Preview → Check → Enforce gives teams time to prepare → Communicate before enforcement - announce changes and timelines well in advance → Coordinate across initiatives - don't overwhelm teams with simultaneous governance changes → Measure absorption capacity - track how many changes teams can handle effectively → Provide transition support - offer documentation, office hours, and remediation guidance

When governance changes are paced and communicated, teams adapt smoothly instead of scrambling for workarounds.

- Keep on Herding, Bob

What's Happening in Cloud Governance 📡

Keys to the Kingdom: A Defender's Guide to Privileged Account Monitoring: Google Cloud's Mandiant team explores why privileged accounts remain top attack targets, with stolen credentials now the second most common initial access method (16% of intrusions in 2024). The guide advocates for a three-pillar defense strategy spanning prevention, detection, and response—but emphasizes that PAM can't be rushed. Organizations must progress through maturity phases (Uninitiated → Ad-Hoc → Repeatable → Iterative Optimization) rather than treating PAM as a one-time technology deployment. Manual credential management doesn't scale; purpose-built PAM solutions require phased adoption to succeed.

Streamlining Cloud Compliance Audits Using AI and Automation: The Cloud Security Alliance examines how AI and automation are transforming compliance from a labor-intensive process (11 working weeks per year, up to 25+ weeks for regulated sectors) into continuous monitoring. Automation can reduce audit prep by 70%, cutting monthly workload from 185 hours to 62 hours, while ML accuracy improved from 78% to 93% for compliance issue detection. However, the 70% time savings only materializes with proper phased adoption—organizations can't flip a switch overnight. Teams need time to integrate AI systems with existing workflows, train personnel, and validate accuracy before relying on automated compliance.

Platform Engineering Is Failing — Here's Why Infrastructure Comes First: Despite widespread adoption, many platform engineering initiatives aren't meeting expectations—not because the approach is wrong, but because teams focus on developer tools while infrastructure becomes an afterthought. The New Stack examines why platforms fail when security, compliance, and cost efficiency aren't built in from Day 1. The emerging Infrastructure Platform Engineering (IPE) approach treats infrastructure as a first-class concern, prioritizing policy-driven, cost-aware Kubernetes infrastructure before building developer experiences on top. Organizations can migrate to IPE by auditing platforms for cost and governance failures, treating infrastructure as a product, and defining policy-as-code from the start.

FinOps at Prudential: Building a Cost-Aware Cloud Culture: Prudential Financial's six-year cloud journey demonstrates the value of paced transformation. Starting with $4M in cloud spend in 2018, the company waited until costs reached $50M before establishing a formal FinOps function in 2021. Rather than mandate cost awareness top-down, the team focused on developers first, providing "starter kits" and a Pricing Bot trained on Prudential data to offer real-time cost estimates. The cultural shift approach—celebrating small wins, using gamification, and automating wherever possible—has helped Prudential avoid 21% of annual cloud expenses. Key lesson: "As with any cultural change, it requires time, persistence, and consistency." Cultural transformation can't be rushed.

Get Involved 👋

Join the conversation on LinkedIn about this newsletter edition.