DevOps is all about culture, right? Yeah, it's developers and operators working more closely, but it's more than that. DevOps is a culture that exudes the principles of Agile, Lean, and Open Source, to deliver higher quality products and services faster, more continuously, and more predictably.
So, if we accept DevOps is a culture, and your CIO gives you a mandate to transform his or her organization to a DevOps organization, you're now effectively responsible for an organizational culture change initiative. That's the situation I found myself in recently, when Red Hat CIO Lee Congdon asked me to lead a DevOps enablement team in his IT organization. At a few weeks in with our new team, called Inception, I'd like to share our approach:
1. Build culture through practices. Culture is the sum of the habits and behaviors of a group. I don't believe culture can be changed through mandates. But, Inception's mission is to change the organization's culture. So, how?
We will focus on the defining and promoting the practices of Continuous Integration and Continuous Delivery (CI/CD). As we march towards CI/CD, I expect we'll uncover many problematic habits and practices across the organization. As we address these one by one, we'll incrementally build the culture we desire. Focus on the desired practices, and the desired culture will emerge.
2. Assemble a dedicated and diverse team. We're lucky: DevOps is a strategic priority for our leadership, up through our CIO. That means I could pull in five full-time people to form this team. Here are the unique backgrounds we assembled in our team:
- System administration
- Information security
- Software development
- Release engineering
- Agile project management
These very different perspectives give our team a firm foundation for the First Way of DevOps culture. If we had five release engineers, five developers, or five sysadmins, we wouldn't have the diversity of opinion and experience necessary to design solutions that meet the needs of our diverse stakeholders in IT.
3. Articulate a clear road map. The anti-pattern here would be the Underpants Gnomes strategy:
- Launch DevOps initiative
- ???
- We have DevOps culture!
Yet, so many organizational change initiatives look like this. So, we thought it important to clearly articulate a road map from the start.
At this early juncture, our road map has "mature CI / CD practice" at the end, "improve environment management" immediately in front of us, and "improve other areas essential to CD maturity" as defined in the CD maturity model put forth in chapter 15 of Jez Humble & David Farley's Continuous Delivery book.
As an Agile team, we're refining the road map and decomposing into epics, stories, and tasks at the last responsible moment, with Humble & Farley's CD maturity model as our compass. But, even standing on the shoulders of these giants, defining a road map for CI/CD that's engaging and believable has been the most difficult part of getting this new team off the ground.
4. Open Source all the things. Red Hat is an Open Source company through and through. That doesn't simply mean we default to Open Source software in IT (we do), it means we embrace the openness and sharing that is core to the Open Source movement--and that is required for a thriving DevOps culture.
For this team, Open Source means we'll share with the world not just our source code, but also our experiences deploying Open Source software to solve DevOps problems in blog posts like this one.
5. Effect change incrementally. Don't get me wrong, we have big ambitions. But, we have the best chance of realizing those ambitions through a long series of small wins aligned to a common strategy, as opposed to promoting sudden, sweeping change.
Change is hard for people and organizations to absorb. And, even with the best of intentions, a team like Inception could easily be perceived as interlopers. By focusing initially on small, non-threatening changes that save a few minutes for a developer here, or avoid one small human error for a sysadmin there, we can earn hearts and minds throughout the organization.
One of the first changes we made was to introduce two small utilities: jsonstats and talook. These small utilities will save a few minutes every week for every developer and release engineer in the organization. This change is small, easy to consume, builds trust and credibility for our team, and inches the larger organization closer towards a DevOps way of being.
Last, as we string together a long list of incremental victories (and failures), we need to maintain a focus on working on IT's tools, processes, and culture--not just the fire hose of work coming into IT. We could easily fall into being the "Level 4 support team" if not actively aware of the risk.
6. Measure your work. In attempting to effect a large organizational change, it's not enough to measure progress with anecdotes or by the seat of our pants. Anecdotes and gut checks are helpful, but not sufficient. We have over a dozen dev teams and a large, globally distributed ops team. For every vocal team member that thinks they're well on their way to CD, there's another one that's completely disengaged, for whom CD isn't on his or her radar.
To take a more holistic and scientific approach to measuring our progress, we're doing two things. First, we've created a survey from the CD maturity model in Humble & Farley's Continuous Delivery book, and asked team members across IT to self-evaluate. We'll use that as a bench mark, and re-evaluate in 6 months.
Second, we're analyzing ways to quantitatively measure benefit to IT's customers. Improving our CI/CD maturity isn't a concept that resonates with our Finance or Marketing teams; improving IT's speed and quality does. Ideas on the table include measuring cycle time in our Agile project management tool, or the number of failed releases per unit time. More to report on that in the near future.
If we demonstrate the resolve to keep plugging away at this incremental change for 18-24 months, and the focus to stay aligned to our goal of a mature CI/CD practice for Red Hat IT, we'll have effected a DevOps culture change in no time.
Last updated: February 7, 2024