"DevOps".
If you're like me, you may be suffering a bit of buzzword fatigue, especially relating to how this word is used (or misused) within the IT community. But for those of us who have been a part of the community for a while, it holds deeper meaning than the oft-repeated platitude of "Software Developers and Sysadmins working together, riding unicorns over rainbows". Okay, while I may have gotten slightly carried away, you get the point.
What is DevOps to the broader community that embraces it, and is helping even now to define it? What does that even mean for Red Hat's IT efforts? We're going to dive deeper into both questions in this installment.
Don't recoil when your boss invokes the word "DevOps"; he may or may not be using it inappropriately, but he's giving you a chance to do the right thing. Become the DevOps subject matter expert and help him to help you. You may be surprised to learn that DevOps didn't originally spring from the business management community, but rather from the IT engineering community.
Not too long ago, the industry was abuzz with another set of buzzwords: "Scrum", which is one of several methodologies of putting Agile into practice. Agile was working, it was making a real difference in transforming software development teams and getting them to produce more relevant software, releasing more frequently, and keeping their customers happier.
It didn't take long for someone on the more traditionally curmudgeonly side of the fence, Operations, to take notice and wonder if some of this Agile magic might help the much-maligned world of Sysadmins. This really started catching on, and for reasons that are becoming painfully easy to see.
You know, I can't tell you how many times, as a Sysadmin, I'd have some Software Developer or Project Manager come over to the IT department and go through the careful ritual of asking for something. The IT Manager would hold court, perhaps with a couple of very conservative Sysadmins nearby, and hear out the plea for making some kind of big change to his otherwise stable empire. The answer would come back, either as a resounding "no", or if it was going to happen at all, the delivery would be at least 6 weeks out (often longer). The Software Developer, who has already been reaping the benefits of Agile software development, is now soberly considering how badly this has just screwed up his next three sprints.
Agile Operations was an approach that attempted to get a better impedance match between the needs of product development teams and operations teams. It got them using similar methodologies and cadences as well as started getting them more aligned on values. Two teams that want the same thing on the same timeline will, after all, accomplish great things together. Right?
This Agile Operations movement really was proto-DevOps. It got Software Developers and Sysadmins working together. Wait, where have I heard that before? Ah, yes... DevOps.
A fellow with a lot of cachet in the DevOps world by the name of Patrick Debois is credited with coming up with the word that is both loved and scorned by many today. As he tells it, he was working as a Sysadmin on an Agile software development project and needed an easy concise term to describe a collaborative organization where Agile practices were being followed beyond just the development effort. Notice we're not talking about some CIO bigwig at a Fortune 100 company here, but an in-the-trenches Sysadmin that was just trying to find an easy way to describe something that he was already doing that made a lot of sense. DevOps did not come out of a few MBA's locked in a room trying to come up with the latest theoretical way to structure our jobs of actually making stuff more tedious!
Bringing Agile into Operations got Sysadmins and Software Developers talking to each other and working more effectively together, as well as QA and security and people in other technical roles. DevOps came in and put a name on this warm fuzzy thing that was already starting to happen. It gave engineers something to rally around, to build on.
What happened next gets some engineers twitchy and others really excited. The DevOps community started rediscovering and incorporating some of the best practices of the industrial revolution, drawing parallels between software development and manufacturing stuff.
The reason it gets engineers twitchy is that this is the vector for a lot of managerspeak getting injected into DevOps. This is where you start hearing things like muda, mura, muri. Or kaizen (is that a Japanese hazing ritual?) and observe-plan-do-check-act. All of this comes from what was learned in the 20th century about effective manufacturing plant management and empowering the workers to improve the way they do their own work.
The thing is, for all of the Lean and Agile managerspeak and hand-wavy pictures of value streams and the like, the principles and practices behind the buzzwords actually work. My colleagues on the Inception team sometimes like to mock me if I use too many of these managerspeak terms in our conversations about how to foment a healthy DevOps culture in Red Hat IT. The challenge here, and the opportunity, is to embrace the really great teachings of gurus like Goldratt, Ohno, Deming, etc. and digest them into the common parlance of the Sysadmin or Software Developer.
When I first started writing this installment on the Inception team's DevOps journey, it was meant to be a deeper dive into The Foundation of DevOps. This is something I can write a good deal about, and have given public talks on the subject that would either fascinate or frustrate depending on your interest in the inevitable managerspeak that goes with such an overview. Partway through, I tossed it all out and decided to accept the challenge thrown down by my colleagues. DevOps in English! We've even gone so far as to hang a big piece of paper on our wall with a Sharpie nearby where we can write down words that Shadowman would never say.
DevOps isn't a role. There is no such thing as a "DevOp". Just like there is no such thing as an "Agile" or a "Lean". DevOps describes the culture of the shop you're working in, not a function that someone performs.
DevOps is also not a combination of Developer and Sysadmin in one role. Nope. Though in a DevOps shop, the lines may blur a bit. But that happens ideally because the wall between the two functions has crumbled, and people from many areas of expertise are working together towards a common goal. Me? I've been a Sysadmin for twenty years now. I'm a Sysadmin now. But because those walls are breaking down, I'm learning some new skills, writing more advanced tools than I ever did in bash because I can (there are no walls to stop me) and because I want to (not only is it personally fulfilling, but it's the right thing to do to help us all succeed together). I'm still, first and foremost, a Sysadmin, but I'm picking up more of a Software Developer skill set to help Red Hat IT succeed in a world where infrastructure is built and defined by APIs and templates rather than racks and bare metal (obligatory use of "cloud"... there, I said it).
So, in a DevOps shop, people are working together towards a common objective. The barriers to effective communication between contributors is very low. There is a cultural allergy to walls. And Software Developers and Sysadmins are no longer set up for constant war because they're sharing values. You might wonder how that works. I alluded to this earlier. Let's foray a little deeper.
The bottom line in the old world of Developers vs. Operations, which can still be seen in shops that haven't started making the leap yet, is pretty simple. Developers are rewarded for and measured by their success in getting new features into the hands of customers. But these changes tend to be big and risky, and happen far apart, so confidence is very low about how likely the new code is to work in production on the first try.
The world of IT Operations is traditionally one that strives for perfect stability, and status quo. Get things super stable, and then don't touch a thing. Any changes made must go through at least one change management committee, requires an outage window, and may carry a lot of risk due to the size and complexity of the change requested by the customer. This foments a culture where the Sysadmin wants to create a perfectly stable hermetically sealed environment where nothing can change once stability has been achieved. With Operations teams being measured first and foremost by uptime, it's no wonder they've been jerks all these years to Software Developers; the system is rigged for strife!
Good grief! No wonder these guys can't get along without getting all stabby! For one to succeed, the other must fail!
A DevOps culture changes the equation. Changes happen all the time, many times per day. But they are really small changes, and they are really well tested. So Software Developers and Sysadmins all understand there is a very low risk of failure, and confidence of success is high. Releases to production happen so often, it's really just a non-event. It's probably a bigger deal to walk from your desk to the break room for a caffeine fix.
Frequent releases reduce risk and increase confidence in success. Artistic value provided by Tim Bielawa
Join me next time in Part 2 where we'll talk about the main areas to focus your attention, in plain English, to get started on this DevOps thing that everybody is talking about.
Last updated: September 8, 2022