DevOps Pitfalls

You've just been given the following assignment from your boss: He's asked you to make the organization more DevOps-oriented. You've gone back and forth on what his definition of DevOps is and what he wants you to accomplish. You're ready to get going, and think, "What could go wrong?" A good bit, it turns out. In the second post of this DevOps series, we are looking into some of the pitfalls and challenges of transitioning from a traditional operations model to a DevOps model.

Dev + DevOps + Ops

DevOps can help your organization when developers and operations are working together toward the same business goals. When you tighten communications, features and issues can be communicated back and forth fluidly between two teams.  Unfortunately, some organizations that have chosen to implement DevOps have placed a DevOps layer between developers and operations. Rather than increasing collaboration and communications between the developer and operations teams, this new team in between could potentially slow communications and the decision-making processes, resulting in failure. It's understandable to want people who are responsible for making this effort work, but rather than create a separate team, choose point people who will make sure that the development and operations teams are communicating effectively.

My Culture Just Ate Your DevOps

Saying that culture is an important piece in ensuring any projects' success is not a new concept. Having team buy-in on DevOps tasks is critical, and it's just as important to have a culture of communications and collaboration. But what if your organization's culture doesn't include that? Shaping an organization's culture, whether for DevOps or something else, can be a long and difficult process. There is not enough space in this post to talk about all of the things that could be done here.  Instead, start with addressing these three premises on how organizations can point their cultural ship in the right direction. First, as leadership, be the culture. You can't expect your organization to shift their cultural norms if the people at the top aren't exhibiting the culture that they want to see. The second suggestion to help you build your culture: Fake it 'til you make it. Lloyd Taylor, VP Infrastructure at Ngmoco said, "Behavior becomes culture." If you keep practicing the desired behaviors, eventually they become the culture. Finally, process becomes behavior. Having well-defined processes for how communications and DevOps interactions should work will build the behavior and in turn the culture.

Your Network Engineer Isn't a Developer

I use the term network engineer pretty loosely in the title of this section. For the purpose of my argument, it could be anyone on the operations side. Some organizations have many people in operations that do impressive scripting work to simplify tasks. These organizations may not have developers on staff and may look to these operators to be the developers in their DevOps initiatives. Don't get me wrong, for some organizations this may be exactly what they need and it may work out just fine. Unfortunately, what sometimes happens is the network engineer that is great at scripting lacks professional developer habits. Error handling, good software documentation, and regression testing are all things that an experienced developer would do out of habit that a network engineer might not think to address. (Okay, I'll admit that good software documentation can sometimes be difficult for experienced developers, too.) Can your operations person be a developer? They might have the skills and training, but don't count on them to be able to pull off all that a fully experienced developer can. If you happen to have an operations person that is a developer, maybe make that operator responsible for the points mentioned above.

Conversely, there are things on the operations side that your developers don't know about. For instance, your developers likely don't have deep knowledge of networking protocols, or even commands that would be required to configure them. One of the people that I spoke to for the last post experienced this very issue. In addition to experiencing long delivery times, his biggest concern was the appearance that the organization's developers were attempting to be network engineers when they were clearly not up for the job.

We Test in Production

It seems almost silly to mention this here, but you need to have a testing methodology for the product of your DevOps efforts. Even though it is basic, it's often overlooked that you shouldn't test your code in your production environment. If a large part of your DevOps program involves automation, and you roll out incorrect code, you roll out bad changes to your entire organization at crazy fast speeds. Yay, automation! Some development platforms allow you to test code to see what the output will be prior to running it in production. However, a solid test bed that replicates key components of the environment is helpful to have. The key is having an environment that you can quickly test and validate code behavior while not significantly slowing down the process for releasing the code to production.

Those are just a few of the pitfalls that frequently come up in a variety of DevOps initiatives. What do you think? What are the biggest challenges you've seen when someone is rolling out DevOps? Comment and let me know!