DevOps Challenges and Overcoming Blockers

Since adopting a DevOps approach at your organization can be a paradigm shift, there are going to be challenges.  Unless you are lucky enough to work for a start-up that gets to build IT processes from the ground up some things must change to shift the organization in such a drastic way.  I've frequently heard IT professionals say "It's hard to turn a battleship around" when describing major changes to the way IT operates.

Various personality types and corporate cultures can affect this, so I'm aware that everyone's journey is different.  Given all the baggage typically attached to IT projects and organizations as a whole, this brings up some very difficult questions as to how IT operates.  Which leads me to today's discussion question:

What are some of the challenges or pain points that you face or have faced in implementing DevOps at your organization and how did you overcome them?

  • Fear of change and being slow to adopt new technology is the biggest roadblock in my experience. At my organization, we have processes and procedures that have been in place for decades, and engineers/management that have been here almost as long. Change can be hard, especially in an IT environment where emphasis is placed on uptime and reliability... it's only natural to hesitate when "rocking the boat" and changing your entire workflow is brought up.

    We've just recently hopped on the Kubernetes train with a few of our web apps, and we just spun up a GitLab instance to house all of our custom code and scripts. We're lagging behind the industry, but we're getting there. DevOps is definitely the future, and it's exciting to be a part of the team that gets to usher in the new age of IT around here. It's taken many engineers and many proofs of concept to convince our leadership that containerization and DevOps practices are "safe" enough to push forward.

    The biggest advice I can offer anyone looking to "rock the boat" is simple: do your research, be prepared, and be ready to have those awkward conversations because they WILL happen.

  • we have processes and procedures that have been in place for decades,

    I think this should say "we have processes, procedures, and people that have been in place for decades." Smiley Or maybe that's just my own interpretation.

    As a long-(enough)-in-the-tooth IT guy, I'm totally happy with change when it comes to technology - frequently it's my favorite part of the gig: learning new tech.  But when there's such a paradigm shift (Yes, I also cringe when I use that phrasing) in the way Information Technology operates as a whole, I can be just as much a stick in the mud. If I'm not careful, I'll start to sound like the people I laughed at who got Novell NetWare certified.

    I feel like the closest we've come to something of this magnitude in recent memory was the rapid shift from Waterfall to Agile development.  But where the move to Agile was concentrated with the development teams themselves, the move to DevOps encompasses all of IT.

    I think you hit the nail on the head for one major thing: do the research and plan everything.  We all have that one team member who Heart documentation and research projects.  This is an opportunity for them to shine.

  • Lack of any manageable process and lack of understanding about how to make processes easier. The red tape and bad process while working for government and large organisations stagnates projects and diminishes efficiency. The endless silos and the WFH culture that we now deal with means we have to fight that bit harder to communicate and put in better ops processes across any org. I've seen cost cutting take away vital DevOps functions in line with government plans and it's hard to keep motivated when they are doing their hardest to ruin what we help build.

  • Public Sector is one of those areas where innovation goes to Skull.

    OK, I'll admit that was probably too harsh.  But, from my own personal experience, any time I've worked with a government agency, the technology they are using (and relying upon for critical works) feels absolutely ancient.  From my friends in PubSec, they tell me that change is slow. In tech, change is fast.  This feels like a Venn diagram with very little overlap.

    I hope I'm wrong and I just haven't had enough exposure to Government, Education, and other Public Sector verticals in the recent past.  If anything, I think that an agile/DevOps approach could help them significantly considering the sheer amount of data they need to process day in and day out.

    I want someone to chime in here and prove me 100% wrong and that I've just been out of the loop for a hot minute.

  • On thing that comes to mind from this was how at the enterprise where I worked through their transition to devops it became pretty clear that a big part of their strategy involved not being afraid to crack some eggs in order to move the ball forward. 

    The company had developed some really business-hostile patterns in some parts of their IT org.  If you wanted a new VM allocated we were looking at months of requesting approvals and hand built process and red tape and a certain amount of kingdom building where departments would sand bag and block efficient automations because they didnt want to lose head count. Sometimes people in big IT orgs can forget that the purpose of all those datacenters and racks of computers is to help the business get things done faster, and if you aren't doing that then they won't need you around. Our leadership was taking advantage of their migration to cloud and devops practices as a cover as they cleaned house on all their most obstinate, thorny IT people and it didn't take long before people understood that just saying "No" to everything wasn't going to be a viable career path anymore.  A fundamental aspect of doing devops right is that it is a continuous loop with feedback and process improvement, you won't get far without a collaborative growth mindset.  Initially it spooked a lot of people and caused a good deal of staff changes, but after about 2 years they had cleared the way forward and operations were really dramatically improved.