There’s Always That One Guy… How to Build Team Accountability and Reduce Firefighting.

katieb over 4 years ago 2 minute read time

Change control. In theory it works. However, there’s always one person who thinks the process doesn’t apply to them. Their justification for going rogue may sound something like, “There’s no time to wait, this has to be done now,” and, “This is a small change, it won’t impact anything else,” or maybe, “This change will make things better.”

But at the end of the day, those changes inevitably end up crashing a service, slowing application performance, or even worse, opening new vulnerabilities. The call will come in, something’s broken and magically no one will know why on earth it’s happening and, they certainly won’t be able to remember if any changes occurred…or who made a change. There goes the rest of your day, looking for the root cause of an issue created by one of your own coworkers.

Recently, Head Geeks Thomas LaRock sqlrockstar and Leon Adato adatole hosted a THWACKcamp session on this exact topic. In their scenario the culprit was “Brad the DBA.” At SolarWinds, we understand this all-too-common scenario and have a tool designed to help.

SolarWinds Server Configuration Monitor (SCM) provides an easy-to-use and affordable way to track when server or application configuration changes are being made, who’s making the changes, and what the differences are between the old configuration and the new configuration. It detects, tracks, and alerts on changes to things like hardware, software, operating systems, text and binary files, Window Registry, and script outputs on Windows and Linux servers.

Additionally, SCM is an Orion Platform-based module, meaning you can quickly correlate configuration changes with infrastructure and application performance metrics in a single view. Helping confirm or illuminate the possibility of a configuration change being the culprit.

These capabilities help provide you with the visibility needed to not only remediate issues faster but, also hold non-process-abiding team members accountable for their actions. If you’re tired of the shenanigans created by your colleagues not following the change control process for your servers and applications, check out a free, 30-day trial of Server Configuration Monitor. And just for fun, if you have a good story of how “Brad” broke your day, feel free to share below!

Top Comments

katieb over 4 years ago +2

Hi jkump ! Thanks for your post. I am very happy to hear NCM is a valuable tool for you. The items you mention (real time reporting, who made the change, and side-by-side comparisons) are all features…
df112 over 4 years ago +1

So true. Hopefully you only have one of those. They always have some super funky way of "making it work" that doesn't work with anything else and causes all kinds of problems in other areas (which don…
rschroeder over 4 years ago +1

Using NCM was a great way to give staff incentive for using Change Management / Change Control. NCM simplified the task of learning what changed and when. Combining it with NPM's log for Solarwinds administrators…

graeme.brown over 4 years ago

working in an MSP environment leads to all kinds of cowboys "fixing things" on the fly and logging almost nothing. DAYS spent tracking down the change, why it happened and is it actually/still needed. sometimes the change was "I was trying it out to see" or "I thought I cancelled that change" or some other nonchalant comment followed by "if you want to change it back go ahead". burns my toast that does.
- Cancel
- Vote Up 0 Vote Down
- More
- Cancel
tallyrich over 4 years ago

I'm a big proponent of change control - but it needs to be well thought out, organized and efficient. I worked with a company once that the initial go-round of change control could take up to 2 weeks to get a change approved - that didn't work for my organization as our customer was used to requesting a change in the morning and having it implemented the same day. They later revised that policy and it became much more efficient and my customer then bought into it. Yes, with CC it now took a couple of days to implement a change, but that was an acceptable price as opposed to weeks.
- Cancel
- Vote Up 0 Vote Down
- More
- Cancel
bobmarley over 4 years ago

We have a pretty strict change policy. First time you make a change without a change control you get a talking to from your boss. The second time you get a box.
- Cancel
- Vote Up +1 Vote Down
- More
- Cancel
katieb over 4 years ago

Hi jkump!
Thanks for your post. I am very happy to hear NCM is a valuable tool for you. The items you mention (real time reporting, who made the change, and side-by-side comparisons) are all features that exist in SCM as well. In fact, the latest release of NCM borrowed code from SCM to better show the side-by-side compare. Real time alerting/reporting on changes and who made the change are available for Windows servers and application via the Orion Agent. Below are a few screen shots from SCM:
Happy to help if you have any questions.
- Cancel
- Vote Up +2 Vote Down
- More
- Cancel
jkump over 4 years ago

I agree with rschroeder on the benefits of NCM. The real-time reporting has made things easier to understand when changes happen and who made the changes. The side-by-side view also allows understanding of what was changed. I am looking forward to seeing if SCM will provide the same benefits.
- Cancel
- Vote Up +1 Vote Down
- More
- Cancel