The Cost of Monitoring - with the wrong tool (part 2 of 2)
Or how SolarWinds saved a company $1 million a year, monitored more than we ever expected, and made me look like a miracle worker.
The company wanted to expand monitoring with their current solution to every single device in the enterprise (roughly 10,000 systems)—a task that would have required an additional $2 million in extra licenses, and would have increase yearly maintenance to $1 million.
I saw it as my first order of business to stop using this expensive-to-own, expensive-to-maintain tool like peanut butter—spreading it across every surface of the enterprise. For those tasks that needed to be everywhere (ping, simple hardware monitoring, etc) we needed a less expensive tool that could scale without breaking the bank.
We evaluated 6 alternatives from both open-source and traditional vendors. We found that just one came in with a solution under $800,000—SolarWinds. Not only that, but in most cases, two SolarWinds modules (Network Performance Monitor and Server & Application Monitor) covered more features than the competition. But the most interesting part was the price tag: Year-one purchase price would be less than 10% of the cost to implement a solution with the incumbent vendor. Maintenance would be about 6% of the predicted cost if we kept the current solution.
These numbers caught the attention of the CIO and the purchasing team. The monitoring group was asked repeatedly if we had somehow evaluated the wrong tools, if we were ordering the wrong number of licenses (we were ordering "unlimited" so that was not the case), if we misunderstood the vendor's quote. They were unable to comprehend how the SolarWinds quote was so low. We finally had to level with them: "Maybe this is normal pricing, and you've just gotten used to being over-charged all this time?"
Once we were able to assure everyone that this was indeed the real deal, the CIO did what companies stuck in a dollar-auction mentality never can: she announced that it was time to walk away. Because the licensing terms of the old tool were so restrictive, we would transition to SolarWinds for all "foundational" monitoring—availability, hardware, and application—and look for other tools to use in niche areas, be it real-time user experience monitoring, closed-architecture applications, or other special purposes.
The installation and migration was performed—not by a team of contractors over 18 months (which is what the previous tool had required), but by existing staff, in parallel with their daily duties on the old system, in just 6 months.
On the day we turned monitoring on, it was doing more than the old system—availability and hardware for 10,000 systems, plus WAN link monitoring and more. In the first 3 months our team was able to develop monitors that application and server owners had been requesting for years.
Monitoring has become an extremely specialized space. You can’t monitor your virtualization infrastructure the same way you monitor a router. Load Balancers require a different set of metrics than a storage array. Trying to force a tool which either wasn’t designed for the task or which hasn’t kept up with the times is not just a matter of “making do with what you have” and calling it “being budget conscious.” In fact, doing so can—and for this customer DID—cost the business an order of magnitude more, as well as not providing the functionality needed.