Last month I took part in our regular #datachat on Twitter. The topic was “Rolling SQL Server® Upgrades”, and my guest was Argenis Fernandez (blog | @DBArgenis) from SurveyMonkey. I’ve enjoyed doing the #datachat for the past year and I’m excited that they will be continuing in 2015.
The discussion that night was wonderful. Lots of data professionals talking about their experiences with upgrades, both good and bad. And the discussion wasn’t just one-way, either. We took the time to field questions from anyone participating in the #datachat hashtag.
When the night was done, and I reviewed the tweets that following day, I found myself grouping many of the tweets into some common thoughts regarding upgrades. Here’s what I saw:
- Have a plan
- Test the plan, repeatedly
- Have a plan for rollbacks
- Understand complexities
- Involve the business stakeholders
Let’s break those down a bit.
Have a plan
That goes without saying…or does it? You’d be surprised at the lack of planning when it comes to upgrades. Many, many times I have seen upgrades go wrong and because there is no actual plan in place the upgrade continues forward. This is just madness, really, as changes are now being hastily applied in production in an effort to get things working as expected.
Test the plan, repeatedly
I’ve seen situations where plans were developed, but not thoroughly tested. Sometimes, the plans weren’t tested at all. The end result is similar to not having any plan in place (see above).
Have a plan for rollbacks
Rolling back is something that must be considered. If your company is unwilling to rollback when things go wrong than you might as well not have any plan in place (see above). The idea that the changes MUST be deployed at all costs is the wrong mentality to have. You might think it is driving the business forward, but the reality is that you are letting chaos rule your change deployment process.
As a server or database administrator you need to understand the complexities of the technologies you are using. The easiest way I have found to get this done is to ask yourself “what if?” at every stage of your plan. What if the scripts fail? What if we upgrade the wrong instance first? What if mirroring breaks while the upgrade is in progress? Answering these questions helps everyone to understand what actions may be needed.
Involve the business stakeholders
I’m kinda surprised at this, but apparently there are shops out there performing upgrades without notifying the business end-users. Perhaps my experience in a regulated industry like financial services means I have some blinders on here, but I cannot imagine that the business is not involved in signing off on the changes when they are completed. You simply must have them involved in some way throughout the upgrade process, if nothing else to serve as reassurance that things are working correctly.
Thanks again to everyone who participated in the #datachat, we always have fun putting these together and I’m looking forward to many more!