Hot fixes and service releases are certainly important to us. We released 9 hotfixes for NPM 10.7, as you mentioned. I'm not sure there is any official process to hot fixes or service releases. We weigh situations as they come and try to make the best decision for our customers. It sounds like in this case it would have been better for you to have a service release for 10.7. I'd like to understand that better. In your environment, it sounds like the testing requirements for deploying a new major release are different vs a hot fix or a service release. Can you elaborate on that a bit? That would help me understand your situation better.
We only run software in production if it runs without any issues. We are deeply dependant on Orion and we therefore expect it to perform without trouble 24/7/365. This then requires in house testing to confirm that there are no issues which would cause downtime, wrong data or performance issues. We have 0 tolerance for software that doesn't deliver.
If there is a known issue and that issue can be addressed with a hotfix then that hotfix would be tested and confirmed to solve the issue as well. Applying a hotfix to a test setup for testing is rather straight forward and in best case scenario would take very little time to carry out. Applying a hotfix to a production system is a whole different story. Not only are we required to create change requests and change documentation but we are also required to test the change for at least a week before going live. 9 hotfixes would then require 9 change requests, 9 change docs, 9 testing procedures, 9 weeks of testing and then 2 more weeks to implement on a production system. You probably think that this is overkill and feel sorry for us, we do too but these are the guidelines we need to follow...
When it comes to deploying a version release, the procedure is similar to hotfix testing, only it takes much longer because bugs and issues are expected to be present in such a release. Only when a service pack is made available do we ever consider starting a test. Example; with NPM 11, we'll only consider testing it for production use once 11.1 or 11.x is released.
Again, we are not the only ones that have expectations to the software that we use. If it doesn't deliver then we won't be using it. Hope that explains the situation.
Just to add to cobrien's response, HFs are very narrow in scope and impact, and in general shouldn't be applied unless you are experiencing the issue that the HF addresses. The critical mass for service release would essentially be a sufficient number of high-impact issues that apply to a broad section of the user-base. This also weighs against how close the next major-release would be that would consume those fixes as well. To your point about requiring time to QA an upgrade, it would be frustrating I'm sure to start QA on a 10.7.5 release just to have the 11.0 release 90 days later.
Call it an extreme coincidence but all the issues that the hotfixes address, bar one, exist on our setup. As far as the 10.7 hotfixes provided so far I can see that all of them are critical to a functional (one that is used by people) deployment. Let me prove it:
Network Performance Monitor v10.7.0 - Hot Fix 1 for Daylight Savings Time problem
This HotFix addresses an issue where polling stops before the shift to the daylight saving time due to the effects of the time change on scheduled polling jobs.
Network Performance Monitor v10.7.0 - Hot Fix 2
This hotfix addresses an issue when adding a node via web console add volumes with "Volume Type" which depends on browser locale. "Volume Type" should always be in Orion's primary locale.
Not having "Volume Type" in the primary locale can for example break alerts filtering.
Network Performance Monitor - Hot Fix 3 for OpenSSL issue
This HotFix addresses the following security issues that occur in OpenSSL: CVE-2014-0224, CVE-2014-0221, CVE-2014-0195, CVE-2014-3470, CVE-2014-0160, CVE-2014-0076, CVE-2013-4353, and CVE-2013-6450.
Network Performance Monitor v10.7.0 - HotFix 4
This HotFix provides a patch for the DB maintenance overdue issue that is caused by Interface Baseline Calculation. This HotFix applies to SolarWinds Orion NPM 10.7.
Network Performance Monitor v10.7.0 - HotFix 5
This hotfix addresses the issue with parsing the VRF Route Distinguisher on some Juniper devices.
Network Performance Monitor v10.7.0 - HotFix 6
This HotFix addresses following three error messages visible in dataprocessor.log.
1. System.InvalidCastException: Specified cast is not valid.
2. NPM.Node.MulticastRouting.Snmp failed. System.OverflowException: Arithmetic operation resulted in an overflow.
3. System.Data.SqlClient.SqlException (0x80131904): Cannot insert the value NULL into column 'MulticastGroupNodeID', table 'SolarWindsOrion.dbo.NPM_Multicast_RendezvousPoints'; column does not allow nulls. INSERT fails.
Network Performance Monitor v10.7.0 - Hot Fix 8
This HotFix improves using Active Directory Groups website login. It introduces configurability of used authentication and Group Membership evaluation methods.
Symptoms: The user is not able to use an Active Directory Group account to log in to the Orion Web Console.
Network Performance Monitor v10.7.0 - Hot Fix 9
This HotFix addresses an issue where some nodes are not polled in NPM 10.7 until you restart the Collector service.
We would much rather start QA on a service release than on a version release because as with all new version releases, they WILL include bugs! That is mainly because new features are implemented in that new version. This is why the stable release concept is important with software that users are heavily dependant on. We always favor stability above and instead of new features.
I will sure support Deltona on that.
Why 9 hot fix and not 15 ? ;-) if there no guideline.
Sure like to have some officel guideline
1.up to X hot fix per GR.
2.up to X days/weeks from last hot fix to SR
3.up to X SR to new GR
4.major bugs should not move from GR to GR.
That will sure help the admin that running large environment to take the call.
How likely is it that a 10.7.1 release will make it to the portal?
With the release of 11.0, we would likely not have a SR for 10.7, as we typically roll previous fixes into the new major release.
What Dev /you opinion on about schedule SR?
What Dev /you opinion on about some "time frame" guides from GR ->SR?
With the dev / QA overhead associated with a release, it is more a matter of having a technical driver for a SR than a time-bound event. We can certainly discuss with our dev management team to see if we can proceduralize for the future.
This is an extremely important feature of the product that should have maximum attention.
Dominoing patches and hotfixes forward into new version releases might be a good exercise to keep everyone "on the boat" but this makes the product certifiably unstable - forever!
We've had stable builds with regular service releases that updated the version code in the past. The stable releases were crucial in the trust building of the product and have been key to keeping customers.
I can't believe how difficult it has become compared to the structure of old. I have no idea what hotfixes have been applied as Add/Remove programs doesn't get updated with patch version codes. Speaking of patches, the hitfixes listed above are in essence patches.
Please push for a stable release, please!
Please don't take this to mean we would not have service releases going forward, and we are also looking at alternate delivery methods for HF for the future. To your point, it would be advantageous to make it easier to understand what HF are applied to the environment.
All I know is that this setup is going to be running crippled and can't be used properly for the next couple of months because the patches/hotfixes/bugfixes we require are only present in the next version release (required patch is not one of the 9 released hotfixes).
We will inevitably find a bug post Q1/2 2015 after finally having upgraded to v11 and that bug would turn out to be addressed in the next version release, v12 maybe. As for the 1 version + 2 service release cycle model, all we are guaranteed to get annually is 1 version release that is guaranteed to have bugs.
Sure it adds up but not to what you'd expect.
I also support the concept of Service Packs. Using our friend Microsoft as an example, they roll a large quantity of patches (hot fixes) into a Service Pack that has been vendor validated to work together and has a pretested installation sequence builtin.
Please consider the idea of Service Packs using one of the following criteria:
- We've not produced a hot fix for 3 months. Lets roll all the fixes together into a SP
- By policy, we release service packs every 6 months.
Either criteria would make Orion administrators lives easier when patching and/or justifying patching to management.
Now NPM 11.0.1 is out very fast or too fast ?
I can see support to SQL 2012 sp2
but no bug fix ...
I still don't get the logic ....
when a update like that could be a hotfix and when it's SR or GR...?
I know you can say no logic..
The driver for 11.0.1 was potential upgrade issues with SQL 2012 SP2. Not something we would be able to address with a HF.
There should be serious consideration for Deltona 's concerns. As I too am part of an environment where patching is acceptable and understood.
Moving to a new Version requires paperwork,scope, testing, planning, a NEW Risk Assessment, implementation and further checking to verify everything is running properly.
It is much easier for us to update the current version rather than dealing with the paperwork of the so called 'Upgrade'.
MS still updates/patches SOME products even after the new version has been released ... this is merely reference of concept as we certainly do not want to use MS testing and/or release structure as a model for SW Products.
So what's up solarwinds ?
NPM 11.1 has 7 HF and know bugs from 10.7 RC
Sure like to see 11.2 SR that fix some old issues....
I've seen the flux of hotfixes come in since 11.0.1 and opted not to update or comment this thread, but since you pointed out this recurring issue I feel compelled to add another 2 cents.
rob.hock, why not follow SAM team's take on hotfixing and at least provide an installer that does the hotfixes doubleclick'omatically? FYI the hotfixes present overlap. Once done applying HF1 and moving on to the next HFs, you are made aware that the next HF overwrites an previous HF. Doh?!
1 of 1 people found this helpful
We are absolutely looking at cumulative HF in the future for other products. The process with SAM is an improvement, but we can certainly improve further.