So after upgrading the primary , secondary poller and additional web server on recommendation from support to address the multiple of bugs I am having - I know see this error from time to time ( Server error: Error: A query to the SolarWinds Information Service failed. ) and when watching the Orion services manager solar winds information service V3 keeps going up and down ...
SO much for painless upgrade again
Solved! Go to Solution.
You may find below HF handy for 11.5.1
SolarWinds Orion Platform 2015.1.1 Hotfix 1
This Hotfix addresses these issues:
1. SWIS V3 Crashing since upgrade to NPM 11.5.1
Apply this Hotfix if you run any of the following SolarWinds products:
- NPM 11.5.1
- UDT 3.2.2
- This Hotfix requires SolarWinds Orion Core 2015.1.1.
- This Hotfix must be installed on the main poller.
- This Hotfix must be installed on any additional pollers/webservers.
- This Hotfix contains the InformationService.msi file with InformationService version 2015.1.236.
- This Hotfix addresses issues for all SolarWinds products located on the same server. As a result this Core platform Hotfix only needs to be applied once per server.
This hotfix should resolve the info service crashing issue, but it is a memory hog and performance is not so good overall.
The resulting "upgraded" Information Service V3 should appear in Control Panel after the change as:
Results of our upgrade:
a) alerts table update broke some reports.. not that much of a problem
b) one additional poller appears broken-- with SW Dev to figure out what happened there
c) misc. config files stomped over by the installer -- separate cases opened for each of these.
d) cannot add any new customize/add resources to a page ("Object reference not set to an instance of an object." errors)
so, I give this upgrade a B+
"cannot add any new customize/add resources to a page ("Object reference not set to an instance of an object." error"
I had this bug in earlier versions of NPM. If you want to give it a shot, he fix (hopefully for you too) may be here: Re: Object reference not set to an instance of an object.
we are considering upgrading from npm 11.0.1 -> 11.5.1 next week. Now that you've been on it for a few weeks, are you glad you upgraded, despite the issues you experienced, or do you wish you waited? appreciate you thoughts.
Our environment is probably one or two orders of magnitude larger than most customers
a) has been fixed by me re-writing reports; the new SDK documentation helped with that
b) I deleted everything solarwinds-related on the servers, moved the directories to one side and did as fresh an install as I could and the problem was resolved.
really, FOE is a pain in the ass to deal with, and I understand what it's doing as we use neverfail elsewhere but I'm not convinced that it's the right failover technology.
[really allowing more dynamic moving of polling from one server to another would be a better solution, but APE cost way more than FOE...]
c) This is annoying, and caused us to lose all of our historical AP data. I can't restore the database because it has the wrong schema. it also broke the topology calculator and business layer, but thos were easily fixed.
d) deleted the inetpub directory and re-ran the install wizard to recreate. fixed the problem.
I do like
a) the new report writer, especially the 'this will general 1000 alerts if you enable this alert now' widget at the end that tells me I've messed up a condition before I light up everyone in the NOC.
b) the ability to define new pollers to cope with odd machine types. I'd like this to be easier to use, but I now have all our werid new stuff polling and being 'correct' rather than as generic net-snmp devices.
In general I'm okay that we upgraded, like I said, I'd give it a B+. I needed to do it so i can get the UDT upgrade scheduled
I would suggest you go to 11.5.2 as that includes some significant patches.
Curious as to what "one or two orders of magnitude larger" means
We have 18 pollers across 3 data centers, 11000 nodes, 75000 elements, almost 100,000 application monitors. Running NPM, SAM, VMAN, NTA (kind of) and just purchased STM/SRM.
Is it that kind of big? I ask as the impact of a botched upgrade on a single poller environment stinks. When you magnify that across 18 pollers it makes life miserable.
You're also not most customers... The University of Washington is much larger than people realize, and the network we mange is even larger.
I only look after the network infrastructure so: we're five pollers (in FOE, making ten upgrades each time we bump a version) across 3 data centers, 6700 nodes; 38,000 elements,
the nodes includes 20 wireless controllers that support over 9000 wireless access points, on any given day we have about 180,000 wireless devices roaming around the network.
Here is just a fraction of our network:
so, about half your size.
I heard a rumour that DanielleH & mrs.alterego have been discussing a special forum just for those of us with enterprise-scale implementations. Why? Not to be exclusive or elitist but simply because we have problems that are unique to enterprise customers. What happens on a single poller or dual-poller environment doesn't always scale well when you get to 10, 15 or 20+ pollers. We have some pretty unique challenges with growth (just look at the updates we had to make to The Ultimate CPU Alert - Reloaded! and The Ultimate CPU Alert...For Linux - Reloaded! when we applied the original to 6000 servers!) and sometimes it we just need to know that the feedback we are getting is from like-sized environments.
Seems like there are more than a few of us who would be a great fit in a new forum space 'Enterprise-scale Deployments'
I would love a special forum for enterprise deployments. We are at 9 APE's, 65K total elements, 30K SAM, 7K nodes. We are adding another additional web site so that we can load balance user traffic better.
(Not really the place for this comment, but no other place or time is better) I would disagree with your comment on a "separate forum" from one perspective I find invaluable. The rest of us should be using your environment (and Richard's) and feedback as a model for ours. While my environment is not nearly the same size, if I build my environment to scale to your size, then I'm prepared to grow. I would say, you, Richard, and anyone else of your size, should get together and document a "best practices" whitepaper the rest of us can use.
I agree with deverts... and on top of that we have enough forums as it is to try and stay on top of. Also I think a lot more environments the tools are used on are bigger than you think. Also as far as the SW isn't just for mom and pop shops anymore... TELL THAT TO EOC ctmidnight!
I 100% agree with you. Larger environments do provide the insight into scaling and we *really* should document a "best practices" whitepaper. (I've been waiting for one of those for years!)
Perhaps I should have been more clear about my intent. The separate forum would still be an open forum (as every forum is already) but with a specific focus on Enterprise-scaling issues. It would *not* replace regular participation on product forums for the exact purpose that you pointed out -- symbiotic collaboration is the key to us being successful as a group.
Sorry if I sounded like I wanted our own little playground just because we are bigger -- that wouldn't be very cool.
You didn't sound like an "elitist," and I wasn't implying you did. My issue is with all the various forums we have already making it difficult to assist others and get assistance. This latest Thwack upgrade is taking away from the very thing that makes it so important...the ability to EASILY collaborate with experts. While I agree, a new forum for large environments would be nice for those that have large environments, all of us share the same issues. And sometimes less is more.
But again, this isn't the place for this discussion. I was just offering my opinion (which I should probably keep to myself most days ).
deverts, why do you think that the latest thwack makes it less easy to collaborate with experts? Feel free to DM if you don't want to continue the discussion here.
Also, just want to remind you all we have a space where you can suggest improvements for thwack, like you do with any other SW product:
It makes sense overall to me to have a separate forum. You have basic users, you have scripting users (such as bluefunelemental, myself) and a separate category for people in significantly large environments. I used to think my 4000+ nodes globally was a large environment but now I see we're simply spread out globally and not so much as giant of an environment.
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community. More than 150,000 members are here to solve problems, share technology and best practices, and directly contribute to our product development process.