Database connection issues

Hi,

Just wondering if anyone else has come across this issue in their use of Orion NPM 9.5 with SP1 and hotfix applied?

I just had to restart my NPM server because it had crashed (or at least the web server was not responding). After restarting the server and logging back into the web console I could see that it looked as though I was missing information from the ends of the graph (ie the most recent polls that would have been completed after the reboot).

I then looked into the event log and found the two following errors in the Solarwinds.Net Event log:

Alerting Service
--------------------
2009-07-01 09:12:27,000 [MainTaskThread] ERROR All - Service was unable to open new database connection when requested.
Exception Cannot open database "NetPerfMon" requested by the login. The login failed.
Login failed for user 'SolarWindsNPM'.
Connection string - Data Source=FTV02LMS\SOLARWINDS_ORION;Initial Catalog=NetPerfMon;User ID=SolarWindsNPM;Password=*******;Connect Timeout=20;Load Balance Timeout=120;Application Name="Orion Alerting Engine";Workstation ID=FTV02LMS

Trap Service
----------------
Service was unable to open new database connection when requested.
Exception Cannot open database "NetPerfMon" requested by the login. The login failed.
Login failed for user 'SolarWindsNPM'.
Connection string - Data Source=FTV02LMS\SOLARWINDS_ORION;Initial Catalog=NetPerfMon;User ID=SolarWindsNPM;Password=*******;Connect Timeout=20;Load Balance Timeout=120;Application Name="Solarwinds Trap Receiver Service";Workstation ID=FTV02LMS

In the application log there was also an error from the Apache Service:
The Apache service named C:\PROGRA~1\CSCOpx\MDC\Apache\Apache.exe reported the following error:
>>> [Wed Jul 01 09:16:29 2009] [warn] pid file c:/program files/cscopx/mdc/apache/logs/httpd.pid overwritten -- Unclean shutdown of previous Apache run? <<<
before the error.log file could be opened.
More information may be available in the error.log file. .

And there was also 4 "Failure Audit" events with a source listed as MSSQL$SOLARWINDS and the following error:

Login failed for user 'SolarWindsNPM'. [CLIENT: 10.8.82.185]; and
Login failed for user 'SolarWindsNPM'. [CLIENT: <local machine>]

What I assume is that there some sort of problem with the "SolarWindsNPM" user account in the SQL database. I don't recall setting this username or password during installation. Nor do I know how to change it so that there is no further connection issues to the database.

Thanks very much for your help.

Find more posts tagged with

web_console

web_server_not_responding

missing_data

database_connection_error

Accepted answers

All comments

bshopp

I would recommend upgrading to SP2 which is now out. After upgrading, see if this issue still persists. If so, please open a support ticket

DeanBennett

We had the same issue on our v9.5 server, after around 30 minutes Orion started responding again, but afterwards one of the Windows nodes had been categorised as "unknown", but had previously been categorised correclty and was being monitored.

Upon further investigation it transpired that the SNMP on the Windows node had become "corrupt". I tried using a MIB tree walker to walk the Windows node's MIB tree, it started to return results, but then returned an error.

So I'm guessing if Orion had previously discovered a node and it's attributes (disks, NIC etc), then upon trying to poll it only gets partial results, it's not going to like it too much.

Would that cause the errors we're seeing ?

wessemat

Hi i have the same Problem is there anz Solution? I am using NPM 9.5 SP5!

Thanks...

stevecera

We have been seeing these same issues with NPM 9.5.1. They also seem to be related to some issues we are seeing with alerts mis-firing. Our alerts are setup based on the owner of the device and the status. We are using several custom pollers to poll SNMP info such as disk state, PSU state from Dell servers and run time remaining on batteries from APC devices. When the alerts mis-fire, not only are the failure conditions present, but the alerts fire to a person other than the listed owner of the device.

Is anyone experiencing any weird incorrect alert firings around the same times that they are seeing the database connection errors mentioned in this thread?

Thank you,

-Steve

Orion NPM 9.5.1

Cosmin

Hi Stevecera,

I'm using Orion v9.1 SP5 and last night I got the same problems with Orion as you. Orion started firing a lot of incorrect alarms about unknown staus of interfaces and nodes...and when I've checked Event Viewer I found out that there are some issues with the connections to the database, like "2010-05-26 03:05:37,077 [MainTaskThread] ERROR All - Service was unable to open new database connection when requested."

I thought of upgrading Orion to 9.5.1...but it seems that the issue still persists in newer versions as well.

Can we do something to solve this problem?

Thanks,

Cosmin

stevecera

Unfortunately, I have not found a solution for this issue yet. SolarWinds tech support tried assisting, but in the end told me that we were overworking our server and recommended that we separate the database out. Unfortunately, a 2nd server is not in the budget for this, so we are currently just dealing with the misfires. They don't seem to happen very often, but when they do occur, they occur in groves. Sometimes they fail to clear automatically as well (since we have all alerts set to clear when the trigger is no longer met), so we have to go in and manually clear them. I wish SolarWinds was able to give us a better answer since our server is way over-powered according to their documented server requirements...

cmiller

Just as an FYI - I know this is for Orion NPM, but i am using Orion NCM. And i am seeing the same issue with the database connection. Unfortunatley NCM does not reconnect until reboot. And my SQL is on a seperate server. NCM is only app. installed on server. and Support tried to tell me the same thing of overworking the server...Knowing this is not possible....

Any solutions? Bueller? Bueller?

pacetti

The following KB article provides a link to Microsoft troubleshooting for SQL connectivity:

knowledgebase.solarwinds.com/.../

Let us know if this helps,