NPM database corruption - remediation options

Question

I've got a looooooong running Orion instance that I've taken over, it started life back in the NPM 8.x era, possibly earlier and has been through many upgrades over the years.  Currently at NPM 12.3.  I've been reviewing all the SQL databases under my purview and noticed Orion's was lacking some basic maint routines, specifically no SQL level backups or checkdbs.  Of course, first checkdb I do, I find issues, and of course it's present in every backup I've got so I suspect it predates me.

In any case, I need to fix it.  The trouble is in SolarwindsOrion.dbo.AlertDefinitions and SolarwindsOrion.dbo.MapStudioFiles.  Based on my read of the errors, and a review of the two tables my current action plan is to:

* Stop all Orion services
* Back up the database in SQL
* Script as a drop and create the two impacted tables
* Drop, and re-add the tables via the scripts
* checkdb
* Restart Orion, review alert counts in admin, and poke at our maps looking for errors

I don't see any triggers on the two tables that would imply I can't do this, but I'm curious what others have encountered.

mesverrum · Answer

Plan looks believable to me.

If it makes you feel better, alert definitions has been deprecated a long time. Map studio files could be a big loss if you already have a lot of atlas diagrams, but I have to admit when I see old instances change hands we almost always find 3/4 of the maps we're way out of date anyway.

jcoombs · Answer

At this point I've got VM level backups of the system, and have been taking SQL level backups as snaps as I poke at the box.  SQL is currently running against the DB, no observed errors and if I hadn't decided to take a peek under the hood there was no indicators of trouble.  I also plan on taking an SQL backup with Orion shut down prior to making any actual changes as a roll back method.

jm_sysadmin · Answer

Maybe a SQL guru will chime in, but I personally would call support if possible.
That said, I would start be ensuring I had one back up I could use going forward in case I make things worse not better.

Then I would make sure that SQL can mount my DB if not, I actually don't know if I would try to rescue it. I might give up and start over.

Assuming SQL can mount it and it just the database contents that are unhealthy, I would re-run the configuration wizard. It gives love to the DB schema does some clean up. It might erorr out though, and support can assist with that.

Good luck!