cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

Merge NCM and NPM Databases

Merge NCM and NPM Databases

NCM now uses two databases to store its data -- NPM/Core db and NCM db. The suggestion is to store all NCM data in the NPM/Core database. The expected benefit is simpler db management; issues connected with node synchronization should disappear.

Please provide your comments.

19 Comments
Level 14

I'm not 100% sure I agree with this.  I definitely see the management benefits and value to consolidated data, but I also see possible performance issues.  NPM 10.4 (not sure about 10.5 yet, I just installed it) already has an issue with fragmentation due to the amount of indexes it creates and deletes.  Additionally, while this poll is all about NPM/NCM, if I consider all the Orion product "modules" and the integrations, I feel a more "Federated" database makes more sense to me.  Keep NPM and NCM data separate, split out NTA (which I believe is coming), split out VNQM, IPAM, and SAM, and create a "Core" DB that contains all the common data elements.  Every element already has an ID number, that is the hook the web page can use to display data.

I know, I know, this would mean a total rewrite of the application, but look at the long term value to integrating all the new stuff you've purchased, like Storage Manager ("Profiler"), LEM ("TriGeo"), and Virtualization Manager ("Hyper9"), etc.  You simply point those products to the "Core" DB and it populates the devices.

Just my 2 cents.

D

Level 14

I share deverts concern about application performance once the DB's are merged. However, since NTA is moving to its own database, perhaps it won't be so bad... By the way, does this mean that the standalone NCM will be discontinued?

Level 18

Hi,

While merging the databases, we also plan to optimize NCM DB. We are very well aware of the performance concerns.

Standalone NCM will be supported. When you have separate NCM now, there are technically two DB's, too -- the NCM one and the Core/"NPM" one.

Regards,

Jiri

Level 14

Jiri,

Thanks for the clarification.  However, your reply makes me wonder why NTA and NPM are treated differently by the SW developers.  What I mean is this... As stated previously, it seems that the SW developers figured out how to preserve the integration of NTA with NPM while at the same time giving NTA its own database (on NTA vnext).  So, it seems reasonable that the same could/should be done w/NCM where it would also be integrated w/NPM while also having its own database.  This would relieve concerns about performance for both NCM and NPM and also provide the integration that many of us are so eager to see between these two.

By the way, the same logic applies to NTA.... That is, if the SW developers found a way to merge the NCM and NPM databases without the app performance concerns...  Why were they then not able to do the same with NTA, so that it would also still reside on the same database as NPM? 

The only possible reason I can think of is that NTA (especially on vnext) will be much more SQL hungry (i.e. RAM, CPU cycles) than NCM.  So, in giving NTA vnext its own database the SW developers intend to address NTA SQL database concerns where the same concerns are not nearly as bad for the NCM database.  Would this be correct thinking?  I would appreciate your feedback on that as well.  Thanks.

Level 18

The situation with NCM/NPM and NTA/NPM is different.

First, both NPM and NTA need a lot of DB traffic in real-time; on the other hand, the vast majority of NCM tasks can be scheduled for off-peak hours.

Second, the nature of NTA traffic is different from both NPM and NCM; as far as I know, SQL DB might not be the only choice for the future.

Therefore I think it makes much more sense to have NPM and NCM DB's on the same SQL server than NPM and NTA.

Jiri

Level 13
  • Make sure that you include routines to clean up after yourselfs. In the past, you left crap in the DB - (think of SAM before it was a standalone product - I STILL have APM tables in my NPM database!).
  • Make sure to perform a backup - even if you tell the end-user to do it - YOU NEED to do it anyway.
  • Make sure you put this as a MAJOR bullet point in the release notes.

I would rather see the ability to use a SINGLE CORE database with the modules having a seperate DB.  For example, we have Orion NPM (server_A) Orion NCM (Server_B) and Solarwinds SAM (Server_C).   For each of these servers, I have a seperate "nodes" database - but they all have the EXACT SAME data - even the custom database fields I created have to be done 3 times - and there isnt, currently, any way to replicate/compare/sync these CORE databases.   I would, personally, like the ability to point all these different products to the same "NODES" database for the static stuff (Name, serial number, etc. etc) and then have a custom/seperate db for each product (Stats/Counters for NPM, Configs for  NCM, etc. )  That makes more sense to me.

Level 9

I am a small shop, but believe that adding NCM to Core DB is fine. Removing NTA as it is a huge IO hog I can see working well, but does NCM really get that much IO?

Level 18

Hi,

For big customers it can. We definitely take this fact into account.

Regards,

Jiri

Level 14

dclick,

Your idea about a single core, and my idea...same. I hadn't seen anyone else chime in on my notion, so I was beginning to think I was all alone is world and possibly crazy (which might still be a little true).

D

Level 14

Jiri,

Your statement here is a little contradictory. To your point, NTA and NPM are very much real-time and DB intensive. Also to your point, NCM is more focused around scheduled batch jobs to do config backups (unless you send configuration change traps from your devices that kick off immediate configuration download jobs). Very different behaviors, I agree. But, outside the obvious management and integration advantages, why would you consider any additional performance increase on NPM at all? You never know when NPM is very busy (its all determined by the environment), and if an NCM job kicks off during a busy time, we start seeing intermittent performance issues. In a small/medium environment, I agree...it doesn't matter where the DB goes because there isn't that much data collection/crunching to make a difference. But Orion is also being used in large Enterprise-class environments where every little bit of performance is crucial.

D