Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials

Issues with Interface Removal script using the Orion SDK

Hello,

I have been working on some scripts to automate some repetitive tasks in relation to merging two distinct Orion NPM platforms, but have hit some issues in getting this script working.

The purpose of the script is to compare all interfaces in two NPM installations and then remove all interfaces from installation B (Destination) that do not exist in installation A (Source). If the script determines that an interface is found to be in both installations it is left unchanged. The result should be that all interfaces on both installations are consistent (Unless there are additional interfaces on the source).

I have this script working perfectly in our development environment, but when executing the script on a backed up production environment I get odd results.

This is what’s happening, which is different from my exhaustive local testing.

The lookup is using the IP address of the Node and the InterfaceIndex as unique identifiers

1. Interfaces exist on both Orion instances with complete match i.e. all 5 Interfaces exist with the same InterfaceIndex

a. Result – No changes are made – this is correct and expected behaviour

2. Interface/s exist on Destination (B) but not on Source (A) AND Interfaces that exist on both

a. ALL interfaces are being deleted and not just the mismatches

i. This is unexpected behaviour

b. The logging output of the script shows the correct assessment and is tagging each Interface correctly i.e. Interfaces with Result of 1 are identified as not existing on both and 0 is where they exist on both and should therefore be left alone

c. Orion logging is showing that ONLY those interfaces that are being matched are being deleted. NO Orion logging is being performed for the Interfaces that a not matched and are deleted

i. This is odd behaviour as if an interface is being deleted via the API, it should be logged regardless

Orion versions v10.3

API v1.5

Powershell script

I would appreciate any help on this.

Regards,

Ben

NPMInterfaceRemoval.ps1

Find more posts tagged with

Accepted answers

All comments

tdanner

I read through your script and it all looks fine to me.

Ultimately, the problem you are seeing is that this:

Remove-SwisObject $destination -Uri $interface.Uri;

Is actually deleting all of the interfaces on its node. That's quite odd. I can't think of anything that would cause that, so I'll just ask a bunch of questions.

Does $interface.Uri look normal when that happens? I assume you would have mentioned if there was something weird there.

In the script you sent, $destinationDataRestrictions = "". Are you using anything complex there in your version?

If you run Remove-SwisObject $destination -Uri $interface.Uri; outside of the script, can you reproduce the behavior?

Is the backed-up production environment a VM that you can repeatedly snapshot and restore? If so, does the script nuke the interfaces on the same node(s) every time? Or does it appear random?

bengreco

Hello there,

Many thanks for your reply.

The Uri does indeed look normal when the problem occurs, I can't reproduce the error outside of the script and checks against smaller subsets of the data work fine.

I have been using the $destinationDataRestrictions field to insert where clauses such as "WHERE NodeID = 1" for testing, and only use conditions based on the NodeID in here and this always works as expected.

When the script is executed the $destinationDataRestriction field is left as an empty string and doesn't affect the results of the script in this state.

The behaviour is not reproduced when running the command Remove-SwisObject $destination -Uri $interface.Uri manually.

The DB is certainly backed up and restorable, but this is not a VM so I am unable to use VM snapshots.

Thanks for your help,

Ben

tdanner

If the problem can't be reproduced with a subset of the data, this is going to be tricky. To investigate this further, I need an Information Service log file at DEBUG level the covers the time period the script ran. Since this is processing thousands of interfaces, that file is going to be enormous, though highly compressible. Use the LogAdjuster to set the "Information Service 3.0" and "Information Service 3.0 - SQL" categories to DEBUG, and increase the file size and number of files. I'm not sure how big they would need to be to cover that period, so I'd err on the big side. Hopefully 100 MB, 10 files would be enough!

tdanner

I got some more details from Caroline Toomey that change the picture somewhat. According to her summary, what you are seeing is that immediately after the script runs, things are in the expected state. But then after database maintenance runs, more interfaces get deleted. In light of that, let's hold off on gathering a giant log file. I'll review the situation based on this information and see if I can identify some possible bad interaction between the API and database maintenance and follow up here.

tdanner

I figured out what's going on. Through a somewhat complicated chain of indirect effects, deleting an interface through the API leads to that interface's NodeID getting added to the DeletedNodes table, which database maintenance uses to delete child objects of that node, including interfaces.

To work around this issue, clear the DeletedNodes table after running your script and before database maintenance runs.

I opened case 166415 for this in our bug tracking system.

bengreco

Good morning,

Many thanks for your help on this, sorry I did not give you the update sooner. We realised that the DB maintentence routine was causing an issue yesterday, but did not manage to trace this back to the entires you mention in the DeletedNodes table. I shall be performing test this morning based on your post and will let you know how I get on.

Thanks again!

Ben

m_roberts

Hi Tim,

I can confirm that the Interface Removal function within the API was causing the NodeID to be added to the DeletedNodes table for us. Truncating this table after running the script has ensured that the additional interfaces do not then get removed when the Orion DB Maintenance task processes.

We now have a merged database which has allowed us to utilise the API to compare two Orion databases and remove the interfaces that do not appear within the current production installation. This issue has caused us heartache, but I am glad to say we are there now.

Thanks for your assistance.

Mark

tuanle55

Remove-SwisObject $destination -Uri $interface.Uri;

I am using SDK 1.5 / NPM 10.4

I use the above and I do not see my nodes being in the DeletedNodes table. I do see my deleted interface in the DeletedInterfaces. I am running NPM 10.4 Has this bug been resolved? I am about to delete a few hundred interfaces from our system.