cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

How to automate the creation of Orion Platform (aka Core) nodes from the API

Level 15

As our customers deploy more SolarWinds products, we see an increasing number of demand for a programmatic way to export and/or import nodes (as well as interfaces, volumes…) from/to an Orion instance.

 

The combinations of products involved in these scenarios cover, but are not limited to combinations such as:

 

  • Export nodes from NCM and import them into NPM network monitor
  • Export nodes from your own CMDB and import them into NPM
  • Export nodes from your own CMDB and import them into NCM
  • Export nodes from NPM and import them into NCM

 

This blog contains and describes a working PowerShell script example, that exports nodes from SolarWinds NPM network monitoring software and imports  them into NCM, but it can be easily modified to accommodate other combinations, including non-SolarWinds sources (CMDB)

 

The required configuration to run this script is:

 

  • NPM 10.2.x or 10.3.x (different versions of the script are required, see below)
  • NCM 7.0.2 or 7.1.x (different versions of the script are required, see below)
  • SDK 1.3 or 1.4 or 1.5
  • Windows PowerShell (ISE is used in this illustration)

 

The script for 10.2.x is located here:PowerShell script that exports Nodes from NPM and imports them into NCM (for Core 2011.2.x and befor....

The version adapted for Core 2012.1 (comes with NPM 10.3) is here: PowerShell script that exports Nodes from NPM and imports them into NCM (for Core 2012.1 and after -...

 

Here is an example of run, which extracted 561 nodes from my NPM 10.2.2 and imported them into my NCM 7.0.2

 

  • The upper panel of PowerShell ISE contains the script. The first 2 blocks define hosts and credential for your data source (NPM in this example) and target (NCM in this example). These need to be changed for your environment, of course.
  • The second panel shows the trace of the run. Should be pretty self explanatory.

 

image_thumb_03D8D49E.png

 

  • After the run, the target system (NCM 7.0.2) had 561 more nodes. As you can see, the nodes added by the API, are “tagged” with a property called “ImportedByAPI”, so you can easily identify, group and mass-modify them if needed.

 

image_thumb_30EDC16C.png

 

A few features of the script:

 

  • Nodes that already exist (same IP) in the target system, are skipped. The trace will send you messages such as this:

 

image_thumb_296251FF.png

  • The script as uploaded, actually ignores interfaces and volumes, but just because the 2 relevant sections are “commented out”. Look for sections like this and remove the If (0) {…} to enable them and start moving interfaces and volumes too (interfaces requires NPM network management software on both sides of the script, of course)

 

image_thumb_41F1EF4F.png

 

  • The script extracts all properties from the Nodes and copies them over to the target Orion.
  • If you run this script against an NCM 7 target, prior to 7.0.2 (i.e. 7 or 7.0.1), the added node won’t be manageable by NCM (they will just be in Core), you will need to add them manually to NCM, using the Add or Manage Nodes UI. NCM 7.0.2 has a new API verb that imports the node up to NCM, so it becomes immediately and automatically manageable by NCM. The last line of the script takes care of this:

 

image_thumb_3583CC26.png

 

I hope you will enjoy and benefit from this script, and I don’t want to finish without thanking Tim and Tomas for their invaluable help!

Try the 30-day free download of NPM and NCM network management software.

44 Comments
MVP
MVP

Excellent!

Level 11

I was running this from NCM (version 7.0.2) server where I installed the SDK (version 1.5) and was receiving this error.

Get-SwisData : There was no endpoint listening at net.tcp://localhost:17777/SolarWinds/InformationService/v3/Orion/ssl that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.

At C:\Program Files (x86)\SolarWinds\Orion SDK\CopyNodes from Orion to NCM.ps1:57 char:18

+ if ((Get-SwisData  <<<< $target "SELECT Field FROM Orion.CustomProperty WHERE Table='Nodes' AND Field='ImportedByAPI'") -eq $null)

When I commented this line of code out, I received the same error on the next Get-SwisData command. The only changes I have made to the script are for username and password and IP Address.

Anyone know what might be going on here?

Level 11

Figured out the issue. NCM 7.0.2 is incompatible with SDK 1.5. I uninstalled 1.5 and installed 1.4. Works like a champ!

Level 19

Sorry for the late reply, but I think it might be helpful for me to explain what's going on here.

In SDK 1.5, the Connect-Swis cmdlet connects to SWISv3 by default. NCM 7.0 does not include SWISv3, so this connection fails.

You can cause Connect-Swis to talk to SWISv2 by adding the "-v2" switch to the command line.

Level 7

fcaron,

Is there a new script for NPM 10.3.1 as the below blog link states *it is not yet available...

Better still a user-friendly feature upgrade option to simplify the process would be great!


http://thwack.solarwinds.com/community/solarwinds-community/product-blog/blog/2012/03/06/how-to-auto...
"This blog contains and describes a working PowerShell script example, that exports nodes from SolarWinds NPM network monitoring software and imports them into NCM, but it can be easily modified to accommodate other combinations, including non-SolarWinds sources (CMDB)

The required configuration to run this script is:

*NPM 10.2.x (this script does not work for 10.3.x, we are working on an update)
•NCM 7.0.2 (does not work with versions prior to 7.0.2)
•SDK 1.3 or 1.4 or 1.5
•Windows PowerShell (ISE is used in this illustration)"

Level 15

The script for 10.3.x was actually there. I just reworded the paragraph about versions for more clarity.

Level 7

Thanks fcaron

I think advance feature would be better though!

Thanks again.

Level 14

Has anyone successfully modified this script to use a non-SolarWinds source?  I'd love to be able to take an Excel spreadsheet with pre-populated data, including custom attributes, and import it directly into NPM.  Our team is going to take a crack at this, but if someone has already done it, I'd love to see your script.

Thanks,

Josh

Level 12

FYI, me and my team will try to modify this script to use it to take fields out of a remote MS Access DB or SQL Server DB and injest into NPM.  We will post on thwack with the outcome.

Level 8

I've been trying to use this script to migrate nodes from one NPM server to another, while the script does work to transpose the data between databases, it does not seem to pull the SNMP data across properly.

While the information is moved between the databases the new nodes do not poll properly. Any ideas on a solution for this?

Level 19

This script will copy the SNMPv1/v2 community string (and read/write community string if you have defined it separately), but it does not copy SNMPv3 credentials. This is because SNMPv3 credentials are not available through the API for security reasons.

Level 8

I can confirm that it is indeed pulling across my community string.

It's an odd issue, it pulls over the Nodes/Interfaces and Volumes but doesn't poll the SNMP data until I go through:

Edit Node -> Test -> Submit

Then it begins polling properly, unfortunately we're trying to move near 8000 objects so that's not really a workable long-term solution.

I've had another check through the database tables this morning;

The corresponding details are being copied into Orion.Nodes and Orion.Pollers so the information is present in the database, but NPM isn't displaying them, nor does it appear to be polling, until we take the steps outlined above.

Level 8

Update:

I've narrowed the problem down to Null or missing data being imported into the Orion.Nodes table, I've found a few threads elsewhere regarding this, such as;

http://thwack.solarwinds.com/thread/53037

I've adjusted the script to perform some basic data validation but it's not fool-proof, does anyone have a solution to this that's cleaner than me adding hundreds of lines of code to check for fields/values? I'm certain there has to be an easier way but after looking at nothing but this script for 24+ hours I'm starting to struggle for focus!

Edit: In the end I found the problem was NULL values being passed from the source to the target database, I added some validation checks to the script which replace the null's with "" and everything works fine now.

Level 8

While the script does work as intended, there are some nuances it wasn't prepared for, I have updated my local copy of the script to cover the issues I encountered but as they are probably unique to the environment I was working in I can't say much.

The script does fundamentally work, but the SNMP polling issue is quite a drastic one, however as no-one else has asked in the comments about it, I will assume as above, it's was unique to our architecture.

Level 11

Anyone know if this script works for NPM 10.4.1?  When I try to run it I get the follow error:

New-Object : Cannot find an overload for "SecureString" and the argument count: "1".

At C:\User\XXXXX\Desktop\NodeMove.ps1:5 char:14

+ $password1 = New-Object System.Security.SecureString  "PasswordRemoved" | Conve ...

+              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodException

    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommand











Level 8

I changed the connection code to;

#Connect to the source and server

$username1 = Get-Credential

$source = Connect-Swis -Credential $username1 -Hostname $hostname1

So all you need to do is define $hostname1 as an IP eg;

$hostname1 = xxx.xxx.xxx.xxx

or ask the host for it

$hostname1 = Read-Host -prompt "Please enter the host IP" etc etc etc

Level 11

Changed it per your rely, I still get the same error message.

Level 8

Could you post your code snippet?

Just the first few lines would be fine, will let me see it in context and try to figure out a solution for you.

Level 11

Sorry, found a solution on Google after I posted the response.  Solution was to change the password line to:

$password1 =  ConvertTo-SecureString "XXXXXXXXXX" -asPlainText -Force

Now it returns a different error at a different location, new error below:

Copying airXXXXs-Rbr.XXXXX.lcl ( XXX.XXX.XXX.XXX )

New-SwisObject : The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter
http://schemas.solarwinds.com/2007/08/informationservice:properties. The InnerException message was 'There was an error deserializing the object of type
SolarWinds.InformationService.Addons.PropertyBag. String must be exactly one character long.'.  Please see InnerException for more details.
At C:\Users\jarrieh\Desktop\NodeMove0.ps1:95 char:15
+     $newUri = New-SwisObject $target -EntityType "Orion.Nodes" -Properties $targ ...
+               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [New-SwisObject], FaultException`1
    + FullyQualifiedErrorId : SwisError,SwisPowerShell.NewSwisObject

Level 8

I had exactly the same problem and even tried using -AsPlainText -Force and still had no joy, what I ended up doing was rewriting the entire connection section of the script as follows.

Original(broken?):

# Set up the hostname, username, and password for the source system

$hostname1 = "10.199.15.78";

$password1 = New-Object System.Security.SecureString  #"" | ConvertTo-SecureString -asPlainText -Force

$username1 = "admin"

# Set up the hostname, username, and password for the target system

$hostname2 = "10.199.15.79";

$password2 = New-Object System.Security.SecureString  #"" | ConvertTo-SecureString -asPlainText -Force

$username2 = "admin"

New (Working):

$hostname1 = "10.199.15.78"

$hostname2 = "10.199.15.79"

#Connect to the source and server

$username1 = Get-Credential

$source = Connect-Swis -Credential $username1 -Hostname $hostname1

#Connect to the destination server

$username2 = Get-Credential

$target = Connect-Swis -Credential $username2 -Hostname $hostname2

Level 11

Thanx for your help.

The new command section returns an Invlaid Username or Password error.

The user validation portion works when I change it to:

# Set up the hostname, username, and password for the source system

$hostname1 = "XXX.XXX.XXX.XXX";

$password1 = ConvertTo-SecureString "XXXXX" -asPlainText -Force

$username1 = "admin"

# Set up the hostname, username, and password for the target system

$hostname2 = "XXX.XXX.XXX.XXX";

$password2 = ConvertTo-SecureString "XXXXX" -asPlainText -Force

$username2 = "admin"

But, it fails to copy, looks like it is trying to reference a URL that

doesn't exist... I get the same results with both SDK 1.5 and SDk 1.7

The error it gives is:

Copying XXXXXXXX (XX.23.13.2 )

New-SwisObject : The formatter threw an exception while trying to

deserialize the message: There was an error while trying to deserialize

parameter

http://schemas.solarwinds.com/2007/08/informationservice:properties. The

InnerException message was 'There was an error deserializing the object of

type

SolarWinds.InformationService.Addons.PropertyBag. String must be exactly

one character long.'.  Please see InnerException for more details.

At C:\Users\jarrieh\Desktop\NodeMove0.ps1:106 char:15

+    $newUri = New-SwisObject $target -EntityType "Orion.Nodes"

-Properties $targ ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:)  [New-SwisObject], FaultException`1

    + FullyQualifiedErrorId : SwisError,SwisPowerShell.NewSwisObject

On Tue, Jul 30, 2013 at 9:13 AM, unclescar <

Level 19

See if SDK 1.7 fixes this. We have a fix that relates to this area that might cover what you are seeing.

Level 19

Ah - I missed that you have already tried SDK 1.7. I'll continue investigating.

Level 11

I appreciate the assistance.  I tried again, with SDK 1.7 still getting the error.

New-SwisObject : The formatter threw an exception while trying to deserialize the message: There was an error while trying to

deserialize parameter http://schemas.solarwinds.com/2007/08/informationservice:properties. The InnerException message was 'There

was an error deserializing the object of type SolarWinds.InformationService.Addons.PropertyBag. String must be exactly one

character long.'.  Please see InnerException for more details.

At C:\Users\jarrieh\Desktop\NodeMove.ps1:87 char:15

+     $newUri = New-SwisObject $target -EntityType "Orion.Nodes" -Properties $targ ...

+               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:) [New-SwisObject], FaultException`1

    + FullyQualifiedErrorId : SwisError,SwisPowerShell.NewSwisObject



I rebuilt the connection section based on previous advice, so that it shows as UncleScar provided.  Still get the same error message.  Here is the code around the lines where it fails. Please let me know if you need the entire code...  it appears to be stopping at the New-SwisObject of the $newUri line of the script.


# Create the node on the target system

$newUri = New-SwisObject $target -EntityType "Orion.Nodes" -Properties $targetNodeProps

$newNode = Get-SwisObject $target $newUri


Level 19

Well, I haven't been able to reproduce this error with NPM 10.4.1 and SDK 1.7, so it may be something specific to your node data. Based on what the error is, it must be related to a single-character field. We don't have very many of those. In fact, the only one on the in the "nodePropsToCopy" list is "CMTS" which relates to cable modems and I don't think anyone even uses. Try removing that property from the list in the script and try again.

Level 11

Awesome, I deleted the CMTS and it copied the node, then fired the same error during the interface copy, but once I removed 64 bit counter from the interface copy, copied the interfaces as well.

It doesn't copy custom properties, but looking through the code I expect that is normal.

It fails at the adding the node to NCM, but I assume I can remark out that section and just manually activate them in NCM.

Invoke-SwisVerb : Verb Cirrus.Nodes.AddNodeToNCM: Cannot find assembly

At C:\Users\jarrieh\Desktop\NodeMove.ps1:178 char:9

+         Invoke-SwisVerb $target Cirrus.Nodes AddNodeToNCM $newNode.NodeID

+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:) [Invoke-SwisVerb], FaultException`1

    + FullyQualifiedErrorId : SwisError,SwisPowerShell.InvokeSwisVerb

Level 13

It's because the NCM is still running through SWISv2, but with newer SDK's the default connection is V3 and V2 must be used as parameter when creating connection.

Create new connection as

$targetV2 = Connect-Swis -Credential $credential2 -Hostname $hostname2 -V2

and invoke then the verb as

if ($targetHasNCM) {

        Invoke-SwisVerb $targetV2 Cirrus.Nodes AddNodeToNCM $newNode.NodeID

    }

Level 8

I'm trying to run this and I was having issues with the script saying incorrect username or password. So, I changed the connection settings to:

$hostname1 = "x.x.x.x";

$password1 = ConvertTo-SecureString "mypassword" -asPlainText -Force

$username1 = "username"

$hostname2 = "y.y.y.y";

$password2 = ConvertTo-SecureString "password" -asPlainText -Force

$username2 = "username2"

After I changed them, it got past the incorrect username part and is now throwing this error:

Get-SwisData : An error occurred when verifying security for the message.

At C:\CopyNodes\CopyNodes.ps1:55 char:6

+ if ((Get-SwisData $target "SELECT Field FROM Orion.CustomProperty WHERE Table='N ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:) [Get-SwisData], MessageSecurityException

    + FullyQualifiedErrorId : SwisError,SwisPowerShell.GetSwisData

Level 19

There are a variety of possible causes for this error. One common one is if the system clocks are not in sync between the client (where you are running the PowerShell script) and the server you are connecting to. Could this be your problem?

Level 11

Export nodes from your own CMDB and import them into NPM   -   I am not seeing a reference to this line item.


I have a cmdb I would like to pull newly add nodes out and get the in the  SAM database.


I am learning the SDK  ,  looking for advice on the process of  exacting a bunch of nodes out of a cmdb and getting them in the SAM database. 


can you share comments and examples please ?

Level 19

Have you looked at the samples in the SDK? It shows how to add nodes to Orion in several languages.

This is an extremely handy script, updated with SDK 1.10. Has anyone managed to modify the latest script to extract to file (.xls perhaps), and then from this file into another instance? This functionality would be invaluable for migration work where the source and destination instances are not logically connected.

Here's an interesting one:

I can get the script to run flawlessly, as long as I don't try to copy any custom properties. As soon as I add in CustomProp1 to the end of the list of properties (the section which starts at line 91 of the script) it errors.

Cannot Resolve Property CustomProp1

I've had a look at the Nodes table, and I can see the columns for the data I need to move, and all the other properties listed in the script are coming from the same table. Consternation! While I could just give up and export/import the required custom properties, it goes against the grain a little as I'm looking for a 'fire and forget' migration script.

Anyone have any ideas? I've prepared the way on the target system by adding in matching custom node properties, but it doesn't make a difference. Perhaps I'm asking for too much from a single script.

EDIT: I'm using version 1.10 of the SDK

Level 19

In SWIS, custom properties are not properties of the Orion.Nodes entity, even though they are stored as columns in the Nodes table in the database (though they are moved to a separate table in NPM 11.5). This is why you are getting that error when you try to copy CustomProp1 from one instance of Orion.Nodes to another.

Ah, I suspected as much. Thanks man. Saves me a shed load of time trying to troubleshoot the impossible

I guess it'll be a two step process after all? Script for the node info, export/import for the custom properties? No other way to do it?

Level 19

You can do it in one script, but it will need separate statements for reading and writing the custom properties. The line in the original script where it sets the "ImportedByAPI" custom property should get you pointed in the right direction.

Right, thanks again.

Level 11

Has anyone tried this with NPM 11.5.2? I am looking at standing up a new system and this would make life a lot easier to bring everything into it. Ditto for standing up a full test environment.

Level 11

Greetings marcrobinson -

did you achieve any success on this challenge

Level 11

Not yet.

Level 7

Hello everyone,

I am trying to migrate from nagios to solarwinds, right now I used WinSCP script to automate the nagios file transfer to the machine with Orion, with all the nodes information in them (ip, hostname...) they are .cfg files, does anyone have a script example that gets Ip adresses from file and gets them on orion to be added as a node. Any help is much appreciated. Hope to hear soon from you guys.

Level 12

Does this still work for moving from NPM 12.1 to NPM 12.2?

Level 19

This is an older post with quite a few comments describing variations. Could you outline what you are trying to do in more detail?

Level 12

Could you please post the changes you made?  Thanks much!

About the Author
Francois has joined the SW product management team in Dec 2010. He has been in the network management space for about 15 years, first in a startup company, then in one of the big 4 and back to a human-size company. Despite his bizarre accent, he is a decent guy to talk to.