Solved: Re: Upgrade from NMP 11.0.1 to 11.5 breaks SDK 1.1...

centirc-jim · ‎03-11-2015

Hello all,

I want to thank those that released the SDK. It's a great tool and we've used it extensively.

We have a powershell script, using v1.10.16.0 of the SDK, which we use to add nodes, setup pollers, and set the nodes Custom Properties. After upgrading Orion Network Performance Monitor from 11.0.1 to 11.5 values for Custom Properties no longer get set. It's as if the initial node creation seems to no longer create the ground work or index for custom properties to be added/edited. When we run the script there's no errors generated.

After a node is added via the PS script, if you go in through the WebGUI, and edit the custom properties of the node, save it and go back, the values for those custom properties have reverted back to the origional values. We are not seeing any issues if a node is manually added via the GUI. I have a feeling some of the tables have been split and it's almost like the '

Has anyone else run into this issue?

# Basic add node, add n-pollers, set some custom props (p.s. what circle of hell did I enter trying to copy and paste into this form's editor?)

#PSSnapin presence check/add

if (!(Get-PSSnapin -Name "SwisSnapin" -ErrorAction SilentlyContinue))

{ Add-PSSnapin SwisSnapin -ErrorAction SilentlyContinue }

# Default values

[string]$AlertingTier = "SysOps Linux - P1 Very Urgent"

[string]$Access_KCNDataCenter = "No"

[String]$KCNHardwareType = "None"

[string]$ResponsibleGroup = "Server Operations (Linux)"

[string]$Community="public"

[string]$SiteAddressState="MO"

ip2guid($ipString) {

$ip = [System.Net.IPAddress]::Parse($ipString)

$src = $ip.GetAddressBytes();

$data = new-object byte[] 16

$src.CopyTo($data, $data.Length - $src.Length)

$dest = new-object byte[] 16

[Array]::Copy($data, 12, $dest, 0, 4)

[Array]::Copy($data, 10, $dest, 4, 2)

[Array]::Copy($data, 8, $dest, 6, 2)

[Array]::Copy($data, 6, $dest, 8, 2)

[Array]::Copy($data, 0, $dest, 10, 6)

return (New-Object Guid (,$dest)).ToString()

}

AddPoller($PollerType) {

$poller["PollerType"]=$PollerType;

$pollerUri = New-SwisObject $swis -EntityType "Orion.Pollers" -Properties $poller

}

# Connect to SolarWinds

$SWhost = "cgslpwiswnpm01.centric.com"

$username = "user"

$password = Read-Host -Prompt "Enter password" -AsSecureString

$cred = New-Object -typename System.Management.Automation.PSCredential -argumentlist $username, $password

$swis = Connect-Swis -host $SWhost -cred $cred

# For simplicity just putting in a host and IP

# In the script we have steps to verify fqdn and ip

$hostname = 'somehost'

$ip = '10.0.0.1'

$ipGuid = ip2guid($ip)

# add a node

$newNodeProps = @{

EntityType="Orion.Nodes";

IPAddress=$ip;

IPAddressGUID=$ipGuid;

Caption=$hostname;

DynamicIP=$False;

Engine=1;

Status=1;

UnManaged=$False;

Allow64BitCounters=$False;

SysObjectID="";

MachineType="";

VendorIcon="";

ObjectSubType="SNMP";

SNMPVersion=2;

Community=$Community;

}

$newNodeUri = New-SwisObject $swis -EntityType "Orion.Nodes" -Properties $newNodeProps

$nodeProps = Get-SwisObject $swis -Uri $newNodeUri

# register specific pollers for the node

$poller = @{

NetObject="N:"+$nodeProps["NodeID"];

NetObjectType="N";

NetObjectID=$nodeProps["NodeID"];

}

### Create N-type Pollers

AddPoller("N.Status.SNMP.Native");

AddPoller("N.ResponseTime.ICMP.Native");

AddPoller("N.Details.SNMP.Generic");

AddPoller("N.Uptime.SNMP.Generic");

AddPoller("N.Cpu.SNMP.NetSnmpSystemStats");

AddPoller("N.Cpu.SNMP.NetSnmpCpuIdle");

AddPoller("N.Memory.SNMP.NetSnmpReal");

AddPoller("N.IPAddress.ICMP.Generic");

## Prepare a custom property value

$customProps = @{

AlertingTier=$AlertingTier;

ResponsibleGroup=$ResponsibleGroup;

KCNHardwareType=$KCNHardwareType;

SiteAddressState=$SiteAddressState;

Comments="Host added by script";

## Build the Custom Properties node URI

$SWNodeURICP = "$newNodeUri/CustomProperties"

## Set Custom Properties

Set-SwisObject $swis -Uri $SWNodeURICP -Properties $customProps

tdanner · ‎03-13-2015

This may just be a side effect of stripping down the script for posting here, but I noticed two problems with how you are creating the pollers for the volume:

1. You are taking the VolumeID value from the $newVolProps object, which doesn't have that property at all. You can get the new VolumeID from $VolProps.

2. You are setting up a property bag for the new poller in the $poller object, but you never actually call New-SwisObject on it.

Please check and see if either of these two issues existing in the original, larger script.

View solution in original post

centirc-jim · ‎03-13-2015

I want to thank Jan Pelousek and TDanner for there help, it was greatly apprecited and relevent. The timing for us adding hosts via the SDK and upgrading from NMP 11.0.1 to 11.5 was just bad timing. We are still having a similar issue with enabling Volume Polling as we were having with Custom Properties and hope someone familar with the new schemas might have a quick fix which is as easy as what Jan provided.

The issue is this, after adding a node (via SDK and ps script), volume properties and setting up the pollers for the volume(s) the volumes do not get polled and you can not enable polling via script or the web interface.

I'm hoping I'm just doing something 'obviously' stupid and one of the great community member will see what I'm doing wrong. Anyway, I've tried many permutations of Volume Properies to get volume polling to be enabled, but as was with Custom Properties, it seems like as if the initial volume creation seems to no longer create the ground work for volume polling to be properly initialzed. We do not see this issue if we add the host (and volumes via the web gui).

Here's the process we're following:

1. Add a node via PS script including adding the volume(s) properites and setting up pollers.

2. The volume(s) get added w/out any errors/warnings, pollers get added, again w/out errors or warning.

3. Query Orion.Volumes by NodeID (the new node that was created) in SWQL studio in. The volume(s) exist in the table and everything looks ok. -- (I've even queried two nodes, one 'good' node and the 'new' node just to compare what's in there to see if we're missing something).

4. Open up the new node in the WebGUI, select 'List Resources', check the box next to the volume(s), submit. Go back to the Node Details page, select 'List Resources' and the checkbox is again un-checked.

Here's a 'stripped down' version of how we're adding volumes. I'm 'manually' setting many of the values I would have pulled from the .csv and I've taken out the part of adding the node (the script from the origional post covers that).

# $volumes = Import-Csv e:\temp\x99ltestsql1.csv
[string]$volumes = "/"
Foreach ($volume in $volumes) {

        $newVolProps = @{
            NodeID=[INT]$NewNodeID;
            VolumeType="Fixed Disk";
            VolumeTypeID="4";
            VolumeSize="0";
            Icon="FixedDisk.gif";
            Index="3";
            Caption=$volume;
            VolumeDescription=$volume;
            # PollInterval="120";
            StatCollection="10";
            # StatusIcon="Up.gif";
            FullName="$hostname -$volume"
            Responding="Y";
            # VolumePercentUsed="";
            # VolumeAllocationFailuresThisHour="";
            # VolumeSpaceUsed="";
            # VolumeAllocationFailuresToday="";
            # VolumeSpaceAvailable="";
            # DiskQueueLength="";
            # DiskTransfer="";
            # DiskReads="";
            # DiskWrites="";
            # TotalDiskIOPS="";
            Status="1";
            # VolumeType=$VolumeType;
         #   VolumeTypeID=[INT]$volumeTypeID;
         #   Icon=$Icon;
         #   Caption=$Caption;
         #   VolumeSpaceAvailable=0;
             RediscoveryInterval=10;
             NextRediscovery=[DateTime]::UtcNow;
        }

        Write-Host "NodeID: $NewNodeID, volume: $volume, volume Index: $Index, volumeType: $volumeType, VolumeTypeID: $volumeTypeID, Icon: $Icon";
        $newVolUri = New-SwisObject $swis -EntityType "Orion.Volumes" -Properties $newVolProps
        $VolProps = Get-SwisObject $swis -Uri $newVolUri

        foreach ($pollerType in @("V.Details.SNMP.Generic","V.Status.SNMP.Generic","V.Statistics.SNMP.Generic"))
        {
            $poller = @{
                PollerType=$pollerType;
                NetObject = "V:"+$newVolProps["VolumeID"];
                NetObjectType="V";
                NetObjectID=$newVolProps["VolumeID"];
            }
        }

        Write-Host "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++";
        Write-Host "newVolUri: $newVolUri";
        Write-Host "newVolProps: $VolProps";
        Write-Host "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++";
    }

}

Here's what's spit out from Write-Host:

NodeID: 4150, volume: /, volume Index: 3, volumeType: , VolumeTypeID: , Icon:
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
newVolUri: swis://localhost/Orion/Orion.Nodes/NodeID=4150/Volumes/VolumeID=6196
newVolProps: NodeID: 4150
Status: 1
StatusLED:
VolumeID: 6196
Icon: FixedDisk.gif
Index: 3
Caption: /
PollInterval: 420
StatCollection: 10
RediscoveryInterval: 10
StatusIcon:
Type: Fixed Disk
Size: 0
Responding: Y
FullName: x99ltestsql1.xxxxx.com -/
LastSync:
VolumePercentUsed: 0
VolumeAllocationFailuresThisHour: 0
VolumeIndex: 3
VolumeTypeID: 0
VolumeType: Fixed Disk
VolumeDescription: /
VolumeSize: 0
VolumeSpaceUsed: 0
VolumeAllocationFailuresToday: 0
VolumeResponding: Y
VolumeSpaceAvailable: -2
VolumeTypeIcon: FixedDisk.gif
OrionIdPrefix: V:
OrionIdColumn: VolumeID
DiskQueueLength:
DiskTransfer:
DiskReads:
DiskWrites:
DisplayName: /
TotalDiskIOPS:
VolumePercentAvailable: 100
MinutesSinceLastSync:
DetailsUrl: /Orion/NetPerfMon/VolumeDetails.aspx?NetObject=V:6196
SkippedPollingCycles:
VolumeSpaceAvailableExp: 0
NextPoll: 3/13/2015 1:43:45 PM
NextRediscovery: 3/13/2015 1:36:45 PM
DeviceId:
DiskSerialNumber:
InterfaceType:
SCSITargetId:
SCSILunId:
SCSIPortId:
SCSIControllerId:
SCSIPortOffset:
StatusDescription: Up
UnManaged: False
UnManageFrom:
UnManageUntil:
Image:
AncestorDisplayNames: System.String[]
AncestorDetailsUrls: System.String[]
StatusIconHint:
Description: Fixed Disk
InstanceType: Orion.Volumes
Uri: swis://localhost/Orion/Orion.Nodes/NodeID=4150/Volumes/VolumeID=6196
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Again I hope it's just me being an idiot and missing something obvious.

Thank you all again,

Jim

tdanner · ‎03-13-2015

This may just be a side effect of stripping down the script for posting here, but I noticed two problems with how you are creating the pollers for the volume:

1. You are taking the VolumeID value from the $newVolProps object, which doesn't have that property at all. You can get the new VolumeID from $VolProps.

2. You are setting up a property bag for the new poller in the $poller object, but you never actually call New-SwisObject on it.

Please check and see if either of these two issues existing in the original, larger script.

View solution in original post

centirc-jim · ‎03-15-2015

TDanner,

Thank you for your response, again it is _greatly_ appreciated. I can't say I understand why making the change you suggested in #1 worked, but it did (w00t). I don't get it, because that's how I add 'N' and 'I' type pollers and that works. But be that as it may here's the chanes I made to the script for volume pollers.

Here's how setup the 'I' and 'N' type pollers:

$iPoller = @{
    NetObject="I:"+$ifaceProps["InterfaceID"];
    NetObjectType="I";
    NetObjectID=$ifaceProps["InterfaceID"];
}

$poller = @{
    NetObject="N:"+$nodeProps["NodeID"];
    NetObjectType="N";
    NetObjectID=$nodeProps["NodeID"];
}

## This works for adding volumes (vol props not shown here) :

# Add volume

$newVolUri = New-SwisObject $swis -EntityType "Orion.Volumes" -Properties $newVolProps

$VolProps = Get-SwisObject $swis -Uri $newVolUri

foreach ($pollerType in @("V.Details.SNMP.Generic","V.Status.SNMP.Generic","V.Statistics.SNMP.Generic"))
{

        # user $VolProps.VolumeID instead of $newVolProps
        $vpoller = @{
              PollerType=$pollerType;
              NetObject = "V:"+$VolProps.VolumeID
              NetObjectType="V";
              NetObjectID=$VolProps.VolumeID;
        }
       $vPollerUri = New-SwisObject $swis -EntityType "Orion.Pollers" -Properties $vPoller
       write-host vpoller info--- vpoller.PollerType: $vpoller.PollerType
       write-host NetObject: $vpoller.NetObject
       write-host NetObjectType: $vpoller.NetObjectType
       write-host NetObjectID: $vpoller.NetObjectID
       write-host $vPollerUri
       write-host +++++++++++++++++++++++++++++++++
    }

# Here's the Output

vpoller info--- vpoller.PollerType: V.Details.SNMP.Generic
NetObject: V:6209
NetObjectType: V
NetObjectID: 6209
swis://localhost/Orion/Orion.Pollers/PollerID=75803
+++++++++++++++++++++++++++++++++
vpoller info--- vpoller.PollerType: V.Status.SNMP.Generic
NetObject: V:6209
NetObjectType: V
NetObjectID: 6209
swis://localhost/Orion/Orion.Pollers/PollerID=75804
+++++++++++++++++++++++++++++++++
vpoller info--- vpoller.PollerType: V.Statistics.SNMP.Generic
NetObject: V:6209
NetObjectType: V
NetObjectID: 6209
swis://localhost/Orion/Orion.Pollers/PollerID=75805
+++++++++++++++++++++++++++++++++

When I clean up the script and add the part where I poll each host for volumes I'll post a complete example Linux folks can use to add hosts as well as N, I and V type pollers.

AGAIN THANK YOU!

Jim

Diner · ‎03-12-2015

Hello, unfortunately you really hit the bug, related to the splitting of the Nodes table and we really apologize for that.

Problem is, that the CRUD action doesn't create the record in the NodesCustomProperties fragment. However here is the workaround how to fix it (until we fix that in the product). Put following lines before the line, where you set the Custom properties values:

[System.String]$sqlCommand=("Insert into NodesCustomProperties (NodeID) VALUES (" + $nodeProps.nodeid + ")")
Invoke-SwisVerb $swis "Orion.Reporting" "ExecuteSQL" $sqlCommand |Out-Null

This should create the correct record, which allows you then to set the Custom Property values.

Btw. here are few things I found during execution of your script. This may potentialy cause troubles in future (and some really made troubles to me, maybe because of different PowerShell version), so it may worth to correct that to avoid those troubles:

1. Function declarations don't start with keyword "function", (for ip2guid and AddPoller functions)

2. Missing } bracket after " Comments="Host added by script";"

3. Node propertybag $newNodeProps contains "Engine" instead of the EngineID

I hope it helps.

Regards,

Honza

rcolanero · ‎03-24-2015

Working on creating new nodes using the API from Ubuntu I was running into the same issue with not being able to set CustomProperties. Running the SQL Insert statement before setting CustomProperty values fixed my issue as well.

bjonescg · ‎03-12-2015

Jan,

Thank you for this workaround. I work with Centric-jim and the work around you provided worked for the WMI windows boxes I am monitoring (centric-jim is still testing in the Linux world), but now I have an issue with adding volumes that previously was not there. Should this code still work or did something else change in the database to prevent this code from executing properly? After execution, I am not seeing any new rows in the Orion.Volumes table through the swql editor. Thank you for any help you can offer.

#we should be passing $poller to make the function prettier, but it still works

function AddPoller($PollerType)

{

$poller["PollerType"]=$PollerType;

$pollerUri = New-SwisObject $swis -EntityType "Orion.Pollers" -Properties $poller

}

$volumes = "C:\"

#this is ran inside a larger loop where $SWnodeID has already been found.

foreach ($vol in $volumes)

{

#This checks for the existence of the C: and M: Drives assigned to the solarwinds node

$SWNodeVolCheck = Get-SwisData -SwisConnection $swis -Query "SELECT NodeID, Caption FROM Orion.Volumes WHERE Caption = '$($vol)' AND NodeID = '$($SWNodeID)'"

If ($SWNodeVolCheck -eq $null)

{

#Volume was not found, create new volume for current node

#$VolAction = "Add $Vol"

$newVolumeProps = @{

NodeID=$SWNodeID;

VolumeType="Fixed Disk";

VolumeSize="0";

Icon="FixedDisk.gif";

Index="3";

Caption=$vol;

VolumeDescription=$vol;

PollInterval="120";

StatCollection="10";

RediscoveryInterval="20";

StatusIcon="Up.gif";

FullName=$vol;

Responding="";

VolumePercentUsed="";

VolumeAllocationFailuresThisHour="";

VolumeSpaceUsed="";

VolumeAllocationFailuresToday="";

VolumeSpaceAvailable="";

DiskQueueLength="";

DiskTransfer="";

DiskReads="";

DiskWrites="";

TotalDiskIOPS="";

}

$newVolumeURI = New-SwisObject $swis –EntityType "Orion.Volumes" –Properties $newVolumeProps

$VolProps = Get-SwisObject $swis -Uri $newVolumeURI

foreach ($pollerType in @('V.Status.WMI.Windows','V.Details.WMI.Windows','V.Statistics.WMI.Windows'))

{

$poller = @{

PollerType=$pollerType;

NetObject="V:"+$VolProps["VolumeID"];

NetObjectType="V";

NetObjectID=$VolProps["VolumeID"];

}

AddPoller $pollerType;

}

tdanner · ‎03-12-2015

That should still work. Are you getting any errors from these commands? It would be very strange for "New-SwisObject Orion.Volumes" to "succeed" but not have actually created the record.

bjonescg · ‎03-12-2015

ERROR: New-SwisObject : Unable to create Volume. Details : Information : Assigned default value '3/12/2015 1:49:41 PM' for NextPoll.

ERROR: Information : Assigned default value '3/12/2015 2:07:41 PM' for NextRediscovery.

ERROR: Error : In property VolumeSpaceAvailable. Reason : Unable to convert '' to System.Double.

ERROR: Information : Assigned default value '0' for Status or StatusDescription.

ERROR: Information : Assigned default value '' for DeviceId.

ERROR:

ERROR: ----------------------------------

updatetest.ps1 (418): ERROR: At Line: 418 char: 41

ERROR: + $newVolumeURI = New-SwisObject $swis –EntityType "Orion. ...

ERROR: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ERROR: + CategoryInfo : InvalidOperation: (:) [New-SwisObject], FaultException`1

ERROR: + FullyQualifiedErrorId : SwisError,SwisPowerShell.NewSwisObject

It helps if you read the full error code . It looks like you can no longer use empty string for VolumeSpaceAvailable. so I commented it out and the information was passed into the database. If this is not an appropriate method, please let me know what I should pass. I currently see the drive in the vital stats page. thank you for your help.

tdanner · ‎03-12-2015

That's a good fix. This property validation is new in the 2015.1 version of the platform. (Thanks, madhavan!)

You can now leave out all of those default values in your script (where you used to have to provide empty string for a lot of properties) and the property validation will fill them in.

c.gura · ‎03-12-2015

I did not realize we could execute raw SQL from SWQL. THANKS A MILLION for revealing this functionality. Nowhere in the documentation does it tell us about this.

This helps me tremendously and I don't have to install a Database drivers now everywhere I want to access custom tables we have created.

contactjt · ‎03-12-2015

procedure stored.

tdanner · ‎03-12-2015

I'm glad you find it helpful. But I do need to caution you that this does not mean we can support whatever-you-use-this-for in general. It's undocumented for a reason - our database schema may change from version to version. I recommend you plan on removing this workaround once we publish a proper fix for this custom property bug. I'll post in this forum when that happens.

tdanner · ‎04-24-2015

NPM 11.5.1 has been released. It addresses this bug. Once you have upgraded to that release, you can and should remove the "Insert into NodesCustomProperties" workaround.

Update May-11: the fix has not been released yet.

RichardLetts · ‎05-08-2015

Does not seem to have made it into 11.5.1 -- we're experiencing the same issue with that version.

tdanner · ‎05-11-2015

You are absolutely right. I thought this fix was supposed to be in 11.5.1, but it is not. It is in 11.5.2, which is not released yet. Sorry for the confusion.

RichardLetts · ‎05-13-2015

this perl code:

my $sqlCommand="Insert into NodesCustomProperties (NodeID) VALUES ($node)";

my $NCPrv= $xs->XMLin($swis->Invoke ('Orion.Reporting','ExecuteSQL',[$sqlCommand]));

is not working; the error returned is:

2015-05-13 16:40:03,209 [54] ERROR SolarWinds.InformationService.Core.InformationService - (null)

Exception caught in method Invoke

SolarWinds.Data.AccessDeniedException: Access to Orion.Reporting.ExecuteSQL verb denied.

at SolarWinds.InformationService.Core.InformationService.InvokeInternal[T](String entity, String verb, Action`1 setupParameters, Func`2 extractReturnValue)

at SolarWinds.InformationService.Core.InformationService.Invoke(String entity, String verb, XmlElement[] parameters)

'

'InformationServiceFaultContract' => {

'xmlns:i' => 'http://www.w3.org/2001/XMLSchema-instance',

'xmlns' => 'http://schemas.solarwinds.com/2007/08/informationservice',

'ErrorCode' => '20',

'FullException' => 'SolarWinds.Data.AccessDeniedException: Access to Orion.Reporting.ExecuteSQL verb denied.

at SolarWinds.InformationService.Core.InformationService.InvokeInternal[T](String entity, String verb, Action`1 setupParameters, Func`2 extractReturnValue)

at SolarWinds.InformationService.Core.InformationService.Invoke(String entity, String verb, XmlElement[] parameters)',

'Message' => 'Access to Orion.Reporting.ExecuteSQL verb denied.',

'ExceptionType' => 'SolarWinds.Data.AccessDeniedException',

'UserMessage' => 'Access to Orion.Reporting.ExecuteSQL verb denied.'

}

does the API user account need more access rights to this? or would to recommend we add a database trigger instead?

tdanner · ‎05-13-2015

Orion.Reporting.ExecuteSQL requires admin rights. Otherwise it could be used to bypass account limitations.

RichardLetts · ‎05-14-2015

thank you;

to fix the nodes I executed the following in SQLStudio:

insert into [dbo].[NodesCustomProperties] (Nodeid)

SELECT nodeid FROM [dbo].[NodesData]

where nodeid not in (select nodeid from nodesCustomProperties )

It should be noted that this cascades into the NCM product as well, since it doesn't seem to be able to display nodes that do not have rows in NodesCustomProperties in the 'Manage NCM Nodes' widget and node pickers for NCM actions.

/RjL

c.gura · ‎03-12-2015

@tdanner I realize this, and I actually take the upgrade scenarios into my design. That is why I create custom Views and create my SQL based off of the views rather than the tables. Then if the schema changes all I have to do is update my View.

How do I do the call using REST I have the Invoke portions? I just need to know if I need to know which values I can make fixed and which to be possibly variable.

What should the payload look like:

my $payload = {

sqlQueryText => $sql,

sqlQueryParameters => ???????,

outputRowsMaxCount => ??????,

schemaOnly => ??????,

};

my $entityName = "Orion.Reporting";

my $verbName = "ExecuteSQL";

$self->invoke($entityName, $verbName, $payload) = @_;

tdanner · ‎03-12-2015

The only required parameter is the sql query, as a string. You can omit the rest.

Upgrade from NMP 11.0.1 to 11.5 breaks SDK 1.10 set CustomProperties