17 Replies Latest reply on May 9, 2019 10:29 AM by aLTeReGo

    Someone please set me straight

    cmblackburn

      So I have several places to set threshold levels in solarwinds for warning and critical for CPU and Memory, but I'm unable to get a visual dashboard of these thresholds out of the box?  I know if I hover over the node it will show me if the memory or CPU has hit the thresholds.

       

      Here is the task, create a dashboard and have it display a visual indicator based upon thresholds I set on each node whether my node is having issues:

       

      Green:  All good

      Yellow: I've reached a warning threshold

      Red:  I've reached a critical threshold 

       

      How could such a simple feature be non-existent, am I missing something?

        • Re: Someone please set me straight
          mesverrum

          For whatever reason the logic the devs used was that the node icon only reflects it's ICMP up/down status.  In cases where I need to change that I've had to make custom reports in SWQL/SQL with case logic that I use to pick my own icon to reflect particular conditions.  See this below example (Paste it into a "Custom Query" resource)

           

           

          select distinct

          n.caption as [Server Name]

          ,'/Orion/images/StatusIcons/Small-' + n.StatusIcon AS [_IconFor_Server Name]

          , n.detailsurl as [_linkfor_Server Name]

          ,case when n.cpuload < 0 then 'Not Polled'

          when n.host.nodeid is not null and n.host.cpucorecount is not null then concat(round(n.host.cpuload,0),'% of ',n.host.CpuCoreCount,' CPU')

          when cpu.[cpu count] is not null then concat(cpuload,'% of ',cpu.[cpu count],' CPU')

          else 'Polling Error'

          end as [CPU Load]

          ,'/Orion/NetPerfMon/CustomChart.aspx?chartName=HostAvgCPULoad&SampleSize=10M&Period=LAST%2024%20HOURS&NetObject=N:'+tostring(n.nodeid) as [_linkfor_CPU Load]

          ,CASE

          WHEN n.cpuload >= n.CpuLoadThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Critical.gif'

          WHEN n.cpuload >= n.CpuLoadThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Warning.gif'

          WHEN peak.cpu >= n.CpuLoadThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Up-Down.gif'

          WHEN peak.cpu >= n.CpuLoadThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up-Warning.gif'

          WHEN n.cpuload < n.CpuLoadThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up.gif'

          END AS [_IconFor_CPU Load]

          ,case when n.percentmemoryused < 0 then 'Not Polled'

          else concat(n.percentmemoryused,'% of ',(round(n.totalmemory/1073741824,0)),' GB')

          end as [Memory Used]

          ,'/Orion/NetPerfMon/CustomChart.aspx?chartName=HostAvgPercentMemoryUsed&SampleSize=10M&Period=LAST%2024%20HOURS&NetObject=N:'+tostring(n.nodeid) as [_linkfor_Memory Used]

          ,CASE

          WHEN n.percentmemoryused >= n.percentmemoryusedThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Critical.gif'

          WHEN n.percentmemoryused >= n.percentmemoryusedThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Warning.gif'

          WHEN peak.mem >= n.percentmemoryusedThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Up-Down.gif'

          WHEN peak.mem >= n.percentmemoryusedThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up-Warning.gif'

          WHEN n.percentmemoryused <  n.percentmemoryusedThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up.gif'

          END AS [_IconFor_Memory Used]

          ,case

          when vol.high is null then 'None Monitored'

          when voldown.down is not null then ((tostring(voldown.down)+' Down of ')+(tostring(vol.[# of Volumes])))

          else (tostring(vol.[# of Volumes])+' Total')

          end as [Volumes Status]

          ,'/Orion/DetachResource.aspx?ResourceID=21218&NetObject=N:' + tostring(n.nodeid) as [_linkfor_Volumes Status]

          ,CASE WHEN voldown.down is not null then '/Orion/images/StatusIcons/Small-Down.gif'

          ELSE '/Orion/images/StatusIcons/Small-Up.gif'

          end as [_iconfor_Volumes Status]

          ,case

          when vol.high is null then 'N/A'

          else (tostring(vol.high) + ' %')

          end as [Fullest Disk]

          ,CASE

          WHEN vol.[High] >= 95 then '/Orion/images/StatusIcons/Small-Critical.gif'

          WHEN vol.[High] >= 80 THEN '/Orion/images/StatusIcons/Small-Warning.gif'

          WHEN vol.[High] >=  0 THEN '/Orion/images/StatusIcons/Small-Up.gif'

          END AS [_IconFor_Fullest Disk]

          ,'/Orion/DetachResource.aspx?ResourceID=21218&NetObject=N:' + tostring(n.nodeid) as [_linkfor_Fullest Disk]

           

          FROM orion.nodes n

          left join (SELECT count(NodeID) as [CPU Count], nodeid

          FROM Orion.CPUMultiLoadCurrent

          group by nodeid) cpu on cpu.nodeid=n.nodeid

          left join (SELECT nodeid, count(volumeid) as [# of Volumes], round(max(volumepercentused),0) as [High] FROM Orion.Volumes

          group by nodeid) vol on vol.nodeid=n.nodeid

          left join (SELECT nodeid, count(volumeid) as [down]

          FROM Orion.Volumes where status=2

          group by nodeid) voldown on voldown.nodeid=n.nodeid

          left join (SELECT NodeID, max(AvgLoad) as [CPU], max(AvgPercentMemoryUsed) as [Mem]

          FROM Orion.CPULoad

          where hourdiff(datetime,getutcdate())<12

          group by nodeid) Peak on peak.nodeid=n.NodeID

           

          --where (n.CustomProperties.Applications like '%${SEARCH_STRING}%' or cont.name like '%${SEARCH_STRING}%' or n.CustomProperties.ApplicationsRole like '%${SEARCH_STRING}%' or n.caption like '%${SEARCH_STRING}%')

           

          order by n.caption

           

          -Marc Netterfield

              Loop1 Systems: SolarWinds Training and Professional Services

          2 of 2 people found this helpful
            • Re: Someone please set me straight
              cmblackburn

              Thanks that's excellent!!!!  Not sure why this isn't built into the product though?!?!

               

              Do you know how I could limit the list to only devices that are over there threshold(s)?

               

              I would also like to use the Nodes Grouping to be able to group these by a couple custom properties, any ideas?

                • Re: Someone please set me straight
                  mesverrum

                  Amusingly enough in the copy of this report that I keep on my system it is grouped by solarwinds groups and application names as a custom property, i removed that all because i thought it might not apply to your situation.  Here is what it looks like when I set it up that way:

                   

                  In this scenario My groups are set up to have the same name as the Applications custom property, and there is a particular application monitor that I wanted to include the status of called "Automatic Services"

                   

                   

                   

                  select distinct

                  cont.name as [Groups]

                  ,cont.detailsurl as [_linkfor_Groups]

                  ,n.CustomProperties.Applications

                  ,n.CustomProperties.ApplicationsRole

                  ,services.name as [Services]

                  ,services.detailsurl as [_linkfor_Services]

                  ,'/Orion/images/StatusIcons/Small-' + services.StatusDescription + '.gif' AS [_IconFor_Services]

                  ,n.caption as [Server Name]

                  ,'/Orion/images/StatusIcons/Small-' + n.StatusIcon AS [_IconFor_Server Name]

                  , n.detailsurl as [_linkfor_Server Name]

                  ,case when n.cpuload < 0 then 'Not Polled'

                  when n.host.nodeid is not null and n.host.cpucorecount is not null then concat(round(n.host.cpuload,0),'% of ',n.host.CpuCoreCount,' CPU')

                  when cpu.[cpu count] is not null then concat(cpuload,'% of ',cpu.[cpu count],' CPU')

                  else 'Polling Error'

                  end as [CPU Load]

                  ,'/Orion/NetPerfMon/CustomChart.aspx?chartName=HostAvgCPULoad&SampleSize=10M&Period=LAST%2024%20HOURS&NetObject=N:'+tostring(n.nodeid) as [_linkfor_CPU Load]

                  ,CASE

                  WHEN n.cpuload >= n.CpuLoadThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Critical.gif'

                  WHEN n.cpuload >= n.CpuLoadThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Warning.gif'

                  WHEN peak.cpu >= n.CpuLoadThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Up-Down.gif'

                  WHEN peak.cpu >= n.CpuLoadThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up-Warning.gif'

                  WHEN n.cpuload < n.CpuLoadThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up.gif'

                  END AS [_IconFor_CPU Load]

                  ,case when n.percentmemoryused < 0 then 'Not Polled'

                  else concat(n.percentmemoryused,'% of ',(round(n.totalmemory/1073741824,0)),' GB')

                  end as [Memory Used]

                  ,'/Orion/NetPerfMon/CustomChart.aspx?chartName=HostAvgPercentMemoryUsed&SampleSize=10M&Period=LAST%2024%20HOURS&NetObject=N:'+tostring(n.nodeid) as [_linkfor_Memory Used]

                  ,CASE

                  WHEN n.percentmemoryused >= n.percentmemoryusedThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Critical.gif'

                  WHEN n.percentmemoryused >= n.percentmemoryusedThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Warning.gif'

                  WHEN peak.mem >= n.percentmemoryusedThreshold.Level2Value THEN '/Orion/images/StatusIcons/Small-Up-Down.gif'

                  WHEN peak.mem >= n.percentmemoryusedThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up-Warning.gif'

                  WHEN n.percentmemoryused <  n.percentmemoryusedThreshold.Level1Value THEN '/Orion/images/StatusIcons/Small-Up.gif'

                  END AS [_IconFor_Memory Used]

                  ,case

                  when vol.high is null then 'None Monitored'

                  when voldown.down is not null then ((tostring(voldown.down)+' Down of ')+(tostring(vol.[# of Volumes])))

                  else (tostring(vol.[# of Volumes])+' Total')

                  end as [Volumes Status]

                  ,'/Orion/DetachResource.aspx?ResourceID=21218&NetObject=N:' + tostring(n.nodeid) as [_linkfor_Volumes Status]

                  ,CASE WHEN voldown.down is not null then '/Orion/images/StatusIcons/Small-Down.gif'

                  ELSE '/Orion/images/StatusIcons/Small-Up.gif'

                  end as [_iconfor_Volumes Status]

                  ,case

                  when vol.high is null then 'N/A'

                  else (tostring(vol.high) + ' %')

                  end as [Fullest Disk]

                  ,CASE

                  WHEN vol.[High] >= 95 then '/Orion/images/StatusIcons/Small-Critical.gif'

                  WHEN vol.[High] >= 80 THEN '/Orion/images/StatusIcons/Small-Warning.gif'

                  WHEN vol.[High] >=  0 THEN '/Orion/images/StatusIcons/Small-Up.gif'

                  END AS [_IconFor_Fullest Disk]

                  ,'/Orion/DetachResource.aspx?ResourceID=21218&NetObject=N:' + tostring(n.nodeid) as [_linkfor_Fullest Disk]

                   

                   

                  FROM orion.nodes n

                  left join (SELECT Name,DetailsUrl FROM Orion.Container) cont on n.CustomProperties.Applications like ('%'+cont.name+'%')

                  left join (SELECT count(NodeID) as [CPU Count], nodeid

                  FROM Orion.CPUMultiLoadCurrent

                  group by nodeid) cpu on cpu.nodeid=n.nodeid

                  left join (SELECT nodeid, count(volumeid) as [# of Volumes], round(max(volumepercentused),0) as [High] FROM Orion.Volumes

                  group by nodeid) vol on vol.nodeid=n.nodeid

                  left join (SELECT nodeid, count(volumeid) as [down]

                  FROM Orion.Volumes where status=2

                  group by nodeid) voldown on voldown.nodeid=n.nodeid

                  left join (SELECT NodeID, max(AvgLoad) as [CPU], max(AvgPercentMemoryUsed) as [Mem]

                  FROM Orion.CPULoad

                  where hourdiff(datetime,getutcdate())<12

                  group by nodeid) Peak on peak.nodeid=n.NodeID

                  left join (select nodeid, name, statusdescription, detailsurl from orion.apm.Application where name = 'Automatic Services') services on services.nodeid=n.nodeid

                   

                   

                   

                   

                  --where (n.CustomProperties.Applications like '%${SEARCH_STRING}%' or cont.name like '%${SEARCH_STRING}%' or n.CustomProperties.ApplicationsRole like '%${SEARCH_STRING}%' or n.caption like '%${SEARCH_STRING}%')

                   

                   

                  order by cont.name desc,n.CustomProperties.Applications desc,n.CustomProperties.ApplicationsRole,n.caption

                   

                  -Marc Netterfield

                      Loop1 Systems: SolarWinds Training and Professional Services

                    • Re: Someone please set me straight
                      mesverrum

                      As far as filtering to only nodes that are over their threshold, you could do a where condition like

                       

                      where n.cpuload >= n.CpuLoadThreshold.Level1Value or n.percentmemoryused >= n.percentmemoryusedThreshold.Level1Value

                        • Re: Someone please set me straight
                          cmblackburn

                          Thanks mesverrum so much for your help with this, if you are EVER in or close to Kentucky I OWE YOU BIG TIME!!!!!!  I would've have never figured this out.  Could you provide any guidance for us that are new to solarwinds how to know the table names and shortcuts that you provided to these objects in your query?  Example I notice that you are referring to an object called n in your query but I don't notice a place where it was created.  I was in the process of creating a SQL query that was doing a bunch of INNER JOIN(s) to connect all the tables I needed but your SWQL seems so much straighter path to get to what I needed.

                           

                          Is there any way to get the functionality of grouping them together into expandable lists?  Like this:

                           

                           

                          Thanks Again!!!!

                            • Re: Someone please set me straight
                              mesverrum

                              To get that kind of interactivity you would most likely need to build an aspx resource and load that into your web console, or put something together in javascript. 

                               

                              To learn how the tables are laid out you will probably want to download and install the solarwinds SDK, GitHub - solarwinds/OrionSDK: SDK for the SolarWinds Orion platform, including tools, documentation, and samples in Powe…

                              For the most part SWQL is like a stripped down  read only version of SQL, a few syntaxes are tweaked but it shouldnt take long to figure out based on that documentation in the github.

                                • Re: Someone please set me straight
                                  cmblackburn

                                  Ok, so when you say an aspx resource do you mean a separate application or is there a way for me to add an aspx resource to a view?

                                   

                                  I’ve been trying to figure out a way to update some custom properties from an SOAP or API call when a page loads, like a page load event from .NET, in order to visualize status of a server on a netscaler dynamically.

                                    • Re: Someone please set me straight
                                      mesverrum

                                      I personally don't do the custom aspx stuff because I don't know how to write .net yet but if you dig around on thwack you can find examples of people who have written their own resources and loaded them into the appropriate directories of the inetpub folders to get them integrated within the Orion web console.  Then you would just add them the same as any other resources you currently use in the views.

                                       

                                      That page load stuff would be tricky because I Imagine you could kick off a fresh poll that is tied to a page load function but then you'd have to wait until the responses came back and refresh the view, typically takes a minute or two for new data to make it back to the web console.

                          • Re: Someone please set me straight
                            ntidwell

                            This is great information. I have been wondering about this for a long time.

                            • Re: Someone please set me straight
                              rschroeder

                              Just . . . wow.

                               

                              You exemplify how users bring value to Thwack by helping each other, sharing their expertise.

                               

                              Thank you for this.  And may your packets be swiftest of all!

                               

                              Rick Schroeder

                            • Re: Someone please set me straight
                              rschroeder

                              I'm missing some basic item here; when I copy & paste in the original SWQL, the resulting report pulls in all my nodes--switches, routers, servers, etc.  and that's not right.

                               

                              With the Basic Selector I selected an option to only include my Windows servers when building the report, but that option goes away when selecting the Advanced Selector that allows me to paste in the SWQL.

                               

                              Might you care to offer a suggestion how I can filter out all the non-server nodes?

                                • Re: Someone please set me straight
                                  John Handberg

                                  I was able to limit the scope to nodes with a specific Custom Property setting.  I have not figured out the filter per vendor yet, but it may be similar.

                                   

                                  Near the beginning of the query i added my custom property resource:

                                   

                                  select distinct

                                   

                                  n.caption as [Server Name]

                                  -- Added custom property to include from original code

                                  ,n.CustomProperties.SupportGroup

                                   

                                  At the end, I added this to the 'where' statement.

                                   

                                  -- Limiting resource scope to our custom property and only nodes where CPU load or memory are over threshold

                                  where n.cpuload >= n.CpuLoadThreshold.Level1Value or n.percentmemoryused >= n.percentmemoryusedThreshold.Level1Value and n.CustomProperties.SupportGroup like '%Windows%'

                                   

                                  order by n.caption

                                  • Re: Someone please set me straight
                                    mesverrum

                                    in swql you could do a where n.vendor='windows' or something like that as a simple filter.

                                  • Re: Someone please set me straight
                                    CourtesyIT

                                    Adding this one to the list.

                                     

                                    Congratulations!!!!

                                     

                                    Awesome stuff here.

                                     

                                     

                                    How to do various customizations with your Solarwinds

                                    • Re: Someone please set me straight
                                      aLTeReGo

                                      NPM 12.5 includes support node thresholds affecting the status of the node, allow you to alert on a node going into a warning or critical state. For more information, see the following post.

                                       

                                      Orion Platform 2019.2 - Enhanced Node Status

                                       

                                       

                                        • Re: Someone please set me straight
                                          borgan

                                          Somehow I missed your post RE: Enhanced Node status back in March. In short, the feature looks very good and should be a long-awaited improvement to how node status is treated in Orion. I could not comment in the referenced post, so please permit me to post a couple of comments/questions here:

                                           

                                          1. Did I miss the indicator for the actual node status in the new Enhanced pop-up? Since the icon now only changes the color of the mini icon at the lower right, I could not immediately tell if the node responded to it's status poll or not.

                                           

                                          2. The matrix that shows how "mixed" status will be determined seems s little confusing. It seems that "mixed" does not actually mean quite the same thing as in the past (which you did explain), but for instance, in the first row, the node status is Down, but the Child 1/2 status can be anything. Why is that not still an Final node status of "Warning"? In this case, it seems that actual node poll status is still more heavily weighted if the final node status is to be set to "Down".

                                           

                                          3. Again, in the fourth row of the Mixed matrix, why is the final node status not Warning if Child 1 status is Up? Having the Child 2 status be Critical and then the final node status also be critical looks more like the old "show worst" status for Groups. Or, do I need to expand and re-think my definitions of "mixed" based on the fact that so many other status types are now included in the Enhanced Node Status?

                                           

                                          Sorry if these questions are not in the correct place.

                                            • Re: Someone please set me straight
                                              aLTeReGo

                                              borgan  wrote:

                                               

                                              Somehow I missed your post RE: Enhanced Node status back in March. In short, the feature looks very good and should be a long-awaited improvement to how node status is treated in Orion. I could not comment in the referenced post, so please permit me to post a couple of comments/questions here:

                                               

                                              1. Did I miss the indicator for the actual node status in the new Enhanced pop-up? Since the icon now only changes the color of the mini icon at the lower right, I could not immediately tell if the node responded to it's status poll or not.

                                               

                                              With enhanced status enabled there is no longer any tiny child sub-status. Anything that's wrong with the node or its associated child entities are rolled up to the node's status itself. To see what is influencing the node status, simply hover your mouse over the node.

                                               

                                              2. The matrix that shows how "mixed" status will be determined seems s little confusing. It seems that "mixed" does not actually mean quite the same thing as in the past (which you did explain), but for instance, in the first row, the node status is Down, but the Child 1/2 status can be anything. Why is that not still an Final node status of "Warning"? In this case, it seems that actual node poll status is still more heavily weighted if the final node status is to be set to "Down".

                                              If the node is down then that would be the most important information to be displayed. Without this higher weighted priority then the worst status a node could ever go into under 'mixed status' would be 'warning'.

                                               

                                              3. Again, in the fourth row of the Mixed matrix, why is the final node status not Warning if Child 1 status is Up? Having the Child 2 status be Critical and then the final node status also be critical looks more like the old "show worst" status for Groups. Or, do I need to expand and re-think my definitions of "mixed" based on the fact that so many other status types are now included in the Enhanced Node Status?

                                              Again, without this higher weighted priority then the worst status a node could ever go into under 'mixed status' would be 'warning'. If it helps, this is also the same formula that is used for calculating the status of Orion Groups.

                                               

                                              Sorry if these questions are not in the correct place.

                                               

                                              No worries. Thank you for bringing this to my attention. I have now enabled comments on this post.