6 Replies Latest reply: Apr 12, 2012 9:57 PM by Andy Ng RSS

ResponseTime_Detail records created for deleted Nodes

dngill

Hi

 

I think our monitoring system is creating ResponseTime_Detail records for nodes that we have recently deleted.

 

I thought that when a node was deleted, it was removed from the Nodes table, and records for that NodeID were removed from the ResponseTime_Detail, ResponseTime_Hourly and ResponseTime_Daily tables.  What seems to have happened recently is that the ResponseTime_Detail and ResponseTime_Daily records are deleted, but ResponseTime_Hourly remain (but are not bundled into ResponseTime_Daily after 180 days).

 

Because we are checking for outages, we have found that deleted nodes are showing as being out.  Records are created every 2 minutes.  If the IP address still exists, all seems fine, but if it does not then Availability swaps between 0 and 100 while PercentLoss is 100.  These records do not appear to get bundled into the ResponseTime_Hourly table after 30 days.

 

My questions are:

  • Is this now normal behaviour?
  • How can we stop records being created for nodes no longer in the Nodes table?

 

Many thanks

dg

 

APM 4.0.2

NPM 10.1.2

 
  • Re: ResponseTime_Detail records created for deleted Nodes
    Andy Ng

    Hi dg,

     

    No, it does not seem to be normal.

    You might want to try manually running Database Maintenance to see if that clears up, as that will remove orphaned information from the database.

    It is normally ran at 2:15am system time by default - it is weird that it did not clear up.

    [Start > Programs > SolarWinds Orion > Advanced Features > Database Maintenance]

     

    To see the logging for the Database Maintenance, you might try to look into C:\ProgramData\Solarwinds\Logs\Orion\swdebugMaintenance.log

     

    HTH.

    • Re: ResponseTime_Detail records created for deleted Nodes
      dngill

      Hi Andy

       

      Thanks for that, it does help.  I'll look into the log, because the job is running (it does do the moving from detail > hourly > daily for existing nodes).

       

      dg

    • Re: ResponseTime_Detail records created for deleted Nodes
      dngill

      Andy

       

      Repeated thanks for the pointer.  I found the swdebugMaintenance.log files in D:\Program Files (x86)\SolarWinds\Orion, but they all seem to be doing the steps for ResponseTime maintenance (including "Removing orphaned data from ResponseTime") without notifying of any failures.

       

      But the records are still being generated for deleted nodes, and the hourly records still exist for deleted nodes.

       

      Thanks again for your help.  But I still need more, if anyone has any more.

       

      dg

      • Re: ResponseTime_Detail records created for deleted Nodes
        Andy Ng

        Hi dg,

         

        If you still have the orphaned information under ResponseTime_Hourly even after the Database Maintenance, it might be that the NodeID is still there?


        Can you check the NodeID that corresponds to ResponseTime_Hourly that is having the orphaned data, and thereafter, cross check it against the 'Nodes' table

         

        SELECT *

        FROM Nodes

        WHERE NodeID = __

         

        Replace __ with the NodeID that you see in ResponseTime_Hourly orphaned data.

         

        Let me know how that goes.

        • Re: ResponseTime_Detail records created for deleted Nodes
          dngill

          Thanks again Andy.

           

          I am not finding records for deleted nodes in the Nodes table using your query.

           

          When I try

           

          SELECT count(*) Count, [NodeID]

          FROM [NetPerfMon].[dbo].[ResponseTime_Hourly]

          where NodeID not in (select NodeID from NetPerfMon.dbo.Nodes)

          group by NodeID

          order by NodeID

           

          I currently get 18 rows.  Against the _Detail table, 72 rows.  Against the _Daily table, none.

           

          Does that provide any clues?

           

          dg

          • Re: ResponseTime_Detail records created for deleted Nodes
            Andy Ng

            Hi dg,

             

            It is weird that you are not seeing the NodeID in the Nodes table, but are still seeing ResponseTime statistics. The Database Maintenance should have removed it.

            Just checking if the output from the query, is the orphaned ResponseTime belonging to the same NodeID, or it involves a few?

             

            I was thinking that it might be a one off issue that might not reoccur after you have dropped those statistics.
            Alternatively, you can try upgrading as that will facilitate in fixing of flat-files (as well as updating), and see if there are improvements.

             

            HTH.