15 Replies Latest reply on Feb 11, 2016 12:14 PM by Craig Norborg

    All Ip Adresses field showing legacy Ip Address

    jacob.johnson

      in a nutshell we were monitoring all our network devices using a polling ip of 10.10.10.xx.  We created a new VLan and added ip addresses 10.10.220.xx to all the switches, we then removed the old ips.  I manually changed the polling IP address in NPM to the new one.  Now the old ip is still showing under all ip addresses for each device, and when I hit edit node it gives me the option to select ip address and shows the old and new one.

       

      what is the trick to remove these old ip addresses?

       

       

      Thank you

        • Re: All Ip Adresses field showing legacy Ip Address
          Craig Norborg

          Try "Rediscovering" and "Polling" the node, might take a few minutes to catch up.   If its in NCM that you're seeing it, you might have to re-Inventory the devices by going to Configuration Management, selecting the devices, and then clicking the "Update Inventory" button.

            • Re: All Ip Adresses field showing legacy Ip Address
              jacob.johnson

              I tried rediscovering, and polling, I tried a reinventory, I have queried 30 different db tables directly looking for one of the old ip addresses and have been unable to find it at this point.

               

              it is showing up under the all ip addresses assigned to caption resource, also when you click edit node the option to select ip addresses shows both ip addresses.

               

              Just need to get them out of there.

                • Re: All Ip Adresses field showing legacy Ip Address
                  Craig Norborg

                  Assuming you checked the configs to make sure when they did the transition they didn't leave the old IP in place as a secondary IP on the interface?

                   

                  You might like this BTW, a potential replacement I made for the "All IP Addresses" resource.  At the very least, the queries should let you know where in the database it might find that information.

                   

                  All IP Addresses on a node - potential replacement resource

                    • Re: All Ip Adresses field showing legacy Ip Address
                      jacob.johnson

                      Thank you very much for the information you provided, I did not implement the SWQL code, but I was able to search the nodeipaddress table you used for the code and I found one of the rogue Ips.  The IPaddresstype, InterfaceIndex, and Subnet Mask are all NULL. (there are total 628 ips with null in the three columns)  So is there a clean way to clean up these old ips.  is it safe to assume since the three columns are null that it is not being used anywhere else in the system?

                       

                       

                      Thank you

                        • Re: All Ip Adresses field showing legacy Ip Address
                          jacob.johnson

                          hate to reply to myself, but I have discovered that nodes managed by ICMP only show up in the nodeipaddress table with the last three columns as null, but also exist in the nodesdata table because that ip address is used for polling.

                           

                          so if I just delete everything from the nodeipaddress table that have the last three columns as null and  does not exist on the nodesdata table I should be good, but this is actually intimidating.  Seems like the out of the box database cleanup should clean something like this up. 

                           

                          Anyone know if it does or should clean it up?

                  • Re: All Ip Adresses field showing legacy Ip Address
                    Craig Norborg

                    How long has it been?   SW does go and clean up after itself a bit later.   From the server you can go to "Advanced Features" from the start menu and do database cleanup.  That might help, not sure...

                      • Re: All Ip Adresses field showing legacy Ip Address
                        jacob.johnson

                        Thank you for recomendation, I have not ran the database cleanup, but I am getting it scheduled now.

                         

                        I also found the rogue IP addresses in the database, crossed checked against several other tables (a lot of them existed in the discovered l3 connections table)  so I have a comprehensive list of which one need removed at least. 

                         

                        If the database cleanup fixes it, it will be a double edged sword

                         

                        thanks and I will update once ran.

                          • Re: All Ip Adresses field showing legacy Ip Address
                            Craig Norborg

                            Yea, not a big fan of removing data from the files manually.  You can really mess things up that way!!

                             

                            You did check the configs on the devices to see if the old IP is a secondary IP already?  Never got an answer on that question...

                              • Re: All Ip Adresses field showing legacy Ip Address
                                jacob.johnson

                                yes, that ip is completely gone from the devices, (searched all the ncm configs, and checked all the ncm tables too)  We do not have layer 2 or 3 topology being discovered, so not sure orion would pull in a secondary ip anyway.

                                • Re: All Ip Adresses field showing legacy Ip Address
                                  jacob.johnson

                                  Just finished running the database maint from the server, it did not correct the issue.

                                   

                                  I guess I have to clear them from the DB, but I wish there was another way.

                                    • Re: All Ip Adresses field showing legacy Ip Address
                                      Craig Norborg

                                      Yea, I would wait a few days at least, and if they don't go away open a support ticket.  You really shouldn't be deleting things directly from the database using SQL unless you're VERY familiar with the dependencies...   

                                        • Re: All Ip Adresses field showing legacy Ip Address
                                          jacob.johnson

                                          I have ran the DB maint, and will wait, but in the mean time I have removed 4 entries directly from the database, after reviewing the data a I pulled out, this is just not a comprehensive list, I will still have several hundred nodes with the old ip address listed.

                                           

                                          I am thinking of implementing the SWQL query you provided, however the current all ip addresses resource shows the polling ip, and your new one does not.  The problem is the polling ip address is actually in a different table, I do not know enough about swql to include the join.  Any thoughts.

                                          • Re: All Ip Adresses field showing legacy Ip Address
                                            jacob.johnson

                                            it looks like some of the nodes are just not in the nodeipaddress table at all, and only exist on the nodedetails table.  Since I discovered we have several reports that only query the nodeipadress table for ip addresses, this problem is way more impactful than expected.

                                             

                                            going to open a ticket as soon as possible. 

                                            • Re: All Ip Adresses field showing legacy Ip Address
                                              jacob.johnson

                                              Not to beat a dead horse, but I found this helpful to at least understand the issue.

                                              Re: NPM's NodeIPAddresses table is not up-to-date

                                               

                                              looks like solarwinds is doing this:

                                               

                                              NPM is able to populate list of IP addresses using DNS server and SNMP or WMI. IP address poll for SNMP node executes these steps:

                                               

                                              1. Perform reverse DNS lookup on Node's IPAddress (from Nodes table) to get fresh DNS name  (no ip domain-lookup configured on affected devices)
                                              2. Perform forward DNS lookup on DNS name from step 1 (I assume this step is not conducted when the first one fails)
                                              3. Query ipAdEntIfIndex OID (1.3.6.1.2.1.4.20.1.2) and ipv6AddrStatus OID (1.3.6.1.2.1.55.1.8.1.5) to get all IP addresses assigned to Node via SNMP (if data is discovered via SNMP it is being collected)
                                              4. Merge those two lists together, store result into NodeIpAddresses (Tables are not being merged, because the data that would be merged is not even collected in step 1 or 2.)

                                               

                                               

                                              That seems to be why the nodeipaddresses table is not complete, or updated.  not sure what the actual solution will turn out to be, but at least I know what is going on.

                                                • Re: All Ip Adresses field showing legacy Ip Address
                                                  Craig Norborg

                                                  Well, steps 1 and 2 are done on the Orion server, not the device your polling, so "no ip domain-lookup" will have no effect on these steps.   The reverse DNS lookup is done on the polling IP and the forward DNS lookup is done on the results from that.

                                                   

                                                  Step 3 is the first place it polls the device and this is what it usually populates the NodeIPAddresses table from.   I don't believe that Orion does any DNS lookups on what it finds in this table.

                                                   

                                                  You'll note in the last message from that link that @planglois says that ICMP only nodes don't end up in the NodesIPAddresses table.  I'd hazard a guess that its because Step 4 doesn't actually happen (ie: merging a list of polling IP's with the results of the SNMP scan).   I think that the NodeIPAddresses table is only the results of the SNMP scan.  So, if a node is ICMP-only, and not polled via SNMP, its address doesn't end up in the table...

                                                   

                                                  That being said, Orion does tend to clean up things where it makes sense, but maybe not on your timetable.  Generally old IP's for nodes tend to go away in my experience, never cared if it took a couple extra days myself...  

                                                   

                                                  But, then again, I could be wrong!!  Open a ticket if you're that curious!!  I'd advise doing that over mucking around in SQL directly, at least at first...