Netflow Query Guidance

We recently upgraded from NTA 4.2.3 to the latest version of NTA running on 2020.2.4.  Prior to the upgrade we generated a bunch of bandwidth reports for clients using the following query.

SELECT SUM(EgressBytes)*8/60 as OutTraffic, TimeStamp FROM Orion.Netflow.Flows(nolock=true) WHERE SourceIPGroupSegmentID = 790
GROUP BY TimeStamp ORDER BY TimeStamp

Since upgrading this query works for some SourceIPGroupSegments and then for others just returns no data.  I know that these should have data as I can see the data on the normal Netflow graphs.  

I think part of the problem is how the Groups are stored in 2020.2.4 vs how they were back in 4.2.3 along with the changes to the netflow DB(fastbit to SQL) and the introduction to some new IP Group and IP Group Segment tables in SQL.  

I need help getting my query repointed in the right direction but I'm just not sure what it needs to look at.  

Parents
  • Hi Shack,

    We are also experiencing problems after switching to 2020.2(fresh installation). Reports that were working correctly and were using the IP Address Groups or ID's are no longer working or are partially working. To me, it looks like the association between IP's and IPGroups/IPgroupSegments is not properly working.

    I can see in the sql netflow database that, for example, one IP that is part of a defined IP Address Group is being assigned sometimes a DestinationIPGroupSegmentID = 0 (which is used for undefined), or SourceIPgroupSegmentID = 0 or any other segmentid instead of correct one. Looks like this process is random.

    One entry:

    DestinationHostname=$hostname1.domain

    DestinationHostnameID=10000000008

    DestinationIPGroupSegmentID=44

    Another entry:

    DestinationHostname=$hostname1.domain

    DestinationHostnameID=10000000008

    DestinationIPGroupSegmentID=2

    Another entry:

    DestinationHostname=$hostname1.domain

    DestinationHostnameID=10000000008

    DestinationIPGroupSegmentID=0

    George

  • You are correct.  The SourceIPGroupSegmentID in dbo.netflowflows does not relate back to the old table named NetflowIPExport any longer.  

    Here is a query you can run that will show you the IP Group and the associated IP Segment ID's.  What we still don't understand is the data in "versions".  When we run the query we are getting multiple entries for the same IP Segment ID but with different version numbers.  The IP Segment ID in this query is what you will see tagged on the netflowflows table but there could be multiple ID's for the same IP range.  That's how it is at least for us.

    select NetflowIPGroups.IPGroupID,RangeID, NetflowIPGroups.Name, NetflowIPGroupRanges.Low, NetflowIPGroupRanges.High, LowDecimal as [Low Decimal], HighDecimal as [High Decimal], IPSegmentID, Version
    from NetflowIPGroups
    Inner Join NetFlowIPGroupRanges ON NetflowIPGroups.IPGroupID=NetflowIPGroupRanges.IPGroupID
    Inner Join [SolarWindsFlowStorage].[dbo].[NetFlowStorageSegments] ON NetFlowIPGroupRanges.LowDecimal=NetflowStorageSegments.Low
    Where Enabled='1'
    order by Name, Low, IPSegmentID

    We have decided to use the hex low and high ranges of the SourceIP from the flows table for generating our reports.  We are also using SQL for reporting now instead of SWQL or the object picker method.  

  • If it was only the different ID's it would have been good, but in our case we have netflows that get labeled with IPGroupSegmentID=0 (IPGroupID=0) which is for ip's that do not belong to the Groups. And these are definately ip's that are part of defined IP Address Groups.

Reply
  • If it was only the different ID's it would have been good, but in our case we have netflows that get labeled with IPGroupSegmentID=0 (IPGroupID=0) which is for ip's that do not belong to the Groups. And these are definately ip's that are part of defined IP Address Groups.

Children
No Data