Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

saving interface utilization data at 1 minute polling frequency for a whole year

We need to run reports and track interface utilization on specific Internet facing ports and preserve the data at one minute polling (no rollups) for up to a year. We know this will generate more data but we also need it for only about 250 interfaces.

I've already changed the rollup frequency or DETAILED STATISTICS RETENTION from one week to 59 days, the maximum.

My idea was to export this data from Solarwinds to Prometheus and build my reports in Grafanna to get the specific information we need.

Wondering if anyone has any thoughts on this approach or might have a better one.

A couple details.

1) I know this could dramatically increase the load on our servers but for the vast majority of our thousands of interfaces, I've set the polling frequency to once per hour. Our servers are currently handling the load comfortably.

2) I am of the opinion that rolling up data, effectively averaging it down to hourly/daily/weekly makes this data useless for any sort of planning.

-Brian

Find more posts tagged with

network_monitoring

orion

network_management

npm

report

Accepted answers

All comments

NVSteven

I would leave the retention at recommended levels. SWQL doesn't have the power to parse that much data. That is why it limits. You can do a query using the SWIS API and pull the data to an external source weekly or whatever time period you like.

cnorborg

Agreed, I wouldn't mess with the levels that much as it will greatly affect performance. Even to move the polling down to 1 minute for 250 interfaces will probably increase your load quite a bit. I think the default is 9 minutes. Increasing it like that is like putting 2250 interfaces on (9*250). Not to mention that's a lot of SNMP polling too. Agreed that if you want to offload the data every 60 days, you could do it just as easily every 7 days and not impact your environment nearly as much.

This also begs the question of whether SNMP is the correct technology for what your really want? Or would you be better off getting Netflow data and somehow summarizing it the way that you want. Netflow is not only more accurate, its driven by the device rather than Solarwinds. You usually want to configure it so it pushes flows at least every minute already, and its done in hardware on the remote device. Plus, if you want it, you get not only how much traffic, but what types of traffic are coming in. The only problem with Netflow is the retention period isn't usually that long. However, you can store all Netflow data on a different database altogether, so it wouldn't affect your main database at all if it was a different server, and you could probably push up the retention period on it, as well as offload it via the API like mentioned?

brianbooher

Thank you NVSteven and cnorborg. I had changed the retention period to 59 days about 2 months ago and have not noticed an appreciable performance impact and my solarwinds db hasn't increased all that much.

But, once I get my import script from Solarwinds db to Prometheus working, i would revert my data retention time back to 7 days and run my script weekly.

I've thought about depending solely on Netflow for the stats I need but haven't entirely trusted the results I'm seeing. If I need to know total utilization on a L3 interface, I wondered if Netflow was only keeping or tracking the highest bandwidth users and ignoring multiple low bandwidth connections, even though in aggregate their impact may be substantial. If I were to trust Netflow data, I'd want to see total bandwidth utilization charts to perfectly match what Solarwinds presents based on SNMP collected data

I'll proceed with my plan to import into Prometheus. So far it's slow going. I'd be interested in what others are doing for long time internet circuit planning and how they are using their solarwinds data to guide spending on an expensive circuit.