How to create an alert based on errors per percent of bandwidth?

Question

I was told it could be done through Advanced Alerting.......but I can't seem to figure out how!!!!

Any help would be really appreciated!!  :-)

vhcato · Answer

Marcy,

OK, here's the poop...

After looking and thinking about it a bit, I don't believe you're going to be able to configure exactly what you're looking for with an advanced alert alone; or at least I couldn't figure out how to make it happen. The problem lies with the fact that in this scenario, a calculation needs to be performed against a pair of data fields for each error and discard type on which you want to alert, and the advanced alerting engine doesn't have an internal mechanism to perform these calculations.

Here are some alternatives:

1) Configure alerts to monitor the number of errors/discards, and once a certain threshold is met, have a trigger action that runs a script to make the determination of whether there is a disproportional amount of errors/discards for the amount of utilization. Depending on the outcome of the calculation performed by the script, the script itself can execute the notification (or not). The scripting piece is beyond my current capabilities, but I can probably come up with the SQL query to perform the calculation. Perhaps you, or someone else in "Thwackville", has the scripting skills..? The thing I don't know is the resource impact of performing these calculations on the Orion server, which is highly variable.

2) Configure and schedule a stored procedure on your SQL server to perform the calculations and populate some custom property fields with the derived values. Once that mechanism is in place, an alert can be configured to monitor and trigger based upon on these values. If you're running SQL on a separate server, this will offload the CPU hit to the SQL server instead of the Orion host server.

3) Re-think the notion of alerting based on a disproportional number of errors/discards relative to utilization, and build them based on the number of errors/discards alone. This is the simplest means of being notified of such conditions.

As a side note, while I agree that in many cases the number of errors/discards may be proportional to bandwidth usage, any errors beyond a very small number should be investigated, regardless of bandwidth. In the case of TX discards in particular, these are most often due to bandwidth over-subscription, which in itself can be tracked and alerted upon. RX discards in a switched environment are most often due to things like inconsistent trunk configurations, are usually not an immediate threat to degrade performance, and can be handled as indicated by a resource on the web site. The caveat to that is in the case where there is a systemic problem with nics, drivers, cabling, or switch hardware and software, which can probably be handled with an alert of less complexity, if the desire is to be alerted.

Those are just my personal thoughts, and opinions will vary, depending on the level of criticality and sense of urgency.

What are your thoughts? Perhaps someone else in Thwackville has some ideas...?

vhcato · Answer

OH, you're gonna make me THINK!!! :)

I think we can do something like that. If you're primarily targeting errors related to oversubscription, I would think you want to track TX Discards, but if you're looking for carrier or physical level problems, I would think you want to track RX Errors. Just my thoughts...

I'll see what I can put together.

mhazelb · Answer

Thanks for the response!!  Sorry for the delay in my response :-)

What I'm trying to do is......I want to monitor our core switches for errors.  But obviously the number of errors will be higher based on the percentage of traffice coming through.  So, what I'm wanting to do is something like.......if 20% of traffic (or some other determined percentage) of traffic contains errors - send an alert.  I'm a dummy when it comes to the Advanced Alerting, no sql skills at all!!

Thanks!

Marcy