cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

I Schooled An Outage Using NetFlow

I Schooled An Outage Using NetFlow

One day, I found myself holding the receiver away from my head as the principal for one of the schools I supported was shouting, “The internet just keeps going down!” I replied, “How often and when?” He said, “2-4 times a day and at random times.”

“Lovely,” I muttered under my breath, “vague AND random. My favorite.”

A quick look at the bandwidth metrics revealed thatyes indeedthe WAN link to the school was becoming saturated at various intervals during the day. Usage would spike for 20-30 minutes, and then drop down until the next incident.

I decided to see if NetFlow (a new tool for me at the time) had anything to tell me. Correlating the spike times from our bandwidth stats, we saw that during the same period, the “top talkers” were 10 MAC addresses which all started conversations with http://www.youtube.com/. Every time there was a spike, it was the same set of MAC addresses. A visit to WireShark’s “OUI Lookup Tool” revealed that all 10 were from Apple, Inc.

I had no more to go on. I called the Principal back and gave him the start and stop times of the spikes, along with the information on 10 Apple products that were to blame.

“Wait, what time was that?” he asked. I repeated the times and he said, “Oh for the love of…I know what the problem is.”

It turns out that the art teacher had been awarded a grant for 10 shiny new iPads. He would go from room to room during art period, handing them out and teaching kids how to do video mashups.

This was one of those rare times when a bandwidth increase really WAS warranted, and after the schools WAN circuit was re-provisioned, the Internet stopped mysteriously “going down.”

Tags (1)
1 Comment
Community Manager
Community Manager
Status changed to: Closed