cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

Network Management stops finger pointing..

Network management is good, but there are some benefits that are not so apparent at first. It gives you the proof you need when that person calls the help desk and says “the network is slow..” at this point you have to clarify the issue with the user and find out exactly what the problem is. Once you have obtained the details, you can then check your NMS, to find out if there is an issue, one of three things is going to happen, you find a problem and you can correct it, or you don’t find a problem and prove the user is incorrect.

The thing you wont have is “ I don’t know if there is a problem“ and that is the one that will drive you crazy. Because the user is sure there is a problem and unless you can offer authoritative proof the finger pointing cycle will begin, and this is a place no one wants to be. You know there is not a problem with the network, but the user needs proof, Solarwinds Orion is what gives you this proof, that can be taken to the client and say, its not my opinion,  that the network is not slow , here are  the facts.. circuit xyz has a utilization of 20% and is nowhere near being saturated.

I had a user call into the help desk and claim the network was slow, he placed a ticket that his mission critical system was slow and this was causing a production problem. The help desk created a high priority ticket and sent it over to my team. I looked at the ticket and checked my Orion server , for the problem area that was reported I saw no issues,  I created a report to backup my findings. I called the user and explained that based on the facts I have obtained , there was no problem.

The user told me , that there was indeed a problem , he had just moved from building 3  to building 1 and when he was in building one he ran a speedtest from speed test.net and it was five times faster in his old building. He even showed me some print screens of his findings.

In reality , there was no problem, but if I didn’t have the facts to back me up, the finger pointing would start and it would just spiral out of control and at that point nobody would be happy . like they always say “the truth shall set you free”  and it sure did in this case.  

Orion, has saved so much time and frustration, and I sleep better at night knowing that yes there will still be network problems, but I will be able to answer that question, Is there a problem? If yes, then I fix it , if not I can prove it. but I never have to deal with I don’t know, and that my friends is a wonderful thing. Bye, bye  finger pointing hello productive day. How much is your time worth?

14 Comments
Level 10

I've been right in the middle of the same situation over the past month or so. Remote site is having horrible network performance which ended up being a combination of things. Both my NMS and netflow I was able to pin-point the issue pretty quick (no QoS on phone ports and old line cards being over-saturated).

I dont know how anyone can manage an infrastructure without these key points of data.

At minimum give me:

-Link usage Tx/Rx (as far back as I can get. Happy with minimum a year)

-Drops, Buffer misses

-CPU/Mem usage trends

-Syslog/traps

This is my first time really working with Netflow but I will never be without it again!

MVP
MVP

So wait... was there a problem? Even if it wasn't a network issue, or a network issue that would have been measured in Orion NPM, surely there was some other reason for the user's performance issues. I can tell you from personal experience that nothing makes me more insane, as a customer, as when I'm experiencing a problem only to be told by support that there is no problem.

Level 21

I find myself in this same situation all of the time; however, I never find it that easy to provide the facts.  The problem I find is that there are so many different factors that can contribute to "slowness".  While we use Orion (NPM, SAM, NCM, LEM, Toolset), there are so many different bits of data that one needs to collect and have available to show that the problem isn't something you are responsible for.

As an example; we often have customers tell us the VM's that we manage for them are slow... I could certainly just pull up a CPU graph and say "no, it's just fine".  The problem is it could be CPU, memory, network, application, customer code, back end storage, etc.  I find just having some performance data isn't always enough and I almost always end up spending a good deal of time on these types of issues despite having a good and well implemented NMS with all of the performance metrics.  In addition to the performance metrics, I always end up looking through logs which is almost always time consuming.

I guess my point is that I don't think there is a silver bullet for these types of things, if there was I wouldn't have a job... at least not this one.

Level 11

Unfortunately, network monitoring isn't always enough to stop the finger pointing.  Generally, I end up going into deep packet analysis to find the actual issue before the network can be exonerated.  I think it stems from a lack of basic networking knowledge, causing other technical teams to push things over on the network, since it's easy to blame something you don't understand.  The other downfall is that my team is usually able to find the issue via packet inspection or basic troubleshooting, so other teams have come to rely on us for answers to problems they can't solve. 

Now, I'm not bitter at all, even though it might seem that way.  At the end of the day, we are all part of the same team, working for the same goals.  My only challenge is to get some of the smaller basic things resolved at level 1 or 2 instead of kicking everything up to level 3 at the first sign of difficulty.

Level 10

You are right Matt. Indeed network monitoring isn't always enough to stop the finger pointing.

Level 12

I agree with what others have said, just because the link utilization was only 20% doesn't mean the user wasn't truly experiencing a performance issue.  To me, that would just be the starting point of the troubleshooting.

Level 19


I'm in agreement with the other folks who say that Network Management alone cannot stop finger pointing. The problem here is that using Network Management alone to show that the issue is not in the network is attempting to prove a negative. Think of the problem of proving that there are no invisible unicorns in the room with you. It can't be done. While comprehensive Network Management can provide a lot of evidence that the network is not involved, it cannot eliminate the network. Now if you add Application Management, Storage Management, and the other IT disciplines the chance of finding the root cause of the issue increases. Once you find the root cause then the network can be eliminated, unless you work for that guy who says, "I still think the network was involved somehow". Then you can only smile and continue choking him in your head.

Level 13

We're only supposed to do that in our head?

Level 13

There is never going to be a way to stop "that guy" from complaining about the network. One of the fun finger pointing thing's that I've gotten to do was have "that guy" complain that it was slow and I got to come back and say "well, thats because you've tried to download over 20 gigabytes from the internet in the last day. What are you downloading thats so big?" It was some bad things involving nudity. Apparently he was downloading it from some place that used password protected zips that get past the network detection.

Andy McBride Sometimes you don't to choke them, but you still get to attach them to another object by an inclined plane wrapped helically around an axis.

Level 12

+1

MVP
MVP

...Once you find the root cause then the network can be eliminated...

Not biased, are we?

Level 19

Hey, I'm a network engineer.

Level 7

I don't think its possible to ever prevent the finger pointing.  Nobody know the network guys unless something is broken other wise they would just assume they stay int he basement playing with their red stapler.  But that's the nature of the work its ok to get pointed at as long as we have the tools to determine when its not us or to fix the problem when it is.

Level 15

Informative!