Skip navigation
1 14 15 16 17 18 Previous Next

Geek Speak

2,277 posts
Leon Adato

Trite Business Lies

Posted by Leon Adato Expert Feb 27, 2017


When someone brings up the topic of lies at work, most of the time people think of the big stuff, like embezzlement, falsifying data, or fudging fantasy football stats. But there's a whole other category of lying that happens all the time. Not only are these lies tolerated, but through some combination of repetition, vehemence, and mind control, employees of all stripes have come to believe them as true.


Part of the reason for this is that the lies are small (and therefore insidious). They often pass themselves off as sage advice or commonly-understood truisms. At worst, they may appear to be clichés or business tropes. Many are hard to argue with unless you've thought it through.


Nevertheless, they are lies, and especially in this era of #AlternateFacts, deserve to be exposed as such.


So take a look below and see which of these phrases match up with your personal experience or reality. If you have stories, reactions, or your own additions, let us know about them in the comments below!


“Perception is reality”

No, it’s not. I perceive myself to be an AMAZING dancer. But 48 years of evidence (as well as testimony from my wife and children. ESPECIALLY my children!) indicate otherwise.


The truer statement is that perception is powerful, and can override facts. Knowing this is so does NOT mean we must kowtow to a requestor’s built-in perceptions or preconceived notions, but rather that we need to understand those perceptions and build compelling stories that bring the person to the actual truth.


“The customer is always right”

The truth is that customers can be, and often are, wrong. This can be due to the fact that they’ve been misled, or un- (or under-) informed, or one of a hundred other reasons why they are asking for the wrong thing, looking for the wrong solution. And then there’s the rare (but not rare enough) case when they are simply bat-guano crazy and want something stupid.


However, the customer IS always the customer. They are always the one who wants to buy, and we are always the one who wants to sell. Understanding this relationship doesn’t mean we have to sell our soul, ethics, or values to sell our product. Rather, it gives us the freedom to find the right customer for our product, and come to terms with the fact that not EVERYONE is a customer (or at least not a customer right now.)


“Work smarter, not harder”

What am I supposed to think here? That I've been working stupider up until now? That I've willfully withheld a portion of my intellectual capacity? Or that I'm just phoning it in? Regardless of which inference you make, it's not a kind reflection on me, nor on what you think of my work product and work habits.


If you think I stink, or that I'm slacking off, or that I've gotten into a behavioral rut, then please just say it. If you think I'm overlooking something obvious, then say THAT. And if you don't think EITHER of those things, don't say this to fill the dead air.


“Think outside the box”

Like "work smarter, not harder," this one has the one-two punch of an insult AND the implication that I can’t solve a particular problem.


I list this as a lie because there IS no box. I may have fallen into a rut or a set of sub-optimal habits. I might be hyperfocused on a particular outcome or trajectory; I could just be lazy and unwilling to put in the extra effort that thinking about something differently might require.


Whatever the reason for my inability to find a novel solution is, it's not "a box." Calling it that doesn't get me any closer to changing my behavior OR solving the challenge.


“Lean and Mean”

Back in 2004, Bob Lewis translated this as "emaciated and unpleasant" ( and that has stuck with me. Lean is all well and good, as long as we mean "healthy" rather than "underfed.”


But "mean" (unless you mean "average,”which you probably don't) is a trait I would not find advantageous in the workplace. When would it be considered organizationally good to be rude, dismissive, short-tempered, or (to use Lewis' term) unpleasant?


Rather, we should strive for our teams and organizations to be healthy, focused, and determined. You can even throw in “hungry” if you want, as long as it’s not due to a lack of sustenance (i.e., resources).


"I just need you to hold out until..."

In my experience, this is a statement that comes in three month cycles. Just keep working on-call for a little longer. Just keep putting in the extra hours. Just deal with this (last) fire drill.


Just put up with this sub-optimal situation.


After two years of ongoing struggle where the answer was routinely, "This is only going to be until the end of quarter,” I finally confronted a manager about how the situation hadn't gotten better. The situation may have changed, but the workload, sense of crisis, etc. had not improved. His response was surprisingly transparent, "Oh, I didn't mean it would get better. I just meant that it would change."


A company, department, or team that has gotten into the habit of trying to wait out bad situations is one that has given up on solving problems. Just remember that.


If you find yourself on the receiving end of this particular lie, one response is to ask for specifics. Such as:

  1. “Then what,”  as in, what is the situation going to be at the end of the <time period>?
  2. How are we going to get there between now and then?
  3. What are your expectations of me during that period?
  4. What will we do if we miss our target (date or end-state)?


By using these questions, and then the logical follow-up responses along with copious documentation, you are asking management to commit to a set of goals and outcomes (this is a technique also known as “managing up”). It also gives you a fairly accurate barometer as to how hard you should be looking for a better situation.


“This company is like family”

No, it isn't. Most companies are too large to have the group dynamics of a family. A company lacks the long-term history, shared genealogy, etc. that create lasting bonds.


That's not to say that people working at a company can't feel a close friendship and camaraderie. But it's still not family.


Those who make this type of comment are typically trying to instill in you a sense of loyalty to them right before they ask you to do something that goes against your personal interests.


“Human Resources”

I know, I know, this is the name of a department. But the name of the group is a misnomer bordering on a trite falsehood. Almost all of the frustration I've ever had (or heard coworkers have) with HR stems from misunderstanding their core mission.


In my experience, HR exists for two primary reasons:

  1. To keep the company out of lawsuits arising from employee interactions
  2. To shield upper management from the messier aspects of employees, including salary negotiations, grievances, etc.


They are NOT there to help you grow as an employee. They are NOT there to provide a sounding board. They are NOT there to help create a positive work environment.


They are NOT a department designed to help the employee in any way, unless "help" intersects with one of the two areas above. Use them in the same way you would use the legal department. Because that's pretty much what they are an extension of.


“Treat Your Users Like Your Customers”

I have a few issues with this. First, when I run my own company I can choose which customers I want to deal with. I do that by deciding how and where I market, which jobs I accept, and which I'm too busy to take on right now, and by setting a price for each job that reflects the level of effort and aggravation I expect to have while doing the work.


Equally, my customers can choose whether to hire me or the person down the street.


Within a company, NONE of those things are true. I can't say no to the accounting department, and they can't find someone ELSE in the company to provide the same set of services.


In addition, customers and vendors come and go. But Sarah in the mail room today will become Sarah the head of accounting tomorrow. So how I treat her today matters for the duration of our time at this company.


“The squeaky wheel gets the grease”

I learned long ago that this is SOMETIMES true. But more often, the more accurate phrase is "The squeaky wheel gets REPLACED!"


When, how, and to whom one squeaks is a lesson many of us learn by experience (meaning: doing it wrong) over time.


So, anyone who blithely tells you this probably wants one of the following outcomes, none of which will be good for YOU:

  1. They don't realize the truth of the situation either
  2. They have the same complaint and know better than to open their mouths. but they are perfectly willing for you to lead the charge and take all the heat.
  3. They want to watch you make a spectacle of yourself for their entertainment


“The elevator pitch”

Brevity is wonderful when it pushes us to create simple elegance. But often it just causes us to stress more, talk faster, widen the margins, shrink the font, and try to jam more into less.


Effective storytelling is partially about knowing when your forum and format fit the story you want to tell. Otherwise, you end up babbling like a fool and ruining your chances of making a case later on.


Some issues, requests, and explanations are complicated and can't be reduced to a 30-second overview. If you run into someone who demands that all of your interactions, requests, etc. be put in that format, then they're not really listening anyway.


“That's not part of our corporate culture/DNA”

There's a famous story about five gorillas (you can read it here).


Phrases like, "that's now how we do things,” or "That's just how it's done here,” or "It's not part of our DNA,”  are all lies, and all fall under the heading of Not Invented Here (NIH). It is often code for, "I don't want to," or worse, "The thought of doing things that way scares me."


As stated earlier, companies are not family. They also aren't organisms and thus don't have DNA. If they have a culture, it is because the people who make up the company actively choose to propagate a set of habits or a particular perspective when doing business. And culture or no culture, a good idea is a good idea. People want to do well, want to succeed, want to get ahead.


How you respond to this lie depends largely on your stake in doing something differently, and your role in the company.


“If you hire good people, they won't require supervision”

Stephen Covey famously said (,

"If you can hire people whose passion intersects with the job, they won't require any supervision at all. They will manage themselves better than anyone could ever manage them. Their fire comes from within, not from without. Their motivation is internal, not external."


That's great. Let's see if they file their expense forms correctly and on time.


People at all levels of the organizational chart require supervision. What they may not need is meddling middle managers.


The best supervisors are part janitor, part secretary, and part cheerleader. They keep things clean (meaning they ensure an unobstructed path for their staff to pursue work); they attend higher level meetings and report back the information honestly and transparently so that staff can take the actions that support the business goals; and they publicly recognize successes so that their team feels validated in their work.


Also, this is insulting in the same way that "work smarter" and "think outside the box" are. It's a form of managerial "negging" ( It implies that if you DO need supervision, you are obviously not passionate enough about your work and may be the wrong person for the job.


“It's easier to ask forgiveness than permission”

Yes. And all planes are equipped to make a water landing. Well, at least once. Whether they can take off after that is just a minor detail, right?


In an unhealthy environment, people do things on the sly and hope they don't get caught. If they DO get caught,  it seems many believe throwing themselves on the mercy of the court is as good a strategy as any.


But in mature, professional, adult environments, asking for permission is always preferable and always easier for everyone.


“Failure is not an option”

Nope, in some situations (often ones where this lie is uttered), it's practically a sure thing!


There is no guaranteed success. There is no outcome that is 100% predictable. Some failures, once they are in motion, are unavoidable no matter how much planning was done beforehand, or how much staff are on hand during the failure to try to save it.


Not only is this statement a lie, it's also not particularly helpful advice.


“So what's the point?”

My point is certainly not that everything, or even most things, said at work are a lie. What I AM saying is that some of these trite and overused clichés have reached the end of their useful period.


Or as Sam Goldwyn said, "Let's have some new clichés."


Ain't that the truth.

Karen Lego Figures (c) Karen Lopez

As promised in my previous post on Better IT, in this series I will be talking about collaboration. Today I'm sharing with you anti-patterns in collaboration.

Anti-pattern - Things you shouldn't be doing because they get in the way of success in your work, or your organization's efforts.  Antonym of "pattern."

In my troubled project turnaround work, when I start to talk about collaboration, I usually get many eye rolls. People think we're going to start doing team-building exercises, install an arcade game, and initiate hourly group hugs. (Not that these would be so bad.)  But most collaboration missteps I see are the result of anti-patterns that show up in how teams work. So in this post, let's look at the not-so-great-things that will get your team and your organization into trouble.


IT admins who don't know who is responsible for what, or can't find them

This is often the case in geo-diverse teams, spread over several time zones, and teams with a high staff turnover. Their processes (their "pattern") is to go on a "responsibility safari" to find the person and their contact information for a resource. On one project, it took me almost a month to find the person, who lived on another continent, who was responsible for the new networks we were going to deploy to our retail locations. By the time I found him, he was planning on moving to another company within a week. Having to hunt down people first, then their tools, then their data, is both costly and time-consuming, which delays one's ability to resolve issues. Having to find people before you find data is not the right way to manage.


IT admins who collect duplicate data about resources and their metrics, often in difficult to integrate formats and units of measure

This is almost always the result of using a hodgepodge of tools across teams, many of which are duplicate tools because one person has a preference of toolsets. This duplication of tools leads to duplication of data.  And many of these tools keep their data locked in, with no way to share that data with other tools. This duplication of data and effort is a huge waste of time and money for everyone. The cost of incompatible tool sets producing data in incompatible formats and levels of granularity is large and often not measured. It slows down access to data and the sharing of data across resource types.


IT pros who want to keep their data "private" 

This dysfunction is one my friend Len Silverston calls "data mine-ing," keeping data to yourself for personal use only. This is derived from the fact that data is indeed power. Keeping information about the status of the resources you manage gives you control of the messaging about those systems. This is a terrible thing for collaboration.


Data mine-ing - Acting in a manner that says, "This data is mine."

- Len Silverston

Agile blocking is horrible

A famous Agilista wants people to report false statuses, pretend to do work, tell teams that "all is good" so he can keep doing what he is doing without interruption. He also advocates for sharing incorrect data and data that makes it look like other teams are to blame. I refuse to link to this practice, but if you have decent search skills, you can find it. Teams that practice blocking are usually in the worst shape possible, and also build systems that are literally designed to fail and send their CEO to jail.  It's that bad. Of all these anti-patterns, this is the most dangerous and selfish.


IT admins who use a person-focused process

We should ensure that all of our work is personable. And collaborative. But "person-focused" here means "sharing only via personal intervention." When I ask them how they solve a problem, they often answer with, "I just walk over to the guy who does it and ask them to fix it." This is seen as Agile, because it's reactionary, and needs no documentation or planning. It does not scale on real-life projects. It is the exact opposite of efficiency. "Just walking over" is an interruption to someone else who may not even manage one of the actual resources related to the issue. Also, she might not even work in the same building or country.  Finally, these types of data-less visits increases the us-versus-them mentality that negatively impacts the collaboration success. Sharing data about an instance is just that: data. It's the status of a set a resources. We can blame a dead router without having to blame a person. Being able to focus on the facts allows us to depersonalize the blame game.


Data will save you

These anti-patterns don't just increase costs, decrease team function, increase risk, and decrease organizational confidence, they also lead to employee dissatisfaction and morale. That leads to higher turnover (see above) and more pressure on good employees. Having the right data, at the right time, in the right format, will allow you to get to the root cause of issues, and better collaborate with others faster, cheaper, and easier.  Also, it will let you enjoy your 3:00 ams better.


Are there other anti-patterns related to collaboration that you've seen when you've tried to motivate cross-team collaboration?  Share one in the comments if you do.

A few years back, I had the privilege of attending InfoSec World in Orlando. I served on a security panel for Tech Field Day Live, where we discussed some game changers in security. It was a strange feeling being there. I don’t think anyone knew who we were or what Tech Field Day was. I was a security guy, at least from my point of view I was. I was sitting next to the former program manager of the CCIE Security, Natalie Timms, Edward Haletky, and Jack Daniel. I had been on Twitter for years, written a few books for Cisco Press, and I still felt out of place.


As I sat at dinner with Stephen Foskett, I came to the conclusion that these security folk were not very social. I think things have loosened up a bit, but it’s still not an easy world to break into. Why is this important? For some of you, it’s not. For me, time and again being on social networking sites has saved my bacon when I found myself in a jam. What about now?


Well, I can’t say that social networking has improved a lot for security professionals. It may have, but I don’t run in those circles. What I can tell you is that security vendors see the value in, and advertise and respond to, direct inquiries. Sometimes their response to social networking questions is days ahead of an email request submitted through a web form.


Still, following a few security accounts on Twitter can’t hurt. Here are a few that I think are beneficial. Trust me when I say that I have followed several, and unfollowed them just as fast. There’s a lot of repetition and noise out there.



Illumio is a security vendor that I learned about while attending a networking field day. They are based out of Sunnyvale, California, and most of their posts are informative and focus on adaptive security for the data center. They have an interesting solution that I should have taken the time to break down a bit more on my personal blog. You can look them up on the Tech Field Day website. The video is worth a watch, as is their Twitter feed.



I’ve followed @infosecuritymag for some time. They also have informative news that tends to be focused on general information security. You’re not going to learn earth-shattering techniques here, but for someone breaking into the world of security, it will give you a variety of topics to discuss with your peers.


Krebs on Security

Brian Krebs is a journalist who frequently covers profit-seeking cybercriminals. He uses his platform to inform on various security topics. He’s become quite a name in the space and can be found on Twitter; he also hosts his own podcast. In late 2016, he was the target of one of the largest DDoS attacks on record. Yep. He ticked off that certain person that made it a point to shut his site, Krebs on Security, down for quite some time.


Cisco Security

If you work with Cisco Security products, you may be interested in following @ciscosecurity on Twitter. The bulk of what you see here is going to be product announcements and write-ups on whatever direction Cisco thinks is important these days. It’s hard to ignore Cisco, even though many will argue the quality of their security portfolio. But when you consider that they house the Talos group, and have tons of customer-provided data that they can analyze, which keeps them up-to-date on emerging threats, it’s worth a little bit of marketing just to get to the good stuff.


Others I’ve Followed

I’ve also followed Bruce Schneier, but after a while his posts don’t really interest me. But you have to admit that the guy is brilliant. Aside from Bruce, there have been others that are more Cisco-specific, but I wouldn’t really recommend them for someone starting out in security.


Final Thoughts

If you are coming from a data networking background and transitioning into a security role, the news is a solid start. Trying to engage individual security professionals as you would networking and virtualization guys is a tough nut to crack. As time goes by, you’ll likely become familiar with peers you meet at conferences and such. Those are the ones to interact with online, if possible. When you find yourself in a bind, these are the folks you want to ask. However you should be aware that the conversation you have may not be in the public eye. Many security professionals are a little but more hush-hush publicly. They understand that certain things that are said could lead people to the wrong conclusions about the organizations they represent. You can’t blame them. The lawyers are going to force their organizations' representatives to be very tight-lipped about possible breeches and such. For example, if you ask a question about some ransomware that you’ve just encountered, and a buddy of yours that happens to work for Wells Fargo tells you how to fix the issue, people may naturally assume that Wells Fargo has experienced a similar breech. Yep, that’s bad for press.


So be patient, interact with those you know personally, and use social networking to keep up on industry trends. Doing so may guide your learning path as you progress as a network security professional.

In my last blog post, I began laying out the case that troubleshooting can be a challenging endeavor, especially when dealing with multiple disciplines (silos) in a larger IT organization. More often than not, it becomes a race to innocence between the groups, with each trying to prove that they are not the problem. The trouble is, nobody is trying to figure out what the problem really is. They just want to know it's not them. As I pointed out, that can largely be attributed to a lack of good tools, though that's less and less of an issue in today's world.


While many organizations have already taken steps to reduce the effect of silos on IT operations, challenges still abound; chief among them is the disparate collection of tools each group uses. Because different tools all represent data in myriad ways, even as cooperation is fostered, roadblocks still remain. Luckily, someone has done something about all of this, and is providing a tool that cuts across multiple disciplines.


Solarwinds' new PerfStack product, a part of the Orion Platform of network monitoring applications, aggregates multiple data points from across the range of products in that suite. As the Orion suite is very widely adopted, centralizing the collected data is a no-brainer. Solarwinds is already deployed in many NOCs, and having the ability to perform cross-discipline troubleshooting from within the same tool should prove invaluable.


While Solarwinds has always been on point with their solutions, and has had systems monitoring dashboards for some time, those dashboards have not necessarily been dynamic. That's not a bad thing, it's just a function of their purpose; NOC monitoring does not typically require quick changes of information displays (dashboards) on a regular basis. Frequently, the dynamic troubleshooting has been handled by various point tools. Perfstack fixes these issues, and more.


PerfStack can now create dashboards on the fly, filled with all of the pertinent pieces of data needed to remediate a problem. More than that, however, they can give another user that same dashboard, who can then add their own bits and bobs. You are effectively building up a grouping of monitoring inputs consisting of cross-platform data points, making troubleshooting across silos seamless in a way that it has never been before.


By way of example, the networking team typically fields initial trouble tickets that involve nebulous symptoms of "can't get there from here," or "something, something, really slow." That team would use whatever point tools at their disposal to look for potential issues, beginning at layer-1 with cabling, optics, etc., and progressing, as needed, up through the stack until a problem is found. At some point, assuming the problem is elsewhere (c'mon, it's never really the network) the trouble ticket gets passed off to security, perhaps, or applications, storage, wherever. But at that point all historical information in the point system is unavailable to the new team who must start from scratch. That is inefficient at best. PerfStack flips that modality on its head, and allows teams to share data from the initial investigation through to an eventual resolution.


With everything in IT connected at such a fundamental level so as to be inseparable on any practical level, the era of point solutions really should be relegated to the IT trash bin, an anachronistic relic of our past. PerfStack is one of the first serious products, built upon a proven technology platform, to usher in this new paradigm in root-cause analysis and remediation. Solarwinds was uniquely positioned to create this product, and while it would disingenuous to say that they've invented this new space, they have certainly capitalized on a need, executed a flawless solution, and dramatically accelerated the time-to-value over what I can now safely call legacy approaches.

In my previous two posts, DDoS and The Broken Internet and The Internet of Hacked Things, we discussed how there are some critical flaws in key services and internet infrastructure that easily allow attackers to cripple large portions of the internet, as well as highlighting how IoT is really the Internet of Vulnerable Things as manufacturers and consumers fail to properly secure these devices. These devices are then easily gathered into massive botnets, which can then be used to exploit the cracks in our infrastructure’s armor.


For all we know, Mirai, or some other massive botnet, could suddenly become self-aware, and then we’re on the path to Skynet and Judgement Day, right?


Now we know what the problem is. So, how do we fix it before we are forced to bow to our IoT overlords?


Who is ultimately responsible?


Some of you wondered if this was an engineering versus marketing problem. The classic battle between engineers and/or designers who want their product to be polished and ready for market, and marketing execs who just want the revenue from getting the product on the shelves as quickly as possible, a few flaws be damned. Nobody can deny the rapid and unfettered uptake of these devices, which are now in the wild numbering several million.


Others blamed the consumer, the person purchasing and installing the IoT devices without “reading the fantastic manual” and properly securing the device from the treacheries of the internet. A problem borne of ignorance. How could we expect the average non-techie consumer to be aware of SSL vulnerabilities?


Security of the IoT, and key internet services as a whole, has to be handled with a layered approach. The mandate of everyone involved, from the designer, manufacturer, consumer, and network operator, should be to ensure that they each are doing their part to secure their piece of this puzzle from continuing to allow massive DDoS attacks to destabilize large sections of the internet.




Designers have to balance ease of use and convenience with security. While it might be convenient to have Telnet and HTTP services on a device, so users can easily configure or manage these IP-enabled devices, it’s irresponsible to have these types of unsecured services exposed to the internet, often with well-documented default passwords. At a minimum, encrypted services like SSH and HTTPS should be used, and the use of default passwords should require a forced change. Hard-coded passwords that cannot be changed are a major foul. Ensuring the use of the most recent and patched versions of these protocols should also be required.




What about a governing body, like the FCC in the United States, or the CRTC in Canada, getting involved to qualify these devices? We’ve all seen the FCC/CRTC stickers on wireless devices, showing the devices meet standards set forth by these agencies, certifying them for use under proper regulations. Why can’t we apply this model to IoT devices, and require they are put through a proper vetting and testing process before being allowed to market? This would keep designers and marketing execs accountable for the products they are putting on shelves, and give the average consumer some comfort in knowing they aren’t inadvertently adding to the problem when purchasing one of these devices.


Service Providers


Service Providers of all types (Internet, DNS, Hosting) need to ensure their infrastructure is not only secure, but resilient, and able to mitigate DDoS attacks before they ever impact their customers. This means investment in their infrastructure and careful monitoring of traffic flow across and through their networks. Top tier providers really have no excuse, they know they are high priority targets, and they should do everything in their power to prevent the negative impact of these attacks.


Enterprise Networks


Enterprise network operators rely heavily on their Service Provider partners to perform their due diligence and clean up a lot of the internet trash thrown their way before it even hits their network edge. That doesn’t remove their responsibility, however, and enterprise networks should be designed and built with as much resiliency as possible, so that if one of their service providers is attacked, or any attack traffic gets through directly to them as a target, that there are backup services in place to allow uninterrupted business traffic flow.


Don’t be part of the problem! Monitoring your own network for compromised endpoints is also critical. Tools such as Solarwinds’ Long & Event Manager can assist with identifying traffic aimed at botnet CnC servers, and identify compromised devices that you can then patch or remove.


A microcosmic layered approach is beneficial as well. A first line of defense that includes cloud services that scrub and pre-filter traffic, followed by deeper levels and layers of defense, such as load balancers, firewalls, IPS, and application filters/firewalls, are all pieces of the protection puzzle that should be integrated.




It’s unlikely that the folks behind massive DDoS attacks like the ones aimed at Brian Krebs, Dyn, or Akamai are going to get bored and stop on their own. It’s equally unlikely that we’ll ever find a single magic bullet that fixes all of the issues with our underlying infrastructure. So, we need to plan for these flaws and ensure that everyone involved has some measure of protection, and that the majority of the malicious traffic can get dumped or filtered out before it has any major impact to the end-user or customer. This can only be accomplished with a multi-faceted approach at all levels of the internet.


We’ve seen attacks over 600Gbps, and current projections of the Mirai botnet speculate that attacks exceeding 1Tbps are presently possible. It seems to be an arms race, and the only way to stop the attackers from continuing to gain strength (literally) in numbers, is to secure IoT as much as possible and limit the attack surface of our own networks.


Are there any other individuals and/or groups that need to be held accountable for the security of IoT?

Back from Germany and on vacation this week. But I'm not going to let a few days off get in the way of browsing the internet for things to share with you. Enjoy!


Mobile Ransomware Jumps 50% in a Year

The article focuses on Android devices, which have always had quality control concerns for apps in their store. Say what you will about Apple, but control over their store leads to lower risk for consumers. And Microsoft doesn't have this issue because they have no apps.


Trump and His Android Phone Putting National Security at Risk

I have no idea why we don't simply take away his toys. It's almost as if Trump has no idea about the risks involved.


German parents urged to destroy data-collecting toy doll

As if having my refrigerator and television spying on me wasn't enough to worry about. Now I have to worry about dolls.


Why buying used cars could put your safety at risk

"The car is smart, but not smart enough to know who it's owner is." Classic design fail here, as the developers never thought about the idea that a car may have more than one owner.


Poor Access Management Leads to $5.5M HIPAA Penalty

Show this to the next person that wants to know why you care so much about data security. Well, that is, if you do. And you should.


Amazon Takes on Skype For Business With Chime But It Won’t Come Cheap

Watching Microsoft and Amazon compete head-to-head on each and every product possible tells me one thing: Amazon is the new Apple, but without the overpriced hardware and underwhelming software. The next 30 years should be fun to watch as Microsoft and Amazon compete for our dollars.


Our workshop was full last week, with 50 people grouping together to do hands-on lab exercises with SQL Server 2016 running on Azure VMs:


I wrote last week about how performance troubleshooting is a problem with multiple dimensions. From the client, across the network, and into the application server, there are many places where application performance can be impacted. One of the key parts of performance troubleshooting is sorting through all those dimensions, then finding the one that is limiting application performance. Unfortunately, there are many more dimensions when your applications are inside virtual machines. Understanding and considering these dimensions is critical to performance troubleshooting in a virtual environment.


The first place we add more dimensions is between the VM and the hypervisor. VM-sizing as well as virtual HBA and NIC selection all play a part in application performance. Then there is the hypervisor and its configuration. A lot of the same issues that affect operating systems also effect hypervisors. Are updates applied? Is the storage configuration correct? How about NIC teaming? Even simple things like storage vendor recommended optimization can make a big difference to the performance of applications in the VMs.


The next dimension is that multiple VMs share a single physical server. So your application’s performance may be dependent on the behavior of other VMs. If another VM, or group of VMs, uses most of the physical CPU, then your application may run slow. If your VM resides on the same storage as a bunch of VDI desktops, then storage performance will fluctuate. We see this noisy neighbor problem most often in cloud environments. But noisy neighbors absolutely can happen in on-premises virtualization. A particular challenge is the invisibility of hypervisor issues to the VM and its applications. The operating system inside a VM will report the clock speed of the underlying physical CPU, even if it is only getting a small fraction of the CPU cycles. A VM that uses 100% of its CPU is not necessarily getting 100% of a CPU, it is just using 100% if what it gets from the hypervisor.


There is a time dimension here, too; one related to the portability and mobility of VMs. Your application server may move from one virtualization host to another over time. So can other VMs. The noisy neighbor VM that caused a performance issue half an hour ago may be nowhere near your VM right now. On most hypervisors, VMs can also move from one piece of storage to another. That performance issue yesterday might have triggered an automated movement of your VM to a better storage location. The migration made the performance problem disappear. This VM mobility can lead to performance issues being intermittent, they only manifest when a specific other VM is on the same physical host.


Another virtualization dimension is when the user’s PC is really a VM. VDI adds more dimension to performance. There is a whole network between the user and their PC, as well as another device right in front of the user that may bring its own performance issues. Users seldom have the right words to differentiate a slow VDI session from a slow application. All of the noisy neighbor issues we see with server VMs are multiplied with desktops.


Virtualization has enabled many awesome changes in the data center. But virtualization has added serious complexity to performance troubleshooting. There are so many dimensions to understand to find and resolve the restricting dimension.

By Joe Kim, SolarWinds Chief Technology Officer


In World War I, the U.S. Army used lumbering GMC® trucks for the first time in combat, which was revolutionary for its time. Today, these vehicles would be considered slow, cumbersome, and archaic in comparison to today's fast, powerful and, most of all, constantly connected war-fighting machines.


In fact, thanks to the Internet of Things (IoT), just about everything that can be connected—from tanks to smartwatches—is connected. The Defense Department’s entire work force depends on thousands of devices that work off of disparate operating systems. The net result is a security risk nightmare for those who must secure government IT networks.


IoT has made establishing a strong security posture much more challenging for beleaguered IT administrators. Still, practicing a few simple strategies can help these administrators beef up security in the face of the IoT onslaught.


Step One: Build security in from the beginning


Last year, the Federal CIO Council’s Mobile Technology Tiger Team released standardized security protocols for agencies that build their own mobile apps. The protocols outline the need to vet applications by building security and functionality together throughout the app development process.


For federal IT administrators, security must be interwoven into the fabric of agency networks. This starts with strategic planning—considering every possible breach scenario, identifying potential threats before they occur, and responding to emergencies. Attention must continue with the deployment of automated tools that scan for and alert users to threats as they occur. Solutions that offer automated, round-the-clock monitoring and real-time notifications help administrators react more quickly to potential threats and mitigate damage.


Step Two: Assess security risks associated with every app


Protection against external applications that constantly track and collect data over secure networks require administrators to be particularly vigilant in the types of apps and devices they allow over these networks.


Managers can create “white lists” of approved apps, and use monitoring tools to alert whenever an unauthorized app requests network access. They can also track those applications back to individual users if necessary.


Step Three: Do the same for devices


IoT takes us well beyond smartphones and tablets into a new realm of connected tools that might not yet be accepted, and administrators must closely monitor the devices accessing the networks. They might permit smartphones and tablets on the network, provided they meet security standards, while eschewing untested or non-essential devices. Simultaneously, they should set up a system to track devices by MAC and IP address, and monitor the ports and switches that those devices use.


Mobile technology has brought us some great things: the ability for fighters to easily communicate and access information from anywhere in the field, opportunities for greater productivity, and collaboration across agencies. But it’s also brought a great deal of headaches. With the IoT here, those headaches could potentially turn into continuous migraines. It is helpful to view the aforementioned strategies as the IT equivalent of an Excedrin tablet: a way to ward off the pain and secure the network before it gets out of control.


Find the full article on Signal.

When a SysAdmin implements a DevOps approach to an environment, there are many things that need to be considered. Not least of which is developing a strong line of communication between the developer and the SysAdmin. Occasionally, and more frequently as time goes on, I’m finding that these people are often one and the same, as coders or scripters are spreading their talents into the administrator’s roles. But these are discreet skills, and the value of the scripter will augment the skills and value of the system administrator.


Process is a huge component of the SysAdmin's job. The ordering process, billing process, build process, delivery process. How many of those tasks could be automated? How easily could these functions be translated to other tasks that are conscripted, and could therefore be scripted?


The wonderful book, The Phoenix Project, is a great example of this. It outlines how an IT manager leverages the powers of organization, and develops a process that helps facilitate the recovery of a formerly financially strapped industry leader.

In the book, obvious communications issues initially bred the perception that IT was falling behind in its requisite tasks. Systems, trouble tickets, and other tasks were not being delivered to the requestor in any sort of accurate time frame. As the main character analyzed both the perceptions and realities of these things falling short, he began questioning various stakeholders in the company. People who were directly affected, and those outside his realm, were solicited for advice. He also evaluated processes within the various groups of the organization, found out what worked and why, then used his successes and failures to help him to establish assets worth leveraging. In some cases, those assets were skills, individuals, and personalities that helped him streamline his functions within the organization. Gradually, he was able to skillfully handle the problems that arose, and fix problems that had been established long before.


A number of key takeaways from the character's experiences in this series of actions illustrate what I was trying to outline. For example, the willingness to question yourself and your allies and adversaries in a given situation, with the goal of making things better. Sometimes it takes being critical in the workplace, which includes scrutinizing your successes and failures, may be the best way to fix issues and achieve overall success. 


What does your process look like for any given set of tasks? How logical are they? Can they be eased, smoothed, or facilitated faster? Evaluating the number of hours it takes to perform certain tasks, and then critically evaluating them, could potentially improve metrics and processes. As I see it, the scriptability of some of these tasks allows us to shave off precious hours, improve customer perception, and gain credibility again. Being willing to embrace this new paradigm offers countless benefits. 


It all begins with communication. Don’t be afraid to allow yourself to be seen as not knowing the answer, but always be willing to seek one. We can learn from anyone, and by including others in our evaluation, we can also be perceived as humble, collaborative, and productive.


Last year I wrote about my experiences at CiscoLive Berlin (or #CLEUR, as it is hashtagged on Twitter).


This year, as I'm sitting on the plane to meet up with patrick.hubbard, ding, stevenwhunt, and the rest of the gang, I thought I would take a few minutes before catching some fitful transatlantic naps and share some of the things I'm looking forward to seeing.


Sharing the PerfStack story

Without a doubt, the thing I'm MOST excited to share with the attendees is PerfStack and the way it allows network engineers to combine data from across the infrastructure and compare it with application insight. This isn't just your usual "MTTI / It's never the network" gag. This is about enabling every IT professional to become a true "full-stack" engineer, and even a data scientist, snapping together data from across the enterprise the way Master Builders snap together "interlocking plastic building pieces" in the Lego Movie.



A close second to PerfStack is my excitement to see what the Arduino booth has in store this year. As I mentioned in my summary from 2016, this activity captured my imagination like nothing that I've seen at a convention in a long time. I'm hopeful they up their game, but even if they just do a repeat of last year's side-quest, I'll be happy to play along. It was the ultimate cross-vendor ice-breaker.



So much has happened in the world of IoT this year (which frequently finds its way onto the pages of Geek Speak), that I'm keen to see how Cisco, the speakers, and even the vendors on the floor are responding. To be certain, we can't keep doing the same old things. But HOW everyone is choosing to respond will be as interesting as anything else.


Clouds and Containers

Finally, the inexorable march continues from pure on-premises through the haze of hybrid IT, to the eventual nirvana of cloud-based everything. As much as this has been a "virtualization, server, and application" type story, network (and Cisco) have had their roles to play and will continue to drive the conversation. I haven't paid as much attention to this sector of the industry as I should, leaning heavily on the experience and insight of fellow Head Geeks kong.yang and sqlrockstar, but it's time for me to start building my knowledge in this area.


That's what I'm looking forward to. Is there something YOU are particularly keen to hear about? Let me know in the comments and I'll try to do some of the legwork for you and report back next week!

Nobody is safe.


From the highest tier service provider to a small business network in rural Iowa, every network is susceptible to a massive scale DDoS attack. Whether as a direct target of the attack, or indirectly as critical services are affected elsewhere on the internet.


In my previous article, DDoS and the Broken Internet, I outlined how the recent attack on Dyn highlighted the interconnected services that we have all come to rely on as network operators, and how their service interruptions affect all of us.


This highlights a major flaw in these networks, and largely in the internet as a whole. And yet, we keep adding to it, adding devices that previously had no need for internet access, yet the Internet of Things (IoT) is here, and it’s exacerbating the flaws in our infrastructure.


On Tuesday, September 20th 2016, Akamai (the world’s largest content delivery network) defended against a DDoS attack that was in excess of 620Gbps. This attack was perpetrated largely by the Mirai botnet, malware that has amassed an army of IP-enabled devices with common open ports on the internet – Telnet, SSH, HTTP, SMTP, etc. These are not malware-infected home computers, mind you. These are vulnerable connected devices like cameras and DVRs. Devices that are either vulnerable to SSH exploits, or don’t use SSH at all, and often use default credentials.


Akamai identified the top protocols used in the attack as SSH and Telnet, both very common protocols. The top usernames used were root, admin, and shell. All common and well-documented defaults. How could so many of these devices be so easily compromised? Many of these devices are purchased and used by consumers who are often unaware of the risks of leaving these settings at their defaults, and, as a result, they are left vulnerable and are easily assimilated into these massive botnets.


To make matters worse, some of the identified IoT devices had hard-coded credentials that could not be changed!


SSH vulnerabilities have been around for a long time, and as new exploits are detailed, patches are released and updates are required. Many of these IoT devices don’t get updates. They are installed by the user and then simply left. Who has time to do a firmware update on the home security camera system they bought at Costco, right?


The attack on DNS provider Dyn in October was also at least partially the responsibility of Mirai and the hacked IoT devices. This attack had far-reaching effects, causing a large number of high profile websites to be inaccessible, including Twitter, Spotify, and Reddit, among others.


The number of IoT devices is increasing exponentially, and with it, the attack potential of massive botnets like Mirai.


How can we secure the Internet of Things, or protect our infrastructure from them?

There is a reason why resolving application performance issues is so hard. Actually, I think there are a lot of reasons, but I’m going to spend a few posts looking at one set of reasons.


A lot of human endeavors are linear: move in one direction until you reach your goal. Troubleshooting application performance is non-linear. It has a lot of different, yet interrelated dimensions. The only way to resolve application performance issues is to address the dimension that is limiting performance. But with the interrelationships between the dimensions, the symptom may be far removed from the constraining dimension. The cause of a performance issue can be a long way from its visible effects.


One dimension is the performance of the user interface. Whether it is a native application or a web browser, every application has a User Interface (UI). Since the UI is close to the user, it is what they perceive as the application. Something as simple as a bug in Adobe Flash or a laptop that hasn’t been rebooted in months can lead to a poor user experience. Of course, the application that is slow for the user may not be the cause. Another application taking up 99% of their laptop’s CPU can be the bully, the application that they are trying to use may be a victim. Just inside the user’s computer, there are multiple performance dimensions.


The next dimension is the network between the user’s device and the application servers. From our desks outside the data center, the network can be very fast. But if your users are on the end of a busy WAN circuit, or over a VPN, or half way around the world,j then they will have a very different experience. What about load balancers, firewalls, and even simple things like name resolution? Having an out of date primary DNS server can add a minute to the application’s initial load time. The network adds still more dimensions to the troubleshooting problem.


Then there are dimensions on the application server. Does it have enough RAM and CPU? Are its disks up to the load of the application? What about the optimization of the operating system? Have the drivers for the storage and network adapters been updated to the latest? Or have they been updated to a faulty release? Is the cause of the performance problem a periodic scan by the anti-virus software? The application server adds more dimensions to any performance problems.


Maybe there is a time dimension? End-users in another country may access the application server at an unusual time. Are there maintenance tasks happening that might make the application slow at exactly the time that your European users need to get their end of day processing done? With all these possible dimensions to be investigated, it is no surprise that performance troubleshooting can take some time.


Performance troubleshooting has many dimensions. Until you identify the limiting dimension you will never fully resolve the performance problem. The number of dimensions increases substantially when you add virtualization to the mix. Next week I will look at some additional dimensions that you need to consider when virtualization comes into play.

There are several security certifications that one can choose from. While the list is long, we're primarily going to touch on five of them here. But for good measure and simply to prove our point, here's a more extensive mound of security certifications that sit before you.


CompTIA Security+

The CompTIA Security+ certification has been around for a long time and is a well-recognized and respected certification in the field. In fact, it meets the ISO 17024 standard and is approved by U.S. Department of Defense to fulfill Directive 8570.01-M requirements. That being said, this certification is more entry level than anything else. You can find the details on the CopmTIA web site. This certification is going to provide you with understanding in the following areas:

  • Threat management
  • Cryptography
  • Identity management
  • Security systems
  • Security risk identification and mitigation
  • Network access control
  • Security infrastructure

All of these areas are powerful in terms of what would be useful in a production environment. You'll probably want to have the Network+ certification first, or at least hold that level of knowledge before this material can fully sink in.


Would the CompTIA Security+ Certification Benefit Me?

If you're in a government job and need to meet certain standards, this certification may prove to be useful.  If you're a newbie to security, this certification will likely offer you a good introduction to security, but many hiring managers understand that this is an introductory certification. This is probably not the kind of certification that's going to dress your resume up enough to demand the big bucks, but it can't hurt to have it. Time learning is usually not time wasted.


GSEC: SANS GIAC Security Essentials

This is another entry-level security course, but it's designed a bit differently. This course is designed to demonstrate hands-on capability in security administration. The certification is good for four years before you need to renew it, and it is much more expensive compared to the Security+. Whereas the Security+ certification will cost you $320.00 USD, the SANS GIAC Security Essentials exam will run you just over $1200.00 USD.

You can find the details on the Web site.

Topics covered by this certification include:

  • Identifying and preventing common attacks
  • Identifying and preventing wireless attacks
  • Access controls
  • Authentication and password management
  • DNS Security
  • Cryptography fundamentals
  • ICMP Security
  • IPv6 Security
  • Public key infrastructure
  • Linux security
  • Network mapping


Would the GSEC: SANS GIAC Security Essentials Certification Benefit Me?

For a lot of people, hands-on is the way to go. In fact, the CCIE Certification Program offered by Cisco has been seen as one of the most credible certifications to hold. Much of that has to do with the fact that it's a hands-on certification, which has the benefit of credibility. If you've passed one of these exams, you must know how to do whatever you were tested on. So if you want to break in at the entry level with a bit more than a sheet of paper, this is the cert for you.


Certified Ethical Hacker (CEH)

The CEH certification is a common certification that is considered intermediate-level. It's not uncommon for organizations to request network security assessments. The CEH certification is a key certification that companies engaged in this type of offering look for.  This certification teaches you the same techniques that hackers use.  Armed with this knowledge you would then be better positioned to identify threats as they come across the network.

Some areas touched on in this certification include:

  • Reconnaissance
  • Scanning networks
  • Enumeration
  • Trojans, worms and viruses
  • Sniffers
  • Denial-of-Service attacks
  • Session hijacking
  • Hacking web servers, wireless networks, and web applications
  • SQL injection
  • Cryptography
  • Penetration testing
  • Evading IDS, firewalls, and honeypots

As you can see, the list is a bit more extensive than the Security+ certification. You'll need to have that general security knowledge before you take on a certification like this. This is another intermediate certification.


Would the CEH Benefit Me?

If you want to be an ethical hacker, this certification is a must. If you want to be a Cyber Security Analyst working in a Security Operations Center, this certification is also valuable because it lets you identify potentially malicious activity much easier than if you didn't have this underlying knowledge.  At the end of the day, I see a lot of people get this for the fun of it rather than to advance their career, but employers still recognize the certification. In specialized environments, they look for it.


Certified Information Systems Security Professional (CISSP)

The CISSP is an advanced-level certification. It's vendor neutral and is one of the certs that's been around the longest. It's been on the "Certifications Most-wanted" list within organizations for many years. Those that hold the CISSP are usually Senior Security Personnel and thus make a bit more cash. Some of the topics you'd be tested on include:

  • Risk management
  • Access control
  • Application security
  • Cryptography
  • Security architecture and design
  • Investigation and ethics


Would the CISSP Benefit Me?

If you have a minimum of 5 years experience in two of what the (ISC)2 called a Common Body of Knowledge domain, or 4 years experience and a college degree, this is your cert.  That's because these are the requirements to obtain this certification. But what are the domains you ask? They are Security and Risk Management, Asset Security, Security Engineering, Communications and Network Security, Identity and Access Management, Security Assessment and Testing, Security Operations, and Software Development Security.


Certified Information Security Manager (CISM)

The CISM certification is designed for anyone that's going to be managing, developing, and overseeing information security systems. This is a newer certification on the scene, but what sets it apart is that its geared toward maintaining the highest quality standards when it comes to audit, control, and security of an organization's security systems. It's not an entry-level certification either. This certification is designed for one with experience. The requirements for this certification include:

  • Agree to ISACA's code of professional ethics
  • Pass an exam
  • Have 5 years experience
  • Comply with a continuing education policy
  • Submit a written application

As you can tell, there's a bit of work included in just obtaining the certification, and that's not counting the actual security knowledge you need.


Would the CISM Benefit Me?

The CISM is a bit more expensive compared to other certifications. If you have the money, have the time, and can meet the requirements, then holding this certification is extremely beneficial.  Hiring managers recognize the certification, and when you combine it with experience, the Infosec Institute ranges the pay from $52,402 to $243,610.  Yes that's a very wide range, but you have to factor experience into the mix. An entry-level position isn't going to pay top dollar, no matter what certification you hold.


Final Thoughts

At the end of the day it's up to you. How much time to you want to commit to certifications vs hands-on experience?  Are you even looking for a job? I knew a guy that had about 40 different certifications and the only reason he got them is because he was bored at work. He had no intention of leaving his high-paying job that was paying for him to become certified. Especially when he didn't have much to do when he did have to work.


Still, one should recognize that employers try to filter through potential candidates, and having a security certification can help shuffle your resume to the top. If you get that far you'll have to prove that you know your stuff in an interview, and that's a whole other conversation.

A few years ago I was working on a project as a project manager and architect when a developer came up to me and said, "You need to denormalize these tables…" and he handed me a list of about 10 tables that he wanted collapsed into one big table. When I asked him why, he explained that his query was taking four minutes to run because the database was "overnormalized." Our database was small: our largest table had only 40,000 rows. His query was pulling from a lot of tables, but it was only pulling back data on one transaction.  I couldn't even think of a way to write a query to do that and force it to take four minutes. I still can't.


I asked him to show me the data he had to show me the duration of his query against the database. He explained that he didn't have data, he had just timed his application from button push to results showing up on the screen. He believed that because there could be nothing wrong with his code, then it just *had* to be the database that was causing his problem.


I ran his query against the database, and the results set came back in just a few milliseconds. No change to the database was going to make his four-minute query run faster. I told him to go find the cause that was happening between the database and the application. It wasn't my problem.

He eventually discovered that the issue was a complex one involving duplicate IP addresses and other network configuration issues in the development lab.


Looking back on that interaction, I realize that this is how most of us in IT work: someone brings us a problem, ("the system is slow"), we look into our tools and our data and make a yes-or-no answer about whether we caused it. If we can't find a problem, we close the ticket or send the problem over to another IT group. If we are in the database group, we send it over to the network or storage guys. If they get the report, they send it over to us. These sort of silo-based responses take longer to resolve, often lead to a lot of chasing down and re-blaming. It costs time and money because we aren't responding as a team, just a loose collection of groups.

Why does this happen?

perfstacksingle.pngThe main reason we do this is because typically we don't have insights into anyone else's systems' data and metrics. And even if we did, we wouldn't understand it. Then we throw in the fact that most teams have their own set of specialized tools and that we don't have access to. I had no access to network monitoring tools nor permissions to run any.  It wasn't my job.


We are typically measured and rewarded based on working within our own groups, be it systems, storage, or networks, not on troubleshooting issues with other parts of infrastructure.  It's like we build giant walls around our "stuff" and hope that someone else knows how to navigate around them. This "not my problem' response to complex systems issues doesn't help anyone.



What if it didn't have to be that way?


Another contributing factor is the intense complexity of the architecture of modern application systems. There are more options, more metadata, more metrics, more interfaces, more layers, more options than ever before. In the past, we attempted to build one giant tool to manage them all. What if we could still use specialty tools to monitor and manage all our components *and* pull the graph of resources and their data in one place so that we could analyze and diagnose issues using a common and sharable way?


True collaboration requires data that is:

  • Integrated
  • Visualized
  • Correlated
  • Traceable across teams and groups
  • Understandable


That's exactly what SolarWinds' PerfStack does. PerfStack builds upon the Orion Platform to help IT pros troubleshoot problems in one place, using a common interface, to help cross-platform teams figure out where a bottleneck is, what is causing it and get on to fixing it.



From <>


PerfStack combines metrics you choose from across tools like Network Performance Monitor Release Candidate @network  and Server & Applications Monitor Release Candidate from the Orion Platform into one easy-to-consume data visualization, matching them up by time. You can see in the figure above how it's easy to spot a correlated data point that is likely the cause of less-than-spectacular performance your work normally delivers. PerfStack allows you to highlight exactly the data you want to see, ignore the parts that aren't relevant, and get right to the outliers.


As a data professional, I'm biased, but I believe that data is the key to successful collaboration in managing complex systems. We can't manage by "feelings," and we can't manage by looking at silo-ed data. With PerfStack, we have an analytics system, with data visualizations, to help us get to the cause faster, with less pain-and-blame. This makes us all look better to the business. They become more confident in us because, as one CEO told me, "you all look like you know what you are doing." That helped when we went to ask for more resources


Do you have a story?


Later in this series, I'll be writing about the nature of collaboration and how you can benefit from shared data and analytics in delivering better and more confidence-instilling results to your organization. Meanwhile, do you have any stories of being sent on a chase to find the cause of a problem?  Do you have any great stories of bizarre causes you've found to a systems issue?

As a longtime IT professional, responding to problems in systems of one type or another has become old hat. It comes with the job description, and one tends to develop habits and methodologies early on in your career. That doesn't mean that I, or anyone else, have developed the best habits, however, and often our methods are quite ineffective indeed.


Standard practice in the industry is for a network operations center (NOC) to monitor some portion of the network for immediate or impending troubles. Companies spend millions of dollars on entire rooms filled with beautiful monitors mounted on walls, desks and workstations built to look as futuristic as possible, low lights at just the right hue, and comprehensive monitoring suites to keep track of it all. The trouble is, this often makes people aware of a problem, but offers nothing in the way of a troubleshooting methodology or tool to actually fix the problem.


As often as not, when some sort of event worth responding to grabs the attention of a NOC engineer, they either call someone or start a trouble ticket, or both. The lucky recipient of the aforementioned prodding then digs into the problem or passes it onto the next person in the chain, with each successive person having to start over in their own domain (compute, network, security, etc.) with new tools and limited information.


This entire approach may seem logical and even expedient, though I suspect that's largely due to a little bit of Stockholm Syndrome and the ever popular "but this is the way we've always done it" argument. I'm not saying that this is a bad approach--or at least that it hasn't always been a bad approach--given the historical dearth of cross-silo troubleshooting tools available on the market. Most of us instinctively knew that this was inefficient, but didn't have a good sense of what we could do about it.


Various tools and paradigms were suggested, developed, sold, and subsequently put on shelves that attempted to fix the full-stack troubleshooting void. Comprehensive network tools are one of the favorites, offering a truly staggering array of dashboards, widgets, alerts, and beautiful graphics in a noble attempt to present the most information possible to the engineers tasked with fixing the relevant problems. Many tools also exist for doing the same thing inside of virtual environments, or on storage arrays, or the cloud, etc., and many are very good at what they do. But they don't do what we need, which is to collapse the silos between IT disciplines into one unified system - until now.


Solarwinds NPM, part of the Orion Platform suite of products, has long been the darling of NOCs everywhere, and with good reason. It is a comprehensive and well-thought-out approach to network and systems monitoring. Collapsing the silos in IT, however, requires more than just a great tool for the NOC, or even a great tool for the network and systems teams. It

requires a tool that is not only useful for all of these teams, but preserves the chain of data (of troubleshooting) as it moves between specialties. In other words, if I'm the systems guy, I want to see the data that the network team is seeing, and the steps they've taken to resolve the problem. The thing is, I want to see it in the system, not a hastily or poorly-crafted email, which is the equivalent of tossing a flaming bag of excrement over the wall on our way out.


NPM 12.1 has taken a stab--a good stab--at solving these problems with the inclusion of a tool called PerfStack. I'll be exploring what this tool can do, and where in the troubleshooting process it fits, in a series of blog posts over the coming weeks. I'll likely also toss in some of my own personal horror stories of troubleshooting problems, as I've had more than my fair share in my past, and confession is cathartic. In the meantime, I'd encourage everyone to check out this already fantastic series of posts on the new tool:

Filter Blog

By date: By tag: