Skip navigation
1 2 3 4 Previous Next

Geek Speak

1,920 posts

I’ve come to a crossroads. Regular SolarWinds Lab viewers and new THWACKcamp attendees might have noticed my fondness for all things programmable. I can’t help smiling when I talk about it; I came to enterprise IT after a decade as a developer. But if you run across my spoor outside of SolarWinds, you may notice a thinly-veiled, mild but growing despair. On the flight back from Microsoft® Ignite last week, I reluctantly accepted reality: IT, as we know it, will die.


Origin of a Bummer


On one hand, this should be good news because a lot of what we deal with in IT is, quite frankly, horrible and completely unnecessary. I’m not referring to managers who schedule weekly all-hands that last an hour because that’s the default meeting period in Outlook®. Also not included are 3:00am crisis alerts that prompt you to stumble to the car with a baseball hat because the issue is severe enough to take out the VPN, too. Sometimes it’s okay to be heroic, especially if that means you get to knock off early at 2:00pm.


The perennial horror of IT is boredom. Tedium. Repetitive, mindless, soul-crushing tasks that we desperately want to remediate, delegate, or automate, but can’t because there’s no time, management won’t let us, or we don’t have the right tools.


All of this might be okay, except for two things: accelerating complexity and the move to the cloud. The skinny jeans-clad new kids can’t imagine any other way, and many traditional large enterprise IT shops also hit the cloud hookah and discovered real benefits. Both groups recognized dev as a critical component, and their confidence comes from knowing that they can and will create whatever their IT requires to adapt and realize the benefits of new technology.


No, the reason this is a bummer – if only for five-or-so-years – is that it’s going to hit the people I have the greatest affinity for the hardest: small to medium business IT, isolated independent department IT in large organizations, and superstar admins with deep knowledge in highly specialized IT technology. In short, those of you who’ve worn all the hats I have at one point or another over the last two decades.


I totally understand the reasonable urge to stand in front of a gleaming Exchange rack and tell all the SaaS kids to get off your lawn. But that’s a short-term solution that won’t help your career. In fact, if you’re nearing or over 50, this is an especially critical time to be thinking about the next five years. I watched some outstanding rack-and-stack app infrastructure admins gray out and fade away because they resisted the virtualization revolution. Back then, I had a few years to get on board, gain the skills to tame VMs, and accelerate my career.


This time, however, I’m actively looking ahead, transitioning my education and certification, and working in production at least a little every week with cloud and PaaS technology. I’m also talking to management about significant team restructuring to embrace new techniques.


Renewed Mission


Somewhere over Louisiana I accepted the macro solution that we’ll each inevitably face, but also my personal role in it. We must tear down IT as we know it, and rebuild something better suited to a data center-less reality. We’ll abandon procedural ticket-driven change processes, learn Sprints, Teaming, Agile, and, if we’re lucky, get management on board with a little Kanban, perhaps at a stand-up meeting.


And if any or all of that sounds like New Age, ridiculous mumbo jumbo, that’s perfectly okay. That is a natural and understandable reaction of pragmatic professionals who need to get tish done. My role is to help my peers discover, demystify, and develop these new skills. Further, it’s to help management stop thinking of you as rigidly siloed and ultimately replicable when new technology forces late-adopting organizations into abrupt shifts and spasms of panicked change.


  But more than that, if these principles are even partially adopted to enable DevOps-driven IT, life is better. The grass really is greener. I’ve seen it, lived it, and, most surprising to this skeptical, logical, secret introvert, I’ve come to believe it. My job now is to combine my fervor for the tools we’ll use with a career of hard-won IT lessons and do everything I can to help. Don’t worry. This. Is. Gonna. Be. Awesome.

Simplifying network management is a challenging task for any organization, especially those that have chosen a best of breed route and have a mix of vendors. I ask my customers to strive for these things when looking to improve their network management and gain some efficiency.


  1. Strive for a Single Source of Truth—As an administrator there should be a single place that you manage information about a specific set of users or devices (e.g. Active Directory as the only user database). Everything else on the network should reference that source for its specific information. Multiple domains or maintaining a mix of LDAP and RADIUS users makes authentication complicated and arguably may make your organization less secure as maintaining these multiple sources is burdensome. Invest in doing one right and exclusively.
  2. Standardization—A tremendous amount of time savings can be found by eliminating one-off configurations/sites, situations, etc. An often overlooked part in this time savings is in consulting and contractor costs, the easier it is for an internal team to quickly identify a location, IDF, device, etc. the easier it will be for your hired guns as well. A system should be in place for IP address schemes, VLAN numbering, naming conventions, low voltage cabling, switch port usage, redundancy, etc.
  3. Configuration Management—Creating a plan for standardization is one thing, ensuring it gets executed is tougher. There are numerous tools that allow for template-based configuration or script-based configuration. If your organization is going to take the time to standardize the network, it is critical that it gets followed through on the configuration side. DevOps environments may turn to products like Chef, Puppet or Ansible to help with this sort of management.
  4. Auditing and Accountability—Being proactive about policing these efforts is important and to do that some sort of accountability needs to be in place. This should happen in change control meetings to ensure changes are well thought out and meet the design standards, safeguards are in place to ensure the right people are making the changes and that those changes can be tracked back to a specific person (no shared “admin” or “root” accounts!) to help ensure that all of the hard work put in to this point is actually maintained. New hires should be trained and indoctrinated in the system to ensure that they follow the process.


Following these steps will simplify the network, increase visibility, speed troubleshooting, and even help security. What steps have you taken in your environment to simplify network management?  We’d love to hear it!

With Data breaches and insider threats increasing, a vulnerable network can be an ideal entry point that puts sensitive data at risk. As a result, federal IT professionals, like yourself, need to worry not only about keeping people out, but keeping those who are already in from doing damage.


But while you can’t completely isolate your network, you can certainly make sure that all access points are secure. To do so, you’ll need to concentrate on three things: devices, traffic, and planning.


Checkpoint 1: Monitor embedded and mobile devices


Although you may not know everything about what your HVAC or other systems with embedded devices are doing, you still need to do what you can to manage them. This means frequent monitoring and patching, which can be accomplished through network performance monitors and patch management systems. The former can give you detailed insight into fault, performance, security and overall network availability, while the latter can provide automated patching and vulnerability management.


According to a recent study by Lookout, mobile devices continue to be extremely prevalent in federal agencies, but an alarming number of them are unsanctioned devices that are being used in ways that could put information at risk. A staggering eighty percent of respondents in a SolarWinds survey believe that mobile devices pose some sort of threat to their agency’s security. But, you can gain control of the situation with user device tracking software, which can identify the devices that are accessing your network, alert you to rogue devices, and track them back to individual users.


Checkpoint 2: Keep an eye on network traffic


Network traffic analysis and bandwidth monitoring solutions can help you gain the visibility you may currently lack. You can closely monitor bandwidth and traffic patterns to identify any anomalies that can be addressed before they become threats. Bandwidth can be traced back to individual users so you can see who and what might be slowing down your network and you can receive automated alerts to let you know of any red flags that might arise.


Checkpoint 3: Have a response plan in place


While it’s a downer to say you should always assume the worst, it’s sadly true. There’s a bright side, though! If you assume a breach is going to happen, you’re more likely to be well prepared when it does. If one has happened, you’ll be more likely to find it.


This will require developing a process for responding to attacks and identifying breaches. Begin by asking yourself, “given my current state, how quickly would I be able to identify and respond to an attack?” Follow that up with, “what tools do I have in place that will help me prevent and manage a breach?”


If you’re uncomfortable with the answers, it’s time to begin thinking through a solid, strategic approach to network security – and start deploying tools that will keep your data from walking out the front door.


Find the full article on our partner DLT’s blog, TechnicallySpeaking.

When it comes to the technical aspects of PCI DSS, HIPAA, SOX, and other regulatory frameworks, the goals are often the same: to protect the privacy and security of sensitive data. But the motivators for businesses to comply with these regulatory schemes varies greatly.

Penalties for Noncompliance


Regulatory Compliance Framework





Governing Body



Payment Card Industry Data Security Standards

Applies to any organization that accepts credit cards for payment


Payment Card Industry Security Standards Council (PCI SSC)[1]

  • Fines up to $200,000/violation
  • Censure from credit card transactions


Health Insurance Portability and Accountability Act[2]

Applies to healthcare-related businesses deemed either covered entities or business associates by law


The Department of Health and Human Services (HHS) Office for Civil Rights (OCR)

  • Up to $50,000 per record
  • Maximum on $1.5M/year


Sarbanes–Oxley Act


Applies to any publicly traded company


The Security and Exchange Commission (SEC)

  • Fines up to $5M
  • Up to 20 years in prison


National Credit Union Association

Applies to credit unions

(r. 2013)

NCUA is the federal agency assigned to enforce a broad range of consumer regulations that apply to federally chartered credit unions and, to a lesser degree, federally insured state chartered

credit unions.[3]

  • Dissolve your credit union
  • Civil money penalties


Gramm-Leach-Bliley Act

Applies to financial institutions that offer products or services to individuals, like loans, financial or investment advice, or insurance


Federal Trade Commission (FTC)

  • $100,000 per violation
  • Up to 5 years in prison


Federal Information Security Management Act

Applies to the federal government and companies with government contracts


Office of Management and Budget (OMB), a child agency of the Executive Office of the President of the United States

  • Loss of federal funding
  • Censure from future contracts



This list only represents a fraction of the entire regulatory compliance structures that govern the use of information technology and processes involved in maintaining the confidentiality, integrity, and availability of sensitive data of all types.


Yes, there are monetary fines for noncompliance or unlawful uses or disclosures of sensitive information – the chart above provides an overview of that – and for most, that alone offers plenty of incentive to comply. But beyond this, businesses should be aware of the many other consequences that can result from non-compliance or any other form of negligence that results in a breach.


Indirect Consequences of Noncompliance


Noncompliance whether validated by audits, or discovered as the result of a breach, can be devastating for a business. Though, when a breach occurs, its impact often extends well beyond the fines and penalties levied by enforcement agencies. It can include the cost of detecting the root cause of a breach, remediating it, and notifying those affected. Further, the cost balloons when you factor in legal expenditures, business-related expenses, and loss of revenues faced by damaged brand reputation.


As if IT pros did not have enough to worry about these days, yes, unfortunately compliance too falls into their laps. But depending on the industries they serve and the types of data their business interacts with, what compliance actually entails can be quite different.


Regulatory Compliance and the Intersection with IT


Without a doubt, there are many aspects of data security standards and compliance regulations that overshadow everything from IT decision-making and purchasing, to configurations, and the policies and procedures a company must create and enforce to uphold this important task.


Organizations looking to comply with a particular regulatory framework must understand that no one solution, and no one vendor, can help prepare them for all aspects of compliance. It is important that IT professionals understand the objectives of every compliance framework they are subject to, and plan accordingly. 


[1] The PCI SSC was founded by American Express, Discover Financial Services, JCB International, MasterCard Worldwide and Visa Inc. Participating organizations include merchants, payment card-issuing banks, processors, developers, and other vendors.

[2] The Health Information Technology for Economic and Clinical Health (HITECH) Act, which was enacted as part of the American Recovery and Reinvestment Act of 2009, prompted the adoption of Health Information Technology. This act is recognized as giving “teeth” to HIPAA as it established stricter requirements by establishing the Privacy, Security, and Breach Notification Rules, as well as stiffer penalties for violations. The HIPAA Omnibus Rule, which went into effect in 2013, further strengthened the OCR’s ability to enforce compliance, and clearly defined the responsibility of compliance for all parties that interact with electronic protected health information (ePHI).

[3] It is important to note that in the financial world, guidance from the Federal Financial Institute of Examiners Council (FFIEC) to a bank is mandatory because the guidance specifies the standards that the examiner will use to evaluate the bank. Credit unions technically fall under a different regulator than banks, however, the National Credit Union Association closely follows the FFIEC guidance.



For a year that started out so very slowly in terms of “The Big Exit,” 2016 has become so compelling.


There have been very few infrastructural IPO’s this year, but there have been some interesting acquisitions, and some truly significant changes in corporate structures. I will highlight a few, and maybe even posit an opinion or two.


The Hyper-Converged market is growing steadily, with new players on a practically daily basis. Nutanix, who has stated from an early position that their exit would be one of Initial Public Offering, has pulled back on their timeframe a couple of times recently. They are consistently viewed as the big dog on the Hyper-Converged Infrastructure. With strong numbers, a loyal fanbase, and a team of salespeople, engineers, and SE’s who’ve at times rabidly promoted their offerings, they come by the stature in this space quite rightly. These statements are not, from me, about the quality, comparison, or reflections on any of the competitors in this growing space. It does seem that the only thing really holding back this company from their desired exit is one of the marketplace shying away from IPO’s, and that if they’d wanted to become an acquisition target, that quite possibly, their brand could have become a part of some other organization’s product line.


The massive companies in the space are not immune to these kinds of changes either. For example, after Dell decided to go private, and made that a reality, they then set their sights toward acquiring the largest storage vendor in the world. After what I’m sure have been long and arduous conversations, much negotiation, and quite a bit of oversight from the financial world, they’ve now made the acquisition of EMC a reality. While we will likely not see the full fallout of which companies stay in the newly created DellEMC, and which get sold off in order to make up some of the costs of the acquisition. The new company will be the largest technology company in the world and comprise so many different services offerings, storage offerings, converged architectures, etc. It’s a truly stunning acquisition, which will theoretically alter the landscape of the industry in profound ways. The future remains uncertain regarding Compellent, EqualLogic, Isilon, ExtremeIO, VNx, and Vmax, to name a few storage only brands, Dell Professional Services, Virtustream, and even VMware can be potentially be spun off. Although, I do suspect that VMware will remain a company doing business as it always has, a free spirit remaining a part of the architectural ecosystem, and beholden to none. VCE itself, could change drastically, as we really don’t know how this will affect the Cisco UCS/NEXUS line.


I recently wrote a posting on the huge and earth shattering changes taking place at HPe as well. Under the guidance of Meg Whitman, HP has now split itself into two distinct companies. Consumer division (Hewlett Packard) comprised of desktops, laptops, and printing as well as other well-known HP brands on one side, while Servers, enterprise storage, and Aruba Networking etc. become part of the other side (Hewlett Packard Enterprise). The transition, first launched at HP Discover 2015, has gone quite smoothly. Channel relationships have, if anything grown stronger. From this man’s perspective, I am impressed. Now, the recent announcement that Enterprise Software will be sold off to MicroFocus, the brand that used to market a version of Cobol, a global presence, will now be the owner of these major software releases. For my money, the operations should be just fine. I don’t really see how it’ll change things. Certainly some of our contacts will change, but as to how smoothly this newest change will transition is left to be seen.


Pure Storage, the last big IPO to transpire in the enterprise infrastructure space, has gone, for the most part very well. These folks seem to have a great vision of where they want to head. They’ve successfully built a second storage platform, essentially on the sly (FlashBlade), meanwhile their numbers have been on the whole, quite good. I’m so interested to see where things go with Pure. I do feel that they’ll handle their future with aplomb, continue to grow their market share, and create new products with an eye to gaps, and customer requirements. Their Professional services group, as well as their marketing have been standout performers within the industry. I do find it interesting, though, that they have been turning the world orange and converting customers away from legacy storage brands to new platform approach, while taking the needs of their customers even in use-cases that hadn’t necessarily been core to their approach gracefully, aggressively, and competently.


Finally, I’ll mention another key acquisition. NetApp, one of those stalwart legacy storage companies, at one time a great alternative to other monolithic storage vendors had gotten stale. By their admission, the reliance they had on an older architecture truly needed a shot in the arm. Well, they achieved this by purchasing SolidFire. SolidFire, a very tightly focused storage player in the All Flash Array market, was accomplishing some truly high-end numbers for a startup, going into datacenters, and service providers and replacing far larger players meanwhile solving problems that in many cases had existed for years, or creating solutions for brand new Cloud related issues. A truly impressive feat for such a lean startup. They’ve proven to be just the key that fit the lock. I’m very interested to see how smoothly this acquisition will go moving forward. I wonder how well that startup mentality will fuse with the attitudes of a much slower moving culture, as NetApp had become. Will attrition force the rockstar development team to slow down, focusing on integration, or will they be allowed to continue along the path they’d cut over the previous few years, run as a separate entity amongst the rest of NetApp? I am curious as to whether some of the great SolidFire people will leave once the dust settles, or if it’s to their benefit to grow within this larger entity. The truth will prove itself.


The current crop of candidates looking to go public all seem to revolve around the Software related cloud business model, companies like Twilio, Blue Apron, and Casper Mattresses appear to be the kind of contender that are poised to transform. They seem to focus on software as their model. From a real IT perspective, I’ve heard Basho mentioned, (A brand new platform for distributed Database), Code42 (the creators of Crash Plan), DropBox (the ubiquitous cloud file share/storage location), and xMatters (A leader in the IOT landscape) as potential candidates for public offering.


As to mergers and acquisitions, there does seem to be a better taste toward companies like Acacia (in Optical Networking), Pandora (the streaming media company), Zynga (video games like Farmville), and a couple of semiconductor firms: Macom and SunEdison.


Updates after writing and before posting: On Tuesday September 20, Nutanix filed (Again) for an IPO, setting the Initial public offering at 209Million based on a corporate valuation at approximately 1.8Billion. Filing paperwork here. And, the VMware vRealize Management Suite, as yet another fallout from the DellEMC deal, has been sold off to Skyview Capital. I’m fairly confident that we’ll be seeing more and more changes in this shifting landscape in the very near future.


We are living in a time of change in tech business. What or who is next for companies like those I’ve highlighted? Who will come up with the next ground breaking tech? And, who’s next to set the world on fire with their huge business news?

There's been a long-standing "discussion" in the world of storage regarding snapshots and backups. Some people say that snapshots can replace backups, while others say that just can't be true. I side with the latter, but the latest industry developments are making me reconsider that stance.


What's a Backup?


A backup isn't just a copy of data. A backup has to be recoverable and reliable, and most snapshots just don't meet that criteria.


What does "recoverable" mean? Backups have to be indexed and searchable by common criteria like date, file name, location, file type, and so on. Ideally, you could also search by less-common criteria like owner, content, or department. But at the very least there should be a file-level index, and most snapshot tools don't even have this. It's hard to expect a block snapshot to include a file index, but most NAS systems don't have one either! That's just not a backup.


Then we have to think about reliability. The whole point of a backup is to protect your data. Snapshots can protect against deletion and corruption, but they don't do much if the datacenter catches on fire or a bug corrupts your storage array. And many snapshot systems don't "snap" frequently enough or keep enough copies long enough to protect against corruption very long. This is why storage nerds like me say "your backup should be on a different codebase and your archive in a different zip code."


Then there's the question of management. Most backup systems have "friendly" interfaces to schedule regular backup passes, set retention options, and execute restores. Many years ago, NetApp showed just how friendly a snapshot restore can be, but options for what to backup and when remain pretty scarce. Although backup software isn't known for having the friendliest interface, you usually have lots more options.


Snap-Based Backups


But array snapshots can be an important part of a backup environment, and many companies are headed in that direction.


Most of today's best backup products use snapshots as a data source, giving a consistent data set from which to read. And most of these products sport wide-reaching snapshot support, from storage array vendors to logical volume managers. This is one source of irritation when people claim that snapshots have nothing to do with backups - of course they do!


Some snapshot systems also work in concert with data replication solutions, moving data off-site automatically. I've enjoyed the speed boost of ZFS Send/Receive, for example, and have come to rely on it as part of my data protection strategy. This alleviates my "different zip code" concern, but I would prefer a "different codebase" as well. That's one thing I liked at this week's NetApp Insight show: A glimpse of Amazon S3 as a replication target.


Then there are the snapshot-integrated "copy data management" products from Catalogic, Actifio, and (soon) NetApp. These index and manage the data, not just the snapshot. And they can do some very cool things besides backup, including test and development support.


Stephen's Stance


Snapshots aren't backups, but they can be a critical part of the backup environment. And, increasingly, companies are leveraging snapshot technology to make better backups.


I am Stephen Foskett and I love storage. You can find more writing like this at, connect with me as @SFoskett on Twitter, and check out my Tech Field Day events.

Thanks to the Internet of Things (IoT), we're on the lookout for invisible devices that are now capable of becoming vectors for all kinds of nasty services. The webcam attack on Brian Krebs is only the beginning. Could you imagine the focal power of a wide variety of IoT devices being brought to bear on Amazon? Or on Google? The potential for destruction is frightening. But it doesn't have to be. It just takes a little effort up front.

If It Talks Like A Duck

One of the best things about IoT devices is that they are predictable. They have static traffic patterns. Thermostats should only ever talk to their control servers, whether they be in the cloud or at a utility service provider. Lightbulbs should only ever talk to update servers. In the enterprise, devices like glucose meters and Point-of-Sale credit card readers also have traffic profiles. Anything that doesn't fit the profile is a huge clue that something is going on that shouldn't be.

Think back to the Target POS data breech. The register payment scanners were talking to systems they had never talked to before. No matter how small or isolated that conversation, it should have been a warning that something fishy was happening. Investigation at that point would have uncovered a breech before it became a publicity nightmare.

IoT devices should all have a baseline traffic profile shortly after they are installed. Just like a firewall port list, you should know which devices they are talking to and what is being transmitted. It should be incumbent on the device manufacturers to provide this info in their documentation, especially for enterprise devices. But until we can convince them to do it, we're going to need some help.

Tools like SolarWinds Network Traffic Monitor can help you figure out which devices are talking to each other. Remember that while NTM is designed to help you ferret out the worst offenders of traffic usage in your network, IoT devices may not always be trying to send huge traffic loads. In the case of the IoT DDoS, NTM should have seen a huge increase in traffic from these devices that was out of character. But in security cases like Target, NTM should be configured to find out-of-profile conversations with things like accounting servers or PCI devices.

You Need To Walk Like A Duck

I know I said it before, but your company absolutely has to have some kind of IoT policy in place. Today. Not after a breech or an incident, but ahead of time. This helps you control the devices flowing into your network from uncontrolled sources. It allows you to remind your executives that the policy doesn't require them to have Hue color-changing lightbulb. Or that they need to remove the unauthorized security camera watching the company fridge for the Lunch Bandit.

Sure, IoT is going to make our lives easier and happier. Until it all falls down around our ears. If I told your security department that I was about to drop 300 unsecured devices onto the network that can't been modified or moved, they would either have a heart attack or push back against me. But if your monitoring system is ready to handle them there won't be any issues. You have to walk the security walk before you're giving the security talk to a reporter from the New York Times.

The IT help desk is the lifeblood of an organization. It assists co-workers and end-users in many critical ways, including troubleshooting, answering questions, solving known problems, and helping the organization maintain productivity. However, if you’re manually managing the IT help desk function for your business, the speed at which you’re able to resolve issues could suffer, leading to delayed ticket resolution and unhappy end-users.


In this blog, I’ll detail five challenges businesses face by manually managing the IT help desk (as illustrated in the infographic), and offer up an alternative solution that’s sure to help desk admins and technicians avoid a lot of unnecessary headaches.


The Trouble With a Manual Help Desk


  1. Routing Tickets – Manually managing the IT help desk can make it difficult to track the availability of technicians to respond to issues. In this case, accidentally doubling down on requests is far from unheard of. And without in-depth knowledge of each individual on your team, assigning tasks based on the criticality and technicality of issues is more of a shot in the dark. Regardless, both scenarios can delay the process of even responding to a ticket, much less meeting a resolution.

  2. Tracking Down the End-user - When relying on a manual system for running the IT help desk, tickets are addressed in-person, which wastes time tracking down the end-user to provide hands-on assistance. Clearly, time spent running around leaves less time for resolving issues.

  3. Tedious, Manual Support – Troubleshooting at the “scene of the crime” (i.e. the end-users’ workstation) can have its benefits, but it leaves end-users twiddling their thumbs while a support technician diagnoses and corrects an issue. Of course, that end-user has much better things to do with their time, including hitting the next deadline, or getting to a meeting, which simply compounds the issue.

  4. Manually Closing Tickets – Another aspect of manually managing the IT help desk involves closing support cases. Though seemingly painless, the process of updating a static spreadsheet somewhere and contacting the supported party to confirm satisfaction can be even more of a time **** for IT support admins than you realize.

  5. Results and Performance – We all know time is money. Therefore, the time it takes to resolve an issue, which can certainly be compounded by all the factors I’ve listed above, means time (and money) wasted because someone is left stranded by IT issues. Plus, when evaluating the performance of the IT help desk organization as a whole, let’s just say downtime is frowned upon.


Two Ways to Run an IT Help Desk


IT Help Desk technicians face challenges on a daily basis. Manually managing the entire help desk function does not have to be one of those challenges. Through the combined use of SolarWinds® Web Help Desk and DameWare® Remote Support, life for IT pros can be far simpler. Take a look at the infographic below to see a comparison of what it’s like to manage a web help desk operation manually versus the use of these two solutions.


A lot can be said about the benefits of enabling remote support capabilities within your help desk solution, especially when IT issues arise. Consider the combined use of these two SolarWinds solutions, if not for the end-users and co-workers you support, but for the productivity of your businesses as a whole.



2 Ways to Run a Help Desk

2 Ways to Run a Help Desk from SolarWinds

Leon Adato

THWACKcamp 2016 Wrap-Up

Posted by Leon Adato Expert Sep 28, 2016


I posted a brief introspective note about THWACKcamp right after the event but now that some time has passed, I wanted to share some of the things that rose to the surface now that I have the benefit of additional time and distance (and let's be honest, SLEEP!).


First and foremost, credit needs to be given where it's due - to YOU. The THWACK community really brought it this year, in numbers and enthusiasm the likes of which we'd never seen before. Being on chat with everyone was like a years' worth of SolarWinds labs all smashed together (which, when you consider we had 15 separate sessions, it pretty much was!); the attendance was through the roof; and the range of questions and conversations have spawned discussions which continue now, 2 weeks later. I will happily don my "Captain Obvious" cape to point out that all of this would have been impossible without this amazing community which you've built and allowed us to be caring stewards of.


Second, kudos has to be given to the SolarWinds team, who worked their heart out to make this event a reality. The video crew, events team, graphics design artists, THWACK administrators, writers, and coordinators of all stripes, shapes, and sizes spent months pouring over every detail, thinking through each session, and crafting the best possible online experience - resulting in the largest conference of monitoring experts in the world - online or off.


With those very important thanks taken care of, here's what lingers in my memory:


1) You're not alone

The theme this year was "Challenge Accepted", but there's a risk of accepting challenges with an "I'll just do it myself" mentality. That's not the spirit of the theme, that's certainly not the spirit of THWACK, and if you were listening, you realized that's not the take-away from any of the sessions. Rarely was there a moment on screen when someone stood alone, and that was by design. As you accept your challenges - from being an accidental DBA to plumbing the depths of log management and even to figuring out how to make monitoring matter to management - you have a team behind you. That team may sit one cube over. Or it may be an IM, forum post, or phone call away. But irrespective of distance or time zone, there are people who share your passion and are ready and willing to help out.


2) The challenge is not unique

I'm not pulling a "Fight Club" reference, trying to tell you that you are not a special snowflake. I personally believe in the unparalleled blessing of recognizing the unique gifts each person brings to the table. But this challenge you've accepted - this problem you have to solve right now? THAT is not special. You may be monitoring one of those "hard to reach places" that seems to be completely off the radar of most tools. Or you are once again playing detective, searching for clues to the elusive root cause. No matter the situation, THWACKcamp taught us that this has happened before and someone has seen it and solved it. That's where item #1 comes into play. You gather your posse, you do the research, and you solve the problem.   Across two days and 15 sessions we heard from Geeks from all walks of life sharing examples of the challenges they faced and what was amazing was how often we shared similar experiences. That should be a huge relief to those of us who operate as "an army of one" at our respective company. It serves as a reminder that this challenge we're working on has been seen before, and solved. Which leads me to the last point:


3) The challenge is never insurmountable

To paraphrase Anthony Hopkins' character Charles Morse in "The Edge" - What one geek can do, another can do.  Someone else has stared this beast in the face and forced it to back down into the primordial slime from which all our worst IT issues arise.


The message after watching THWACKcamp this year was that this challenge HAS been accepted and resolved in the past, it CAN be solved by you today, and there's a TEAM standing beside you to help you out.


So let's all resolve to accept that next challenge, gather our team, and start building some epic stories and incredible solutions to talk about.


Because THWACKcamp 2017 can't be that far away.


[EDIT: Tip of the hat to designerfx for reminding me to add a link to the videos:]

To relive the excitement of THWACKcamp 2016, head over to this link: Pop some popcorn, watch each video as many times as you like, and bask in the #MonitoringGlory!

I'm at Microsoft Ignite this week, so if you are at the show please stop by the SolarWinds booth and say hello! I'd love to talk data, databases, and bacon with you, not necessarily in that order.


Here's a bunch of stuff I thought you might find interesting, enjoy!


Oracle's Cloudy Future

Last week was Oracle Open World, and we saw Larry Ellison talk about how much better Oracle Cloud is when compared to AWS. Read this and understand why Larry is wrong, and why Oracle is falling behind. In short, they are the new IBM.


The War On Cash

Long, but worth the read. My first thought was about the Star Trek future where money wasn't needed, and perhaps eliminating cash is the first step towards that. If so, then things will get worse before they get better.


Oh, ****, git!

Using Git? That's great! Make a mistake? You might want to read this.


Bad Security Habits Persist Despite Rising Awareness

Because you can't fix stupid, no matter how hard you try. Apparently.


Yahoo Says at Least 500 Million Accounts Breached in Attack

See preceding comment.


Top 10 ways to secure your mobile phone

Good list of tips here for anyone with a mobile phone. I like the idea of the remote wipe, will be adding that to my toolbox.


A Digital Rumor Should Never Lead to a Police Raid

I had never thought about the civil liberties aspect of a home raid brought about by IP address details. That's probably due to the fact I've never been worried about my home being raided, either. Anyway, interesting debate here.


This is how I imagined gminks rolled into work this past Monday:




SolarWinds and Vyopta now integrate so that you can monitor live data from your video infrastructure and access switch interface for any problem call in any conference room for Polycom or Cisco endpoints.


Key Features:

  • Simple API-level integration
  • Single click from any Cisco or Polycom endpoint to NPM interface details page
  • Live call stats, video device status, camera/display connection data, and registration info.


Eliminate the Video Conference Blind-Spot

Do you ever enter a never-ending blame game with your A/V team about why video conferences fail? Are you responsible for the video infrastructure in your environment? Perhaps even if you don’t want to be? Tired of those codecs and video infrastructure being a black-box in terms of actual call statistics and quality metrics? Want to bridge the visibility gap between your voice / video and the rest of your network infrastructure? Well perfect - because Vyopta’s real-time video monitoring now integrates with SolarWinds Network Performance Monitor.
     With this integration you are now able to monitor, alert, and diagnose end-to-end video call issues through Vyopta AND identify whether it is a network problem or a video device problem. Furthermore, with one-click access to NPM from every video endpoint, you can diagnose and fix the issue if it is a network problem. On the Vyopta side- call quality and hardware statistics are pulled directly from your endpoints and bridges via API. Whether you are using Cisco, Polycom, Acano, Pexip or Vidyo in whichever flavor, your data is combined and normalized in real-time. Based on this broad dataset, you are able to assess end-to-end call quality per call or determine whether an issue may be systemic within your video environment. Perhaps it’s as simple as the screen or camera being disconnected on the endpoint. Maybe the user dialed the wrong number. In Vyopta, you can get alerted for and diagnose the following issues at a glance:

  • Camera/Display disconnect
  • Endpoint becomes unregistered (unable to make calls)
  • Endpoint down
  • Bad call quality from gateway, bridge, or endpoint (packet loss or jitter)
  • High Packet Loss




Vyopta’s built in dashboards you can also quickly evaluate the health of your bridging infrastructure. Perhaps one of your MCU’s is at capacity, or you have a spike in traversal calls:


RT Capacity.png


If the issue isn’t with an endpoint or bridge, you can click on the helpful SolarWinds link next to the endpoint to take you right to the connected access-layer switch interface in NPM:



Once in NPM, you can determine if there is a common interface-level issue (VLAN / duplex / etc) or start to drive upstream into the infrastructure. Enhance your situational awareness with Netflow data or perhaps proactive UDP IPSLA transactions in VNQM. Recent config change bork DSCP tagging? NCM has you covered.


Screen Shot 2016-09-10 at 2.53.50 PM.png


So next time users start rumbling that those darn vidcons “don’t work” or the CEO’s call drops in the middle of a board meeting, know that your video infrastructure doesn’t have to be a black-box. With Vyopta and SolarWinds integration, it’s easy to troubleshoot. No more chasing phantom issues - isolate the root cause of video conference issues in just a few clicks.

There is hardly a government IT pro who has not seen sluggish applications create unhappy users.


Because the database is at the heart of every application, when there’s a performance issue, there’s a good chance the database is somehow involved. With database optimization methods -- such as identifying database performance issues that impact end-user response times, isolating root cause, showing historical performance trends and correlating metrics with response time and performance -- IT managers can speed application performance for their users.


Start with these four database optimization tips:


Tip #1: Get visibility into the entire application stack.

The days of discrete monitoring tools are over. Today’s government IT pros must have visibility across the entire application stack, or the application delivery chain comprising the application and all the backend IT that supports it -- software, middleware, extended infrastructure and especially the database. Visibility across the application stack will help identify performance bottlenecks and improve the end-user experience.


Tip #2: See beyond traditional infrastructure dashboards.

Many traditional monitoring tools provide a dashboard focused on health and status, typically featuring many charts and data, which can be hard to interpret. In addition, many don’t provide enough information to easily diagnose a problem -- particularly a performance problem.


Tools with wait-time analysis capabilities can help IT pros eliminate guesswork. They help identify how an application request is executed step-by-step and will show which processes and resources the application is waiting on. This type of tool provides a far more actionable view into performance than traditional infrastructure dashboards.


Tip #3: Reference historical baselines.

Database performance is dynamic. It is critical to be able to compare abnormal performance with expected performance. By establishing historic baselines of application and database performance that look at how applications performed at the same time on the same day last week, and the week before that, etc. , it is easier to identify a slight variation before it becomes a larger problem. And, if a variation is identified, it’s much easier to track the code, resource or configuration change that could be the root cause and solve the problem quickly.


Tip #4: Align the team.

Today’s complex applications are supported by an entire stack of technologies. And yet, most IT operations teams are organized in silos, with each person or group supporting a different part of the stack. Unfortunately, technology-centric silos encourage finger-pointing.


A far more effective approach shares a unified view of application performance with the entire team. In fact, a unified view based on wait-time analysis will ensure that everyone can focus on solving application problems quickly.


Remember, every department, group or function within an agency relies on a database in some way or another. Optimizing database performance will help make users happier across the board.


Find the full article on Government Computer News.

Following my review of Solarwinds Virtualization Manager 6.3, the fair folks at Solarwinds gave me the opportunity to put my hands on their next planned release, namely VMAN 6.4. While there is no official release date yet, I would bet on an announcement within Q4-2016. The version I tested is 6.4 Beta 2. So what’s new with this release?


From a UI perspective, VMAN 6.4 is very similar to its predecessor. Like with VMAN 6.3, you install the appliance and either install VIM (Virtual Infrastructure Monitor component) on a standalone Windows Server, or integrate with an existing Orion deployment if you already use other Solarwinds products. You’d almost think that no changes have happened until you head over to the « Virtualisation Summary » page. The new, killer feature of VMAN 6.4 is called « Recommendations » and while it seems like a minor UI improvement there’s much more to it than it looks like.


While in VMAN 6.3 you are presented with a list of items requiring your attention (over/under-provisioned VMs, idle VMs, orphan VMDK files, snapshots etc. – see my previous review), in VMAN 6.4 all of these items are aggregated in the « Recommendations » view.


Two types of recommendations exist: Active or Predicted. Active Recommendations are immediate recommendations that are correlated with issues that are currently showing up in your environment. If you are experimenting memory pressure on a given host, an active recommendation would propose you to move one or more VMs to another host to balance the pressure. Predicted recommendations, on the other hand, focus on proactively identifying potential issues before they become a concern, based on usage history in your environment.


The « Recommendations » feature is very pleasant to use and introduces a few elements that are quite important from a virtualisation administrator perspective:


  • First of all, administrators have the possibility to apply a recommendation immediately or schedule it for a later time (out of business hours, change windows, etc.)
  • Secondly, an option is offered to either power down a VM to apply the recommendation or to attempt to apply the recommendation without any power operations. This features comes in handy if you need to migrate VMs, as you may run into cases where a Power Off/Power On is required, while in other cases a vMotion / live migration will suffice
  • Last but not least, the « Recommendations » module will check if the problem still exists before actually applying a recommendation. This makes particularly sense in the case of active recommendations that may no longer be relevant by the time you decide to apply the recommendation (for example if you decide to schedule a recommendation but the issue is no longer reported by the scheduled time)


A nice and welcome touch in the UI is a visual aid that shows up when hovering your mouse over the proposed recommendations. You will see a simple & readable graphical view / simulation of the before & after status of any given object (cluster, datastore, etc.) in case you decide to apply the recommendation.


Max’s take


The “Recommendations” function, while apparently modest from an UI perspective, is in fact an important improvement that goes beyond the capacity reclamation and VM sprawl controls included in VMAN 6.3. Administrators are now presented with actionable recommendations that are relevant not only in the context of immediate operational issues, but also as countermeasures to prevent future bottlenecks and capacity issues.


A few side notes: if you plan to test the beta version, reach out to the Solarwinds engineers. The new “Recommendations” function is still being fine-tuned and you may not be able to see it if you integrate it with your current VIM or Orion environment. Once you install VMAN 6.4, you should let it run for approximately a week in order to get accurate recommendations.

Flash storage can be really, really fast. Crazy fast. So fast that some have openly asked if they really need to worry about storage performance anymore. After all, once you can throw a million IOPS at the problem, your bottleneck has moved somewhere else!


So do you really need to worry about storage performance once you go all-flash?


Oh yes, you definitely do!


All-Flash Storage Can Be Surprisingly Slow


First, most all-flash storage solutions aren't delivering that kind of killer performance. In fast, most all-flash storage arrays can push "only" tens of thousands of IOPS, not the millions you might expect! For starters, those million-IOPS storage devices are internal PCIe cards, not SSD's or storage arrays. So we need to revise our IOPS expectations downwards to the "hundred thousand or so" than an SSD can deliver. Then it gets worse.


Part of this is a common architectural problem found in all-flash storage arrays which I like to call the "pretend SSD's are hard disks" syndrome. If you're a vendor of storage systems, it's pretty tempting to do exactly what so many of us techies have done with our personal computers: Yank out the hard disk drives and replace them with SSD's. And this works, to a point. But "storage systems" are complex machines, and most have been carefully balanced for the (mediocre) performance characteristics of hard disk drives. Sticking some SSD's in just over-taxes the rest of the system, from the controller CPU's to the I/O channels.


But even storage arrays designed for SSD's aren't as fast as internal drives. The definition of an array includes external attachment, typically over a shared network, as well as redundancy and data management features. All of this gets in the way of absolute performance. Let's consider the network: Although a 10 Gb Ethernet or 8 Gb Fibre Channel link sounds like it would be faster than a 6 Gb SAS connection, this isn't always the case. Storage networks include switches (and sometimes even routers) and these add latency that slows absolute performance relative to internal devices. The same is true of the copy-on-write filesystems protecting the data inside most modern storage arrays.


And maximum performance can really tax the CPU found in a storage array controller. Would you rather pay for a many-core CPU so you'll get maximum performance or for a bit more capacity? Most storage arrays, even specialized all-flash devices, under-provision processing power to keep cost reasonable, so they can't keep up with the storage media.


Noisy Neighbors


Now that we're reset our expectations for absolute performance, let's consider what else is slurping up our IOPS. In most environments, storage systems are shared between multiple servers and applications. That's kind of the point of shared networked storage after all. Traditionally, storage administrators have carefully managed this sharing because maximum performance was naturally quite limited. With all-flash arrays, there is a temptation to "punt" and let the array figure out how to allocate performance. But this is a very risky choice!


Just because an array can sustain tens or even hundreds of thousands of I/O operations per second doesn't mean your applications won't "notice" if some "noisy neighbor" application is gobbling up all that performance. Indeed, performance can get pretty bad since each application can have as much performance as it can handle! You can find applications starved of performance and trudging along at disk speeds...


This is why performance profiling and quality of service (QoS) controls are so important in shared storage systems, even all-flash. As an administrator, you must profile the applications and determine a reasonable amount of performance to allocate to each. Then you must configure the storage system to enforce these limits, assuming you bought one with that capability!


Note that some storage QoS implementations are absolute, while others are relative. In other words, some arrays require a hard IOPS limit to be set per LUN or share, while others simply throttle performance once things start "looking hot". If you can't tolerate uneven performance, you'll have to look at setting hard limits.


Tiered Flash


If you really need maximum performance, tiered storage is the only way to go. If you can profile your applications and segment their data, you can tier storage, reserving maximum-performance flash for just a few hotspots.


Today's hybrid storage arrays allow data to be "pinned" into flash or cache. This delivers maximum performance but can "waste" precious flash capacity if you're not careful. You can also create higher-performance LUNs or shares in all-flash storage arrays using RAID-10 rather than parity or turning off other features.


But if you want maximum performance, you'll have to move the data off the network. It's pretty straightforward to install an NVMe SSD in a server directly, especially the modern servers with disk-like NVMe slots or M.2 connectors. These deliver remarkable performance but offer virtually no data protection. So doing this with production applications puts data at risk and requires a long, hard look at the application.


You can also get data locality by employing a storage caching software product. There are a few available out there (SanDisk FlashSoft, Infinio, VMware vFRC, etc) and these can help mitigate the risks of local data by ensuring that writes are preserved outside the server. But each has its own performance quirks, so none is a "silver bullet" for performance problems.


Stephen's Stance


Hopefully I've given you some things to think about when it comes to storage performance. Just going "all-flash" isn't going to solve all storage performance problems!


I am Stephen Foskett and I love storage. You can find more writing like this at, connect with me as @SFoskett on Twitter, and check out my Tech Field Day events.

Screen Shot 2016-09-20 at 11.34.34 AM.png


This week I will be in Atlanta for Microsoft Ignite, splitting time between the Microsoft and SolarWinds booths in the exhibit hall. I have the privilege of delivering a session in the Community Theater on Tuesday, the 27th of September, from 2:50-3:10PM EDT. The title of the talk is "Performance Tuning Essentials for the Cloud DBA," and it's a story that's been on my mind for the past year or so.


First, "cloud DBA" is a phrase I borrowed from Rimma Nehme who mentioned the term during her PASS Summit keynote in 2014. Dr. Nehme was reinforcing an idea that I have been advocating for years, and that is for database administrators to stop thinking of themselves as DBAs and start thinking of themselves as data professionals. To a DBA, it shouldn't matter where the data resides, either down the hall or in the cloud. And for those of us that are accidental DBAs, or accidental whatevers, we know that there will soon be accidental cloud DBAs. And those accidental cloud DBAs will need help.


And that help begins with this 20-minute session at Ignite tomorrow.


During that session, you are going to hear me talk about the rise of hybrid IT, the changing face of IT, and how we won't recognize things in five years. An accidental cloud DBA will be overwhelmed at first, but we will help provide the structure they need for a solid foundation to perform well in their new role. And I will share some tips and tricks with you to help all cloud DBAs to be efficient and effective.


So if you are at Microsoft Ignite this week, stop by to chat with me in the booth, or after my session Tuesday. I'd be happy to talk cloud, data, databases, and technology in general.

Filter Blog

By date:
By tag: