How much storage do you have? What storage are you using? When will you need more?
These seem to be simple questions, and that's often how they're approached. But if you're a systems administrator "on the hook" for storage, you know they're actually devilishly complex to answer. Storage utilization metrics have been a challenge from the beginning, and it's only getting more complex in our world of storage virtualization, shared storage, and cloud storage! Let's get back to the basics and think about the root questions here and how we can solve them.
Storage Utilization: Why Do You Care?
Here's a radical statement coming from a "storage guy" like me: Don't obsess about storage capacity for its own sake, since it's basically free these days.
Now that I've given that a moment to sink in, let's consider the value of a byte. I recently bought 10 TB of storage at retail for under $250, including tax. That's a whole lot of storage! In fact, it's enough capacity to hold the core financial and operational data of a midsize company! 5 TB, 6 TB, 8 TB, and even 10 TB hard disk drives are readily available today, and they're available for under $1,000!
With prices like these, why do we even care about storage capacity anymore?
- Storage performance remains very expensive, even though widespread use of flash and SSD's have radically opened up what's possible in terms of performance
- Advanced storage features are expensive, too, and they're what we're really paying for when we buy enterprise storage
- IT budgets, approvals, and group ownership remain confounding factors in making efficient use of storage
In other words, even though storage capacity is free, everything else still costs a whole lot of money! Storage management has never really been about the bytes themselves. It's all about making data available to the business on demand, every time. And that's a much bigger issue!
Yet all of those things (performance, features, and bureaucracy) are linked to those core questions at the top of the page. When people ask how much storage they have, how much they're using, and how much is available what they're really asking is, "can the storage environment support my needs?" Therefore, answering capacity questions requires thought and consideration. And the answer is never a simple number!
How Much Storage Do You Have?
You have more than enough storage today. Otherwise, you'd be in "crisis mode" installing more storage, not reading blog posts! But it's painfully difficult to answer even this basic question!
Most businesses purchase storage on a per-project or per-department basis, and this is probably the worst possible way to do it. It encourages different groups to be misers, hiding storage from each other lest someone else take it. After all, if engineering paid for a storage array, why should sales be able to use some of it?
Even if people have the best intentions, "orphan storage" is common. As servers and applications come off-line, "shared" storage systems are the last to go. It's typical for companies to have many such storage arrays still soldiering on, supporting a few leftover servers in a corner somewhere. Some storage is "orphaned" even before it's ever used, having been purchased and saved for an application that never needed it.
A few years back, I used to do audits of enterprise storage environments, and I always started with an in-person tour of the data center. My most-common question was, "what's that storage array over there?" And the answer was always illuminating and a little embarrassing for someone. But that wasn't my goal; I just wanted to get a baseline of what was in place today.
What Storage Are You Using?
Even storage systems in active use might not meet your needs. Outdated and over-burdened arrays are just as common as orphans. And duplicate data is everywhere in a modern datacenter!
Companies often purchase systems that match their budgets rather than their needs, leading to odd mis-matches between application criticality and system capability. And vendor sales representatives are as much to blame, pushing inappropriate systems on companies.
The key question in terms of storage in use is suitability to task, not just capacity used. Are applications I/O constrained? There's plenty of storage performance to be had, thanks to SSD's and flash storage arrays! Do you need to actively share and move data? Today's arrays have fantastic snapshot, cloning, and replication features just waiting to be exploited! And there's specialized software to manage storage, too!
My storage audits also included a census of data in active use. I had the systems and database administrators tell me how much data was actively used in their applications. Once again, the answer was shocking to jaded managers used to talking about petabytes and usually exabytes. Indeed, most large companies had only a small amount of active data: I often recorded single-digit percentages! But this too is misleading, since modern systems need test and development, backup and archiving, and space to grow.
When Will You Need More Storage?
A critical question is, "what kind of storage will you need in the future?" Applications are going in all directions today: Some need scalable, API-driven cloud storage, others need maximum performance, while more need integration and data management features. The old standby storage arrays aren't going to cut it anymore.
Don't just look to expand existing infrastructure, which is likely out of date and inappropriately used anyway. Consider instead what you want to do with data. There are lots of wonderful solutions out there, from large and small companies, if you're willing to look beyond what you have.
Encourage your company to move to a "storage utility" model, where a general storage budget goes to IT instead of being doled out on a per-project basis. Then you can develop a multi-product storage service with SAN, NAS, Object/Cloud, and even Software-Defined Storage. And maybe you can stay ahead of the question in the future.
Another option is to purchase per-application but be very conservative when selecting. Try to keep variety to a minimum and don't over-purchase. Over-buying leads either to orphans or "hop-ons", and neither is good for business.
Regardless, make sure the storage meets the needs of the application. Isn't that better than just worrying about capacity?