15 Replies Latest reply on Apr 10, 2014 2:15 PM by esther

    Have you ever made a mistake because a storage array was hard to use?

    plankers

      I’ve spent a lot of time working on, with, and around storage arrays, and I have to say that almost none of them qualify as “easy to use.” Some of the interfaces are shiny, but difficult to use because the prettiness doesn't help get things done. Most of the time the interfaces are written by a software engineer who will never actually use the product in the field, to a spec written by people who won’t use the product in the field, and are complicated and convoluted.

       

      “Who cares?” you might ask. You didn’t buy an array because it’s easy to use, you bought it because it has five NICs instead of four, or autotiers, or is $1000 cheaper than another. The problem is that these arrays are the foundations of everything else in the data center, so the scope of an error is usually pretty large. Take, for example, part of my history. A long time ago we had one of IBM’s first Enterprise Storage Servers (a 2105). One day our DBAs asked for more space on a database server. “No problem,” I thought. I allocated it, grew their filesystems, done.

       

      A few days later that database server had massive filesystem corruption on the volume I’d extended. What happened was that the admin interface for the 2105 was this Java app that didn’t enforce any common UI standards, and had tiny fonts. So when I was scrolling through the giant list of server aliases I inadvertently selected another server, didn’t realize it, and when I finally clicked on the right server it didn’t unselect the other one (as all other multi-select boxes do, everywhere). As it turns out, the admins of that other, inadvertently-selected server had been waiting for a storage allocation, and when they saw it they formatted it… Many, many people sat idle that day while we restored the database.

       

      Don’t you think that if storage interfaces were easier to use the chance of someone making a giant mistake would be a lot lower? Do you think that’d be a selling point? Would you be willing to pay more for a UI that isn’t just fancy, but is truly usable? Why don’t we audition storage for ease of use?

       

      Or, if you don’t have answers to those questions, just tell me a story.

        • Re: Have you ever made a mistake because a storage array was hard to use?
          jeremymayfield

          I bought a Dell Equallogic for that ease of use possibility, only thing is i bought it refurbished.  Bigger issue is although it has a nice shinny interface it has no or very little support.   Now i have a what was once a very expensive Dell Storage SAN and its taking a long time to configure as i don't have anyone to lean on in the weeds to get this configured.  I find that storage is simply a necessary evil, our architecture is built around a software or hardware platform that is built around a specific architecture.   Really we become victims of a not learning from our own history or others mistakes.  

           

          What would be easy to use?  And what would the sacrifice be to have it.  It wouldn't just be dollars.   I would guess there would have to be significant congestion on some other point in the chain to make this a easier process.  Seems to me I would consider giving a few more dollars, but what is a few more.  $10K, $20K?  That simply would be workable in my small company.   It needs to be and stay affordable and work. 

          • Re: Have you ever made a mistake because a storage array was hard to use?
            zackm

            When I was a contractor at one of the major Federal agencies; there were 2 people in the nation who knew how to use the storage GUIs. Both were in Seattle, and one has since left to work for EMC.

             

            Think of it, an entire agency can go down at any moment (not the critical part, just the behind the scenes stuff like logistics and HR, etc) simply for a lack of training and/or ease of use. Seems a little crazy, but it is a reality. Though I've never personally been in a storage admin position, I have felt a bit of the stress on relying on a single person in a large agency.

             

            Usability is something that is absolutely lacking

              • Re: Have you ever made a mistake because a storage array was hard to use?
                Carlo Costanzo

                zackm wrote:

                 

                Though I've never personally been in a storage admin position, I have felt a bit of the stress on relying on a single person in a large agency.

                 

                Usability is something that is absolutely lacking

                I guess the Good and BAD thing is that the storage responsibilities are slowly getting filtered down to more people thanks in big part to Virtualization.  Most of the virtualization platforms are doing a good job at bringing SAN functionality INTO the HyperVisor Management GUIs (Which in my opinion are somewhat better designed).  This of course is BAD for the Storage Guy/Gal who may be crafting unique architecture designs and performance profiles only to have us Virtualization guys come blow it out with new LUNs and VM data.

                 

                From a SAN UI perspective, I've also had the unfortunate pleasure of having to hunt down 11 different versions of Java and Browser Upgrades to even get some of the SAN UIs to even function. 

                 

                CARLO

              • Re: Have you ever made a mistake because a storage array was hard to use?
                wluther

                I am waiting for Apple's Yahooglesoft cloud of life.  Then I will completely do away with all hardware (phones included), and run everything on hopes and wishes.  Seems to be the most efficient way to go about it.

                • Re: Have you ever made a mistake because a storage array was hard to use?
                  byrona

                  We use NetApp for our storage systems and while the product is rock solid and the support is fantastic, the interface(s) certainly leave something to be desired.  NetApp suffers from interface fragmentation.  They have several different interfaces (GUI's), some web based and others client based with a bit of overlap between them; its never clear which interface I should be using for which activity and in the end there are still many thing that require you to drop to a CLI.

                   

                  Unfortunately we have had problems that were the result of mistakes made issuing commands to our NetApp; I fully believe a more user friendly interface would have helped in most if not all of these cases.  I certainly believe there would be value in a good interface for managing storage systems; how much I would be willing to pay is a difficult question to answer since I am not the one responsible for making those decisions at my company.

                   

                  What I don't understand is why storage manufacturers seem to struggle with a good interface for managing their systems.  I have worked with other interfaces used for much more complex tasks that put the storage system GUI's to shame.  Is finding a team of interface developers such a difficult thing to do?  When you think about it, storage system vendors are really in the software business because actual storage itself is all very similar, its the different software and capabilities that sets them apart from each other, yet they can't mange to put together a single user friendly GUI.  Am I missing something here?

                   

                  Disclaimer: I am relatively new to storage management and have really only worked directly with NetApp and a tad bit with Hitachi.

                  • Re: Have you ever made a mistake because a storage array was hard to use?
                    Scott Sadlocha

                    I don't have a ton of experience with storage networks directly, but do have some, most of it indirectly. Dell Equallogic is being used at my current company, but we ran NetApps at my last two companies, one of which I left a couple weeks ago. One of my other duties was endpoint protection management, and we used SEP for that. We were a manufacturing company with multiple plants in all major regions of the world, and each of those plants had a common infrastructure, which included a NetApp filer. One day I started getting a ton of SEP alerts for our NetApp in Wuhu, China. In looking it over, I found that the filer was implemented with no AV at all installed.

                     

                    In sitting down with our filer admin, I found that he was not aware of exactly how AV could integrate with the NetApp, and the previous admin (who implemented the systems) didn't even give endpoint protection a thought. Anyways, in sitting down to discuss the situation, we went through the interface, and I was a bit surprised. In this day and age of incredible interface functionality, I was left thinking "This is the best they can do?"

                     

                    We eventually found a SEP solution for endpoint protection of the filers, but in looking at the interface, I can definitely see how a mistake might be fairly easy to make.

                    • Re: Have you ever made a mistake because a storage array was hard to use?
                      tomiannelli

                      1. Don’t you think that if storage interfaces were easier to use the chance of someone making a giant mistake would be a lot lower?

                      Not really. I think that you have to change the culture of an organization so it understands that not all storage is created equal. There are real performance and functional reasons why storage arrays are complex. Organizations need to realize that to truly manage enterprise storage it takes dedicated skilled staff. Unfortunately most people look like a deer in the headlights when you start discussing the details of your storage array compared with the portable 1 TB disk they carry around in their brief case or backpack. The complexity will lead to mistakes just because the person that is managing your storage is the same one managing the rest of your infrastructure and hosted applications, not leaving enough time to focus and ultimately details will get overlooked and mistakes made. You will never make a software interface idiot proof they are always making better idiots.

                       

                      2. Do you think that’d be a selling point?

                      The management interface is always a point of interest if you let the administrators get involved in the evaluation. An important point is the integration with the rest of your infrastructure and how well the interface lets you do that. I really do not like it when 80% of what you need to do is available via a GUI and the other 20% is only available via a CLI. I do not mind using either, but the functionality should be the same from both. If a vendor told me that I can doo all of this from a GUI and then do the same thing in batch using a CLI, it would really get my interest.

                       

                      3. Would you be willing to pay more for a UI that isn’t just fancy, but is truly usable?

                      I might, but one customer is not enough to drive that market. If it were then Novell GroupWise and Corel's WordPerfect would be market leaders today. Their functionality outpaced the competition in past, but no the "fancy" stuff.

                       

                      4. Why don’t we audition storage for ease of use?

                      Investment and time. If you put storage in place you would have to have a large T&D environment so you could put it through some legitimate use cases. If your not dealing with a large enough user group you won't encounter the variety of storage requirements to test the UI out. If all you are doing is going to create some LUNs to present to hosts and maybe do some DR configuration, your not really putting it through its paces. Then if you do not like the UI you have to deal with the data disposition, uninstall it, and return it to the vendor. That costs the business money in your time and effort, as well as others, all for a UI that 99% of the business really does not care about. Unless of course you are all about providing storage to your customers then I guess it would matter.

                      • Re: Have you ever made a mistake because a storage array was hard to use?
                        bgenzoli

                        Yes, I made a mistake on a Dell Storage Array due to the GUI. This was a couple of years ago (maybe 8 or so), and it really wasn't intuitive or well worded. I lost about 5TB of Data that I had to restore. Bye bye holiday weekend!

                        • Re: Have you ever made a mistake because a storage array was hard to use?
                          jspanitz

                          So I was "fortunate" enough learn / use EMC Clariion arrays and can tell you nothing is more user unfriendly or confusing than that interface.  And I've gone through multiple versions of it.  Even the newer Unisphere is a disaster.  Now that being said I also used DELL EqualLogic and even more recently Compellent - which is now DELL but was not when we bought them.  And I can only say that the EqualLogic was SO MUCH easier to use / deploy.  It'a also not in the same class as the CX / VNX arrays.  But the Compellent is and it's interface and the way it is managed it like icing on the ease of use cake.

                           

                          Now, that being said, it has room for improvement.  There are plenty of areas that could be polished.  Managing multiple arrays is one of them.  The tools are there and are pretty good but could be better.

                           

                          Don’t you think that if storage interfaces were easier to use the chance of someone making a giant mistake would be a lot lower?  No, we still would require those responsible to be properly trained.

                          Do you think that’d be a selling point? Hell Yeah

                          Would you be willing to pay more for a UI that isn’t just fancy, but is truly usable? Sure.  But we paid MUCH LESS with Compellent and got everything you are asking about.

                          Why don’t we audition storage for ease of use? I don't do auditions

                           

                          To answer the OPs other question, yes, we've made mistakes due to the complexity of the EMC GUI.  Luckily for us it was with VMware and it was with only a single node in a cluster.  So we did not loose any data or noticeable downtime.

                          • Re: Have you ever made a mistake because a storage array was hard to use?
                            svindler

                            I think I've made equally many stupid mistakes on userfriendly GUI interfaces, as I have from the command prompt.

                            That being said, I do believe that there is an expectation that a decent GUI would make a warning in case you are about to do something irreversible, whereas people generally don't have the same expectation when doing the same changes from a CLI.

                            • Re: Have you ever made a mistake because a storage array was hard to use?
                              Kurt H

                              No mistakes here yet. But we should be getting a sotrage array here soon, so I am hoping I do not make any mistakes getting the device configured the way it should be.

                              • Re: Have you ever made a mistake because a storage array was hard to use?
                                Hans De Leenheer

                                One of the things that was very welcome for me (&colleagues) was the possibility to set MPIO defaults in vSphere. It wasn't waterproof when you had multiple types of storage but if you had Equallogic or EVA for example that did help not having to go into each datastore to check if MPIO was set correctly.

                                 

                                But I do have a war story you might like: I was HP Enterprise Storage certified so I did know one or two things about my stuff. So on that day that one of our EVA's at a customer (local gov wth 1000 active desktop clients) got stuck on a RAID rebuild at around 38% I called support. The only thing they needed to do was giving the controllers a little kick so that it would restart the rebuild. Nothing fancy. They would send an "HP Enterprise Storage certified" engineer to get it done. You get where we are going here? "No problem, just get me through the procedure. I can handles this".

                                 

                                So that's what we did. Hooking up the laptop at the serial port of the controller and going into config mode. Now enter this 4 digits hexadecimal command and we are done. The array reboots, one controller at a time and ... it's EMPTY! The whole config is gone, all RAID groups have disappeared, no more users, volumes, ... Guess what happened? I switched 2 of the 4 digits and magically that was another recognizable command that flushes the whole configuration. It took us 48hrs with a level XX R&D guy with lots of weird scripts to get everything back.

                                 

                                Lessons learned: if support offers you to take responsibility - GIVE IT!

                                • Re: Have you ever made a mistake because a storage array was hard to use?
                                  ElevenB2003

                                  I corrupted a bunch of View desktops while adding a new ESXi host to an array. After I added the host, it required a "rescan" of all HBA's to allow LUN's to be mapped to the new host - So I did it.  Re-scanning on this particular box resets the HBA's on the SAN side, thus causing all of the desktops I was hosting on it to drop their connections to storage.  The good thing was that the pool that got screwed up was just something I was doing some testing with but I found out the hard way that this particular SAN didn't have a very user friendly interface nor was it very informative about what it was going to do.