9 Replies Latest reply on Mar 19, 2014 3:38 PM by sevier.toby

    Are roll-your-own distributed storage systems a good idea?


      As projects like OpenStack progress we’re starting to see more scale-out filesystems, like Ceph and Gluster, providing inexpensive storage services for "the cloud." For these, you take commodity compute hardware, pack it full of disk, and turn it into a managed storage system. These have certain advantages, they've very custom, they tend to be less expensive, and performance often scales with growth of the storage cluster. They have definite disadvantages, too, like being complex, having little support (though each of those examples has support that can be purchased), only having a CLI, and often running on operating systems that aren't as well understood in the enterprise (Linux).

      Contrast that with the traditional array, monolithic but single point of management, fewer controllers but larger, purpose-built software to drive the array. Performance often gets worse as the array gets larger and there is more traffic on the internal links, but management interfaces are standard. There's always support from vendors, and training, and almost always a GUI for management.


      Do distributed storage systems of the roll-your-own variety really save that much money? Do their advocates ignore the realities of staff time spent managing them? From your point of view which one is better?

        • Re: Are roll-your-own distributed storage systems a good idea?

          I have to say that i've never done roll your own storage but I have done it within other areas of IT.  And I can also say after having been there and done that I truly feel there are few reasons to roll your own.  The complexity and the (usually unmeasured) maintenance required rarely outweigh using a vendor provided solution.


          Perhaps I am getting older, but I really dislike roll your own for almost any situation.

          • Re: Are roll-your-own distributed storage systems a good idea?

            I've never been a fan of roll-your-own solutions in the enterprise, simply because anything in use in the enterprise needs to have vendor support and SLAs behind it.  Buying a bunch of mid-range Dell desktops and packing them full of whatever drive you can get your hands on sounds like a recipe for failure.  What happens in 2 or 3 years when a drive or part dies and you can't find suitable replacement?  Do you self-stock replacements on site or pray to the FedEx gods that a replacement can be delivered in a timely manner? 


            That said, in a small business where cost is almost as important as reliability, there may be room for these roll-your-own solutions.  Maybe for a cost-focused business, saving thousands of dollars is worth having a random 2 day downtime sometime in the future, but for large enterprises, it's not.

              • Re: Are roll-your-own distributed storage systems a good idea?

                I have to agree with matt.matheus on this one.  I don't think that roll-your-own is a good solution for the enterprise for all of the reasons that he has stated.  Commercial vendor support so important in an enterprise environment, I don't know what I would do without it.


                While I have worked here our company has gone from a small roll-your-own company to a much larger enterprise grade company so we have had both types of solutions.  The enterprise solutions scale much better and free up your staff hours to work on customer facing projects and services versus trying to maintain and manage the internal home brew systems.


                The other great thing about enterprise solutions is that as part of your maintenance fees you also get constant feature enhancements allowing your organization to keep up with the constantly changing industry.

              • Re: Are roll-your-own distributed storage systems a good idea?

                I've done roll-your-own storage.  It can be great, for some purposes like archival data.  I'm not sure I'd want to trust it over an enterprise solution for my production servers, but sometimes you have a budget that you have to work within.

                • Re: Are roll-your-own distributed storage systems a good idea?

                  Definitely not a fan. I have been on the receiving end of roll your own twice in my career. What I mean is, the engineer who rolled out the solution jumped ship for better career options and left me to care for the entire system, with nothing more than a username and password...


                  Vendor support is absolutely priceless.

                  • Re: Are roll-your-own distributed storage systems a good idea?

                    IMO Roll your own is not a good idea for any form of Enterprise storage solution. First question asked should always be What will this be like in three years? Roll your own will end up a mish mash of technologies as parts fail, are no longer available, and stuff is jerry rigged to work, be compatible etc.

                    Using a Vendor gives some element of consistency and upgrade path.

                    • Re: Are roll-your-own distributed storage systems a good idea?
                      Scott Sadlocha

                      I can say that I have never worked with roll your own storage, but just reading the comments, I have to wonder how this would ever be a viable solution for an enterprise production system. I can see it being useful for a small shop where cost is the primary motivator. However, I can see it getting to a point where there is no solution but to replace it because of lack of support or replacement drives/parts.


                      At my current job and my previous job, the companies are medium sized (about 4500 and 1500 employees, respectively), and both have environments where vendor support is critical. As a matter of fact, my last company not only didn't use it, but had a policy against using such solutions because vendor support was instrumental in implementation. Personally, I would never use a solution for a business critical system. A span of several years can fly by, and it seems that, after that amount of time, the reliability of a roll your own solution would diminish incredibly.

                      • Re: Are roll-your-own distributed storage systems a good idea?
                        Scott Sadlocha

                        Hmm, it seems my last post didn't register as a reply for the Ambassador Monthly Engagement. I'm going to see if this one does (sorry for the pointless reply email reminder folks).

                        • Re: Are roll-your-own distributed storage systems a good idea?

                          Datacore SANMelody is a good product.  I maybe biased as a DCIE (Datacore Certified Infrastructure Engineer) but building your own solution is great if you have access to good left over hardware, memory, licensing, switches, etc.  This old equipment will provide a SAN that equals official branded SANs.  However, Dell EqualLogic has been offering impressive solutions recently that would be a better option than building your own.