Virtualization is one of the fastest growing “new technologies” in the I.T. world, but it’s not cheap to implement, mostly because of storage costs. CPU and Memory capacities of today’s systems are more than sufficient to handle a host with several running virtual machines, but very few systems have adequate internal storage services to support the number of virtual machines that can run in the available memory space.
External storage is inevitable
One way to address the storage challenge is with external storage. Large virtualization implementations typically implement Fibre Channel or iSCSI SANs, but these are fairly expensive implementations, and quite often out of the reach of many organizations. File sharing solutions can be very cost-effective for more modest virtualization implementations, but they’ve not always been functional solutions for more than experimentation.
Another factor that encourages the use of external storage is the growing prevalence of both Microsoft Hyper-V and VMWare ESX/ESXi implementations in the same data center. Building out individual host servers with internal storage capacities gets very expensive. The differential in pricing between servers with large numbers of internal bays and a minimal number of internal bays can be a factor of ten. Shared storage can be a great cost-effective strategy for addressing the needs of hybrid environments.
Back in the really early days of virtualization, I recall experimenting with putting virtual disks on a file server, and found that to be a functional solution for a single virtual machine, but it quickly hit the wall with additional machines, due to limited network bandwidth. At that time, Gigabit Ethernet was not within easy reach of the masses, and only a limited number of virtual disks could be supported on a 100Mb/sec Ethernet connection. The only thing available at that time that could really provide the services needed were Fibre Channel SANs. Today available Ethernet network bandwidth exceeds the capacity of Fibre Channel.
Previous limitations of file sharing
Until recently, though, the file sharing technologies available for using external storage were the primary bottleneck. If you were working with Microsoft Hyper-V, the capabilities of SMB in Windows Server just didn’t measure up. In fact, some people believe that SMB v2 (introduced in Windows Server 2008) was actually a step backward in performance, and the SMB v2.1 patches done for Server 2008 R2 merely brought us back to the performance levels of Server 2003.
For VMWare ESX/ESXi the opportunities weren’t much better. The ESX/ESXi external file sharing model uses Network File System (NFS) v3. NFS has been the file sharing system in Unix and Linux for dozens of years, but it can be complicated to configure and tune performance. NFS v3 is almost 30-years old. It also has limitations with respect to security, performance, statefulness, and cluster support – all things that would be particularly useful when hosting virtual disks.
File sharing improvements.
In April, 2003, the IETF introduced NFS v4 (RFC 3530) which addressed security and performance, and introduced a stateful protocol model. In January, 2010, the NFS v4.1 (RFC 5561) enhancements added capabilities for clustered server deployments and scalable parallel access to file distributed among multiple servers, making NFS v4.1 a very robust file sharing protocol for use with virtualization. Concurrent with the NFS developments, Microsoft was hard at work making improvements to the SMB protocol, and has introduced those with SMB v3. Some of these improvements compare to the scale of improvements seen in NFS from v3 to v4.1.
NFS v4 and SMB v3 together
Combining the capabilities of SMB v3 and NFS v4.1 into a single file server brings an exceptionally powerful external storage solution to virtualization environments that are budget conscious, and Microsoft has made both of these protocols available in Windows Server 2012. Now, a single file server can provide storage services to both Hyper-V environments and ESX/ESXi environments, using each hypervisor’s ‘native’ file sharing protocols.
Today, Hyper-V v3 can use either SMB v3 or NFS v4.1, and soon (we hope) ESX/ESXi will be enhanced to also take advantage of these significant improvements in NFS v4.1. If you’re looking to expand your virtualization infrastructure, shared file storage might be something to include in your environment.
For those that are using file sharing to support your virtualization environments, what are your experiences?