VMware – managing virtual switches inside your ESX Server

From talking to customers, we know that many-possibly most-of you work in organizations that have embraced virtualization to some extent. VMware's ESX server is especially popular. One interesting aspect of ESX servers that hasn't gotten a lot of attention are the virtual switches inside that ESX server. We here at SolarWinds are wondering what the Thwack community thinks about virtual switches and virtual switch management.

As network engineers, how do you manage those virtual switches?
As network engineers, do you monitor virtual switches?
What visibility do you have into the network traffic inside the ESX server?
- Do you wish you had more?
- Have you ever experienced problems related to traffic inside the ESX servers?

Virtualization is only going to grow, and we'd like to hear your thoughts on how its impacting networking and network engineers.

Find more posts tagged with

VMware

esx

Accepted answers

All comments

ceclark

Denny-

It's a good thought, but from what I understand, the VM switches are generally dumb. They aren't configurable (like a physical switch), aren't manageable, they don't pass STP, etc. I have never experienced an issue with the virtual switches, but our VM guy is pretty good so he knows how to configure them. Even on my test system, I've never had a problem, but I have not purposely tried to break it either. I'd be interested in hearing if/how we can monitor them.

Collin

JeanC.

We currently are using VMware's ESX Servers. We are not monitoring the virtual switches. Should we?

FormerMember

the VM switches are generally dumb. They aren't configurable (like a physical switch), aren't manageable, they don't pass STP, etc

True, but don't worry about the how.

I have never experienced an issue with the virtual switches

That's a good data point.

but I have not purposely tried to break it either

Do you usually try to break your stuff?

I'd be interested in hearing if/how we can monitor them.

What would you want to know? What would make monitoring it a priority?

FormerMember

We are not monitoring the virtual switches. Should we?

I don't know. Should you?

ceclark

but I have not purposely tried to break it either

[quote]Do you usually try to break your stuff? ;-)[/quote]

Try sometimes, generally break, yes!

I would not see it as a priority for me. I would rather see enhancements in other areas (integration with Cirrus, web admin on Orion, etc)

slbrodbeck

I personally would like to have a better handle on the traffic that flows across the vSwitches in our VMware environment. We have a number of servers that our application staff swears is our fault for performance. We are looking at additional tools to dive into the server and disk systems in VMware, better network information would be beneficial as well.

As an aside, if the APM also provided information about the VMware environment, that would be beneficial. Again, our application developers have figured out how to write code in such a way that 100% of the fastest machines on the planet are still not enough to handle their code.

Details include, vCPU time assigned, Real CPU time, I/O wait time, Memory stats such shared pages, the ballon. On top of all this the reservation vs available statistics.

I would really need to sit down and think about what stats would be the most important. I bet if we had them all I could find a nugget of useful information like I did with Orion and Netflow when I discovered the 2 to 3GB file copies across a T-1 every time a person logged on.

rob.borias

We have several VM ware server here with multiple VM switches. We would like to see up/down, traffic flow, packet loss to them.

Yea I can see the guest servers and their nic's but seeing this info for the host would be great.

epenney

Physical servers with dedicated networks ports are rapidly becoming a thing of the past. On a daily basis our organization is virtualizing its physical servers. This is making network services job of providing feedback/performance monitoring very difficult. Any and all visibility, performance reporting that Orion can do for ESX infrastructure is so desperately needed.

SamuelB

I would like to see at least the same information for virtual switch interfaces that we can currently see with physical switch interfaces in Orion. Being able to identify the VMs that are the top bandwidth users would assist me in telling the Server Admins that the network is not the problem. We haven't experienced any problems yet, but I just know (based on history) that it is only a matter of time.

amm_orion

Indeed, I agree, I think it would be phenomenal to be able to get statistics from the VM Switches.

Then I'm guessing that VMWare would have to create some form of management access to them.

To answer your questions:

1) The server teams manage the VM Switches.
2) Network teams don't monitor them.
3) No visibility.
4) Yes wish Network Teams were able to look deeper into them.
5) Yes.. Had to monitor at the aggregate port on the Cisco switch that the server was connected to.

I personally think we need to see the VM Switches managed like any other switch on the network, with relevant security controls, port mirroring etc. that could be enabled.

craig999

Thats a great idea. We would love to see it.

makrez

Just an FYI, you can enable netflow on the virtual switches...I haven't done it yet

www.vmware.com/.../vi3_35_25_netflow.pdf

Thanks,

mark

FormerMember

Yes, you can. It's still listed as "experimental", but we are working with VMware on their efforts to make the support official.

jdedmundson

We are looking at Reflex Security http://www.reflexsecurity.com/ but we would like to monitor the virtual switches within Orion. As a network engineer, we are having a hard time managing the virtual switches. Server team is not too happy right now because we will not trunk all the VLANs to them. Because we just not for sure of how it will route.
We would like to monitor the virtual switches within Orion.
We do not have any visibility within the virtual network unless we use the Reflex Security.
Yes we want more.

Our biggest concern is that the server team will configure one of the virtual switches wrong and we could end up with routing problems. As it stands now they have to ask the Network team for the change, we make sure that the change will not conflict with any thing and after the change we monitor the traffic from the change to make sure we do not have any problems. But with virtual switches that is out of the Network team hands.

Our Tech Support Team likes Orion very much and they like that they can go to one system now and see everything that is going on with both the servers and the network.

So please Solarwinds give us the tools to look into the virtual world.

FormerMember

We have spoken to VMware about this very issue several times. Unless they expose the APIs for us to manage their switch, there is little we can do.

They announced their strategic approach at VMworld 2008. They intend to allow vendors to create their own virtual switches. And Cisco has first to announce a virtual switch for VMware ESX Servers. It's called the Nexus 1000v, and it runs their NX-OS and should be manageable just like any device running the nexus OS. It supports SNMP, NetFlow, etc.

I don't want to push Cisco's product. You can learn more here. Other vendors will certainly come out with similar solutions for VMware. This is VMware's direction and what they intend to support. We'll work with them on ths strategy.

unclegio

Cisco has a product comming out that will replace the current VMWare VSwitch. Then you can actuall hand off your networking at the virtual switch to your infrastructure team and just focus ont he servers (hosts and guests). the Cisco line will be called Nexus.