What is the difference in the guest OS measurement of CPU and memory in a virtual machine (VM) versus the CPU and memory utilization of the host server? Why does each matter? Each serves their own purpose. Perspective is key to properly utilize them as you optimize your virtual environment.
At a high level, the CPU and memory utilization reported by the host server takes into account all the VMs on that system, their scheduling, and any privileged instructions as consumption takes place against that host’s resources. CPU and memory reported from this perspective is at a system level. Optimization requires having a holistic view of the system, the VMs on it, and their applications since over-commitment of system resources can cause bottlenecks.
The guest OS measurement of CPU and memory is measured from the perspective of the VM with respect to those resources that have been provisioned for it. These metrics have neither awareness of the VMkernel and its scheduling nor the physical system’s overall system metrics and the system’s scheduling though it can definitely be impacted by what happens on the system. Point in case, noisy neighbor VMs. Optimization focuses on understanding the behavior of the VM and its application. For instance, the threadedness of the application can come into play. A single-threaded application does not benefit from additional vCPUs because the app won’t be able to take advantage. Plus, those unused vCPUs waste pCPU and can hurt overall system performance, which in turn, can affect that VM's performance.
The end-goal remains delivering application Quality-of-Service that is acceptable to end-users. Both points-of-view are important in how one designs and implements their virtualization infrastructure as well as proper resource allocation to VMs and across clusters of host systems. They both serve purposes in root-causing bottlenecks and proper remediation of those issues.
Share your thoughts in the comments section. Below are additional materials for reference.
VMware’s CPU scheduler white paper: http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vmware-vsphere-cpu-sched-performance-white-paper.pdf
VMware Knowledge Base Article: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2032