Wednesday, August 21, 2013

Nested Virtualization with VMWare Workstation and KVM

I once intended for this to be a daily blog.  I see it's been two years since my last entry.  Ooops.

Anyway, I overcame a technical challenge (i.e., snafu) today for which Google wasn't much help.  So here we go.

Scenario:  I have a PC.  It has 32 GB of RAM.  I use it both for games and other stuff requiring Windows, and for certification labs, which I run on WMWare Workstation because I got a free license when I passed my VCP.

I recently began studying for my RHCSA and eventually RHCE.  The books for these certifications come with lab virtual machines.  The lab VMs are in KVM format.  KVM is a Red Hat exam certification topic, and it doesn't run on Windows.

I didn't want to convert the VMs to VMWare format because I need to learn KVM anyway.  But I considered it.  Everything I found on Google was about converting VMs the other direction.  This doesn't bode super-well for the market value of my VMWare certification, but never mind.

The free VMWare converter does not recognize KVM format.

Giving up on the conversion idea (though I'm sure there is some way to make it work) I moved on to a nested scenario - running the lab VMs on a CentOS host which is itself a guest of VMWare Workstation under Windows.  The RedHat lab guide (the McGraw Hill one, Michael Jang is the author) explicitly says not to do this; that KVM will not install on a VM, or if it does it will act badly.  My inner first-world anarchist accepted the challenge.

In fact, it worked immediately, but it sucked (technical term).  Performance was unbearable. So...

Objective: to make this nested virtualization scenario pleasant enough to use.

Resolution:  It became clear pretty quickly, though I don't recall how, that this was a hardware virtualization problem.  That is, the virtualization boost that modern processors provide to virtualization guests wasn't being relayed on to the guests of the guest.  There is an option to do this in  VMWare called "Virtualize Intel VT-x/EPT or AMD-V/RVI".



Unfortunately it was greyed out on my VMs.  Fortunately adding it is just a matter of adding the following line at the end of the .vmx file for the VM:

 vhv.enable = "true"

From there, shut down and re-enable the RedHat guest host (host guest?) VM.  The RedHat host guest (guest host?) should now see the its CPU as having virtualization assist.  You can verify like so:

  [user@CentOS64-VHost ~]$ virt-host-validate
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking for device /dev/kvm                                          : PASS
  QEMU: Checking for device /dev/vhost-net                                  : PASS
  QEMU: Checking for device /dev/net/tun                                      : PASS
   LXC: Checking for Linux >= 2.6.26                                            : PASS

The last thing that must be done is to enable the vmx processor option (for the passthrough hardware assist) in the virtual machine settings of the guests (guest guests?) in KVM.  The easiest way to do this is to set advanced processor options when you create a VM in KVM.  Then use the "Copy Host CPU Configuration" button.



Restart the KVM guest if you need to, and you're good.  Fast nested guests.