Nested VMware 64bit VMs. Time to run Hyper-v or KVM inside VMware ESXi5.0 and Workstation 8.0

Working with multiple hypervisors has been great in experimenting with certain features or just gaining an understanding how to configure something similar to a customers environment. Until now my lab environment is littered with multiple servers  or harddisks with different images upon them but having the ability to work  with these technologies with minimal hardware in lab environment or even mobile out and about would be great. Until recently this was not possible due to the fact that nested 64bit VMs were not possible with Workstation 7.x or even ESX 4.x for that matter. But now with ESX5i and Workstation 8 this is now possible.

One caveat to add though is that your hardware must be VT-x capable which of course most are these days but with one additional and really important CPU instruction set of EPT (Enhanced Page Tables) or RVI from AMD must be present, searching the various chip manufactures sites it seems that if your running Nehalem or i7 from Intel at least then your good to go. Luckily I have both an i7 on my laptop and Nehalem in my HP ML G6 home test lab. So lets take a look at ESX5i first and see what needs to be set for Hyper-v to load.

VMware ESX5i configuration.

1. First you need to enable hardware virtualization by modifying the etc/vmware/config file. Enable SSH via tech support mode and putty to the ESX5i server

2. Once connected with putty  :

# echo ‘vhv.allow = “TRUE” ‘ >> /etc/vmware/config

3. Next create your Virtual Machine hardware, I personally used hardware version 8 to make things easier with configuration.

4. Before you get to booting up the VM and installing Hyper-V you need to add two lines the virtual machines config file .vmx

You can either add these via the vSphere Client in the settings of the virtual machine > Configuration Parameters, or doing it from command-line

To add them using command-line move back in SSH > change into the directory where you Hyper-V VM is installed

For example config file where my VM is located is called Hyper-V.vmx. Type the following commands:

# echo ‘monitor.virtual_exec = “hardware” ‘ >> Hyper-V.vmx

# echo ‘hypervisor.cpuid.v0 = “FALSE” ‘ >> Hyper-V.vmx

5. Next there are a couple of changes to be made with the CPU configuration.

in the VM settings > Options > CPU/MMU
Virtualization
make sure you select the option to pass the Intel EPT feature.

6. Next move to the Options area > CPUID Mask click on Advanced

 

Add the following CPU mask Level ECX: —- —- —- —- —- —- –H- —-

8. Finally you are now ready to install Windows  2008/R2 and enable the Hyper-V role.

VMware Workstation 8 configuration.

For configuration with Workstation 8.0 things are a little easier.

1. Start off by creating a new VM again I used version 8 hardware

2. Configure the VM with RAM and Disk etc….

3. Important step here select Windows 2008 R2 x64  and not ESX5i as you probably did with nested 32bit VMs.

4 Under the settings of the VM > CPU, make sure  you have the option to pass-through the Intel VT-x/EPT feature.

6. Make sure you have set the VM to boot from  Windows 2008 R2 x64 media ISO.

7. Before booting, you should edit the config file  .vmx and add the parameter: hypervisor.cpuid.v0 = “FALSE”

8. Now Boot and Install Windows 2008 R2 x64.

Next stop for me is to see if I can get KVM working also and then I will be happy.

About virtuallylg

Hello, my name is Lorenzo Galelli, I have been working with availability and virtualization solutions for Symantec for over a decade now and its amazing to see the impact virtualization has brought to the world of IT. During my time at Symantec I have worked as a systems engineer for customers big and small and seen a vast array of different virtualization projects. I am currently Technical Product Manager for ApplicationHA for VMware and KVM and I also have focus on VDI especially with Symantec's VirtualStore and FileStore technologies. Follow my blog for all things Symantec and virtualization. Opinions expressed here are my own.
This entry was posted in VMware. Bookmark the permalink.

5 Responses to Nested VMware 64bit VMs. Time to run Hyper-v or KVM inside VMware ESXi5.0 and Workstation 8.0

  1. Axel Elfner says:

    Hi Lorenzo,

    Thank you for this excellent reference. I’m attempting to create 2nd level VMs in KVM on top of ESX5. I know of someone who has done this successfully on Workstation 8, but I am having issues. The 2nd level KVM VM hangs on boot and the qemu proc on the 1st level VM spins it’s CPU at 90+%. Appreciate if you have any insight.

  2. Hi,
    We are actually, trying to work with nested VMs and KVM. Our model is, VMWare Workstation and a Ubuntu Linux version on it as VM and our KVM hypervisor of type 2 on Ubuntu and Archlinux on this KVM. so our concern, is how efficient will our system utilize and work, with respect to hardware virtualization capabilities of the main host system. How can we have a measure of this?

  3. Magued says:

    Hi,

    did you manage to get KVM running on workstation?
    I am trying now with no luck.

    thanks,
    Magued

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s