Hardware Accelerated Virtualization with QEMU and KVM

In this post I will take you through a series of steps for setting up hardware accelerated virtualization on your PC, if it is supported, along with some background on virtualization. The assumption that I make here is that you already have a Intel/ AMD based processor with virtualization support with a GNU/Linux installation (preferably Debian).

Although it may sound little tricky but believe me, it is not as difficult to set it up, as it seems, given you follow the right steps.

A Must Read for those are planning to buy new PC hardware

Even if you don’t intend to set up virtualization on your PC, it will be a good read and probably you will think about Virtualization support before buying a new PC.

Why should You care?

Usually people prefer to stick with an Operating System which they are more comfortable with and even though they may setup a dual boot machine, they hate to switch between them frequently. At the same time, once in a while you require the other OS for certain things…for example to test some software, to learn an OS (for example if you are a Windows user and want to familiarize yourself with GNU/Linux), to isolate an environment for a software (say a crack for a software which you do not want to infect your windows installation :) ) etc. , Thus having Hardware Accelerated Virtualization will definitely make the experience of using a Virtual or Guest OS a breeze and will avoid any kind of frustation due to slow emualation without h/w accelerated virtualization.

What is Virtualization?

Virtualization is a mechanism which allows running one kind of environment inside another environment. It can be of several types:

1. API Virtualization:

It is quite like cygwin, where GNU/Linux API calls are translated to equivalent windows API calls. But the apps need recompilation to run on cygwin. Another example is WINE (windows emulator) which does the reverse, i.e it runs unmodified windows applications to be run on Linux.

2. OS Virtualization / Full Virtualization:

It is the kind of virtualization where a complete Operating System runs within another Operating System. It could be any combination of OS (Linux inside Linux, Windows inside Linux, Windows inside Windows with 32, b4 bit combinations as well). it is most popular form of virtualization and is usually required by a lot of people for testing softwares. Examples of this kind of virtualization are: QEMU, VMWare workstation

3. OS Level Virtualization:

It is the application level virtualization provided by the OS itself for the applications. Examples of this include ‘chroot’ jail, where an application runs inside a ‘jail’ with access to the OS resources only in that jail. This is also useful for application testing.

4. Emulators:

This kind of virtualization is usually referred, if not limited to, to the kind of virtualization where a complete environment is replicated. An example of this is Palm OS emulator, Android emulator, Nintendo Emulator etc. Even QEMU is an example of Emulator but it is usually referred to in the context of x86 instruction emulation or OS /Full Virtualization.

5. Desktop Virtualization:

In this kind of virtualization the desktop of a user is emulated / exported to another remote system from where it can be manipulated. Examples of this are: VNC viewer, Windows Remote Desktop. I would even consider eyeOS to be a kind of desktop virtualization.

For a detailed and complete list of different types of Virtualization, refer to Wikipedia article http://en.wikipedia.org/wiki/Virtualization

Hardware Accelerated Virtualization

Hardware Accelerated Virtualization is a kind of Full Virtualization which is accelerated with some hardware support. Not too long ago, it was difficult to do emulation of certain kind of instructions, without modifications to the Guest OS. The requirement for a system to be really called Full Virtualization system is that the Guest OS shouldn’t be modified. Apparently doing this was not so easy and even if implemented, the instructions were trapped and then emulated in software itself. This made the performance of a Virtual OS really slow in comparison to the native OS running on the system.

Hardware accelerated virtualization, solved this problem by proving certain instructions / extensions so that the ‘trap and emulate’ of instructions could be done in hardware. AMD and Intel processor added extensions called vmx or svm respectively to provide support for hardware assisted virtualization. To know if your processor supports these extensions type the following on a shell prompt:

$cat /proc/cpuinfo | grep “vmx \| svm”


flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority

flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority

Essentially the command above lists the info about your CPU and greps for flag which tells if virtualization extension is supported or not.

In short Hardware Accelerated Virtualization dramatically increases the speed of a ‘Guest OS’, so that it is as much responsive and usable as the Native / Host OS.

Hardware Requirements for Hardware Accelerated virtualization

The requirements for hardware accelerated virtualization are listed below:

  1. CPU support (found using cat /proc/cpuinfo | grep “vmx \| svm”)
  2. BIOS support –> Usually most recent Motherboards which support Core2 Duo processors have virtualization support but cross check this information by reading the motherboard manual (available online for most popular motherboard manufacturers).

Software Requirements for Hardware Accelerated virtualization

  1. QEMU –> Qemu is the emulator which acts as the emulator on which the virtualized or ‘Guest’ OS runs. It is similar to VMware client
  2. Linux kernel supporting kvm
  3. kvm.ko and kvm_intel.ko or kvm_amd.ko kernel modules which utilize the vmx | svm capability for h/w acceleration
  4. An OS image to run / test.

How to set up hardware virtualization:

Assuming you have hardware which is capable of accelerating virtualization,

Warning: Since the exact steps to setup hardware virtualization could vary from one Linux distribution to another, the steps below are only indicative and you may need to figure out how to set it up for your distribution. But the principle remains the same. The configuration on which I have been able to setup virtualization is listed below:

Intel Core2 Duo E8400 (3.0 Ghz). –> supports vmx extension

Asus P5P43TD motherboard –> BIOS for this motherboard does support virtualization and is enabled by default)

Debian 5.0.3 (lenny)

At the time of writing this post, kvm support in Qemu downloaded from Debian repository (0.9.1-10lenny1) is broken. Ideally one should be able to download and install Qemu and run it without any issues and Qemu will automatically use h/w acceleration, but that is not the case and hence you require some changes to your setup.

  • Compile 2.6.32 kernel, with Virtualization support enabled. Please note that the kernel that we build here is not the official Debian kernel wherein certain code is removed from mainline kernel to keep up with the Debian software policy.

      • $tar -jvxf linux-2.6.32.2.tar.bz2
      • cd linux-2.6.32.2
    • Copy your existing kernel config to untarred folder i.e. linux-2.6.32.2/.config
    • Run menuconfig, which is a ncurses based configuration chooser (lets you choose various features to be enabled / disabled in the Linux kernel). If it asks for questions for enabling / disabling of features, just choose the default, unless you know what you are doing.
      • make menuconfig
    • Select Virtualization support in the menuconfig (see the screenshots below).
    • Install any build dependencies. This will install g++
      • sudo apt-get install build-essential fakeroot
    • Compile and install the kernel the Debian way or you can compile it the usual way and install it (in that case skip this step). The following step will build the kernel and automatically do the changes to your bootloader (grub or lilo), copy the kernel modules into /lib/modules/2.6.32.2/, setup links in / pointing to appropriate vmlinuz, prepare initrd automatically. This actually makes building kernel very easy on a debian system.
      • fakeroot make-kpkg –initrd –revision=custom.1.0 kernel_image
      • sudo dpkg -i ../linux-image-2.6.18_custom.1.0_i386.deb

make menuconfig screen - 2

make menuconfig screen - 2

Screenshot when kernel source is configured with make menuconfig for virtualization support

  • Download, build and install the qemu-kvm package

    • Download the qemu-kvm package from http://sourceforge.net/projects/kvm/files/. In my case I downloaded qemu-kvm-0.12.1.1.tar.gz, which works fine for me.
    • Ensure that package checkinstall is already installed on your system. This is not essential as you can install a package on your system using usual ‘make install’ which will simply copy the required files in appropriate folders. But installing it through checkinstall ensures that you can get the status of your installed package and you can remove it like any standard debian package. To install ‘checkinstall’:
      • sudo apt-get install checkinstall
    • Do
      • $qemu-kvm-0.12.1.1.tar.gz
      • cd qemu-kvm-0.12.1.1
      • ./configure
      • make
      • checkinstall
    • Chekinstall asks some questions like the name of the package etc. which you can provide so as to keep track of the installed packages.
    • Once you do dpkg –list it shows the following:
      • dpkg –list qemu-kvm
        Desired=Unknown/Install/Remove/Purge/Hold
        | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
        |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
        ||/ Name                              Version                           Description
        +++-=================================-=================================-==================================================================================
        ii  qemu-kvm                          0.12.1.1-1                        qemu-self-compiled
    • To see the list of files installed by this package, run the folowing command:
      • dpkg –list qemu-kvm
      • This lists a lot of files of which the following is important
      • ….
      • …..
      • …..
      • /usr/local/bin/qemu-system-x86_64
        /usr/local/bin/qemu-nbd
        /usr/local/bin/qemu-io
        /usr/local/bin/qemu-img
        /etc/init.d/qemu-kvm
  • Now reboot your system
  • Add the user to kvm group. This is essential so that the qemu can read the device /dev/kvm, which by default has permissions 600. Simply changing the permission to 666 is not a good option as this will be reset to 600 on next reboot.
    • sudo adduser divkis01 kvm
  • Before you can us QEMU + KVM, you need to have an OS image to be able to boot it with QEMU. To prepare images,
  • First make an ISO image of your XP, Ubuntoo CD or alternatively if you already have an Iso Image skip this step
    • dd if=/dev/cdrom of=~/MyMachines/winxp.iso
  • Now create an image in which the OS installation will be done.
    • qemu-img create -f qcow2 winxp_x86.img 10G
  • Install the OS on the image created in last step
    • qemu-system-x86_64 -hda winxp_x86.img -cdrom ~/MyMachines/winxp.iso -boot d -m 512
  • Once the installation is complete, you are ready to run Qemu with hardware acceleration. You can boot the OS using the following command:
    • qemu-system-x86_64 ~/MyMachines/windows/winxp_x86.img

See the screen shots below which shows Windows XP and Ubuntoo booted on a Debian GNU/Linux host

QEMU running Windows and Ubuntoo images on Debian host

Caveats

  • If you see that your native OS (on which you are running QMU) is fast enough but the guest OS runing on QEMU is too slow, then it is possible that hardware acceleration is not being used. There are several ways to figure out if that is the case or not.
    • Listing the currently loaded module and grepping for kvm, should show non zero entries in the right most column. For example on my machine, it shows:
      • lsmod | grep kvm
        kvm_intel              31979  6
        kvm                   181149  1 kvm_intel
    • On running QEMU, if QEMU complains with the following message
      • qemu-system-x86_64 ~/MyMachines/Linux/ubuntoo-9.10.x86.img &
        divkis01@divkis01-pc:/home/data/downloads/isos/debian-x86$ open /dev/kvm: No such file or directory
        Could not initialize KVM, will disable KVM support
    • The performance is too low of the guest OS. I have found drastic difference at least 10x perceived difference with and without hardware acceleration
  • The reason for h/w acceleration not available could be due to any of the following reasons:
    • kvm not properly installed
    • user running QEMU is not part of kvm group i.e. the user does not have access to device /dev/kvm
    • kvm is installed but the kernel modules for kvm (kvm.ko and kvm_intel.ko / kvm_amd.ko) are not properly loaded
    • The installed kernel does not have kvm built as module
    • Or probably h/w acceleration is not supported by your processor or motherboard.

Useful Links

http://en.wikipedia.org/wiki/Hardware-assisted_virtualization

http://www.linux-kvm.org/page/Main_Page

http://www.nongnu.org/qemu/user-doc.html

http://kernel-handbook.alioth.debian.org/

http://wiki.debian.org/QEMU

If you face any difficulty setting up kvm or QEMU, don’t hesitate to send me questions. I will be glad to help. And if you find this page useful, please feel free to drop a comment.

Happy QEMUing!!!!

    2. Download, build and install the qemu-kvm package
About these ads

9 thoughts on “Hardware Accelerated Virtualization with QEMU and KVM

  1. hi divick,

    wonder full post, your compilation of all the information at one place about virtualization is must read for everyone in the industry.
    Really great work. Keep it up dude.

    pankaj anand

  2. Hi Divick,
    Lovely post. I have a question . I am able to install the guest O/S but when I try to boot the guest O/S I get the following error:open /dev/kvm: No such file or directory.
    I have a Acer 5738G laptop with Intel processor. When I do cat /proc/cpuinfo | grep vmx I don’t get anything. Can you let me what is the problem.I am running qemu in my system.I have ubuntu 10.04 installed in my system.

    Regards,
    Sumit.

    • Hi Sumit,
      though your question is quite old but just in case it might be needed by someone else. Without seeing the exact h/w specification, it appears that your CPU/motherboard does not support h/w acceleration. Moreover even if it supports you need to have the support for h/w acceleration enabled in your kernel.

      Hope that helps,
      Divick

  3. Kindly do provide me a link which can help me install kvm properly.. As u said above in caveats, my guest OS is slow.. and i feel that i might have not installed kvm properly.. N also how do i check if H/W acceleration is being used or not??

    O/P for “lsmod | grep kvm” was:
    kvm_intel 46456 0
    kvm 286599 1 kvm_intel

    As there is a zero entry, does it mean that h/w acceleration is disabled??

    • Hi Ashwath,
      looks like I missed your comment somehow. If it is showing 0 then it simply means that is not being used. The number over their show how many instances are using the module. It does not mean that h/w acceleration is disabled. You need to run the qemu and then do lsmod and see if the number goes up. If it does not then h/w acceleration is not being used.

      Hope that helps,
      Divick

      • Thats exactly my issue, the obvious question is, how do I make it use the h/wacceleration? I guess it isnt automatic as I havent specifically told it not to.

  4. HI. DIvick.

    Nice information about qemu. I have a question.

    I am having trouble time with qemu.

    I am using KVM in Fedora 13 64 bit. My Guest Machine works fine and fast while I open it through Virtual-Machine-Monitor. But when I try to open Guest machine through the command line, my Guest OS becomes extremely slow. It takes 20-25 minutes to boot up. My hardware supports virtualization. It does not even show me the activity bar. I use the following command to run Guest:

    qemu fedora.img -m 1024

    Urgent help is needed. I need to run my Guest through command possibly because I need to pass a lot of parameters to the Qemu…

    • Hi Mohit,
      from the information you have provided it is hard to guess what’s going on but my guess is that you are not using the right binary for running the qemu. Check do you have qemu-system-x86_64 binary installed. Also check the Caveats section in my post. Do lsmod and see what does it print.

      Hope that helps,
      Divick

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s