Some useful vim plugins and keymaps

Over the years I have used couple of fancy plugins and key mappings but here I am listing a few which are almost always required. Everytime setting it up from scratch becomes a pain so I am listing it here so that I can just do copy paste directly from my blog and in general if it is useful to anyone else.

  • NerdTree
    1. Download it from http://www.vim.org/scripts/download_script.php?src_id=17123
    2. Unzip into your .vim directory
    3. In your .vimrc you can have the following key maps
map <F2> :NERDTree<CR>
let NERDTreeIgnore = ['\.pyc$', '\.lo$', '\.o$', '\.la$']

The first one maps F2 key to open up the NerdTree Browser and the second one does not show the binary object, .pyc files.

nnoremap <silent> <C-Right> <c-w> l
nnoremap <silent> <C-Right> <c-w> h 
nnoremap <silent> <C-Right> <c-w> k 
nnoremap <silent> <C-Right> <c-w> j

These keymaps make it easier to do move from one window to another when multiple windows are open. I find it an extra key to be pressed for moving using ctrl + w and then pressing the arrows or jklh keys. Moreover due to overuse (specially because of ALT + Tab) my thumbs start to ache and pressing CTRL + w requires pressing CTRL from my thumb and w using one of my finger. So to avoid using my thumb I find this mapping quite better.

  • Cscope
      1. Download the vim plugin for cscope from http://cscope.sourceforge.net/cscope_maps.vim
      2. Place the plugin in your .vim/plugin directory
      3. Now you can generate the files for which you want to search in and build the database. I generally store the cscope database and generated files in a separate directory in home. And then I run the cscope pointing to the cscope.files or the projectname.files and generate the cscope database files also in the ~/cscope directory. Finally I create a couple of scripts in my ~/bin directory to set the CSCOPE_DB environment variable to point to the cscope database file for each project.
mkdir ~/cscope
find full-path-to-project -name "*.[ch]" -o -name "*.cpp" -o -name "*.cxx" -o -name "*.hpp" > ~/cscope/projectname.files
cscope -i ~/cscope/projectname.files -q -k -b -f ~/csope/projectname.out
export CSCOPE_DB=~/cscope/projectname.out
  • Session Saver plugin
    1. Download the plugin from http://www.vim.org/scripts/script.php?script_id=3150
    2. Place it in your .vim/plugin directory
    3. That’s all. Everytime you open vim without specifying file, it will ask you to open your last saved session and when you quit vim, it will ask you to save your session. If you don’t want it to automatically load or save your session then you can set the following in your .vimrc
let g:session_autosave = 'no'
let g:session_autoload = 'no'

Apart from this some of the settings that I generally keep in my .vimrc are:

syntax on
filetype plugin indent on
set expandtab
set sw=4
set ts=4
set textwidth=80
set autoindent
autocmd BufEnter * lcd %:p:h

set wildignore=*.o,*.swp,*.pyc,*.svn,*.git
set title
Advertisements

How to install skype on debian wheezy with x86-64 / amd64 machine

I have been trying to install skype on my debian wheezy installation on amd64 a.k.a. x86-64 bit machine. It has taken several days of my productive time just to install skype just because the skype version available for download on Linux Debian is an i386 package and due to dependencies issues, it fails to install. Even though the download page says that the package is for multiarch but actually it is a i386 based package. I have tried installing other .deb packages available on skype.com but still it does not configure the skype. dpkg reports that it is leaving the package unconfigured and suggests that you should run:

sudo apt-get install -f

But that also does not help as it simply uninstalls skype. I have tried the instructions as listed here http://wiki.debian.org/skype but that simply does not help.

Finally the options that I was left with was:

  1. Upgrade my whole system to sid (which at the time of writing is quite unstable, whicih usually is the case after a release).
  2. Make my system dual boot with sid and the other as wheezy so that I have at least one OS running even if sid breaks.
  3. The last option was to install sid in a chroot and run skype from within chroot.

Even though the last option seems inelegant but that is the best unless you want to live on the edge.

To setup a sid chroot one can use debootstrap which makes it easier to setup chroot and follow the instructions listed here:

http://wiki.debian.org/Debootstrap

Once that is done, you can also install schroot package which makes it easier to run chroot. Once I did that, I was still unable to run skype because the xserver would not allow connection to the chrooted system so that the gui for skype shows.

To be able to run skype, you first need to allow xserver to let a client connect. To do so simply run the following on the main system:

xhost +

Viola! Now the gui for skype shows up but wait … the sound card still does not work.

For it make sure you have libpulse installed. If it is not installed, install it using:

apt-get install libpulse0:i386

The other thing that you need to do is configure alsa.

. Configure alsa in the chroot to use pulse by default by creating/editing /etc/asound.conf. Put the following in the file asound.conf:

pcm.pulse {
    type pulse
}
ctl.pulse {
    type pulse
}
pcm.!default {
    type pulse
}
ctl.!default {
    type pulse
}

The other thing that you might need to do is copy the file /var/lib/dbus/machine-id from host to chrooted system.

Once you do these steps run skype again and it should now have sound working on skype. This last part is based on the following post:

http://blog.einval.com/debian/misc/pulseaudio_chroot.comments

Hopefully once you have this, the skype should be running on your debian installation 🙂

How to install Java 6 on ubuntu 12.04 / 12.10

Source of Confusion: Many Versions of Java

You will find many methods to install java / jre/ jvm on web and thus it can be very much confusing to a user which method to use. Given there are many implementations of Java available, adds to the confusion. The different implementations of JDK available are:

  1. Java from Oracle (earlier provided by Sun which has been acquired by Oracle
  2. Java available from IBM
  3. OpenJDK
  4. GNU’s JDK

On an ubuntu system, one expects Java to be available as standard debian package installable with apt-get install <package-name>. But on ubuntu packages installed this way is the Open Source Version. If that suits your purpose, you can simply install it using:

$ sudo apt-get install openjdk-6-jdk

However if you wish to install the “Standard Java (earlier Sun Java)”, then it too can be installed either directly from binary distribution or from ubuntu alternate package repository.

Most Common Way:

The simplest way to install java is from a third party repository with

$ sudo add-apt-repository ppa:webupd8team/java

$ sudo apt-get update

$sudo apt-get install oracle-java7-installer

Caveat

But this works only for Java 7 as Java 6 is not available from ubuntu repositories because of licensing issues. If you download and install Java 6 locally on you machine in your home directory then it is not available to other users on the system and sometimes launching an application which is dependent upon java would be unable to find the java installed in your home directory. Moreover you have to mess with JAVA_HOME, CLASSPATH etc. environment variables. In such a case the best method is to install Java6 manually by following the steps below:

UPDATE: Today I was trying to build android from source and discovered that Java 6 is available now from webupd8team repository. You can simply install oracle / sun java 6 with

$sudo apt-get install oracle-java6-installer

How to install Java 6 Manually

Download the jdk binary from oracles website and follow the steps below:

 $ chmod u+x jdk-6u34-linux-i586.bin
 $ ./jdk-6u34-linux-i586.bin
 $ sudo mkdir -p /usr/lib/jvm
 $ sudo mv jdk1.6.0_34 /usr/lib/jvm/
 $ sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.6.0_34/bin/java" 1
 $ sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.6.0_34/bin/javac" 1
 $ sudo update-alternatives --install "/usr/lib/mozilla/plugins/libjavaplugin.so" "mozilla-javaplugin.so" "/usr/lib/jvm/jdk1.6.0_34/jre/lib/i386/libnpjp2.so" 1

Want to install Multiple Versions of Java?

If you already have some other version of java installed on your system say like the Java 7 or say the OpenJDK then you can choose the default java to be used on your system using the update-alternative command. Running the update-alternative shows the list of alternatives for the command and you can choose one of the options listed. As an example if say there are multiple programs fulfilling the same functionality then which one is to be used by a program is not clear. For example say you have installed Aptana Studio IDE on your system which is Eclipse based then to run Aptana it requires jre. If there are multiple versions of jre installed on your system then it is not clear which one should be used to run Aptana. This is where update-alternatives comes into play. For more details check the man page of update-alternatives. To select the version of java to be used by default run the following commands and select the appropriate version.

$ sudo update-alternatives --config java
$ sudo update-alternatives --config javac
$ sudo update-alternatives --config mozilla-javaplugin.so
$ sudo update-alternatives --config javaws

Running the commands above would show a prompt something like this. The details may vary on system to system.

There are 2 choices for the alternative java (providing /usr/bin/java).  
Selection Path Priority Status 
———————————————————— 
* 0 /usr/lib/jvm/java-6-openjdk/jre/bin/java 1061 auto mode 
1 /usr/lib/jvm/jre1.7.0/jre/bin/java 3 manual mode  

Press enter to keep the current choice[*], or type selection number: 1

Why debian rocks

It has been quite some time, actually years, since I started using Debian. Earlier I have used various flavors of Linux like SuSE, Red Hat, RHEL, Fedora, Ubuntoo but my experience with Debian has been so far the best. I just thought to write the reasons for which I really like Debian compared to other systems.

1. Excellent package management tools

I find that the package management tools that Debian offers is really fantastic. There are host of tools like Synaptic, Aptitude, Apt, dpkg, dselect etc. which one can choose based on how much one is comfortable with command line, TUI or GUI. One can easily choose and configure from where to install the package i.e. CD, standard Debian repositories hosted around the world, or some special Debian package repositories. Given I have my Debian CDs downloaded in ISO format, I can choose to loop mount them and install packages from them without the need to put the CDs for installation every time into the CDROM and without the need to download the package from internet every time. Above all the package dependencies are automatically resolved and installed / removed making the installation process a real breeze.

Apart from this there are host of packages like checkinstall (although available on other systems as well), localepurge, module assistant, debootstrap, pbuilder etc., which work so well in conjugation with other Debian package management tools that it is hard to describe in just one line or paragraph. Each of these tools really deserve a separate blog entry 🙂

In the past I have had real troubles specially with RPMS and local self installed packages cluttering the system, unresolved package dependencies that I felt a lot many times that I am spending too much time maintaining my Linux system than actually doing some work and had a strong desire to switch back to Windows.

2. Flexibility with the choice of installation of debian itself

Debian offers various ways to install Debian. One can choose either installing directly from internet with a minimal net install CD or one can install using the CD or DVD downloaded from internet or even from CD / DVD images downloaded using Jigdo. I think so is the case with other distributions, but I found installation of Debian offers much more flexibility. There are distributions like Gentoo which even build complete packages from sources but I believe that it is a waste of time and bandwidth unless you really want every bit of your computer resource to be utilized to its maximum possible extent.

3. Very active community of Users and Developers

The Debian users / developers community is so huge and active that most problems that you are going to face while installation, would have already been resolved or someone from the community will be able to help you out.

4. Larger support base

A lot of companies which provide support for their H/W on Linux, usually provide support for Debian apart from other major Linux distributions. That is at least they provide Debian specific packages or installers which specifically support Debian.

5. More suitable for programming and development

In the past having used SuSE, which offered a host of gizmo stuff for managing, installation and other utilities by default, it made me feel that I was no longer in control of my Linux System. Everything was so opaque and hidden from the user. Debian is simple in comparison and which is why it is so powerful. I really feel I am in control of my system.

6. Ports available for a large number of architectures and target device

Debian is available for a variety of architectures and platforms, which further increases the user base as well as the support base. Ports of debian exist for most common architectures like ARM, MIPS, PPC, x86, x86_64 etc. There are variety of GNU/Linux distribution which are Debian based, most notable of them is Ubuntoo, which further increases the support and user base.

The net result of having so many ports of debian is that one can run Debian on systems ranging from Servers to Desktops to Embedded Devices.

Some Debian specific useful links

http://www.debian-administration.org/

http://wiki.debian.org/

http://www.emdebian.org/

http://atterer.net/jigdo/

Hardware Accelerated Virtualization with QEMU and KVM

In this post I will take you through a series of steps for setting up hardware accelerated virtualization on your PC, if it is supported, along with some background on virtualization. The assumption that I make here is that you already have a Intel/ AMD based processor with virtualization support with a GNU/Linux installation (preferably Debian).

Although it may sound little tricky but believe me, it is not as difficult to set it up, as it seems, given you follow the right steps.

A Must Read for those are planning to buy new PC hardware

Even if you don’t intend to set up virtualization on your PC, it will be a good read and probably you will think about Virtualization support before buying a new PC.

Why should You care?

Usually people prefer to stick with an Operating System which they are more comfortable with and even though they may setup a dual boot machine, they hate to switch between them frequently. At the same time, once in a while you require the other OS for certain things…for example to test some software, to learn an OS (for example if you are a Windows user and want to familiarize yourself with GNU/Linux), to isolate an environment for a software (say a crack for a software which you do not want to infect your windows installation 🙂 ) etc. , Thus having Hardware Accelerated Virtualization will definitely make the experience of using a Virtual or Guest OS a breeze and will avoid any kind of frustation due to slow emualation without h/w accelerated virtualization.

What is Virtualization?

Virtualization is a mechanism which allows running one kind of environment inside another environment. It can be of several types:

1. API Virtualization:

It is quite like cygwin, where GNU/Linux API calls are translated to equivalent windows API calls. But the apps need recompilation to run on cygwin. Another example is WINE (windows emulator) which does the reverse, i.e it runs unmodified windows applications to be run on Linux.

2. OS Virtualization / Full Virtualization:

It is the kind of virtualization where a complete Operating System runs within another Operating System. It could be any combination of OS (Linux inside Linux, Windows inside Linux, Windows inside Windows with 32, b4 bit combinations as well). it is most popular form of virtualization and is usually required by a lot of people for testing softwares. Examples of this kind of virtualization are: QEMU, VMWare workstation

3. OS Level Virtualization:

It is the application level virtualization provided by the OS itself for the applications. Examples of this include ‘chroot’ jail, where an application runs inside a ‘jail’ with access to the OS resources only in that jail. This is also useful for application testing.

4. Emulators:

This kind of virtualization is usually referred, if not limited to, to the kind of virtualization where a complete environment is replicated. An example of this is Palm OS emulator, Android emulator, Nintendo Emulator etc. Even QEMU is an example of Emulator but it is usually referred to in the context of x86 instruction emulation or OS /Full Virtualization.

5. Desktop Virtualization:

In this kind of virtualization the desktop of a user is emulated / exported to another remote system from where it can be manipulated. Examples of this are: VNC viewer, Windows Remote Desktop. I would even consider eyeOS to be a kind of desktop virtualization.

For a detailed and complete list of different types of Virtualization, refer to Wikipedia article http://en.wikipedia.org/wiki/Virtualization

Hardware Accelerated Virtualization

Hardware Accelerated Virtualization is a kind of Full Virtualization which is accelerated with some hardware support. Not too long ago, it was difficult to do emulation of certain kind of instructions, without modifications to the Guest OS. The requirement for a system to be really called Full Virtualization system is that the Guest OS shouldn’t be modified. Apparently doing this was not so easy and even if implemented, the instructions were trapped and then emulated in software itself. This made the performance of a Virtual OS really slow in comparison to the native OS running on the system.

Hardware accelerated virtualization, solved this problem by proving certain instructions / extensions so that the ‘trap and emulate’ of instructions could be done in hardware. AMD and Intel processor added extensions called vmx or svm respectively to provide support for hardware assisted virtualization. To know if your processor supports these extensions type the following on a shell prompt:

$cat /proc/cpuinfo | grep “vmx \| svm”


flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority

flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority

Essentially the command above lists the info about your CPU and greps for flag which tells if virtualization extension is supported or not.

In short Hardware Accelerated Virtualization dramatically increases the speed of a ‘Guest OS’, so that it is as much responsive and usable as the Native / Host OS.

Hardware Requirements for Hardware Accelerated virtualization

The requirements for hardware accelerated virtualization are listed below:

  1. CPU support (found using cat /proc/cpuinfo | grep “vmx \| svm”)
  2. BIOS support –> Usually most recent Motherboards which support Core2 Duo processors have virtualization support but cross check this information by reading the motherboard manual (available online for most popular motherboard manufacturers).

Software Requirements for Hardware Accelerated virtualization

  1. QEMU –> Qemu is the emulator which acts as the emulator on which the virtualized or ‘Guest’ OS runs. It is similar to VMware client
  2. Linux kernel supporting kvm
  3. kvm.ko and kvm_intel.ko or kvm_amd.ko kernel modules which utilize the vmx | svm capability for h/w acceleration
  4. An OS image to run / test.

How to set up hardware virtualization:

Assuming you have hardware which is capable of accelerating virtualization,

Warning: Since the exact steps to setup hardware virtualization could vary from one Linux distribution to another, the steps below are only indicative and you may need to figure out how to set it up for your distribution. But the principle remains the same. The configuration on which I have been able to setup virtualization is listed below:

Intel Core2 Duo E8400 (3.0 Ghz). –> supports vmx extension

Asus P5P43TD motherboard –> BIOS for this motherboard does support virtualization and is enabled by default)

Debian 5.0.3 (lenny)

At the time of writing this post, kvm support in Qemu downloaded from Debian repository (0.9.1-10lenny1) is broken. Ideally one should be able to download and install Qemu and run it without any issues and Qemu will automatically use h/w acceleration, but that is not the case and hence you require some changes to your setup.

  • Compile 2.6.32 kernel, with Virtualization support enabled. Please note that the kernel that we build here is not the official Debian kernel wherein certain code is removed from mainline kernel to keep up with the Debian software policy.

      • $tar -jvxf linux-2.6.32.2.tar.bz2
      • cd linux-2.6.32.2
    • Copy your existing kernel config to untarred folder i.e. linux-2.6.32.2/.config
    • Run menuconfig, which is a ncurses based configuration chooser (lets you choose various features to be enabled / disabled in the Linux kernel). If it asks for questions for enabling / disabling of features, just choose the default, unless you know what you are doing.
      • make menuconfig
    • Select Virtualization support in the menuconfig (see the screenshots below).
    • Install any build dependencies. This will install g++
      • sudo apt-get install build-essential fakeroot
    • Compile and install the kernel the Debian way or you can compile it the usual way and install it (in that case skip this step). The following step will build the kernel and automatically do the changes to your bootloader (grub or lilo), copy the kernel modules into /lib/modules/2.6.32.2/, setup links in / pointing to appropriate vmlinuz, prepare initrd automatically. This actually makes building kernel very easy on a debian system.
      • fakeroot make-kpkg –initrd –revision=custom.1.0 kernel_image
      • sudo dpkg -i ../linux-image-2.6.18_custom.1.0_i386.deb

make menuconfig screen - 2

make menuconfig screen - 2

Screenshot when kernel source is configured with make menuconfig for virtualization support

  • Download, build and install the qemu-kvm package

    • Download the qemu-kvm package from http://sourceforge.net/projects/kvm/files/. In my case I downloaded qemu-kvm-0.12.1.1.tar.gz, which works fine for me.
    • Ensure that package checkinstall is already installed on your system. This is not essential as you can install a package on your system using usual ‘make install’ which will simply copy the required files in appropriate folders. But installing it through checkinstall ensures that you can get the status of your installed package and you can remove it like any standard debian package. To install ‘checkinstall’:
      • sudo apt-get install checkinstall
    • Do
      • $qemu-kvm-0.12.1.1.tar.gz
      • cd qemu-kvm-0.12.1.1
      • ./configure
      • make
      • checkinstall
    • Chekinstall asks some questions like the name of the package etc. which you can provide so as to keep track of the installed packages.
    • Once you do dpkg –list it shows the following:
      • dpkg –list qemu-kvm
        Desired=Unknown/Install/Remove/Purge/Hold
        | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
        |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
        ||/ Name                              Version                           Description
        +++-=================================-=================================-==================================================================================
        ii  qemu-kvm                          0.12.1.1-1                        qemu-self-compiled
    • To see the list of files installed by this package, run the folowing command:
      • dpkg –list qemu-kvm
      • This lists a lot of files of which the following is important
      • ….
      • …..
      • …..
      • /usr/local/bin/qemu-system-x86_64
        /usr/local/bin/qemu-nbd
        /usr/local/bin/qemu-io
        /usr/local/bin/qemu-img
        /etc/init.d/qemu-kvm
  • Now reboot your system
  • Add the user to kvm group. This is essential so that the qemu can read the device /dev/kvm, which by default has permissions 600. Simply changing the permission to 666 is not a good option as this will be reset to 600 on next reboot.
    • sudo adduser divkis01 kvm
  • Before you can us QEMU + KVM, you need to have an OS image to be able to boot it with QEMU. To prepare images,
  • First make an ISO image of your XP, Ubuntoo CD or alternatively if you already have an Iso Image skip this step
    • dd if=/dev/cdrom of=~/MyMachines/winxp.iso
  • Now create an image in which the OS installation will be done.
    • qemu-img create -f qcow2 winxp_x86.img 10G
  • Install the OS on the image created in last step
    • qemu-system-x86_64 -hda winxp_x86.img -cdrom ~/MyMachines/winxp.iso -boot d -m 512
  • Once the installation is complete, you are ready to run Qemu with hardware acceleration. You can boot the OS using the following command:
    • qemu-system-x86_64 ~/MyMachines/windows/winxp_x86.img

See the screen shots below which shows Windows XP and Ubuntoo booted on a Debian GNU/Linux host

QEMU running Windows and Ubuntoo images on Debian host

Caveats

  • If you see that your native OS (on which you are running QMU) is fast enough but the guest OS runing on QEMU is too slow, then it is possible that hardware acceleration is not being used. There are several ways to figure out if that is the case or not.
    • Listing the currently loaded module and grepping for kvm, should show non zero entries in the right most column. For example on my machine, it shows:
      • lsmod | grep kvm
        kvm_intel              31979  6
        kvm                   181149  1 kvm_intel
    • On running QEMU, if QEMU complains with the following message
      • qemu-system-x86_64 ~/MyMachines/Linux/ubuntoo-9.10.x86.img &
        divkis01@divkis01-pc:/home/data/downloads/isos/debian-x86$ open /dev/kvm: No such file or directory
        Could not initialize KVM, will disable KVM support
    • The performance is too low of the guest OS. I have found drastic difference at least 10x perceived difference with and without hardware acceleration
  • The reason for h/w acceleration not available could be due to any of the following reasons:
    • kvm not properly installed
    • user running QEMU is not part of kvm group i.e. the user does not have access to device /dev/kvm
    • kvm is installed but the kernel modules for kvm (kvm.ko and kvm_intel.ko / kvm_amd.ko) are not properly loaded
    • The installed kernel does not have kvm built as module
    • Or probably h/w acceleration is not supported by your processor or motherboard.

Useful Links

http://en.wikipedia.org/wiki/Hardware-assisted_virtualization

http://www.linux-kvm.org/page/Main_Page

http://www.nongnu.org/qemu/user-doc.html

http://kernel-handbook.alioth.debian.org/

http://wiki.debian.org/QEMU

If you face any difficulty setting up kvm or QEMU, don’t hesitate to send me questions. I will be glad to help. And if you find this page useful, please feel free to drop a comment.

Happy QEMUing!!!!

    2. Download, build and install the qemu-kvm package