Some useful vim plugins and keymaps

Over the years I have used couple of fancy plugins and key mappings but here I am listing a few which are almost always required. Everytime setting it up from scratch becomes a pain so I am listing it here so that I can just do copy paste directly from my blog and in general if it is useful to anyone else.

  • NerdTree
    1. Download it from http://www.vim.org/scripts/download_script.php?src_id=17123
    2. Unzip into your .vim directory
    3. In your .vimrc you can have the following key maps
map <F2> :NERDTree<CR>
let NERDTreeIgnore = ['\.pyc$', '\.lo$', '\.o$', '\.la$']

The first one maps F2 key to open up the NerdTree Browser and the second one does not show the binary object, .pyc files.

nnoremap <silent> <C-Right> <c-w> l
nnoremap <silent> <C-Right> <c-w> h 
nnoremap <silent> <C-Right> <c-w> k 
nnoremap <silent> <C-Right> <c-w> j

These keymaps make it easier to do move from one window to another when multiple windows are open. I find it an extra key to be pressed for moving using ctrl + w and then pressing the arrows or jklh keys. Moreover due to overuse (specially because of ALT + Tab) my thumbs start to ache and pressing CTRL + w requires pressing CTRL from my thumb and w using one of my finger. So to avoid using my thumb I find this mapping quite better.

  • Cscope
      1. Download the vim plugin for cscope from http://cscope.sourceforge.net/cscope_maps.vim
      2. Place the plugin in your .vim/plugin directory
      3. Now you can generate the files for which you want to search in and build the database. I generally store the cscope database and generated files in a separate directory in home. And then I run the cscope pointing to the cscope.files or the projectname.files and generate the cscope database files also in the ~/cscope directory. Finally I create a couple of scripts in my ~/bin directory to set the CSCOPE_DB environment variable to point to the cscope database file for each project.
mkdir ~/cscope
find full-path-to-project -name "*.[ch]" -o -name "*.cpp" -o -name "*.cxx" -o -name "*.hpp" > ~/cscope/projectname.files
cscope -i ~/cscope/projectname.files -q -k -b -f ~/csope/projectname.out
export CSCOPE_DB=~/cscope/projectname.out
  • Session Saver plugin
    1. Download the plugin from http://www.vim.org/scripts/script.php?script_id=3150
    2. Place it in your .vim/plugin directory
    3. That’s all. Everytime you open vim without specifying file, it will ask you to open your last saved session and when you quit vim, it will ask you to save your session. If you don’t want it to automatically load or save your session then you can set the following in your .vimrc
let g:session_autosave = 'no'
let g:session_autoload = 'no'

Apart from this some of the settings that I generally keep in my .vimrc are:

syntax on
filetype plugin indent on
set expandtab
set sw=4
set ts=4
set textwidth=80
set autoindent
autocmd BufEnter * lcd %:p:h

set wildignore=*.o,*.swp,*.pyc,*.svn,*.git
set title

How to install skype on debian wheezy with x86-64 / amd64 machine

I have been trying to install skype on my debian wheezy installation on amd64 a.k.a. x86-64 bit machine. It has taken several days of my productive time just to install skype just because the skype version available for download on Linux Debian is an i386 package and due to dependencies issues, it fails to install. Even though the download page says that the package is for multiarch but actually it is a i386 based package. I have tried installing other .deb packages available on skype.com but still it does not configure the skype. dpkg reports that it is leaving the package unconfigured and suggests that you should run:

sudo apt-get install -f

But that also does not help as it simply uninstalls skype. I have tried the instructions as listed here http://wiki.debian.org/skype but that simply does not help.

Finally the options that I was left with was:

  1. Upgrade my whole system to sid (which at the time of writing is quite unstable, whicih usually is the case after a release).
  2. Make my system dual boot with sid and the other as wheezy so that I have at least one OS running even if sid breaks.
  3. The last option was to install sid in a chroot and run skype from within chroot.

Even though the last option seems inelegant but that is the best unless you want to live on the edge.

To setup a sid chroot one can use debootstrap which makes it easier to setup chroot and follow the instructions listed here:

http://wiki.debian.org/Debootstrap

Once that is done, you can also install schroot package which makes it easier to run chroot. Once I did that, I was still unable to run skype because the xserver would not allow connection to the chrooted system so that the gui for skype shows.

To be able to run skype, you first need to allow xserver to let a client connect. To do so simply run the following on the main system:

xhost +

Viola! Now the gui for skype shows up but wait … the sound card still does not work.

For it make sure you have libpulse installed. If it is not installed, install it using:

apt-get install libpulse0:i386

The other thing that you need to do is configure alsa.

. Configure alsa in the chroot to use pulse by default by creating/editing /etc/asound.conf. Put the following in the file asound.conf:

pcm.pulse {
    type pulse
}
ctl.pulse {
    type pulse
}
pcm.!default {
    type pulse
}
ctl.!default {
    type pulse
}

The other thing that you might need to do is copy the file /var/lib/dbus/machine-id from host to chrooted system.

Once you do these steps run skype again and it should now have sound working on skype. This last part is based on the following post:

http://blog.einval.com/debian/misc/pulseaudio_chroot.comments

Hopefully once you have this, the skype should be running on your debian installation :)

How to install Java 6 on ubuntu 12.04 / 12.10

Source of Confusion: Many Versions of Java

You will find many methods to install java / jre/ jvm on web and thus it can be very much confusing to a user which method to use. Given there are many implementations of Java available, adds to the confusion. The different implementations of JDK available are:

  1. Java from Oracle (earlier provided by Sun which has been acquired by Oracle
  2. Java available from IBM
  3. OpenJDK
  4. GNU’s JDK

On an ubuntu system, one expects Java to be available as standard debian package installable with apt-get install <package-name>. But on ubuntu packages installed this way is the Open Source Version. If that suits your purpose, you can simply install it using:

$ sudo apt-get install openjdk-6-jdk

However if you wish to install the “Standard Java (earlier Sun Java)”, then it too can be installed either directly from binary distribution or from ubuntu alternate package repository.

Most Common Way:

The simplest way to install java is from a third party repository with

$ sudo add-apt-repository ppa:webupd8team/java

$ sudo apt-get update

$sudo apt-get install oracle-java7-installer

Caveat

But this works only for Java 7 as Java 6 is not available from ubuntu repositories because of licensing issues. If you download and install Java 6 locally on you machine in your home directory then it is not available to other users on the system and sometimes launching an application which is dependent upon java would be unable to find the java installed in your home directory. Moreover you have to mess with JAVA_HOME, CLASSPATH etc. environment variables. In such a case the best method is to install Java6 manually by following the steps below:

UPDATE: Today I was trying to build android from source and discovered that Java 6 is available now from webupd8team repository. You can simply install oracle / sun java 6 with

$sudo apt-get install oracle-java6-installer

How to install Java 6 Manually

Download the jdk binary from oracles website and follow the steps below:

 $ chmod u+x jdk-6u34-linux-i586.bin
 $ ./jdk-6u34-linux-i586.bin
 $ sudo mkdir -p /usr/lib/jvm
 $ sudo mv jdk1.6.0_34 /usr/lib/jvm/
 $ sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.6.0_34/bin/java" 1
 $ sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.6.0_34/bin/javac" 1
 $ sudo update-alternatives --install "/usr/lib/mozilla/plugins/libjavaplugin.so" "mozilla-javaplugin.so" "/usr/lib/jvm/jdk1.6.0_34/jre/lib/i386/libnpjp2.so" 1

Want to install Multiple Versions of Java?

If you already have some other version of java installed on your system say like the Java 7 or say the OpenJDK then you can choose the default java to be used on your system using the update-alternative command. Running the update-alternative shows the list of alternatives for the command and you can choose one of the options listed. As an example if say there are multiple programs fulfilling the same functionality then which one is to be used by a program is not clear. For example say you have installed Aptana Studio IDE on your system which is Eclipse based then to run Aptana it requires jre. If there are multiple versions of jre installed on your system then it is not clear which one should be used to run Aptana. This is where update-alternatives comes into play. For more details check the man page of update-alternatives. To select the version of java to be used by default run the following commands and select the appropriate version.

$ sudo update-alternatives --config java
$ sudo update-alternatives --config javac
$ sudo update-alternatives --config mozilla-javaplugin.so
$ sudo update-alternatives --config javaws

Running the commands above would show a prompt something like this. The details may vary on system to system.

There are 2 choices for the alternative java (providing /usr/bin/java).  
Selection Path Priority Status 
———————————————————— 
* 0 /usr/lib/jvm/java-6-openjdk/jre/bin/java 1061 auto mode 
1 /usr/lib/jvm/jre1.7.0/jre/bin/java 3 manual mode  

Press enter to keep the current choice[*], or type selection number: 1

A tutorial on Virtualenv to isolate python installations

  1. You don’t have sudo / root access on a development machine?
  2. You want to try out a new package without polluting the system directories?
  3. You want to test or experiment with a new version of a library without uninstalling the previous version?
  4. You want to test your code in an isolated environment before you deploy it to a production server?

If your answer to any of the questions above is YES, then this post if for you!

What is virtualenv?

Virtualenv is a tool to create isolated Python environments quite like chroot jail on Unix systems. In a chroot, programs cannot access anything outside of chroot but in virtualenv as the name implies, it creates isolated environments only with respect to libraries, but the programs can still access the files and folders normally.

Imagine if you are working on your next Web2.0 project and you want to upgrade to a newer version of a library to test your code with it first without breaking your existing python installation. Or say you want to ensure that you have same set of libraries as on the systems where you will be deploying your code. Moreover if your hosting service is shared, hence you don’t have sudo access to install any package in global python installation. Virtualenv, comes to your rescue in all these scenarios.

Virtualenv setup and usage

Step 1: Install virtualenv

$ pip install virtualenv

Step 2: Create a virtualenv

The following command creates a clean virtualenv devoid of any packages in a directory called “ve”. However you can use any name you want for the name of virtualenv directory.

$ virtualenv --no-site-packages ve
The --no-site-packages flag is deprecated; it is now the default behavior.
New python executable in ve/bin/python2.6
Also creating executable in ve/bin/python
Installing setuptools............done.
Installing pip...............done.

Note that –no-site-packages is the default behavior so it is not really necessary to pass this option but I included this for clarity.

The way it works is by creating symlinks to your global python installation, as you can see clearly using the tree utility.

$ tree -d ve/
ve/
|-- bin
|-- include
| `-- python2.6 -> /usr/include/python2.6
`-- lib
 `-- python2.6
 |-- config -> /usr/lib/python2.6/config
 |-- distutils
 |-- encodings -> /usr/lib/python2.6/encodings
 |-- lib-dynload -> /usr/lib/python2.6/lib-dynload
 `-- site-packages
 `-- pip-1.1-py2.6.egg
 |-- EGG-INFO
 `-- pip
 |-- commands
 `-- vcs

15 directories

Step 3: Use the isolated python environment

To use or enter the environment, use the following command:

$ source ./ve/bin/activate

Now you are in an new environment and any globally installed packages won’t be available here. If I list the packages before entering the virtual environment it shows all the installed packages in my global python installation.

$ pip freeze
Brlapi==0.5.5
CouchDB==0.8
Django==1.3
Fabric==1.3.4
GnuPGInterface==0.3.2
Mako==0.3.6
MarkupSafe==0.9.2
PAM==0.4.2
PIL==1.1.7
Scrapy==0.14.1
South==0.7.3
Twisted==12.0.0
...

After I run enter the environment, I see that no system packages exist in this installation/ virtualenv:

$ source ./ve/bin/activate
(ve)$ pip freeze
(ve)$

Also note that the command prompt on the shell now has a prefix ‘ve’ (the name of your virtual environment) which indicates you are in the virtual environment.

Step 4: Install the required packages

Now you can simply install the packages using pip and does not require sudo access as all the packages are installed in the virtualenv ‘ve’ created above:

(ve)$ pip install <package-name>

or from a requirements / dependency file:

(ve)$ pip install -r requirements.txt

Now you can continue your to develop and test your code in this environment.

Step 5: Exit the virtual environment

To leave the virtual environment simply run:

(ve)$ deactive
$

Notice that the command prompt has now reverted back to $ without the prefix ‘ve’.

Summary

Virtualenv and Pip are must use tools for advanced as well as beginners in python development. It seems daunting but both Pip and Virtualenv are quite easy to setup and use. Using them in combination would avoid a lot headache in the long run.

A short tutorial on Python packaging

If you are a beginner to Python, then you might find it confusing that there are several ways to install Python packages and which one to use. They are:

  1. distutils - With distutils you would typically download the package, untar it and then run:
     $ python setup.py install

    This will install the library files (packages and modules) into the ‘site-packages’ directory of the Python interpreter. On my ubuntu11.04 the location of site-packages is /usr/local/lib/python2.6/site-packages/. Note that distutils does not install dependencies.

     

  2. easy_install (part of setuptools package) - It automates the process by downloading a named package from PyPi (which is a repository of Python Packages). To install packages using easy_install run:
    $ easy_install <package-name>

    and it downloads and installs the package in ‘dist-packages’ directory of the Python interpreter.

     

  3. pip - succeeds easy_install as it has several advantages. A few of them are:
    • support for uninstallation of packages, which is not possible with easy_install
    • installs dependencies
    • partial / incomplete installation do not happen with pip
    • pip automatically installs all packages specified in a file with pip install -r requirements.txt

There are other benefits of pip which are listed here: http://stackoverflow.com/questions/3220404/why-use-pip-over-easy-install

To install packages using pip simply run:

$ pip install <package_name>

or to install all packages listed in a file run:

$ pip install -r requirements.txt

To see the list of installed packages run:

$ pip freeze

Apart from the tools listed above there are several others which succeed like disutils2 and distribute but they are not used very commonly.

Summary

Pip is the future and is the recommended way to install python packages. For best results use pip with virtualenv. I will talk about virtualenv in another post of mine in this series.

Coding? One size doesn’t fit all …

No, really I mean it!!!

Over the years having worked on a lot of projects, big and small, I have observed a few things that programmers tend to overdo, resulting in patchy code, huge code size doing too little work, unnecessary logs and sometimes insidious bugs as well. What follows might look like that I have something against organized code, patterns, code design but IMHO, there has to be a balance between time, money, code clarity / organization to achieve the results.

Lets talk some business first. Even though not in the realm of programmers but it is very important that programmers involved in a project must know:

1. The target audience and platform for the product / software that is being written.

2. Time and money allotted to a project is finite.

3. Code evolves. No one writes best code for the very first time, at least not me :) By best I mean the code which is efficient, clean, structured, modular, commented and has not bugs, unimplemented functionality.

4. The lifetime of a product is finite.

Sometimes programmers have a tendency to think that as long as a company is paying them salary (assuming you work for an organization), they need not worry about completion of the project instead focus on writing the best code. Though most of us think this way and probably it can be argued that to get the best out of someone, this is the right approach. But it is not true!!!

Programmers DO need to know and remember at all times the points listed above. Now let us move on to know what really happens if the points listed above are not taken care of when designing and writing code.

1. What really happens if the programmers do not know the target audience and platform?

You start off on a project. Now you gather requirements for the project. Figure out various building blocks of your software, sometimes using open source libraries and sometimes writing everything from scratch and sometimes doing a mix and match. Once you have figured out the various building blocks of your project, you start off writing code.

The first software engineering principle that comes to your mind is, make the code modular. What people really understand by modularity: implement every functionality separately. Since the intrinsic of all pieces is not clear for the very first time, you create lots of classes / files for every functionality you want in your project.

Most often or not, this is how most people write code:

/*

* Product: A really over ambitious project

* Author: An aspiring author

* Created: Some auspicious day

*/

Class A{

B objectOfClassB;

C objectOfClassC;

D objectOfClassD;

public:

A(…args);

/**

* Author: <Great Author 1>

* Date: Some auspicious day

* Upated Date: Some other auspicious day

*/

Init() {

Log.enter(Entering ….

Log.leaving(Entering ….

}

Destroy(){

Log.enter(Entering ….

Log.leaving(Entering ….

}

Start(){

Log.enter(Entering ….

Log.leaving(Entering ….

}

Stop(){

Log.enter(Entering ….

Log.leaving(Entering ….

}

Functionality A(){

Log.enter(Entering ….

Log.leaving(Entering ….

}

Functionality B(){

Similarly for all functions…

}

Functionality C();

Functionality Z();

}

Let us see what is wrong with the above code:

1. Comments: The comments in the code above are redundant and are problematic to update. The reason they are redundant are because things like author, date, product, updated functions and date are all captured by any decent SCM software like svn, svs, perforce etc.

2. Logging: In the code fragment above, you will see that there are logs while entering and leaving. Most often such logs are not needed in very small projects or do not really help in debugging unless there are lots of threads in the app interacting with each other. Sometimes they do help in debugging / tracing but most often I have found only the code debugger helps you recover from insidious bugs instead of logs. Overall logs do not offer much benefit unless your application runs day and night on some sort of server doing lots of transactions every day / hour / minutes.

3. Empty functions: Having too many functions being empty or probably not doing much doesn’t help as it adds complexity and code size. Such functions can be added on the fly.

Next software engineering principle that comes to your mind is make the code platform independent. Most often this is not the goal of a project. When managers and people having decision powers think of platform portability, they really mean that is should be easily portable to other platforms but what gets conveyed to programmers  is that write code portable to all platforms for all platforms in the beginning. What people really do, wrap every OS functionality into a class / module so that only OS files need change when ported. The next thing that they do is write code inside each such class for a few OS  / platforms. Let me show you what I mean:

Class Mutex{

#if LINUX

private pthread_t thread;

#elif WINDOZ

SOME_WINDOWS_SPECIFIC_THREAD thread;

….

#elif IRIX

#elif WHATEVER_ELSE…

}

Similarly other classes for other platform features…which can be really a huge list say for semaphores, Timer, Network , File Read/ Write, Logging, etc etc.

You might wonder what is wrong with the above piece of code. Well there is nothing wrong as such with the code but most often such platform portability targets are either unachievable or achievable only if there are ample funds available. The reason is even if you get to write such piece of code, it takes a lot of time to really test it on all the platforms, without which the code holds no value no matter how good it has been written.

There are numerous other scenarios and code snippets that I could have posted / explained here but I guess you got the point so I will save that for later.

The other issue that I see most often with most project is not really with the code but with the mindset. In an effort to write very clean, modular code, what most code designers really do is the following:

1. Over abstraction –> This usually is manifested in the form of having too many classes with deep nested hierarchies, a lots of abstract classes or interfaces and the core work being done by only a few classes.

2. Over encapsulation –> Sometimes to hide out the functionality people tend to over encapsulate. This is often manifested in the form that people tend to separate out every functionality in separate class of file. Though nothing wrong with this but in the end each such function tends to do very little and most often the overhead is communication between various objects, classes, files, functions. Moreover extra functionality has to be added to the various classes, modules for communication and interaction.

3. Unknown fears –>A lot many times people have unknown fears that sometime may not work as expected if it has not been designed completely. It depends again on whether such a system already exists or not. If it is something new that has never been done before and has lots of dependencies on some external factors then it is most likely that your design may never finish no matter how hard you try. So if you think you will start coding only when you have designed completely the in such a situation your project is already dead. The best methodology to adopt in such a situation is to build a working prototype first even though it might not be the best piece of software but it will give you a good idea of how you could prioritize and redesign your prototype into a full-blown product. Most often people think that moving from prototype to a complete product never happens and it always leads to patchy code. Again this is not always true as no product ever gets made for the first time unless you have loads of money to finish the product. In such a situation the best is to release working code often so that at every stage you and your customers know how the product is coming along. It also gives you an idea of how much time it is going to take to finish rest of the functionality.

To summarize one has to think of the scope, lifetime, funding / cash inflow, time to market for the project before starting to write or design code. Thus same coding style methodology does not suit all projects. Most often Agile methodology suits most projects and developers.