A tutorial on Virtualenv to isolate python installations

1. You don’t have sudo / root access on a development machine?
2. You want to try out a new package without polluting the system directories?
3. You want to test or experiment with a new version of a library without uninstalling the previous version?
4. You want to test your code in an isolated environment before you deploy it to a production server?

If your answer to any of the questions above is YES, then this post if for you!

What is virtualenv?

Virtualenv is a tool to create isolated Python environments quite like chroot jail on Unix systems. In a chroot, programs cannot access anything outside of chroot but in virtualenv as the name implies, it creates isolated environments only with respect to libraries, but the programs can still access the files and folders normally.

Imagine if you are working on your next Web2.0 project and you want to upgrade to a newer version of a library to test your code with it first without breaking your existing python installation. Or say you want to ensure that you have same set of libraries as on the systems where you will be deploying your code. Moreover if your hosting service is shared, hence you don’t have sudo access to install any package in global python installation. Virtualenv, comes to your rescue in all these scenarios.

Virtualenv setup and usage

Step 1: Install virtualenv

$ pip install virtualenv

Step 2: Create a virtualenv

The following command creates a clean virtualenv devoid of any packages in a directory called “ve”. However you can use any name you want for the name of virtualenv directory.

$ virtualenv --no-site-packages ve
The --no-site-packages flag is deprecated; it is now the default behavior.
New python executable in ve/bin/python2.6
Also creating executable in ve/bin/python
Installing setuptools............done.
Installing pip...............done.

Note that –no-site-packages is the default behavior so it is not really necessary to pass this option but I included this for clarity.

The way it works is by creating symlinks to your global python installation, as you can see clearly using the tree utility.

$ tree -d ve/
ve/
|-- bin
|-- include
| `-- python2.6 -> /usr/include/python2.6
`-- lib
 `-- python2.6
 |-- config -> /usr/lib/python2.6/config
 |-- distutils
 |-- encodings -> /usr/lib/python2.6/encodings
 |-- lib-dynload -> /usr/lib/python2.6/lib-dynload
 `-- site-packages
 `-- pip-1.1-py2.6.egg
 |-- EGG-INFO
 `-- pip
 |-- commands
 `-- vcs

15 directories

Step 3: Use the isolated python environment

To use or enter the environment, use the following command:

$ source ./ve/bin/activate

Now you are in an new environment and any globally installed packages won’t be available here. If I list the packages before entering the virtual environment it shows all the installed packages in my global python installation.

$ pip freeze
Brlapi==0.5.5
CouchDB==0.8
Django==1.3
Fabric==1.3.4
GnuPGInterface==0.3.2
Mako==0.3.6
MarkupSafe==0.9.2
PAM==0.4.2
PIL==1.1.7
Scrapy==0.14.1
South==0.7.3
Twisted==12.0.0
...

After I run enter the environment, I see that no system packages exist in this installation/ virtualenv:

$ source ./ve/bin/activate
(ve)$ pip freeze
(ve)$

Also note that the command prompt on the shell now has a prefix ‘ve’ (the name of your virtual environment) which indicates you are in the virtual environment.

Step 4: Install the required packages

Now you can simply install the packages using pip and does not require sudo access as all the packages are installed in the virtualenv ‘ve’ created above:

(ve)$ pip install <package-name>

or from a requirements / dependency file:

(ve)$ pip install -r requirements.txt

Now you can continue your to develop and test your code in this environment.

Step 5: Exit the virtual environment

To leave the virtual environment simply run:

(ve)$ deactive
$

Notice that the command prompt has now reverted back to $ without the prefix ‘ve’.

Summary

Virtualenv and Pip are must use tools for advanced as well as beginners in python development. It seems daunting but both Pip and Virtualenv are quite easy to setup and use. Using them in combination would avoid a lot headache in the long run.