18 October 2014

Virtual environments for real developments, playing with virtualenv

I don't know what you do but I usually develop multiple projects at the same time. Problem is that sometimes you need libraries for a project that are incompatible with those in another. That's why virtualenv exists in Python world.

Thanks to virtualenv you can keep all the Python versions, libraries, packages, and dependencies for a project separate from one another. It does it creating an isolated copy of python for your project directory without worry of affecting other projects. That is useful too if you have to develop and install libraries in a linux system where you have no sudo/root access to install anything.

Using virtualenv as part of your usual developments tools will keep you away future headaches.

First step about using virtualenv is to be sure about which version of python are you going to use in your project. Original virtualenv doesn't run right with Python 3, so if you use it you should keep yourself in Python 2.7. Fortunately, since 3.3 version Python includes it's own flavour of virtualenv built-in, called venv. Be aware that 3.3 version's venv install with no pip support but thankfully that support was added in Python 3.4. We are going to cover both of them in this article: first virtualenv and venv afterwards.

You can install virtualenv using your linux native package manager or python's pip installer. In Ubuntu you can install it through package manager doing:
dante@Camelot:~$ sudo aptitude install python-virtualenv

Using pip, should be enough with:
dante@Camelot:~$ pip install virtualenv

You can check which version you installed with:
dante@Camelot:~$ virtualenv --version

To use it, just navigate through console to your project directory an run following command:
dante@Camelot:~/project-directory$ virtualenv --no-site-packages env
New python executable in env/bin/python Installing setuptools, pip...done. dante@Camelot:~/project-directory$

That command will create a directory called env where binaries to run virtual environment are placed. The --no-site-packages flag truly isolates your work environment from the rest of your system as it does not include any packages or modules already installed on your system. That way, you have a completely isolated environment, free from any previously installed packages.

Before your work can start you should start your virtual environment running activate script:

dante@Camelot:~/project-directory$ source env/bin/activate (env)dante@Camelot:~/project-directory$
You know you are working in a virtualenv thanks to the directory name surrounded by parentheses to the left of the path in your command line. While you stay in  virtualenv all packages you install through pip will be stored in your virtual instance of python leaving your operating system python alone.

To leave virtualenv just type deactivate:

(env)dante@Camelot:~/project-directory$ deactivate dante@Camelot:~/project-directory$

You should create a virtualenv each time you start a new project.

Using venv in Python 3.4 is not so different. Nevertheless be aware that Ubuntu 14.04 comes with a broken version of venv. If you try to create a virtual environment with venv in Ubuntu 14.04, you'll get an error like this:

dante@Camelot:~/project-directory2$ pyvenv-3.4 env Error: Command '['/home/dante/project-directory2/env/bin/python3.4', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1 dante@Camelot:~/project-directory2$

The only way I've found to fix this problem is upgrade Ubuntu release to 14.10:

dante@Camelot:~/project-directory2$ sudo do-release-upgrade -d

Several hours later you'll get an upgraded Ubuntu 14.10 system. There you may find that venv needs to be installed from a python package:
dante@Camelot:~/project-directory2$ cat /etc/issue Ubuntu 14.10 \n \l dante@Camelot:~/project-directory2$ pyvenv-3.4 env El programa «pyvenv-3.4» no está instalado. Puede instalarlo escribiendo: sudo apt-get install python3.4-venv dante@Camelot:~/project-directory2$ sudo aptitude install python3.4-venv Se instalarán los siguiente paquetes NUEVOS: python3.4-venv 0 paquetes actualizados, 1 nuevos instalados, 0 para eliminar y 0 sin actualizar. Necesito descargar 1.438 kB de archivos. Después de desempaquetar se usarán 1.603 kB. Des: 1 http://es.archive.ubuntu.com/ubuntu/ utopic/universe python3.4-venv amd64 3.4.2-1 [1.438 kB] Descargados 1.438 kB en 0seg. (1.717 kB/s) Seleccionando el paquete python3.4-venv previamente no seleccionado. (Leyendo la base de datos ... 237114 ficheros o directorios instalados actualmente.) Preparing to unpack .../python3.4-venv_3.4.2-1_amd64.deb ... Unpacking python3.4-venv (3.4.2-1) ... Processing triggers for man-db (2.7.0.2-2) ... Configurando python3.4-venv (3.4.2-1) ... dante@Camelot:~/project-directory2$ pyvenv-3.4 env

That way venv runs with no errors:
dante@Camelot:~/project-directory2$ source env/bin/activate (env)dante@Camelot:~/project-directory2$

Once in your virtual environment you can install every package you need for your development using pip and without messing python libraries of your operating system. Modern linux distributions make heavy use of python applications, so it's a good practice to keep your operating system's python libraries clean with only what is really needed by your linux and mess with application specific libraries for your developments through their respectives virtual environments. For instance, you would use virtual environment if you needed to develop in Python 3 while keeping your operating system's default python interpreter in 2.7 version:

dante@Camelot:~/project-directory2$ source env/bin/activate (env) dante@Camelot:~/project-directory2$ python Python 3.4.2 (default, Oct 8 2014, 13:08:17) [GCC 4.9.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> exit() (env) dante@Camelot:~/project-directory2$ deactivate dante@Camelot:~/project-directory2$ python Python 2.7.8 (default, Oct 8 2014, 06:57:53) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>

Don't be shy with virtual environments. You can use them in production environments, in fact they are what is actually recommended for production situations where you don't want an update of your operating system libraries break your running developed application dependencies.