Django, Pinax, virtualenv, setuptools, pip, easy_install and requirements.txt
This article describes how to build project dependencies with virtualenv. It is appliable to Pinax. Pinax uses virtualenv by default. It lets the developer have a project-specific python directory, including binaries, packages etc … OpenSuse for example supplies very bad Pinax packages which allows the user to mess with his operating system. It is much better to isolate project dependencies from a project to another for more granular control of the maintenance cost; for example if a package upgrade breaks the user project.
pip installs python packages, it replaces easy_install. It is recommanded for Django and Pinax development because it is more flexible and natively supports requirement scripts. pip is complementary with virtualenv, and it is encouraged that you use virtualenv to isolate your installation.
A working toolchain (particularely gcc, glibc, binutils) is needed for python modules implemented in C, like mysql-python module.
Use the virtualenv command to create a virtualenv as such:
Activate a virtualenv as such:
Now your “python” command should be the one from my_env_name/bin/python. And every package you install with the virtualenv activated will install in the virtualenv and not mess with the OS.
If you want to avoid conflicts with the system-wide site-packages, you might as well use the (update: now default)
--no-site-packages option to create the virtualenv.
Pip Usage pip help Usage: pip COMMAND [OPTIONS] # [snip] bundle: Create pybundles (archives containing multiple packages) download: Download packages freeze: Output all currently installed packages (exact versions) to stdout help: Show available commands install: Install packages unzip: Unzip individual packages zip: Zip individual packages
Simple package installation
Install the latest django-extensions release in the virtualenv:
pip install django-extensions
Simple package upgrade
The pip install command option to upgrade a package in the virtaulenv:
-U, --upgrade Upgrade all packages to the newest available version
Upgrade to the latest django-extensions release in the virtualenv:
pip install -U django-extensions
Simple package installation from a VCS
The pip checkout option:
-e VCS+REPOS_URL[@REV]#egg=PACKAGE, --editable=VCS+REPOS_URL[@REV]#egg=PACKAGE Install a package directly from a checkout. Source will be checked out into src/PACKAGE (lower-case) and installed in-place (using setup.py develop). You can run this on an existing directory/checkout (like pip install -e src/mycheckout). This option may be provided multiple times. Possible values for VCS are: svn, git, hg and bzr.
For example, to install Django trunk:
pip install -e git+git://github.com/django/django#egg=django
The “egg” argument is required, but the user is free to use whatever name it judges good. The most important is to keep egg names unique per package.
By default, pip will checkout in your my_env_name/src/egg_name directory, and run setup.py install in the resulting directory.
It is important to hardcode the revision that the project needs in the requirements file, ways and reasons to do it are explained in the next section of this article.
pip help install Usage: pip install [OPTIONS] PACKAGE_NAMES... # [snip] -r FILENAME, --requirement=FILENAME Install all the packages listed in the given requirements file. This option can be used multiple times.
Basically, pip will prefix each line of the requirements plain text file with “pip install “. Consider this example requirements.txt:
--find-links=http://pypi.pinaxproject.com django-email-confirmation==0.1.3 django-mailer==0.1.0 django-announcements==0.1.0 django-pagination==188.8.131.52
pip install -r requirements.txt
Is equivalent to making a shell script with:
pip install --find-links=http://pypi.pinaxproject.com pip install django-email-confirmation==0.1.3 pip install django-mailer==0.1.0 pip install django-announcements==0.1.0 pip install django-pagination==184.108.40.206
Pinax copied requirements.txt to your project root when cloning a project. Now it uses requirements/project.txt. Each additionnal package that you install should be appended to requirements/project.txt. It is easier to run pip install -r requirements.txt than to install all dependencies manually, or using svn:externals or git submodules.
It is important to specify the version of each package because it is common to break backward compatibility in the Python world. It is important to add “barpack==1” instead of just “barpack” in requirements.txt pf a project depends on function barfunc provided by barpack version 1, because installing “barpack” later might install version 1.1 which might break backward compatibility. Hardcoding versions in the requirements file works around backward compatibility breaks. It allows granular deployement and maintenance resource management.
Use case: Django release upgrade for Pinax
Pinax 0.9 ships a dependency to Django 1.3.1. Django 1.2 for example should be upgraded through pip if the project requires it. The pip install command option to upgrade is -U:
pip install -U django==1.3.1
Use case: install and develop from your branch of an app
The pip install option for checkout (-e) is convenient for development because it installs the package in the env/src directory in which you can edit the code, push commits and so on.
pip install -e hg+ssh://email@example.com/jpic/django-authority/#egg=django-authority cd env/src/django-authority
Any change done in env/src/django-authority is directly useable in the virtualenv and by mercurial in that case, making development convenient.
The requirements file should be set up normally:
- without —no-install
- with a hardcoded revision identifier.
Thanks Brian Rosner shared his experience on the #pinax channel.