We are well known for our work with Ruby and Rails here at thoughtbot, butgenerally we always try to use the most appropriate language or framework forthe problem at hand. With that, I’ve recently beenexploring machine learning techniques so I’ve been working a lot morein Python.
One of the big differences between working on Ruby projects and Python projectsis the way that dependencies are typically managed. There currently isn’tanything similar to Bundler or Gemfiles in the Pythonuniverse so usually a Python developer will create a virtual environmentusing Virtualenv, and then annotate a requirements.txt
text file witha list of dependent packages, which they can then install using Pip.
This approach works fine but sometimes it can be a juggling act, as you have tomanually install or remove packages with particular versions, and remember toregularly update the requirements.txt
file, in order to keep the projectenvironment consistent. Especially when there are Python packages you wantinstalled in your virtual environment, but not necessarily associated with theproject itself. Moreover, some projects sometimes maintain two versions of therequirements.txt
file – one for the development environment and one for theproduction environment – which can lead to further complications.
Fortunately Kenneth Reitz’s latest tool, Pipenv, serves tosimplify the management of dependencies in Python-based projects. It bringstogether Pip, Pipfile and Virtualenv to provide astraightforward and powerful command line tool.
Getting started
Begin by using pip
to install Pipenv and its dependencies,
pip install pipenv
Then change directory to the folder containing your Python project andinitiate Pipenv,
cd my_projectpipenv install
This will create two new files, Pipfile
and Pipfile.lock
, in your projectdirectory, and a new virtual environment for your project if it doesn’t existalready. If you add the --two
or --three
flags to that last command above,it will initialise your project to use Python 2 or 3, respectively. Otherwisethe default version of Python will be used.
Managing Python dependencies
Pipfiles contain information about the dependencies of your project,and supercede the requirements.txt
file that is typically used in Pythonprojects. If you’ve initiated Pipenv in a project with an existingrequirements.txt
file, you should install all the packages listed in that fileusing Pipenv, before removing it from the project.
To install a Python package for your project use the install
keyword. Forexample,
pipenv install beautifulsoup4
will install the current version of the Beautiful Soup package. A packagecan be removed in a similar way with the uninstall
keyword,
pipenv uninstall beautifulsoup4
The package name, together with its version and a list of its own dependencies,can be frozen by updating the Pipfile.lock
. This is done using the lock
keyword,
pipenv lock
It’s worth adding the Pipfiles to your Git repository, so that ifanother user were to clone the repository, all they would have to do isinstall Pipenv on their system and then type,
pipenv install
Then Pipenv would automagically locate the Pipfiles, createa new virtual environment and install the necessary packages.
Managing your development environment
There are usually some Python packages that are only required in yourdevelopment environment and not in your production environment, suchas unit testing packages. Pipenv will let you keep the twoenvironments separate using the --dev
flag. For example,
pipenv install --dev nose2
will install nose2, but will also associate it as a package that is onlyrequired in your development environment. This is useful because now, if youwere to install your project in your production environment with,
pipenv install
the nose2 package won’t be installed by default. However, if another developerwere to clone your project into their own development environment, they coulduse the --dev
flag,
pipenv install --dev
and install all the dependencies, including the development packages.
Running your code
In order to activate the virtual environment associated with your Python projectyou can simply use the shell
keyword,
pipenv shell
You can also invoke shell commands in your virtual environment, withoutexplicitly activating it first, by using the run
keyword. For example,
pipenv run which python
will run the which python
command in your virtual environment, and display thepath where the python
executable, that is associated with your virtualenvironment, is located. This feature is a neat way of running your own Pythoncode in the virtual environment,
pipenv run python my_project.py
If you’re like me and shudder at having to type so much every time you want torun Python, you can always set up an alias in your shell, such as,
alias prp="pipenv run python"
Keeping it simple
I hope this post has shown you how to manage your Python projectswith Pipenv. It has been around for less than a month now, so I, forone, will be interested to see how it develops over time. I certainly don’twant, or expect, it to become exactly like Bundler for Ruby, but I’lldefinitely champion it for simplifying the management of dependencies in Pythonprojects. I hope you do too!