My Transition to Python 3 and Pyenv, goodbye Virtualenvwrapper
2014-11-30 von Artikel von HannesHannes

  • Howto
  • Tech

Python 3 has been out there for some time, but the main excuse for not using it has always been the lack of support by 3rd party projects. In 2014 that changed drastically and it seems it's a good time to start doing serious stuff with Python 3.

OK LET'S ONLY CODE 3 FROM NOW ON! ehhh no, sadly not possible..

My biggest problem is that I have a lot of Python 2 code untested with Python 3 and that I also want to continue supporting Python 2. You properly have the same issue. Before 3 the world was easy. I installed Python 2 with brew and used virtualenv to isolate all the dependancies of my Python 2 projects. And virtualenvwrapper made this simple and easy..

But when you start using multiple Python versions you realize that that things are getting complicated. You need something like virtualenv with support for isolating your Python installation itself, so that you can use different versions on a per project basis. As you can imagine pyenv is exactly that. It takes the pain out of having multiple versions of Python installed at the same time.

All the stuff I will tell you assumes that you are using OSX. Linux should be slightly different and Windows is sadly only a second-class citizen.

So let's get started! You will see that the new world is actually pretty nice.

Install pyenv and pyenv virtualenv

To install pyenv we simply clone the pyenv project to the folder .pyenv in our home directory. We also clone the pyenv-virtualenv plugin to the plugins folder inside that directory.

git clone git:// ~/.pyenv
git clone ~/.pyenv/plugins/pyenv-virtualenv

Further you have to add the pyenv bin directory to your PATH and initialize pyenv. To do this simply add the following lines to your ~/.zshenv or ~/.zsh_profile.

# pyenv root
export PYENV_ROOT="$HOME/.pyenv"

# Add pyenv root to PATH
# and initialize pyenv
if [[ -d $PYENV_ROOT ]];then
    # initialize pyenv
    eval "$(pyenv init -)"
    # initialize pyenv virtualenv
    eval "$(pyenv virtualenv-init -)"

We should be all set now. Let's start playing!

Install new Python Versions

The first thing you want to do is to install some new Python versions.

env PYTHON_CONFIGURE_OPTS="--enable-framework CC=clang" pyenv install 2.7.8
env PYTHON_CONFIGURE_OPTS="--enable-framework CC=clang" pyenv install 3.4.2

This should install the Python versions 2.7.8 and 3.4.2 to ~/.pyenv/versions. The option enable-framework causes pyenv to build the framework version of Python, which is the standard on OSX.

To verify that the versions are available use pyenv versions.

pyenv versions
* system

Set the global python version

To define the default Python version use pyenv global. New shells should now use this Python version by default.

pyenv global 2.7.8
pyenv version
2.7.8 (set by /Users/YOU/.pyenv/version)

Change the version for current directory or shell

You can also use pyenv shell to set the python version of the current shell..

pyenv version
2.7.8 (set by /Users/YOU/.pyenv/version)

pyenv shell 3.4.2
pyenv version
3.4.2 (set by PYENV_VERSION environment variable)

.. or pyenv local to set the python version for the current folder.

mkdir test && cd test
pyenv local 3.4.2
pyenv version
3.4.2 (set by /Users/YOU/test/.python-version)

cd ..
pyenv version
2.7.8 (set by /Users/YOU/.pyenv/version)

Note that pyenv shell sets an environment variable PYENV_VERSION, while pyenv local creates the file .python-version in the current folder to set the python version.

How pyenv determines the python version

  1. Check for PYENV_VERSION environment variable
  2. Check for .python-version file in current directory
  3. Check for .python-version file in parent directories
  4. Use the global python version from ~/.pyenv/version

OK, but where is my virtualenv?!

Now we can change our current python in a flexible way, but the package isolation part virtualenv provides is still missing. This is why we also installed the pyenv-virtualenv plugin before. The plugin allows us to create virtual environments based on existing pyenv python versions. Let's create a virtualenv.

mkdir mytestvenv
cd mytestvenv
virtualenv 3.4.2 mytestenv
pyenv activate mytestenv

We created a new virtualenv named mytestenv based on version 3.4.2 and activated it. The new virtualenv is also located in the folder ~/.pyenv/versions. You can now install packages into that virtualenv the way you are used to it using pip.

One behavior I missed from using virtualenvwrapper is that the projects virtualenv is automatically activated when I change into the project-directory, but it turns out we can use pyenv local to achieve exactly that.

pyenv local mytestenv
cd ..

# now we change into the directory again..
cd mytestvenv
pyenv-virtualenv: activate mytestenv
(mytestenv) mytestvenv »

# ..and out again
(mytestenv) cd ..
pyenv-virtualenv: deactivate mytestenv

From now on our virtualenv will be automatically activated when we change into that directory. This happens because of the .python-version file that was created when we called pyenv local mytestenv.

If you think about it this is particularly great when you combine it with git. This allows you to very easily setup a new python 3 branch on your existing python 2 project. Since the active pyenv is determined by the .python-version file in your project folder, you can use a python 3 based virtualenv in your python 3 branch, so that your python setup will automatically change based on which branch you are working on.

Create Aliases

One thing I always do for my projects is to create an alias to change into the project directory.

alias mytestvenv='cd ~/mytestvenv'

In combination with all the stuff I just showed you this will..

  • change into the project directory
  • configure the correct python version
  • activate the correct virtualenv
pyenv-virtualenv: activate mytestvenv
(mytestvenv) python

Python 3.4.2 (default, Nov  9 2014, 16:14:18)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Wrap up

I hope I could convince some of you who, like myself, struggled with the 2 to 3 transition to give pyenv a shot. Python 3 has a lot of interesting stuff to explore and it is definitely the future. As I see it the language shouldn't stand still and we as developers should neither. Pyenv provides us with a convenient solution to make our codebase fit for the next 10 years.

I will use that opportunity to learn some new stuff and to improve my code. I hope you will too.