A Guide to Building Python 2.x and 3.x Extensions for Windows

2

I am a proponent of not rebuilding binaries when possible (thanks to Gohlke’s Unofficial Windows Binaries for Python Extension Packages site and the Python wheels), but there are many times I need to rebuild a binary Python extension, either to fix a problem or to get a package (or version of a package) that’s not on there. I’ve been around the block and back trying to build Python extensions for Windows using the various flavors of MinGW with (very) limited success, and have come to the conclusion that the best way is the direct way – build them using the same Windows SDKs that are used to build CPython itself…

It turns out that’s a pretty painless process, but first, some assumptions:

  • You’re running Windows 7 (though this probably works just as well on Windows 8)
  • You’ve already installed Python 2.x (e.g., 2.7.8) or 3.x (e.g., 3.4.1) from Python.org

I’m using 64-bit Windows… I would think using the 32-bit compilers with 32-bit Python would work as well on this platform, whereas is you’re using 32-bit Windows I doubt you’ll get far with 64-bit anything.

First, you might want to install the Visual Studio Express version(s) appropriate to the Python you’ll be building against:

Now install the appropriate Windows SDK(s):

You’ll be selecting one of two types of ISO based on the OS of your build machine

  • Running 64-bit Windows? Select GRMSDKX_EN_DVD.iso
  • Running 32-bit Windows? Select GRMSDK_EN_DVD.iso
  • (e.g., I’m running 64-bit Windows so I installed the 64-bit ISO to build both x86 and x64 targets)

Get one or both SDKs depending on the version(s) of Python you’re targeting:

Note: Windows 7 and above can open ISOs much like a ZIP file – extract it and run setup. You can skip the documentation and samples, and note that if you have newer versions of certain tools installed it will ask if you want to deprecate them (just say no).

If you installed Visual Studio 2010, you should grab the post-install KB2519277 fix for the compilers as well. Run Windows Update to get topped off with patches.

Problems installing the SDK? It can be quite a pain sometimes… I encountered this on one of the machines I was setting up for this. First, rebooting can actually help, if you haven’t done so in a while or if something else got installed/updated in the interim. Try removing the MS VC++ 2010 redistributables (via Programs and Features) and reinstalling it again. If you have VS 2010 you’ll notice the compilers specifically fail to install – because you’ve already got compilers!

You can use the SDK command prompt shortcuts as they were created. Here’s what those shortcuts look like on my setup:

  • Python 2.x:
  • cmd.exe /E:ON /V:ON /T:0E /K "C:\Program Files\Microsoft SDKs\Windows\v7.0\Bin\SetEnv.cmd"
  • Python 3.x:
  • cmd.exe /E:ON /V:ON /T:0E /K "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd"

After starting the SDK console, you need to select the target build of interest (x86 = 32-bit, x64 = 64-bit). Here, I select the x64 release environment:

setenv /x64 /release
set DISTUTILS_USE_SDK=1

After running these commands the text colors in the console should change from yellow (signifying “debug” target mode) to green (signifying “release” target mode)

At this point (within the SDK command prompt!) we can build those tricky binary extensions ourselves or use pip (or pip3.4) to build and install them.

 

Example time: Let’s build the pycrypto extension for Python 2.7 and/or 3.4! (This is particularly useful as finding the binary is notoriously difficult.)

First, download the source tarball from the website and extract it somewhere useful.

I want to build the Python 2.7 version… Start the SDK 7.0 environment as discussed earlier and switch to the folder you extracted the source code to, e.g.:

 cd /d %USERPROFILE%\Desktop\pycrypto-2.6.1

Thoroughly remove the previous build artifacts to ensure a clean build (if you don’t you can get some odd problems, and unfortunately the “clean” target doesn’t always work as you expect it to).

rmdir /q /s build dist

Run the build for Python 2.7

setenv /x64 /release
set DISTUTILS_USE_SDK=1
C:\Python27\python.exe setup.py build
C:\Python27\python.exe setup.py bdist --format=wininst

(I used the “wininst” format above to create setup executables like you get at the Unofficial Windows Binaries site mentioned earlier – I tried the msi format and it was less forgiving IMHO.)

Now I want to build the Python 3.4 version… Start the SDK 7.1 environment as discussed earlier and switch to the folder where the code is. Then run the build for Python 3.4:

setenv /x64 /release
set DISTUTILS_USE_SDK=1
C:\Python34\python.exe setup.py build
C:\Python34\python.exe setup.py bdist --format=wininst

I now have two easily-installed artifacts:

.\dist\pycrypto-2.6.1.win-amd64-py2.7.exe
.\dist\pycrypto-2.6.1.win-amd64-py3.4.exe

And that’s all there is to it!

(In an upcoming post I’ll go through a practical example of creating an extension using Cython.)

(With gratitude to the Cython wiki on building extensions in Windows, where I finally found a build framework that worked!)

Find this content useful? Share it with your friends!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.