How To Package Your Python Code

[gvalkov]

I've wondered why using a dictionary to initialize setuptools.setup() isn't more advocated in such guides. I understand this is superficial, but imho, this looks much clearer than the author's suggestion [1]:

    from setuptools import setup

    kw = {
        'name'         : 'funniest',
        'version'      : '0.1',
        'description'  : 'The funniest joke in the world',
        'url'          : 'http://github.com/storborg/funniest',
        'author'       : 'Flying Circus',
        'author_email' : '[email protected]',
        'license'      : 'MIT',
        'packages'     : ['funniest'],
        'zip_safe'     : False,
    }

    if __name__ == '__main__':
        setup(**kw)

Here are a few of my setup.py files that follow this convention [2] [3] [4].

[1]: http://www.scotttorborg.com/python-packaging/minimal.html#cr...

[2]: https://github.com/gvalkov/harstats-graphite/blob/master/set...

[3]: https://github.com/gvalkov/jenkins-autojobs/blob/master/setu...

[4]: https://github.com/gvalkov/python-evdev/blob/master/setup.py

[michaelhoffman]

Adding spaces to make things line up is proscribed by the Python Style Guide. Which is entirely optional, but many people follow it.

http://www.python.org/dev/peps/pep-0008/#pet-peeves

[borntyping]

Because the only difference there is neater indentation, which you could do using the keywords method.

Additionally, there's no point checking that the script is the main file, as there's only ever one use-case for the script, and that's to call setuptools.

[JulianWasTaken]

Taking a quick look through this it looks pretty good! I find that beginners have tons of trouble navigating the slightly tricky waters that is packaging in Python, so more resources is great.

One thing I think is missing is to point out not to use setuptools, but to use the less broken, more maintained, drop in replacement distribute (http://guide.python-distribute.org/). That website also has some further information on the packaging ecosystem that's worth flipping through.

[storborg]

Thanks, I'll definitely add that to the "See Also" section.

Can you elaborate on the places where setuptools is broken? I wanted to keep things as simple as possible for the sake of this tutorial, and I think the described constraints should avoid any setuptools bugs.

[borntyping]

Doesn't distribute miss out on a few of setuptools really useful features, like develop installs?

[pak]

The fact that there is no good, canonical guide for doing this in Python is one of the reasons I moved away from it and toward Ruby. With Ruby I had no problems writing and forking gems using the canonical guide (http://guides.rubygems.org/make-your-own-gem/). With Python, navigating the hell that is packaging is just demoralizing when it comes time to give end users a complicated chunk of code. (Setuptools is truly "a hack on a top of a bad design", not my own words.) I hope for Python's own sake that distribute (http://packages.python.org/distribute/) catches on and people are simply shamed out of using older tools, but since Python already has a problem with getting everyone to migrate code away from old things, I'm not holding my breath.

I'm still not sure how to do certain things "right", e.g., distribute a modified version of somebody else's package with my own package. In Ruby, I can do this with bundler specifying a git repo for my own fork of the gem (and then submit a pull request for the original fork, if it's a patch that's useful upstream).

[cdavid]

if setuptools is a hack on top of bad design, distribute has exactly the same issues, since it is simply a fork that was started because of the impression that setuptools dev was stalled. None of the (numerous) design issues from setuptools have been fixed in distribute.

[chewxy]

Here are other good packaging guides for Python (by Tarek Ziade). I think Scott Torbog has joined the ranks of these greats:

http://www.aosabook.org/en/packaging.html

http://guide.python-distribute.org/

[mixedbit]

Are setuptools suitable for packaging the whole web application (for example in Django) together with html/js/css files, configs for uwsgi and some management scripts?

I once tried to do something like this and failed and I had an impression that setuptools are not really intended for such things, but mainly for packaging plain Python modules.

Is this a right impression? If yes, what is a good way to package the whole Python based web application?

[grosskur]

I find setuptools works great for packaging whole apps with HTML/JS/CSS. You can put these data files in your Python package directory, list them in MANIFEST.in, and set 'include_package_data=True' in setup.py. Then you can reference them in your code relative to __file__. And you can include a run script for your app using 'scripts' in setup.py, or let setuptools generate it using 'entry_points'.

Take a look at https://github.com/getsentry/sentry for a good example. After doing 'pip install sentry', you can run the included gunicorn-based HTTP server via bin/sentry ('sentry init && sentry start'). A uWSGI-based server would have worked, too.

I like to create a virtualenv under /opt, install the package and its dependencies with pip, and then run fpm to create an RPM or DEB. Then I use Chef or Puppet to install the package, deploy config files, deploy a service script (for upstart or daemontools), and enable the service.

See also http://www.12factor.net/ for some principles on how to structure your app. These can be applied whether or not you use setuptools. For example, Heroku's Python buildpack will deploy your 12-factor app in-place using requirements.txt and Procfile in your app root, ignoring setup.py. I find 12-factor apps are easier to deploy whatever method you choose, since configs/services/logs are cleanly separated from your app itself.

[borntyping]

It sounds like you're trying to package an entire application, not a python package. If thats the case, maintaining and distributing it may be simplest to do with a git repository.

[storborg]

Although setuptools is capable of that kind of scenario, sometimes it's not worth the overhead.

A good rule of thumb is that if you have defined access points for others to consume your code with (command line tools, Python modules, or Setuptools entry points), it's a good idea to make a Python package like this. Otherwise, skip it.

[feniv]

I know this isn't exactly related to packaging the code, but any thoughts on utilities like py2exe (http://www.py2exe.org/), cx_Freeze (http://cx-freeze.sourceforge.net/) or pyinstaller (http://www.pyinstaller.org/) for distributing finished python modules to end-users?

[rhettg]

You guys might also find this github project useful: https://github.com/splaice/py-bootstrap

[maxjaderberg]

ive been writing python for ages but never found a simple introduction to how to package modules - always thought it was voodoo, so thanks a lot! so simple...

[indiecore]

It is but now you know the correct incantations ~computers~ :)

[dearmash]

I apologize for the spam, but apparently I'm unable to save articles for some reason. I'd like to read this later, and I want to see if a comment will work.

[doesnt_know]

If only there was some way to save a url within the software that we use to browse the web. That would be an incredibly useful feature, someone here should consider developing an addon or put in a feature request for such a thing.

[borntyping]

While I've already been doing things a similar way, it's nice to have a clean, clear explanation of how to package stuff.

[norlowski]

Awesome job. I had been stressing about getting my code up there but your tutorial made it easy.

[phryk]

Thanks, this is exactly what I was looking for the last few weeks and couldn't find :)