Moderate Realism

From time to time, the idea of a standard Python “Enum” object is raised on the Python lists. You know the kind of thing: a lightweight means of mapping numbers to labels so you can do set_colour(Colours.red) without having a long module of manifest constants or magic numbers lying around all over your codebase.

It all sounds very straightforward, and Barry Warsaw had an existing module which seemed like a fairly good starting point, so PEP 435 was started and it all looked like it was just a formality.

Now, literally *hundreds* of mailing list posts and endless, endless threads later, GvR has just pronounced his approval of the PEP and it’s good to go.

If you — like me — thought “this one won’t be controversial”, then just point your search engine of choice at mail.python.org/pipermail/python-dev and look for “enum” or “435″, or just look at the archive for May alone (which only represents the final few days of details being thrashed out) to realise just how much discussion and work is involved in what appears to be quite a simple thing.

Of course, part of the problem is precisely the fact that the idea is so simple. I’m sure most people have rolled their own version of something like this. I know I have. You can get up and running with a simple “bunch” class, possibly throw in a few convenience methods to map values to names and then just get on with life. But when something’s got to go into the stdlib then it all becomes a lot more difficult, because everyone has slightly (or very) different needs; and everyone has slightly (or very) different ideas about what constitutes the most convenient interface.

And there’s always the danger of the “bikeshed” effect. If a PEP is proposing something perhaps quite fundamental but outside most people’s experience, then only people with sufficient interest and knowledge are likely to contribute. (Or argue). But an enum: everyone’s done that, and everyone’s got an interest, and an idea about how it should be done.

But, bikesheds aside, I’m glad that the Python community is prepared to refine its ideas to get the best possible solution into the standard library. As a developer, one naturally feels that one’s own ideas and needs represent everyone else’s. It’s only when you expose your ideas to the sometimes harsh winds of the community critique that you discover just how many different angles there are to something you thought was simple.

Thankfully, we have a BDFL (or, sometimes, a PEP Czar) to make the final decision. And, ultimately, that means that some people won’t see their needs being served in the way they want. But I think that that’s far preferable to a design-by-committee solution which tries to please everybody and ends up being cluttered.

(This is because I use mercurial rarely enough and commit to Python even more rarely; so I always forget what incantations I used last time…)

hg clone http://hg.python.org/cpython hg.python.org
hg clone hg.python.org issue1234
cd issue1234

hg up 3.3
hg import –no-commit http://…/fixedit.patch
# Do whatever
hg commit -m “… (Patch by …)”

hg up default
hg import –no-commit http://…/fixedit.patch
# Do whatever
hg commit -m “… (Patch by …)”

hg push ssh://hg@hg.python.org/cpython

You’ve probably seen the mailing list announcements and the tweets which declare that Python 3.3 is entering its beta stage. For core developers this means that no more features can be introduced into this version of Python (at least not without the connivance of the Release Manager: Georg Brandl). Bug fixes are still allowed — which sometimes leads to rather creative labelling of changesets.

Have a look at the preliminary “What’s New?” doc.

Up to now, Python core developers have been championing Python 3 over Python 2 largely on the basis of a significant amount of cleanups, rationalisation and cruft-clearing. Now all this is very good, but mostly pleases people who have to work with the underlying Python code & data or who are fond of housekeeping. Clearly there have been other additions. (In fact the more I read through the “What’s New?” documents the more I realise that we’ve been doing something of a poor job of advertising our own achievements). But there hasn’t perhaps been anything which is really structurally groundbreaking.

Now we’ve got built-in virtual environment support, proper namespace packages, a much clearer OS exceptions hierarchy, and — at long last — a way of yielding from other iterators. (That thing where you had to loop over an iterator yielding its values: you can now just “yield from”). Any one of those alone is surely worth the entrance price. It’s actually starting to become attractive to use Python 3.x not just because it’s cleaned-up (on the moral highground but without much of a change of scenery) but because it’s got cool new features that I actually need (wow! look at that view).

To be sure there are literally hundreds of other small changes, ranging from one-off bugfixes to large-scale rewrites: bz2 has been rewritten from scratch; Brett Cannon’s finally achieved his objective of reimplementing the import mechanism in Python; Unicode support under the covers is cleaner and faster; the useful-but-slower decimal module is now useful-but-faster thanks to a version written in C, the indefatigable Stefan Krah getting the credit there. And after much bouncing around, Peter Moody’s ipaddress module is now integrated in the stdlib.

For Windows users, two significant changes have made it in (although they haven’t reached the What’s New? docs yet). One is the implementation of PEP 397 — a Python launcher for Windows, conceived by Mark Hammond, with implementation work by Vinay Sajip and worked up into its final form by Martin von Loewis. The other, implemented by Brian Curtin, is the addition of python.exe to the system PATH. This one is a long-standing gripe especially for novice users who can now just type “python” at the command prompt and get the latest version up. In fact, the PEP 397 launcher makes this perhaps less important but it’s still good to have the option there in the installer.

I’m trying to help along a couple of changes to the Win32 implementation of some os pieces under the guise of bugfixes, but even if I don’t get them past the eagle eye of the release manager, this’ll still be a version to look forward to.

There’s work afoot by Mark Hammond with encouragement from others to produce a Python launcher for Windows. The work’s being specified under PEP 397 (not yet in the PEP index) and a reference implementation is being tracked under issue11629 on the Python issue tracker.

The need arises from the difficulties of having an easy way to associate Python .py/.pyw files with the appropriate version of Python on Windows, exacerbated by the advent of Python 3. The idea is that rather than having .py associate with c:\pythonxy\python.exe, it would associate with \py.exe which would attempt to read a shebang line (#! python.exe) from the top of the file dropping through various levels of default to choose a meaningful version of the Python executable to run.

From what sounds like a simple requirement there are lots of details to iron out. The (draft) PEP explains the levels of drop-through which the launcher will employ to decide what to run. There’s what seems to be a sort of awkward stand-off between not caring about cross-platformability and honouring common conventions such as /usr/bin/python. For my part I rarely if ever write code which needs to run on *nix — it’s so Windows specific that it wouldn’t even import all its dependencies, let alone do anything useful. So that side of things matters less to me.

The other detail which has caused much discussion is the mechanism by which the launcher process starts the Python interpreter. Ignoring such esoterica as COM instantiation, you have basically two options: have the launcher load the appropriate Python .DLL; or have the launcher call CreateProcess on the appropriate python .exe, passing through the parameters, open handles, etc. Each has its pros and cons but Mark’s currently plumping for creating a child process and using the Windows Job Object API to ensure the child dies with its parent.

If you’re at all invested in developing or deploying Python under Windows, have a look at the PEP and try out the Python reference implementation.