Moderate Realism

I keep half an eye on various of the channels through which new or updated Python projects are announced: planet.python.org, the PyPI feed, and c.l.py.announce. While I am occasionally irritated by the noise which PyPI produces (nested lists, anyone?) it’s helpful to see projects go past, just to remind myself what’s out there.

One phenomenon I’ve experienced is that I’ll not even have *heard* of some underlying library or web service until someone writes a Python interface to it. Glancing randomly at the current PyPI front page, I see “Python yubico client” which refers to some kind of one-key logon service which I’d never heard of. Then there’s “sunburnt - a Python interface to Solr” which, it happens, I have heard of. And a project which refers to the “Comprehensive Knowledge Archive Network” which, even after I’ve visited the website, I’m still not very clear why I might want to use it. But I’m sure it’s really good for the people who do.

This is great news for two reasons at least: people are using Python for all sorts of things, and not just the big-named everyone-does-that stuff; and there’s an inadvertent advertising effect when they publish their work, which brings a possibly interesting project to the attention of a wider audience.

Which brings me to the main point of this post: from time to time I see a project go past which really grabs the programmer in me, and I just wish I had an excuse to use. Occasionally I actually get to fulfil this wish, but usually I just look wistfully on and check as each new update is released to see if there’s any way I can squeeze it into whatever I am paid to do (or have time to do otherwise).

• Kamaelia - I’ve followed this one since its owner (Michael Sparks) presented it at a London Python meetup some years ago. I love the idea of simple objects and network pipelines between them. (Who wouldn’t?). I recall a thread a few years ago where Michael wondered why more people hadn’t taken it up (and why they’d gone for Twisted instead which seemed so much less intuitive).
• Celery - obviously not unrelated to the previous entry. My interest in lightweight distributed message passing probably goes back to my days using Modula-2, the language I used at University and which led to my discovering Python. (Modula-2 has built-in support for coroutines).
• requests - bit of a newcomer but it it exemplifies one of the things I admire most about Python code: it makes it really easy to do something which is conceptually simple. In this case, making HTTP requests. I think this is a key differentiator when comparing Python to other useful and successful languages: it just doesn’t take much code to get something done. It’s something I try for in my own libraries, and is, sadly, one of the reasons why the built-in logging module gets such a bad press. (In spite of its author’s ready willingness to answer questions, address issues, improve things, and update docs).
• Whoosh - a pure Python search engine. For no very good reason, the idea of doing this in pure Python appeals to me. As it happens I’ve rolled my own search facilities for several small projects. (And who hasn’t?). Unfortunately, the cognitive overhead of replacing those by Whoosh is just too great at the moment to justify the small risk in changing. One day…
• PyMongo (or any other of the several interfaces to MongoDB). As a daytime database developer, I’m as intrigued as the next man by the various schemaless / distributed databases currently going under the “NoSQL” umbrella. I’ve played with CouchDB (very RESTful interface but map-reduce all the way down) and MongoDB (less pure but more practical) and I even toyed with re-implementing our Helpdesk system using one of them, but as ever the risks were too great and there’s no real benefit to the business. (Only to me :) ).

There are loads more; these are just some that occur to me. Others include the kind of popup notifiers inspired by Growl, lkcl’s recently-discussed pyjamas and even some Windows-specific functionality within the pywin32 modules.

If only time and opportunity allowed…

Yesterday’s Dojo was as entertaining as ever. It almost went wrong before it started, as Fry-IT — who continue to provide the venue and the food & drink — had no working projector. Fortunately, my company were willing to lend one for the evening which got over that hurdle.

We started with a couple of lightning(ish) talks. Ravi had been on the StartupBus and he and the team had produced the final winner: TripMedi. Ravi explained the way the thing works and various of the challenges involved. He also explained how YCombinator works — something I’d only been vaguely aware of. [On a personal note, I can’t even imagine the kind of person who’d actually want to make use of what TripMedi provides, but I suppose it takes all sorts…] UPDATE: Ravi has just tweeted that the team has made it through to the next stage of YCombinator. Congratulations!

I then gave a presentation on contributing to Python core development and how you could go about it. Thanks to the efforts of Jesse Noller, Doug Hellmann and others, there is now the Python Insider blog, whose intention is to give greater visibility of some of the wider-reaching discussions taking place on python-dev. And the new core-mentorship mailing list is specifically intended to welcome would-be contributors who aren’t too sure how to go about things, or what the ground rules and policies are.

Once all the talking was done, the trouble started :) Nicholas, the indefatigable organiser of the London Python Code Dojo, has been quite keen on promoting XMPP (aka Jabber) to implement some sort of network-ability for our nascent multi-Dojo game engine. For the previous Dojo, his team had produced a simple XMPP Component which implemented a to-do list: the challenge for that Dojo. This time, Nicholas was keen for everyone to have a go with the same thing. He’d put code on Github, added a readme, and recommended the lua-based Prosody server and SleekXMPP for the Python end.

So we broke up into small groups (as they say), fired up git, pip and whatever, and dove in with about 75 minutes to produce something.

About 45 minutes later, most of us were still diving in, trying to get Prosody to listen to anything and failing to get the component talking to Prosody.

At this point, Mr Tollervey very wisely decided that we should cut our losses. Fortunately, John Chandler was willing to go up front and ask for a code review of the code he had produced. Which turned out to consist of two lines, one of which was “import random”. Being social coders, we were more than willing to advise him on the many ways in which he could make this simple and correct code into an over-engineered mess of obfuscation.

Since we’re in the middle of PyWeek, it seemed apt to have a demo from Daniel Pope of the game he’s developing on the theme of “Nine Times”. Impressive as always. (If you haven’t seen it, check out Bamboo Warrior, one of his previous efforts).

As always, we ended with a raffle of name-stickers for an O’Reilly Book and — this time — an additional donation from Mr Tollervery himself. Ironically Jon “I don’t like XMPP” Ribbens got the O’Reilly book on XMPP.

Watch the python-uk mailing list or follow ntoll on Twitter to know when tickets are available for the next Dojo (it usually takes place on the first Thursday of the month).

There’s work afoot by Mark Hammond with encouragement from others to produce a Python launcher for Windows. The work’s being specified under PEP 397 (not yet in the PEP index) and a reference implementation is being tracked under issue11629 on the Python issue tracker.

The need arises from the difficulties of having an easy way to associate Python .py/.pyw files with the appropriate version of Python on Windows, exacerbated by the advent of Python 3. The idea is that rather than having .py associate with c:\pythonxy\python.exe, it would associate with \py.exe which would attempt to read a shebang line (#! python.exe) from the top of the file dropping through various levels of default to choose a meaningful version of the Python executable to run.

From what sounds like a simple requirement there are lots of details to iron out. The (draft) PEP explains the levels of drop-through which the launcher will employ to decide what to run. There’s what seems to be a sort of awkward stand-off between not caring about cross-platformability and honouring common conventions such as /usr/bin/python. For my part I rarely if ever write code which needs to run on *nix — it’s so Windows specific that it wouldn’t even import all its dependencies, let alone do anything useful. So that side of things matters less to me.

The other detail which has caused much discussion is the mechanism by which the launcher process starts the Python interpreter. Ignoring such esoterica as COM instantiation, you have basically two options: have the launcher load the appropriate Python .DLL; or have the launcher call CreateProcess on the appropriate python .exe, passing through the parameters, open handles, etc. Each has its pros and cons but Mark’s currently plumping for creating a child process and using the Windows Job Object API to ensure the child dies with its parent.

If you’re at all invested in developing or deploying Python under Windows, have a look at the PEP and try out the Python reference implementation.

I’m fairly new to github; to git, in fact. Truth be told, I wouldn’t even have bothered with it if it weren’t for the fact that ntoll organises the code for the London Python Dojo around it. And that the game engine we had decided to use was hosted there. While I’m all in favour of DVCSes (you just *know* what’s coming when someone says “I’m all in favour…”, don’t you?), the problem is that there’s only one Subversion, but there are at least two if not three — depending on your view on bzr — DVCSes with fairly widespread support, and they’re all subtly or radically different. I more-or-less committed to learning Mercurial, because it’s what Python’s moving to, but that doesn’t help me when other people are using git. Oh well…

Previously on London Dojo: we cloned the crawlr engine from axedcode’s github repo. And found, ironically, a maze of twisty passages all alike. We went north, south, east and west in search of treasure but found only darkness… Fortunately, the man himself spotted that his game engine was flavour of the month and got in touch to ask why, and to offer a newer, better version: axengine2. Which is a lot cleaner, even if there are some questionable design choices…

So we did that. And I spent half an hour at the beginning of last Thursday’s Dojo going over key pygame concepts — rects, surfaces, sprites, and sprite groups — and giving a whistle-stop tour of the codebase which, slightly surprisingly, reimplements the Python class inheritance model in JSON, backed up by Python classes. Sort of. There was half a chance we might even have a live Skype connection to the author at the beginning of the session — before he started work over in the States — but timing didn’t work out so it didn’t come off. Shame.

We set to work again, in teams as usual, each team having a “Dojo Quest” and 90mins to complete it. Naturally, everyone was still frantically coding at the 89-minute mark. (I know we were, having made a critical breakthrough in our attempt to introduce a joystick-controlled second player [*]). We had the usual show-and-tell of: a menu system, some background sound, the two-player mode, hidden treasure, and something else I’ve forgotten. By which time it was getting on for 10.30pm and there was only just time to pull a name out of the hat for the winner of the O’Reilly Book-of-the-Month on XMPP, which we hope to use for the inter-dojo gameplay.

The Dojos are always fun; they’re also nothing like the disciplined Dojos which others favour. Basically, it’s a social coding exercise with a fair bit of laughter and passenger-seat driving. We seem to be nudging the 30 mark every month at the moment, so watch the Python UK mailing list for the announcement that tickets are available and get in there quick.

[*] We delayed ourselves a little by confusing the KEYUP / KEYDOWN constants — which indicate depression and release of keys — with the constants indicating the up and down-arrow keys :(

What technical stuff am I trying to pick up at the moment? I suppose many techy types are like me in that they keep a more-or-less close eye on the various technologies and libraries just to know what’s around. Without an immediate need; and without the time or perhaps the inclination to dive into any one of them. But from time to time some kind of incentive or motivation (or need (or job)) comes along and you start to pick something up to see if it’s worth pursuing further.

At the moment, then, I’m reading up on and trying out:

• Jabber / XMPP - for a future London Python Code Dojo
• C# - for possible work and for career prospects (and just because…)
• BI / Data Warehousing - for work
• Django - just because

Things I want to find the time for (and the money to buy the books):

• Windows Internals - to have a better understanding when I’m helping people out
• New aspects of WMI & AD - for work and to help other people

All of which adds up to a fair amount of reading and so on. And that’s on top of everything I’m trying to do. Good thing I’ve got a 50-minute train journey in the morning.