Archive for Books

Python Testing Cookbook

I blogged previously that I was having a look at the Python Testing Cookbook courtesy of the eager marketeers at Packt Publishing. Well I’ve finished reading it — in installments on the Tube — and I think I’ve come to a conclusion.

The short version is: I’m not mad about the layout; it’s not entirely what I expected from a Cookbook; but I think it does what it does very well.

Since I’m overall positive, I’ll just pick up on the very slight negatives first. In a previous Packt review I commented (adversely) on the style of the layout and so on, and I’m afraid that it hasn’t really changed. Obviously, this is somewhat subjective, but I can’t be the only person who’s subconsciously affected by the font and layout choice. (Bear in mind that I was reading it via PDF which may make a difference). To be clear: it’s not awful; it’s not even bad; it’s just suboptimal.

The other slight negative is hardly a negative at all: that the “recipes” in this cookbook are far broader — in some cases, chapter-length — than I’d expected. Subjective expectation, to be sure. But I was a very little bit surprised.

Ok, that’s the downsides. Now the upsides:

* Nice division of chapters
* The right pacing
* Pretty much one example used throughout
* “Why didn’t you do this?” question sections
* Good re-use or alternative use of previous examples or approaches
* Interesting spread and combination of toolsets

The chapter divisions are: unittest, nose, doctest, BDD, UAT, CI, coverage, load-testing, good habits. In broad terms, each chapter builds on previous ones, but thanks to the reuse of the a simple shopping cart example, repeated in nearly every chapter, you can take a chapter on its own without too much difficulty. In addition, different chapters sometimes offer alternative approaches to the same problem when it makes sense to illustrate one point rather than another. One chapter, for example, walks you through creating a nose plugin to show how to do it; the next chapter introduces you to a ready-made one since the focus of that chapter is not nose plugins.

And that brings me to the variety of toolkits & modules mentioned throughout the text. This is not a book which simply uses stdlib code to perform various tasks (which I had rather thought it might be when I started). It introduces the ideas mentioned in the paragraph above. And puts forward Python modules (such as nose or mock) or external tools (such as Hudson / Jenkins or TeamCity) as best-of-breed or useful tools.

One final point which really put the icing on the cake is the occasional introduction of pull-out sections answering the very question which was occurring to me as a reader: “Why didn’t you use this technique here?” or “How is this different from that?”. Perhaps it’s just me, but I find this smoothes my progress through a book considerably which otherwise would be (mentally) jarred by my wondering “Now why did he do that?”.

I can’t but recommend this book. There’s a sample chapter here if you want to have sniff. As a final note, it’s very slightly unfortunate that the timing of its production precluded the author from picking up on Michael Foord’s magnificent work in producing unittest2 / unittest for 2.7 & 3.2. Perhaps there’s a “missing chapter” opportunity in there for someone…

Python Testing Cookbook - having a look

Thanks to the energetic folks in the Packt Publishing Marketing department, I’m currently looking at the Python Testing Cookbook with Python Web Development waiting in the wings. The copies I have are PDFs which are never my favourite, so I end up printing chunks out to read on the train or elsewhere, but it’s great to get to read other people’s insights and ideas. I’m about halfway through PTC so look out for a review soonish.

IronPython in Action

I received my copy of IronPython in Action (henceforth IPiA) in the middle of the week, and I’ve spent the last few days going through it on the Tube. What that means, in practice, is that I’ve simply skimmed sections which are code-heavy: I’ve not had my laptop with me and trying to read even a modest one-page explanation of a Windows Form app I find daunting unless I can tap the code in as I go. But I’ve gone through everything else. And I like what I see.

By way of disclosure, I’m given a couple of blushingly generous footnote credits by Michael which naturally leave me feeling well-disposed towards the book as a whole. But even without those, I’d be giving it the thumbs-up. As the authors note early on, a book of this sort is trying to fulfil two expectations: to inform existing .NET users about Python; and to inform existing Python users about .NET. With a very few exceptions I believe it manages at least the second of those two. (I’m not competent to judge the first).

As I noted in my review of Tarek Ziade’s Expert Python Programming, people want different things from their programming books. What suits me in IPiA might not suit someone else. But I find the style to be lightweight enough to avoid pomposity (my own cardinal fault in writing) while not descending into jokey asides so often as to distract. It also keeps the code examples fairly short. Altho’ I personally do prefer self-contained examples every time, I recognise that this isn’t always easy or even possible. And that some people like one growing example app to run through a book.

But the most important win, I think, is managing to write a book about IronPython, not about Python or .NET. Naturally there is an element of explanation involved in both directions when some feature is being introduced or compared. But for the most part you can refer to the appendices which give summaries of Python/.NET if an unfamiliar term arises. For me, this achievement is key to the success of a book like this. If I want to learn Python or .NET I’ll turn to another book or website. Here I want to know what IronPython can do which Python can’t (or better, or faster, or worse) and what IronPython can do which C# can’t (or better etc.). I don’t want to know what .NET can offer as such, altho’ real-word examples are obviously great illustrations. The one place I believe IPiA falls down just a little in this regard is in the chapter on Databases and Web Services. The problem is that IronPython doesn’t seem to bring enough to the table here to distinguish it from the equivalent C# code.

As a long-time user of Python on a Windows platform, I’ve obviously been umming-and-ahing about IronPython for a while. I’m happy with CPython, familiar with it; I’ve written no few lines of code around the Win32 API, all of which disappears when you enter the world of IronPython/.NET. That said, it’s clear that .NET is the future of Windows. As it happens, it looks like we’re about to undertake a project at work based around the Juggernaut Sharepoint and I hope this will provide the incentive for me to have a go with IronPython and see what it can do.

Expert Python Programming

(Review copy from Packt Publishing)

Preamble: re-reading this review I realise that it comes across as rather negative. Unfortunately, the things about this book which put me off put me off sufficiently that they outweighed the many parts which I did find useful. Since I know that there are quite a few other reviews around the Python community which are more positive, I don’t feel the need to recast anything in particular: prospective purchasers who aren’t deterred by the aspects which deterred me will find enough useful material in other reviews. My sincere compliments to the author on producing the book at all: I hope he considers my comments constructive and not merely negative or nit-picking.

When I first saw it being noised about, this book’s title (and its subtitle: “Best practices for designing, coding… “) suggested to me that its target audience was people who already had an amount of experience of Python and who wanted to learn some nifty tips and tricks and some good design practices in general. And I thought to myself that it was high time for such a book. It seems to me, without an exhaustive survey, that the majority of Python books out there are either in the “learn from scratch” camp or are aimed at a particular segment of the market: either a particular toolkit, such as Twisted or sqlalchemy; or a particular discipline, such as Bioinformatics. It’s almost as though, having launched you out into the mainstream of Python programming, you’re left to trawl the internet for resources.

The book falls into three main sections: the first few chapters covering various syntax choices, especially those which have been introduced more recently; the middle chapters covering a sort of project-management approach focusing a lot on the use of eggs and buildout; and a final section covering aspects of optimization and useful patterns. For reasons I’ll explain below, I find the first and last of the these sections the most useful and have less time for the central section.

To start a hare on python-list / python-tutor, you only need to ask a question like “What book should I read to learn Python?”. You’ll get as many answers as people who answer. Obviously there are a few favourites, but even these will have a detractor: someone who found it didn’t suit them at all. And that’s the way with books. One person likes every piece of illustrating code to be completely self contained; another prefers an example to run all the way through the book. One person prefers the code to be spelt out completely in the course of the book; another prefers a downloadable .zip or a CD. One person prefers a cookbook-style set of recipes; another prefers a more narrative approach punctuated with examples. One person prefers a jokey manner, littered with witty allusions and community in-jokes; another prefers a more straightforward just-tell-me-what-I-need-to-know style.

Expert Python Programming’s visual layout, font choices,etc. seemed to me a little pedestrian. The section headings are just a little too big, and the occasional pull-out [Notes] paragraphs remind me of late 1980s DTP more than I’d have expected for a professional publication. That the author’s first language isn’t English is apparent from the — extremely few — errors and very occasional awkwardness of style. (That his English is very good is equally apparent). None of this is showstopping, but it all forms part of the effect. I was surprised that the opening chapter detailed installing Python. I would have expected anyone reading a book on Expert Python Programming to have reached the point already where they could install Python and use it. Perhaps this is my misreading of the target audience. Likewise the advice on configuring a text editor (a notoriously personal issue).

The next few chapters cover newish syntax choices such as iterators, generators, decorators and context managers (contextors, anyone?). This is nicely pitched although I feel that the author is sometimes too didactic in his advice. For example: “Every time a loop is run to massage the contents of a sequence, try to replace it with a list comprehension”. Certainly, the “try” in that recommendation softens the effect slightly but he does the same thing a little later with generators which “should be considered every time you deal with a function that returns a sequence or works in a loop” (emphasised in the original). Again, the “considered” obviously doesn’t make it a mandate, but it does seem a little less nuanced that it might be. There are certainly places where you’ll trip yourself up if you just throw a generator at a sequence/loop.

This section includes a lot of examples, most small and self-contained and following the layout of an interpreter session. This is useful, although I feel that showing the entire interpreter session tends to introduce a lot of noise into the example (including all the “>>>” and “…” prompts. It also tends to result in quite vertically long code segments which will more easily cross over pages, making it more difficult to follow. Might work better if the font size was a point size smaller.

There follows a set of chapters covering syntax choices around classes, including subclassing built-in types, using super, slots and metaclasses. Possibly because this is an area where my own ignorance is more exposed I found the narrative and explanations here mostly useful. Unfortunately, stemming possibly from that same ignorance, I found the clutter in the code examples more noticeable and distracting. Try making sense, for example, of the explanation of a descriptor on page 75. Not sure what I’d have done to improve it, although removing the inline comments is probably not a bad idea especially as most of them are simple restatements of the next line of code.

Chapter 4 covers good naming and module / package organisation. And introduces the two things which I like perhaps most and least about this book: reference to useful external packages; and eggs. I like this reference to exernal tools because the stdlib doesn’t have everything – and nor should it, I say – and it’s always good to find out about tools other people have found useful. On the other side of the coin, I’m one of those people for whom setuptools / eggs and the subcommunity which has built around them are solving a problem I don’t have. That could be because I’ve never built a package with sufficient sophistication. But chapter 5 on “Writing a Package” plunges straight in with “Therefore, all packages can be built using egg structures” which I’m afraid put me off. Personally, I plan to go back and re-read this section when I’ve found myself trying to manage a package which is something more than a set of useful modules. As far as it goes, for those who are interested in learning about eggs and their interaction with PyPI, this chapter seems to give lots of practical advice. I would have preferred it to have started off slimline, showing a straightforward distutils setup.py without setuptools, without Paster.

Chapter 6 starts an application which runs through the rest of the book on-and-off: Atomisator, a feed aggregation toolkit. The author walks through creating the package, introduces sqlalchemy as a way to model the data and then discusses the API before talking about using a setup.py to declare interdependencies between subpackages and upload to PyPI. Chapter 7 extends this simple start by introducing zc.buildout, a build/release mechanism the author favours.

The next couple of chapters cover code management via version control systems – the author cannily focuses on Mercurial which has just been selected by GvR as Python’s VCS of the future. And speaks about continuous integration, showcasing buildbot, the Python-based system used by the Python core development among others. The chapters interleave higher-level explanations with low-level step-by-step installations. I feel that the latter actually detracts from the book a little: for one thing, projects move sufficiently fast that anyone reading this in a year’s time will very likely be seeing very different prompts and responses to those which the author illustrates. In addition, it leaves less space for real discussion of some key pros and cons or tips and tricks or Python-specific hints. Credit to the author, certainly, for being thorough, but I fear it could be counterproductive.

The next chapter neatly covers possible lifecycle models and illustrates the use of Trac as a project management tool: this is a nice choice as it’s a bit of a poster-child for Python and does lots of useful things well. Chapter 10 on Documenting is the free online chapter and I imagine many people had, like myself, read it before coming to the complete book. If one again cuts out the slightly over-explanatory example sections, this is the author at his best: there are concise and clear guidelines (for which he gives due credit to the author of Agile Documenting). More useful tools are introduced, including the invaluable Sphinx. Chapter 11 covers test-driven development and is nicely split into an “I Don’t Test” section followed by an “I do test” for new or existing converts, running over stdlib tools and nose / py.test. The chapter concludes with Fakes & Mocks and I think gives just the right amount of space to all of its sections and recommended tools: enough to give you a flavour of the tool without weighing you down with an extended example which might be better hosted on the tool’s own site.

The final section covers optimisation techniques, and covers approaches, data structures, caching, and multi-whatever programming touching on each one for long enough to give good enough feel of the pros and cons. More examples of useful tools which help to analyse the results of profiling and memory use. Here again I think the author’s approach shines: he’s doing a kind of question-and-answer session by himself and the motivation behind a number of the recommendations becomes a lot clearer. The very last chapter covers a number of the common Design Patterns, and I’m afraid I’ve never found the famous Patterns a particularly digestible topic so I’ll skip any commentary here.

In conclusion “Expert Python Programming” is definitely worth reading. If, like me, you’re not a fan of eggs and the like, then borrow it from a friend or the library lest you feel that you’ve wasted half the price. If you are a fan of eggs or are neutral, then buy the book. The best parts for my money are those where the author is covering speedily several different approaches or tools or issues and illustrating each one briefly.

As a postscript: the author’s examples are geared to Unix. Early on, he recommends installing MSYS and if you are a Windows user and a sufficient novice not to know what to substitute where, that might be your best solution. Personally I give MSYS & Cygwin the go-by as much as I can, unless forced (eg when trying build the ffmpeg libraries). Just one of those things.

Sadly, “The Dark is Rising”

When I was younger, I loved Susan Cooper’s fantasy sequence The Dark is Rising. I remember reading a few years ago that some studio had picked up the option to make a cinema-release film out of it and I was just about as enthusiastic as I could be. Then the thing seemed to go quiet (which seems to happen quite a lot with films). Earlier this year, Walden Media of Narnia & Holes fame announced that they were going ahead with the film, based on the second book after which the sequence is named. Naturally I was thrilled. While I didn’t think their first Narnia film was the best thing since, at least it was moderately faithful to the spirit of the author’s created world. And Holes was pretty much spot-on.

Alas, I should have known better than to trust Hollywood. When the trailer appeared on the internet, I scrambled for it and hunched agog over my laptop screen… Well, I nearly cried. About the only thing the film and the book have in common as far as I can see are the title and the names of some of the characters. Will Stanton, that understated but very normal Buckinghamshire lad, youngest son of a boisterous but affectionate family, has become Will Stanton the American schoolboy who discovers superman-like powers under the tutelage of a bunch of frankly weird-looking adults who seem to lack any of the dignity and gravitas of their literary counterparts.

I realise that a film doesn’t have to be — arguably shouldn’t be — a simple rendering of the book onto (digital) celluloid. But surely audiences can appreciate the appeal of something which stands out from the teenage-superhero mould and which shares a background with a host of British folk legends? Obviously I haven’t seen the film, but if the trailer’s anything to go by, we’ve lost any of the magic of the original, to be replaced by nascent telekenesis, pyrokenesis and an apparently obligatory love interest.

Oh well. Another dream bites the dust.

UPDATE: Evidently, I’m not alone.