musings

I feel a bit like an journalist digging up stuff from the trenches and presenting it to a wider audience for evaluation (i.e. posting about portal_view_customizations) but there has been another topic lingering in the background and coming up now and again on several mailing lists. There is a sentiment (and lament) to it, similar to the TTW issue.

I noticed it first, when I released the first beta version of Quills 1.6") ca. one month ago. Being a "Plone 3.0 only" release, I had worked on it using a custom buildout configuration which was just fine and dandy... for me.

When releasing the beta I initially hadn't even included a tarball, instead (thinking I was making it easier for potential beta testers) I simply referred to aforementioned buildout and included even a handy one-liner that would get you going.

The sentiment I was following (unconsciously at the time) seems to be prevalent among other developers, namely that buildout is easy to use, even for end users. "I mean, c'mon, it's only one line of copy-and-paste... how hard can it be?!"

Well, that's certainly true, if viewed in isolation. But the truth is, as usual, far more complex. I received several questions (and even complaints) both on and off-list afterwards. Just as with the TTW story, there was this notion of "This used to be easier! Why are you developers forcing this complicated new stuff upon us? Sure, it may be easier for you, but what about us?" (I'm grossly paraphrasing here, mainly because I think these complaints are indeed very valid.) The issues that came up fell mainly into two categories: either buildout itself gave rise to trouble (Windows related or python related) or people wanted to test Quills in their existing setup and were left puzzled as how to achieve that.

Well, one reason is, that things have indeed become more complicated with Zope 3. Let's not mince words here:

What (error prone, hard-to-debug, rot-in-hell-evil) magic Zope 3 takes away from Zope 2 it has to make up with additional paperwork.

Namespace-packages, .zcml, .cfg, the whole "multicomponadapterfaces" shebang. While most of this is only of concern to developers (who mostly are eager to make that trade-off - believe me, once you got used to the 'new style' you don't ever want to go back) some of this (currently) also has repercussions for end-users and integrators. Packaging (finally coming back to the title of this post) being one of them.

Let's (continue to) take Quills as an example. This used to be just a bunch of ordinary 'capital-"P"-Products' that you downloaded as a bundle and popped into your Products folder - presto! Now it's made up of several components (5 products and 4 packages). From a developers point of view that is a good thing, as each component does less and is easier to debug and maintain. For an integrator its (again) just extra sit-ups: "What needs to be copied where? Do I need to copy the whole quills.core directory or just quills.core/quills/core?" etc. pp.

Personally, I've always found the "unpack it and stick it into Products" approach rather inelegant but most people have accepted this as the de facto method of distribution and installation of Plone add-ons. This tolerance might start to wear thin, though, now that it's basically coming to "unpack it and stick certain bits into Products and others into lib/python/ (wherever that is on your system, good luck with finding it...) and stick something called zcml-slugs into etc/site-packages".

Like with the TTW issue I'm just playing the messenger here, though. I just want to work on Plone features and cool Plone products that users out there (hopefully) will find useful... if they can be bothered to get them installed, that is...

It's not as if the designers of Zope 3's component architecture hadn't come up with a design solution to this mess, though. Their answer is: use eggs. That way end users and integrators just need to download a few zip files (and stick them into some 'magic' folders... sound familiar?).

The problem is, Plone won't be entirely eggified for quite some time, as I understand it. 'Old-Style' Products can't be eggified, either and not all Products can simply be converted into packages, etc. etc.

As I mentioned, tolerance is starting to wear thin, adoption of Plone 3 is being slowed down. Somebody needs to make sure that the new Zope 3 way of doing things is followed all the way through to the end user. Currently it's stuck somewhere at the developer's feet. This needs to change.

So, I'm afraid this might turn into some sort of 'signature ending' of mine, but I'm ending this post, too with... "Any takers?" ;-)

In response to optilude saying "Plone 3 Rocks Hard" I feel compelled to add that developing with Plone 3.0 is (at least) just as enjoyable. Witsch and I have migrated a client project based on Plone 2.1 to 3.0 (trunk) ca. eight weeks ago and have been working with the Plone 3.0 API on a daily basis ever since. And while we did pay some penalty by aiming at a moving target that penalty has well paid off by the niceties we've been enjoying since.

From a developer point of view that change is perhaps not as dramatic as for an end user, as things like the Zope 3 component architecture (via Five), Generic Setup, membrane etc have been available already since 2.1. It's more the fact that Plone itself uses these technologies now to a much larger degree, i.e. their benefits are no longer limited to your own code. Just take a look at the plone.* namespace for instance, that now makes up the core of what is Plone 3 (as opposed to just CMFPlone which otherwise would undoubtedly have become dangerously bloated and monolithic).

By using Zope 3 components Plone is becoming a very rich development framework where you as a developer can 'mix and match' and chose from a wide array of components and adapt them for your needs without having to bother with (much) of the (Zope 2) baggage that comes with using a full-grown CMS. (instead of using some lighter weight framework such as Django, Turbogears or Ruby on Rails).

On a gut level it comes down to this for me: both the code that I am writing as well as the pages that are generated from it have become lighter, more readable and more elegant. I make less compromises in my daily work and the satisfaction that I get from it has risen drastically during the past six weeks. I'd ten times rather risk being labelled as a fanboy than shutting up about how cool Zope 3 and Plone 3.0 are. And now back to work (yay!)...

• Cache-Fu is entirely Five-agnostic. This means, that it won't set any caching headers for Five based templates. I.e. also not for basesyndication's feed templates.
• Feeds get a whole lot of repeated access, but change much more rarely than they get accessed (even on more prolific blogs than mine ;-)
• fatsyndication's default adapter has a very generic but also very expensive implementation for a feedsource's modification date (it simply queries all of its contained objects and then sorts them by modification date) because it doesn't (and shouldn't!) "know" anything about specific implementation details
• tomster.org currently contains roughly 500 blog entries.
• tomster.org/blog's feeds currently receive ca. 30.000 requests per month combined

Having completed two Plone-based projects yesterday I now want to take some time and reflect how these two projects went down and what I’ve learned during their course – not so much on a technical level but more from a management perspective.

Both projects differ quite from each other technically. One is basically just a “skin job” for a site using standard ATCT objects and a non-customized version of the Plone Tableless skin for editors, while the other is based on a multipage UML-diagram and requires customized editing interfaces and workflows).

Both fulfill their specification – but neither are finished. In my – for the time being ;-) – humble opinion I now think that it’s basically futile to try to release a 1.0 version of a site because you can’t create a polished interface without customer feedback. And – at least in my experience – most clients find it hard to provide specific and valuable feedback on beta stages of a site. I found that I’m doing nobody a favor when submitting half-finished inbetween-versions of a product for customer review. In the end I just confuse him (or her) and waste everybody’s time.

In these two specific cases we’ve now all happily arrived at 1.0 and are now (just as happily) taking ‘em for a ride and are collecting a wish-list of improvements that will turn them (hopefully!) into really cool sites. The key point here – and my lesson to take home – is to communicate this approach to the client from the start. As a developer you not only have to manage ressources and time but – perhaps first and foremost – expectations – i.e. tell the client clearly that the “1.0” will be a “rough diamond” – valuable but not necessarily pretty, yet.

In conclusion, here’s my proposed “life cycle” of a plone project:

1. initial consulting (What is Plone? Is it the right tool for the job? free)
2. specification (based on hourly rate and thus variable – but small in scope. Under no circumstances complete this step without having read Joel Spolsky’s Article Painless Software Scheduling first!)
3. implementation of 1.0 (the largest part of the work but based on the specification and thus fixed price and fixed date)
4. specifying, what’s still missing based on the real experience with the 1.0 – version (most work in this part done by client – no charge here, either, but no fixed time frame)
5. cleaning up to 1.1 (based on the findings in step 4 – fixed price and, again, fixed date)

I think this represents a fair trade-off between offering realistic compensation for the developer on the one side and budget security for the client on the other side. Because there are always technical uncertainties involved in any project that could endanger its success and in my view it falls into the developer’s responsibility to protect the client from these (i.e. if I have to spend some extra time during phase 3 wrestling ArchGenXML or learning how to write test cases that shouldn’t raise the cost of the project – especially since the client has absolutely no influence over this part anyway – the client needs to be protected from the developer, so to speak.

Not so with steps 2 and 4, though: here it is almost entirely up to the client to dictate the terms and thus the price. Here it’s the other way around: the developer needs to be protected from the client ;-) (by using an hourly fee in step 2 and by creating a second specification for step 5.)

Now I know for a fact, that meanwhile some of the more seasoned Plone and Zope developers happen to read this blog, too and I’d specifically like to ask you to share your opinion and/or experiences on this subject.

I’m sure we all can profit from being able to deliver slick Plone sites on time and within budget ;-)

I’ve recently been on the lookout for a pythonic webframework to add to my toolbag (which currently pretty much consists of one shiny-but-clunky multi-tool).

Just then I noticed Titus Brown’s article Picking a Webframework (which neatly spares me the effort of summing up features of Django, TurboGears, CherryPy etc.)

My motivation for looking for alternatives to plone were simply that I’ve come to realize that Plone really is overkill for some of the projects that I’ve currently got in my pipeline. For instance one will be a simple web-presence of a homeopathic practitioner with a simple guestbook. I really like the idea of starting with a clear table and then just add what is necessary. (With plone I always start with a – powerful – behemoth and then start stripping away… not the best way to build something, for sure.)

But Aaron Swartz’s discussion around writing his own web framework web.py has given me a valuable insight: even the lighter frameworks such as Django or TurboGears contain a lot of code and complex API’s you need to deal with.

It seems to me, that if I went ahead and tried to implement that simple website with, say, Django, I’d have to make a sizeable investment of time and energy in groking Django and still end up with lots of code but without enjoying the comforts Plone has to offer.

In the end Titus Brown decided on CherryPy simply because web.py isn’t released yet and he needs to move forward now. I, on the other hand, am in no real hurry and have thus decided to remain in the comfy world of Plone for now and keep web.py on my radar. I mean, just look at the tutorial – it really looks a lot slicker and simpler than the others. And in the mean time I can get my hands dirty with Cheetah, the templating engine recommended for web.py. Currently we create all our config files (for web-, dns- and mailhosting) via XSLT and/or directly printing to files via Perl and Python. Rewriting that crud using a sane (and pythonic) templating engine has been on my todo-list for pretty much a year already…