The 2010 mod_wsgi world road show.

FYI. I will be travelling to US, Canada and Malaysia for a holiday from late September. If any Python user group or company is interested in hosting a mod_wsgi talk while I am there, drop me an email at Graham.Dumpleton at Gmail. Cities where I will possibly have the time available are Los Angeles (??), New York and San Francisco in the US. Toronto in Canada and Kuala Lumpur in Malaysia. The trip is primarily a family holiday so specific days I may have available will be limited, but we might be able to organise something if there is enough interest and people can help me get to where ever any talk may be held so I don't get lost.

Light hearted look at status of WSGI on Python 3.

From the #pocoo IRC logs. We can only hope it doesn't come to that. :-)

2010-08-01T18:06:29  <jokey> mm what's the state of current wsgi for python3 btw? what are the open points?

2010-08-01T18:09:17  <DasIch> jokey: str vs. bytes

2010-08-01T18:10:43  <DasIch> jokey: one side says everything should be str, the other says everything should be bytes, another side says it should be a mix of the two and all of those sides don't have anything to show which is the best solution

2010-08-01T18:11:33  <DasIch> jokey: then there is also the group of people that says that wsgi was even in python 2.x not that good in certain points and wants to create something else altogether

2010-08-01T18:12:28  <DasIch> jokey: as you can see it's only a slight disagreement which should resolve once we have only one side because everybody else died

Oh and please don't let this start up another discussion just yet. I am still trying to find some time to do one of my state of the union summary posts which covers in detail the issues around the use of % encoding with SCRIPT_NAME and PATH_INFO.

Sydney PyCon mod_wsgi talk slides available.

UPDATE: Video now available at http://blip.tv/file/3840484.

Sydney PyCon is over and my talk on mod_wsgi went okay. For those interested the slides can be downloaded from the mod_wsgi site.

Rather than try and explain how mod_wsgi worked, I concentrated on showing how to setup some basic things, but in doing so deliberately hit all the common problems that some people have with setting up a WSGI application under Apache and mod_wsgi. The slides therefore capture the details of various obscure error messages one will find in the Apache error logs when things do go wrong.

To illustrate issues related to the Python module search path and virtual environments I used Django as an example. The slides therefore are especially valuable if you are trying to deploy Django using mod_wsgi.

The mod_python project is now officially dead.

At the Apache Software Foundation June board meeting which took place on the 16th June 2010, the following resolutions were passed unanimously.

B. Terminate the Apache Quetzalcoatl Project (to Attic)

This was the umbrella project for mod_python and as such mod_python is now no more.

Long live mod_wsgi. Or at least until I get bored of it. ;-)

The mod_python project soon to be officially dead.

The mod_python project last had a release February 2007. There has more or less been no developer activity since. Under the rules of the Apache Software Foundation if a project becomes inactive then it can be moved into what is called the Apache Attic. Come the next meeting of the ASF board a proposed resolution will be put up to dissolve the Quetzalcoatl project management committee and projects which it overseas, namely mod_python. Thus, in some respect one can say that mod_python will be officially dead.

What will this really mean. Well, no more development on mod_python, but then that isn't happening now. Linux distributions will likely still carry the package, but they will need to apply any changes themselves to ensure that it continues to work for newer versions of Apache. This actually is already being done as a bug in mod_python was exposed by some changes to the Apache runtime library and since no mod_python release has been made since then, only option was for distributions to patch it themselves.

I suspect though that this patching by distributions will only extend to Apache 2.2.X as Apache 2.4 changes the internal APIs enough that getting mod_python to compile will need more significant changes than a minor patch. You also will never see a version of mod_python for Python 3.X as that is going to require a radical rewrite.

For platforms like MacOS X where the last release of mod_python doesn't build properly, the only option will be to checkout mod_python source code from the subversion repository as that incorporates a fix for those issues, although it has since been shown to not be a complete fix and so you may still have some problems. In other words, the subversion repository will still exist, but it will be made read only and the location may potentially move.

What options exist if you want to move away from mod_python?

If you are only using mod_python as a means of hosting within Apache a distinct Python web framework or application and it supports the WSGI interface, then the obvious candidate is to move to mod_wsgi instead.

If you are using mod_python to implement custom access, authentication or authorization handlers, then you may also be able to get away with using mod_wsgi. You may have to make compromises though as mod_wsgi doesn't currently allow you to write full blown Apache style handlers and instead only implements the Apache authentication and authorization provider interfaces. This actually makes it easier to do most things, but you loose the ability to do some more complicated stuff which depends on using different error status values or custom error pages. You can partly get away with using ErrorDocument directive to return custom error pages, but not always.

Although proper support for Apache style handlers could be added to mod_wsgi, and a lot of work has been done in that direction already through the implementation of SWIG bindings for Apache, whether the remaining work to allow the actual hooking of handlers into mod_wsgi will be done will really depend on interest in it. Up till now, there hasn't been sufficient interest to justify doing the final bit of work and there has actually been some resistance put up to the idea of extending mod_wsgi too far beyond the core goal of providing WSGI hosting.

If you are not using the basic handler mechanisms of mod_python and are instead using the CGI handler, the publisher handler or PSP handler, then there aren't really any options at this point except for rewriting your application on top of a WSGI framework. If you have to do such a rewrite, but like the low level that one works at when using mod_python, then would strongly recommend you perhaps look at Werkzeug and Flask.

If you are using mod_python input or output filters there simply isn't any replacement. Frankly though I always thought that writing input or output filters in mod_python was a really bad idea. Yeah it may work, but it wouldn't exactly be efficient. You would be much better writing a proper Apache module in C to do what is required, certainly if performance is an issue.

Finally, the ability to use Python code with Apache server side include mechanism also has no real equivalent. But then, not sure anyone ever actually used that feature anyway. Something similar could possibly be implemented in mod_wsgi but not sure there would really be a point. If you are only using ability to include Python code, you would be much better off using a proper templating system that works with a WSGI framework.

So, the writing is on the wall so to speak and if you are using mod_python, you really should be starting to plan how to move away from it as eventually it will likely not be an option you can use if you want to keep up to date with Apache and/or Python.

Could it yet be saved by a white knight. Well, yes it could as the Apache Attic does allow projects to be resurrected or forked with it then being maintained outside of the ASF. If someone does do the latter though, you would need to be mindful of the name as the ASF in some respects has rights over the mod_python name. As such, you would likely need to rename the project when you take it over.

If you fork mod_python, you would also want to think about capturing all the details of the significant number of still open bug reports for mod_python in its issue tracker, as am not sure what will happen to that and whether it will be made read only or whether it will be completely closed down.

Anyway, will be interesting times ahead. I should point out though that according to Google trends (currently borked), searches on WSGI finally overtook those for mod_python recently. Searches for mod_wsgi haven't yet, but not far off occuring. It is also the common belief that WSGI is the way to go for Python web applications now, so perhaps it is simply time to just move on from mod_python. If we could only just get newbies, who even now keep using mod_python even though better options exist, to understand this, then maybe it can finally be left to die in peace.

I will be presenting at Sydney PyCon.

The conference schedule has just been announced for PyCon in Sydney this June. If one looks closely one will find I decided to get off my butt this time around and actually put my hand up for doing an introductory talk on getting started with mod_wsgi. I usually like to hide under a rock and avoid such things, but since this is the first time PyCon is being held where I live in Sydney, thought I better put in the effort.

Although I am not presenting any talk, I am also going to the PyCon in Singapore this June as well. So, will be a busy month.

BTW, I will be in Singapore for a week and not just the conference days. If members of any local Python user group is interested in meeting up outside of the conference days to talk about mod_wsgi, I could be interested. I could perhaps use you as Guinea pigs for my Sydney talk. Shopping with my family will take precedence, but can see how things go. :-)

Time to find a new job.

I have been in my current job for 8 years. I am thinking it may be time to move on.

The problem is that in this job I have slowly been becoming somewhat deskilled, no longer being buzz word compliant. If I were to rely purely on the employment history in my resume, I am not sure how successful I would be at actually finding a decent job. On top of that, the local job market isn't particularly flash as far as interesting roles anyway.

One of the reasons on specualating about changing jobs is that my interests don't actually intersect with what I have been doing in that position. I have hardly even got to do much programming at all in that position for quite a long time. The only way I have managed to keep myself challenged is through the work I do on Open Source projects in my own time.

If you talk to local head hunters in my area though, some are quite adverse to Open Source. I have even had one bluntly say to me that I should stop doing all Open Source work as it will get me no where. I sort of got the impression that he felt that when working for a company that everything you do should belong to them and that if you work on Open Source in your spare time that you must be stealing from the company.

You also get head hunters and/or companies that seem to put more emphasis on your ability to answer a mini exam on basic programming or system factoids commensurate with you having just left university. That you may have been working for a long time and have a diverse experience with working on many systems and integrating them together, especially Open Source software, and can show signficant activity as a developer or contributor of respected Open Source projects seems to mean nothing.

Anyway, you can see I have a rather dim view of head hunters. Because of this and because my current contract doesn't end until October (although can leave earlier) I think I might like to try an experiment. Rather than see what I can do through the local head hunters, I would like to see what others out there who know me and what I do in the Open Source community have to directly offer me or can point me to that may be of interest.

Obviously if it is in the area that my own Open Source projects relate to, ie., Apache module development, Python web development and web hosting, then all the better. Just be aware that my focus has very much been on the issues around hosting. This means I am not an expert on specific Python web frameworks such as Django, Pylons or TurboGears, although obviously I would have no problems learning what I need to know.

At the same time though, a job in a hosting company simply maintaining existing web hosting infrastructure will likely not be of interest. Instead my preference is probably for working on development of new or better ways to support web hosting, in particular for Python. If the company is a big user of Open Source and contributes back to the Open Source community rather than keeping everything proprietary, then would be a bonus.

What are the restrictions? Well, I cant relocate from where I am in Sydney, Australia. So, would have to be a company with a local presence. Alternatively, company would need to be able to cope with remote teleworking such that I could work from my home. I accept that if working remotely that trips to company offices may be needed occassionally, although the USA is quite a hike from here for me, Asia not so much and Europe may as well be on a different planet.

So, what job opportunities do you know of out there in your own companies that you think I might be interested in?

As I said, my resume probably doesn't say much. Thus am relying on my reputation from developing my own Open Source projects OSE and mod_wsgi and my involvement in mod_python as well as contributions to other Open Source projects over the years. Providing some recognition of my contributions to the Open Source community is the fact that I am a member of both the Apache Software Foundation and Python Software Foundation, membership of both being by nomination only by existing members.

If you need to ask for my resume, then it sort of defeats the purpose of my experiment and so likely wouldn't be interested if that is all the company is going to rely upon. Also don't submit me as referral into any companies recruiting programs. The last thing I want to deal with is some faceless people who want to herd me through some never ending recruitment interview process. If you are going to approach me, then I want to know about the actual projects you have which I might work on if working at the company. Am not interested in just the prospect of being able to work for some well known company without even knowing what I would be doing. Obviously, any head hunters themselves also need not apply.

Let us see how this works out. If no bites, then I guess I will just have to see what the local head hunters do have to offer. That or just stick it out where I am working now. :-)