Friday, October 9, 2009

WSGI issues with HTTP HEAD requests.

The HTTP protocol defines the GET and HEAD method types for requests. As outlined in the RFC-2616, the HEAD method is specified as:
The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification.

The response to a HEAD request MAY be cacheable in the sense that the information contained in the response MAY be used to update a previously cached entity from that resource. If the new field values indicate that the cached entity differs from the current entity (as would be indicated by a change in Content-Length, Content-MD5, ETag or Last-Modified), then the cache MUST treat the cache entry as stale.
Key in this statement is that it says 'the server MUST NOT return a message-body in the response'. Problem is that many web application authors take it upon themselves to treat differently the HEAD request. What they do is make the decision not to return any content for the response, rather than leaving it up to the underlying web server to discard it and not return it to the client.

Taking this action in web applications can have unintended affects. The issue here being that the response headers for a GET and HEAD are supposed to be identical.

For WSGI where it is possible to write WSGI middleware that wraps other WSGI components or applications, this can result in the WSGI middleware being deprived from seeing the response content. If that WSGI middleware is generating response headers based on the response content, then it can generate results for a HEAD request which then differ to that for a GET request.

There has in the past been some debate about whether a WSGI middleware that allows for compressing of response content is allowed, but various example abound. One such example is in the Pylons book. The idea with such a middleware is that rather than just pass back the response content, it will accumulate it and compress it. From the result of that it will then set an amended Content-Length response header providing the new length of the returned content, then yield that modified content.

Imagine now where such a WSGI middleware is used to wrap another WSGI component or application which decides to discard response content for a HEAD request. In this case there will be no content to compress and so the WSGI middleware will generate a Content-Length of '0' rather than the same value that would have been returned for a GET request.

So, should a WSGI components or application generating content never distinguish between GET and HEAD and always treat a HEAD request as a GET request? Although it is arguable that they should treat them the same, various of the larger Python frameworks do treat them differently and will throw away response content for a HEAD request. I can't see that they are likely to change their code.

What to do then? Simple answer is that since we are never going to be able to trust users not to treat GET and HEAD the same, a general guideline is that if your WSGI middleware must always see the response content, even for a HEAD request, then it should change the 'REQUEST_METHOD' variable in the WSGI environment passed to the enclosed WSGI component or application from 'HEAD' to 'GET'. This is the only way you will know things will work as intended.

This problem isn't isolated to just WSGI middleware though. When using Apache, there can be configured output filters that want to do exactly the same sort of thing as the compressing WSGI middleware. For example, there is the 'DEFLATE' output filter provided by the mod_deflate module and which also handles compression of response content. In this case though the output filter is outside of the realm of the WSGI stack.

Apache/mod_wsgi has recognised this problem for a while though and as a result when it detects that an output filter has been registered with Apache that processes the response content, it will forcibly change 'REQUEST_METHOD' from 'HEAD' to 'GET' even before it is passed into the WSGI stack. The WSGI application as far as it can tell got a GET request and is none the wiser.

Although this detection is part of mod_wsgi, it isn't going to occur when using a WSGI adapter on top of mod_python, FASTCGI, SCGI or CGI. As a result, using those other types of hosting systems means that you still risk getting different results for GET and HEAD if a WSGI component or application decides itself to discard the response content.

This isn't the end of our troubles with respect to a HEAD request. In the WSGI specification it says:
If the application does not supply a Content-Length header, a server or gateway may choose one of several approaches to handling it. The simplest of these is to close the client connection when the response is completed.

Under some circumstances, however, the server or gateway may be able to either generate a Content-Length header, or at least avoid the need to close the client connection. If the application does not call the write() callable, and returns an iterable whose len() is 1, then the server can automatically determine Content-Length by taking the length of the first string yielded by the iterable.
This all sounds good and looks like a good optimisation. Problem is that it itself can cause different results to be returned for the HEAD request than the GET request and thus is arguably wrong and should partly be stricken from the WSGI specification.

To understand the issue, imagine a WSGI application which returns an iterable of length 1 and which for a GET request yielded a string of length '666'. If no 'Content-Length' response header is set, then the guideline above says that the WSGI adapter can create the 'Content-Length' header itself, which it would with the value '666'.

Now for a HEAD request, if that same WSGI application decided not to return any response content, that is the yielded string was empty, but still didn't set 'Content-Length', then the WSGI adapter would blindly set 'Content-Length' to '0'. So, the result will be different to that for the GET request.

One might think that one could just not generate a 'Content-Length' for a HEAD request, but having a 'Content-Length' for GET and not for HEAD is still leaving the two responses different in some way.

To me then, it seems that no WSGI adapter should perform automatic generation of 'Content-Length' response header. Alternatively, and to avoid the whole problem, WSGI adapters should always say it is a GET request, rather than passing on the HEAD request.

I don't know how other WSGI like systems such as RACK, JACK and PSGI, or even other web systems, deal with this issue, but definitely think that WSGI has got something wrong here.

Anyone else got any comments on this? Should I be going and modifying mod_wsgi to remove that feature whereby it adds 'Content-Length' where it can? Is the WSGI specification wrong in saying that?


Friday, October 2, 2009

New release candidate for mod_wsgi 3.0 is now available.

Version 3.0c5 of mod_wsgi is now available. This is the 5th release candidate for mod_wsgi version 3.0. The process of developing and releasing version 3.0 of mod_wsgi has dragged on for quite a long time now because of the uncertainties over how Python 3.X support should work. So much so that a lot of the fixes and changes were back ported to mod_wsgi 2.X and released in mod_wsgi 2.4.

Version 3.0 of mod_wsgi will support both Python 2.X and Python 3.X. So, the big question at this point is what is being done about Python 3.X support.

For starters, Python 3.0 will not be supported, you will need to at least be using Python 3.1 if you want to use Python 3.X at all. This is in part because I am assuming the PSF itself would prefer people not to use Python 3.0, but also because am relying on 'surrogateescape' mechanisms from PEP 383 in some of the ways that unicode strings representing file system paths are dealt with where they are passed to WSGI applications under Python 3.X and this was only added in Python 3.1.

As to the whole argument about whether bytes or unicode strings should be passed for values in WSGI environment, and if unicode what encoding should be used, I am effectively sticking with the original proposal put up by PJE about this. This is the same definition that mod_wsgi 3.0 development code has implemented all along and which people have been using already. That no one has complained that how it was implemented was causing problems for them very much suggests for me that it is a reasonable compromise for how it should work.

Just so we are clear on what this definition is, I include it below. This was labelled Definition #3 in my post about WSGI roadmap.

1. The application is passed an instance of a Python dictionary containing what is referred to as the WSGI environment. All keys in this dictionary are native strings. For CGI variables, all names are going to be ISO-8859-1 and so where native strings are unicode strings, that encoding is used for the names of CGI variables.

2. For the WSGI variable 'wsgi.url_scheme' contained in the WSGI environment, the value of the variable should be a native string.

3. For the CGI variables contained in the WSGI environment, the values of the variables are native strings. Where native strings are unicode strings, ISO-8859-1 encoding would be used such that the original character data is preserved and as necessary the unicode string can be converted back to bytes and thence decoded to unicode again using a different encoding.

4. The WSGI input stream 'wsgi.input' contained in the WSGI environment and from which request content is read, should yield byte strings.

5. The status line specified by the WSGI application should be a byte string. Where native strings are unicode strings, the native string type can also be returned in which case it would be encoded as ISO-8859-1.

6. The list of response headers specified by the WSGI application should contain tuples consisting of two values, where each value is a byte string. Where native strings are unicode strings, the native string type can also be returned in which case it would be encoded as ISO-8859-1.

7. The iterable returned by the application and from which response content is derived, should yield byte strings. Where native strings are unicode strings, the native string type can also be returned in which case it would be encoded as ISO-8859-1.

FWIW, this is also the same definition as the 'wsgiref' library in Python 3.X has been using, albeit that how it was implemented in Python 3.0 was somewhat broken.

Up until now, even in the absence of a definition of what WSGI should be for Python 3.X, the mod_wsgi implementation and the 'wsgiref' implementation have been carrying the 'wsgi.version' tag in WSGI environment of '(1,0)'. As such, this has already more or less become the defacto standard of what WSGI 1.0 is on Python 3.X.

Going forward though I am no longer going to set 'wsgi.version' to '(1,0)', instead I am going to label it as '(1,1)'. That is, WSGI 1.1. This will also apply to WSGI as implemented for Python 2.X in mod_wsgi version 3.0.

The reason for calling it WSGI 1.1 for both Python 2.X and Python 3.X is to make it clear that the implementation supports the amendments I previously described for WSGI 1.0. Actually, mod_wsgi has always been implemented inline with those amendments but has more or less been impossible for WSGI developers to rely on things working that way. Thus, using the WSGI 1.1 moniker is to give some certainty about what is actually implemented.

I fully realise that I am making my own decision here to use WSGI 1.1 name and not defer to the Python WEB-SIG to agree on something and some may complain I have no right to do so. Given though that the discussions on the Python WEB-SIG was moving to any more significant update beyond the definition above being called WSGI 2.0, even if it retained 'start_response()', I believe it is a reasonable call on my part and at least gives some short term certainty for developers about what to expect.

If people want to vehemently complain about it being called WSGI 1.1, then I guess I will just need to call it something other than WSGI from now on.

As to all the other discussions about WSGI 2.0 and WSGI 3.0 on the Python WEB-SIG, if anything does come of that, I will not be implementing them in mod_wsgi itself.

This doesn't mean that one wouldn't be able to use WSGI 2.0 or WSGI 3.0, whatever they may turn out to be, with mod_wsgi, it just means that you will need to use an adapter to bridge between what mod_wsgi implements and the latter versions.

Part of the reason for taking this approach is that it has to be understood that mod_wsgi has no Python code in it, it is actually all C code. Thus, the proposal such as put forward by Ian Bicking is actually a PITA when it comes to implementing them in mod_wsgi.

Don't get me wrong, I am not saying anything against the proposal as it is as reasonable as some of the others, just that for mod_wsgi and how it is implemented as C code it isn't necessarily as trivial to implement as for pure Python web servers. There is as such some sense to taking a layered approach for mod_wsgi with it only implementing the current WSGI specification and deferring anything else to higher level provided by way of a separate package implemented as Python code.

So that it isn't too painful to try and use any newer version of the specification, and in pursuit of making mod_wsgi even easier to use than it is, I am though working on a separate deployment interface for mod_wsgi. Take this is a blending of ideas from Paste and buildout.

The intention would be to have a Paste or buildout style WSGI deployment description file in place of the current WSGI script file, with what version of WSGI being used and how one adapts a specific WSGI application and setup virtual environments being specified by way of plug and play recipes.

The end result would be that instead of having to construct your own WSGI script file, you would just use pre canned recipes that do all the hard work for you. You would just need to, as per the requirements of the recipe, specify such things as where your WSGI application instance is located etc.

Anyway, that is what is happening with Python 3.X support. As to what else mod_wsgi 3.0 contains, you can find the details in the CHANGES file for now. I'll be making other posts about some changes as I need to solicit some feedback so can determine what default behaviour should be for some changes.

So, please go forth and see what issues you can find in this release candidate. Because the process has been long, it is actually appearing to be very stable already, even with Python 3.X. Hopefully I can wrap everything up in the next few weeks and finally release something. Then I can look more at this new deployment mechanism and so maybe once again the discussion around WSGI 2.0 and WSGI 3.0.

Details on WSGI 1.0 amendments/clarifications.

In my last gargantuan post I tried to lay out a possible roadmap for the future of WSGI. That generated another large discussion on the Python WEB-SIG list. I don't want to go into any outcomes of that discussion just yet though. Instead, I want to focus on one part of that roadmap and go into it in more detail. That part was the set of amendments/clarifications I listed towards the end of the roadmap. Although these were thrown in towards the end of the post, they actually stand independent of the bigger issue of how WSGI should look for Python 3.X.

The extent to which this is the case is that even if nothing is worked out about Python 3.X and how URI variables should be represented, we could still bring out a revision of WSGI 1.0 called WSGI 1.1, which contains just those amendments. In fact, like trying to come out with changes related to Python 3.X support, getting out a revision of the WSGI specification with some of those changes has also been attempted in the past and nothing has ever come of it.

To better understand what those changes are about, I will list of each of them and explain the reason behind the suggested change.

1. The 'readline()' function of 'wsgi.input' may optionally take a size hint.

The WSGI specification as it is written states 'The optional "size" argument to readline() is not supported, as it may be complex for server authors to implement, and is not often used in practice'.

Unfortunately since that was written the 'cgi.FieldStorage' class in the Python standard library was modified to make use of the ability of 'readline()' of a file like object to accept a size argument. Because many WSGI frameworks make use of 'cgi.FieldStorage' for the purposes of parsing the content of POST requests, this has resulted in those WSGI framework being in conflict with the WSGI specification as they indirectly rely on a feature which the API specification says a complying WSGI doesn't need to support.

2. The 'wsgi.input' must provide an empty string as end of input stream marker.

The WSGI specification as it is written states 'The server is not required to read past the client's specified Content-Length, and is allowed to simulate an end-of-file condition if the application attempts to read past that point. The application should not attempt to read more data than is specified by the CONTENT_LENGTH variable.'

In other words the WSGI server can provide an end of file indicator in the way of an empty string, but is not required to. On the basis that such an end of file indicator is not guaranteed to be present, the WSGI application is not allowed to read more content from 'wsgi.input' than is specified by CONTENT_LENGTH. In the case where CONTENT_LENGTH was not defined or is an empty string, then CONTENT_LENGTH is supposed to be interpreted as being '0'.

As far as a WSGI application code goes, this means that any WSGI application has to keep track of how much data it has read from 'wsgi.input' in a variable, incrementing that by the block size it uses for each read. It has to then compare that to CONTENT_LENGTH, ensuring it doesn't read more than it should. This may mean that it has to deal with a partial read on last iteration where CONTENT_LENGTH is not a multiple of the block size used by the code when performing reads.

There are few problems with this. The first is that although the WSGI application notionally should not rely on there being an end of file indicator in the way of a read returning an empty string, it still has to cater for that situation. This is because premature closure of the client connection may not always result in an exception being raised, indicating a failed read. Instead it may simply result in the return of an empty string. This could still occur in cases where an implementation doesn't return an empty string when all the request content has been consumed. If a WSGI application does not deal with an empty string being returned for this case and assumes it can just keep reading until amount of data specified by CONTENT_LENGTH is returned or an error occurs, then it can find itself in a tight loop when the read returns an empty string instead.

Short story is that the WSGI application has to deal with an empty string anyway, so why not make it mandatory for a WSGI adapter to ensure one is always supplied.

The scenario where this may be problematic for a WSGI adapter is where HTTP/1.1 and keep alive connections are supported. In this case, if 'wsgi.input' corresponds to the socket connection from the client, one can't rely on the send half of the connection being closed when all request content is sent. If a WSGI application were to read more data than specified by CONTENT_LENGTH in this case, it would potentially cause the WSGI application to hang forever, or at least until the client timed out and closed the whole connection. This is effectively why the WSGI specification says that WSGI application should not read more data than it specified by CONTENT_LENGTH. That is, it relies on WSGI applications being well behaved and once all data specified by CONTENT_LENGTH is read, that it processes the data and returns any response. That same socket connection could then be used for a subsequent request.

Note that for correct operation, not only is it relied upon that the WSGI application not read more data than specified by CONTENT_LENGTH, it is actually necessary that the WSGI application read exactly that amount of data when available. If the WSGI application doesn't consume the exact amount of data, then it will still be present in the buffers of the socket connection and could be erroneously interpreted as being the start of a subsequent request over the same socket connection.

A new requirement to make it mandatory for a WSGI adapter to ensure an empty string is returned as end of file sentinel, means that raw socket can now no longer be supplied as 'wsgi.input', instead 'wsgi.input' will need to be supplied as an object that wraps the socket connection. This wrapper object will need to count how much data has been read, and when the amount of data reaches that as defined by CONTENT_LENGTH, any subsequent reads should return an empty string instead. Having the WSGI adapter do this though is good, as it ensures that a WSGI application can't hang. Further more, the WSGI adapter can at the end of a request ensure that any data supplied with the request which wasn't consumed is read and discarded and that if the socket connection is reused for a subsequent request, that any residual data doesn't get wrongly interpreted as being part of that subsequent request.

Forcing the requirement for an empty string to be returned as end of file sentinel, also resolves a potential problem when 'readline()' is used without a size argument. The issue here is that since a size argument doesn't have to be supplied, it is possible to inadvertently attempt to read more more data than specified by CONTENT_LENGTH. For code such as 'cgi.FieldStorage', the only reason a problem doesn't occur is that the request content has its own end marker within the input stream to represent the end of data and this always ends with a newline. Thus 'readline()' will always supply the end marker on any final read. If for some reason that newline wasn't supplied, or the request content was malformed in some way, it again is possible that a WSGI application could hang due to attempting to read more data than it should.

As you can see, mandating that a WSGI adapter be required to return an empty string as end of file sentinel avoids a number of potential problems. Such a change also has the potential to simplify WSGI application code. This is because at present a WSGI application has to maintain a variable tracking how much data it has read so as to ensure it doesn't read more than CONTENT_LENGTH. If an empty string as end of file sentinel is guaranteed, it doesn't necessarily need to do this any more. Although, that said, robust WSGI applications may want to still do validation based on amount of data read in case request content was truncated due to client closing connection prematurely.

Depending on an empty string for end sentinel also opens up possibility for a few new features which at present aren't strictly allowed by WSGI specification.

The first of these is the concept of mutating input filters. These are web server level input filters or WSGI middleware components, which modify the request content as it is being read in.

The best example of this is the mod_deflate module for Apache, which implements the ability to handle compressed data in the request content. In this scenario the CONTENT_LENGTH would actually be the original size of the compressed data sent by the client. When that is uncompressed the actual amount of data would normally be greater than that specified by CONTENT_LENGTH. If the current WSGI specification is followed, an application is only allowed to read up to CONTENT_LENGTH so it would end up truncating the input, but possibly think that all was fine. Further, because it hadn't read all the actual request content, the residual when HTTP/1.1 is used could be wrongly interpreted as start of next request.

End result is that an Apache module such as mod_deflate which mutates the input stream, turning it into something which is of a different length, can't be used with WSGI applications which only read up to length specified by CONTENT_LENGTH. If instead the WSGI application just kept reading data until an empty string was returned, then it would all work fine.

Amusingly, because 'cgi.FieldStorage' uses 'readline()' and so effectively ignores CONTENT_LENGTH, the mod_deflate module can actually be used in the case where it is known that 'cgi.FieldStorage' would always be used to interpret the request content.

That such inconsistencies exist gives further weight to there being a requirement that an empty string always be used as end sentinel and that WSGI applications give greater weight to that, than the value of CONTENT_LENGTH. The result is that CONTENT_LENGTH would be mostly ignored, perhaps only being used where a WSGI application wants to block a request before reading anything if the original amount of request content was greater than a certain amount, in that case returning a 413 error response.

The other feature that could be supported is the ability of a request to use chunked data. That is, where the HTTP request uses 'chunked' as the transfer encoding.

In this type of request there is no content length and as the specification is now, a WSGI application should treat that as zero length and would believe there is no data at all to read. Some WSGI adapters work around this by reading in all the request content, calculating the length and setting CONTENT_LENGTH before the request is passed through to the WSGI application. To do this though means that all the request content has to be held in memory or possibly subsequently written to disk if it turns out to be a large amount of data.

Having to do this complicates the WSGI adapter and denies the WSGI application the ability to stream the request content and process it as it arrives as the WSGI adapter will already have read it in.

By having an empty string be required as an end sentinel, the WSGI application itself seeing that CONTENT_LENGTH wasn't specified, but seeing that HTTP_TRANSFER_ENCODING was 'chunked' could just go ahead and read the request content, processing it as it goes, until the empty string as end sentinel was reached.

The ability to handle chunked request content is likely to grow in importance as some of the smart phones now available fallback to using chunked request content when the amount of data being sent is greater than some predefined value. That predefined value is a part of the software on the phone however, is not controllable, nor is use of chunked request able to be turned off.

If people want to be able to use Python to implement web applications that support such devices, then WSGI specification needs to be able to handle chunked request content.

The only issue with this is that existing WSGI applications are going to see CONTENT_LENGTH being missing as indicating there is no data at all. If chunked request content is now made permissible under WSGI specification by virtue of empty string as end sentinel being mandatory, then existing applications will appear to fail and it will not be obvious what is going wrong.

As such, although WSGI should be able to handle chunked request content, it may be wise that WSGI adapters have such ability to pass through chunked requests as being optional, with the server needing to be configured explicitly to allow them.

Apache/mod_wsgi currently takes this stance with mod_wsgi 3.0 because in prior versions it always blocked chunked requests anyway, as was known that weren't strictly supported by WSGI specification. But then, wsgiref server has never treated chunked requests any differently so it already tried to pass them through, but with WSGI applications always treating them as requests where no content was supplied.

So, although would want to be able to handle chunked request content, not obvious whether such requests should always be passed on, with potential for silent failures, or whether should require it to be explicitly enabled in the server.

3. The size argument to 'read()' function of 'wsgi.input' would be optional and if not supplied the function would return all available request content. Thus would make 'wsgi.input' more file like as the WSGI specification suggests it is, but isn't really per original definition.

This one is reasonably self explanatory. The WSGI specification says that 'wsgi.input' is supposed to be file like yet doesn't support this basic feature of such objects. In practice it may not be overly useful except for quick and dirty scripts, but if we are going to clean up 'wsgi.input' so it is more file like in other areas, should also align this as well.

4. The 'wsgi.file_wrapper' supplied by the WSGI adapter must honour the Content-Length response header and must only return from the file that amount of content. This would guarantee that using wsgi.file_wrapper to return part of a file for byte range requests would work.

The WSGI specification currently says in respect of 'transmission should begin at the current position within the "file" at the time that transmission begins, and continue until the end is reached.' In other words, all data with file from that point on should be returned.

The problem with this though is where the 'Content-Length' response header as specified by the WSGI application defined a value which was actually less than the amount which could be read from the file.

Strictly speaking a WSGI application shouldn't be returning more data than specified by the 'Content-Length' response header, yet the WSGI specification is effectively saying the WSGI adapter should do exactly that.

The WSGI specification should be more precise in the area of how much data is allowed to be returned in relation to what is defined by the 'Content-Length' response header. One benefit of doing that would be that 'wsgi.file_wrapper' could then reliably be used as a means of returning a range of a file in an optimised way to satisfy byte range requests.

Right now it is a bit of an unknown whether a WSGI adapter which supports 'wsgi.file_wrapper' is ignoring the 'Content-Length' as WSGI specification suggests should be done, or whether they see common sense prevailing and not allow more data than specified by 'Content-Length' to be returned.

Because of HTTP/1.1 and pipe lining of requests, allowing more data to be returned than specified by 'Content-Length' could cause problems with any additional data returned being interpreted as the start of the response corresponding to any subsequent request.

5. Any WSGI application or middleware should not return more data than specified by the Content-Length response header if defined.

This is just enforcing the rule about 'Content-Length' above to WSGI middleware. Albeit that it perhaps only need to apply where the WSGI middleware is inserting its own wrapper around the generator returned to perform a transformation on the returned data, or to calculate a response header based on the returned content.

6. The WSGI adapter must not pass on to the server any data above what the Content-Length response header defines if supplied.

Again, same rule, but this time as fallback saying that the WSGI adapter shouldn't return more data than it should. This is to catch where WSGI middleware or application isn't implemented correctly and is returning wrong amount of data.

Anyway, that is what the amendments were about and what issues they were trying to address.

In addition to these, there have been a couple of suggestions made by others as well.

The first of these was that a mutating input filter should override CONTENT_LENGTH and set it to the value -1. Whether using such a value would cause problems is a bit of an unknown given that value is no longer positive. For Apache at least, it also isn't possible to reliably detect when an Apache input filter mutates the input and changes its length, so no way to even do that.

The second suggestion was that 'wsgi.input' should supply the additional method 'tell()'. This would allow one to find out how much data had already been read. Technically this wouldn't be too hard as many WSGI adapters would be explicitly counting the amount of data read anyway in order to simulate end of file sentinel, so would be available. Introducing this method though may signal to some that 'wsgi.input' should also support 'seek()', but rewinding input isn't generally going to be able to be implemented. Also, with using empty string as end sentinel, there would be a movement away from needing to know how much data was read as it is being read. Either way, it certainly isn't a requirement for things to work, so perhaps best to not add it.

So, as far as those other two suggestions go, I am not convinced they are worth including.

Now, I always take time to get blog posts out these days, but I hope to follow this post up soon with another which details how the above and other discussions about WSGI and Python 3.X fit into my mod_wsgi version 3.0 release plans. I was never going to hang around waiting forever until decisions are made, so have decided that I am going to make some of my own. :-)