Solving some of the problems with wsgi.file_wrapper.

The WSGI (PEP 333) specification describes an optional extension for high performance file transmission. Sounds great, but it has various issues in practical use. As a result, not many frameworks or applications try and make use of it, or when they do, because of other features they try and implement on top of the WSGI interface, they actually stop it working properly.

I am not going to go through all the problems with wsgi.file_wrapper here. Some of the problems I have described previously in this blog and also in posting to the Python WEB-SIG. I'll summarise all the problems another time. Today I want to revisit the specific problems around the close() method of the generator returned when wsgi.file_wrapper is used.

The idea behind wsgi.file_wrapper is that if a WSGI server has a means of providing a better platform or server specific way of delivering up static content than simply reading content from a file like object in chunks and writing it back to the client, that it add to the WSGI environ dictionary the key 'wsgi.file_wrapper'. This key should reference a callable object which could then be invoked by a WSGI application, passing it a file like object, which is then somehow wrapped or tracked. The result of the callable should then be an iterable object which is returned from the WSGI application.

The typical implementation would be such that the iterable returned by the wsgi.file_wrapper callable would encapsulate the file like object, with the iterable being of a specific class type. When the server is processing the iterable returned from a WSGI application it would recognise that it is an instance of this specific class type, and instead of iterating over the iterable, it would instead retrieve the file like object directly out of the iterable object and use some high performance transmission mechanism such as sendfile() to directly write the content addressable by the file descriptor enclosed by the file like object over the socket connection back to the HTTP client.

The WSGI specification provides an example of this special file wrapper class type as:

class FileWrapper:
    def __init__(self, filelike, blksize=8192):
        self.filelike = filelike
        self.blksize = blksize
        if hasattr(filelike, 'close'):
            self.close = filelike.close
    def __getitem__(self, key):
        data = self.filelike.read(self.blksize)
        if data:
            return data
        raise IndexError

with the WSGI server performing a check something like:

environ['wsgi.file_wrapper'] = FileWrapper
result = application(environ, start_response)

try:
    if isinstance(result, FileWrapper):
        # check if result.filelike is usable w/platform-specific
        # API, and if so, use that API to transmit the result.
        # If not, fall through to normal iterable handling
        # loop below.

    for data in result:
        # etc.

finally:
    if hasattr(result, 'close'):
        result.close()

In this case the value associated with the 'wsgi.file_wrapper' key in the WSGI environ dictionary is the special 'FileWrapper' class type itself and as you should know, when a class type is executed, it has the result of creating an instance of that type.

Although the value associated with the key in this case is the class type itself, the WSGI specification doesn't mandate that. Instead, it need only be a callable, and so it could actually be a function which internally then creates an instance of what ever type is used to produce a suitable iterable. The details of the file like object need not even be encapsulated within that iterable object and could be stored elsewhere, so long as the WSGI server knows how to obtain it later for this specific request and guarantee that it is the value originally used in creating the iterable from the wsgi.file_wrapper callable that is actually ultimately returned by the WSGI application.

Implementing all this can be tricky but luckily only the WSGI server implementers need do it. Even so, not all WSGI servers that attempt to implement it get it completely right. Apache/mod_wsgi as far as I know get its right. uWSGI however as far as I can tell gets it wrong in a number of situations, granted though that they aren't typical use cases. Part of the problem here is that the WSGI specification doesn't actually go into how one would implement it, and so WSGI server implementers have to work it out for themselves, and if you don't think it through properly, it is easy to get it wrong. Even with mod_wsgi for Apache it took a few iterations to get it right, helped out by some users who were trying to do some out of the ordinary things with wsgi.file_wrapper. Anyway, how to actually implement the performance optimisation is not the point of this post, so we will move on.

The actual topic I want to talk about is the close() method of the iterable returned by wsgi.file_wrapper. This method was meant to mirror the purpose of the close() method for generators proposed in PEP 325. That is, to provide a means for releasing of resources when everything had been consumed from the iterable, or the iterable otherwise was no longer required. As it turned out, PEP 325 got rejected and got replaced by PEP 342, which ended up implementing that aspect of generators in a different way.

For the case of the FileWrapper class above, the close() method was setup so as to close the enclosed file like object. For the method to actually be called though, requires that the WSGI server invoke the close() method, if one exists, of any iterable returned by the WSGI application. This was a requirement regardless of whether the iterable returned from the WSGI application was an instance of the special file wrapper iterable, or a distinct type of iterable.

As far as the special file wrapper iterable returned by the WSGI application goes, problems start to arise with this though when a WSGI application stack decides to interject a WSGI middleware into the response pipeline. There are many variations on what a WSGI middleware could be doing, but I'll focus on the one problem where a WSGI middleware wishes to perform its own cleanup actions at the very end of handling a request after all response content has been written back to the HTTP client.

This sort of requirement can be because it genuinely has resources to release, but also can occur where a WSGI middleware is being used to capture metrics about how long a request runs for, or in an attempt to provide a means to run arbitrary small queued up tasks outside of the context of the actual request.

A naive programmer may approach this problem by using code such as:

def _application(environ, start_response):
    status = '200 OK'
    output = 'Hello World!'

    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)

    return [output]

def application(environ, start_response):
    try:
        return _application(environ, start_response)
    finally:
        # Perform required cleanup task.
        ...

That is, invoke the inner WSGI application component inside of a try/finally block and perform the cleanup action with the finally block.

This might even be refactored into a reusable WSGI middleware as:

class ExecuteOnCompletion1:
    def __init__(self, application, callback):
        self.__application = application
        self.__callback = callback
    def __call__(self, environ, start_response):
        try:
            return self.__application(environ, start_response)
        finally:
            self.__callback(environ)

The WSGI environment passed in the 'environ' argument to the application could even be supplied to the cleanup callback as shown in case it needed to look at any configuration information or information passed back in the environment from the application.

The application would then be replaced with an instance of this class initialised with a reference to the original application and a suitable cleanup function.

def cleanup(environ):
    # Perform required cleanup task.
    ...

application = ExecuteOnCompletion1(_application, cleanup)

All this though will not have the desired affect as the cleanup task would run when the WSGI application component returns, but before any data from the iterable has been consumed by the underlying WSGI server and written back to the HTTP client. Because the iterable can be a generator and so yielding of the next value could actually involve more work on the part of the WSGI application, the cleanup actions could release resources that are still going to be required to generate the content for the response.

In order to have the cleanup task only executed after the complete response has been consumed, it would be necessary to wrap the result of the application within an instance of a purpose built generator like object. This object needs to yield each item from the response in turn, and when this object is cleaned up by virtue of the 'close()' method being called, it should in turn call 'close()' on the result returned from the application if necessary, and then call the supplied cleanup callback.

class Generator2:
    def __init__(self, iterable, callback, environ):
        self.__iterable = iterable
        self.__callback = callback
        self.__environ = environ
    def __iter__(self):
        for item in self.__iterable:
            yield item
    def close(self):
        try:
            if hasattr(self.__iterable, 'close'):
                self.__iterable.close()
        finally:
            self.__callback(self.__environ)

class ExecuteOnCompletion2:
    def __init__(self, application, callback):
        self.__application = application
        self.__callback = callback
    def __call__(self, environ, start_response):
        try:
            result = self.__application(environ, start_response)
        except:
            self.__callback(environ)
            raise
        return Generator2(result, self.__callback, environ)

Now imagine that our original WSGI application was doing the following and you might start to see the problem.

def _application(environ, start_response):
    status = '200 OK'

    response_headers = [('Content-type', 'text/plain'),]
    start_response(status, response_headers)

    filelike = file('/usr/share/dict/words', 'r')
    blksize = 8192

    if 'wsgi.file_wrapper' in environ:
        return environ['wsgi.file_wrapper'](filelike, blksize)
    else:
        return iter(lambda: filelike.read(blksize), '')

def cleanup(environ):
    # Perform required cleanup task.
    ...

application = ExecuteOnCompletion2(_application, cleanup)

In this case what is happening is that the inner WSGI application, where the wsgi.file_wrapper key exists in the WSGI environ dictionary, is creating an instance of the file wrapper object and returning it. The WSGI middleware wrapper however, in order to replace the close() method to trigger its own cleanup action, has replaced the return iterable with its own. This results in the WSGI server not seeing that wsgi.file_wrapper was used.

If it was the case that the WSGI middleware wrapper was actually modifying the response content this wouldn’t be a big deal as it legitimately needs to consume content from the file wrapper iterable to modify it. In this case though, purely in order to add its own cleanup action, it has to consume and pass through unmodified the content from the file wrapper iterable.

The end result therefore is that any performance optimisation which could be derived from using wsgi.file_wrapper is lost. The question is, can anything be done to save the situation. Unfortunately, with how the WSGI specification is written presently it cannot. With a few changes however to the WSGI specification, then new possibilities are opened up, and as long as WSGI middleware were changed then a way could be provided for cleanup actions to be attached to a file wrapper iterable without preventing it from working.

The first change that would be required in the WSGI specification to support this would be to make it mandatory that the value associated with wsgi.file_wrapper in the WSGI environ dictionary be the class type used to construct the file wrapper iterable. That is, exactly like the example in the WSGI specification showed. As explained before, right now this isn’t a requirement and the value for wsgi.file_wrapper can be any callable so long as it returns an iterable that the WSGI application can return.

By making this change, it would firstly allow for a WSGI application to accurately detect when it is dealing with an iterable created by wsgi.file_wrapper. This is because the WSGI middleware can say:

file_wrapper = environ.get('wsgi.file_wrapper', None)
if file_wrapper and isinstance(result, file_wrapper):
    ...

Based on that, the WSGI middleware could do something different based on whether it knew it was dealing with an instance of a file wrapper iterable.

With the way the WSGI specification is currently written, there is no guarantee what the value against wsgi.file_wrapper is. The only way one could try and do a type comparison to determine if you were dealing with a file wrapper iterable would be to create a second iterable and compare the type of both.

file_wrapper = environ.get('wsgi.file_wrapper', None)

if file_wrapper:
    file_wrapper_instance = file_wrapper(StringIO(‘’))
    if type(result) == type(file_wrapper_instance)
        ...

Although this may work it is clumsy and your are dependent on WSGI servers implementing support correctly such that they don’t get confused when wsgi.file_wrapper is used more than once in the context of a specific request. For that reason, requiring that wsgi.file_wrapper be the class type for the iterable is a better solution.

By ensuring that wsgi.file_wrapper is a class type, it also guarantees that one can create instances of it via the class type. That is, by invoking it with required arguments to create an instance of it, namely, the file like object and block size. Being able to do that, then we can write the following.

def FileWrapper1(iterable, callback, environ):
    class _FileWrapper(type(iterable)):
        def close(self):
            try:
                iterable.close()
            finally:
                callback(environ)
    return _FileWrapper(iterable.filelike, iterable.blksize)

file_wrapper = environ.get('wsgi.file_wrapper', None)

if file_wrapper and isinstance(result, file_wrapper):
    return FileWrapper1(result, cleanup, environ)
else:
    ...

What we have done here is create a new file wrapper iterable, but one which is an instance of a class derived from the type of the file wrapper iterable returned by the inner WSGI application component.

The result of this is that we can now provide a close() method which triggers our own callback function on completion of the request, while still allowing the WSGI server to see that it was actually a file wrapper iterable that was used. As such, the WSGI server can then still perform any performance optimisations that it can.

The sharp eyed amongst you will also have noticed that this code does however rely upon the file wrapper iterable having a ‘filelike’ and ‘blksize’ member such that the original values can be retrieved. This is true and would be another new requirement which would need to be added to the WSGI specification.
Combining this together with our original WSGI middleware to trigger cleanup actions on completion of the request, we end up with:

class ExecuteOnCompletion2:
    def __init__(self, application, callback):
        self.__application = application
        self.__callback = callback
    def __call__(self, environ, start_response):
        try:
            result = self.__application(environ, start_response)
        except:
            self.__callback(environ)
            raise
        file_wrapper = environ.get('wsgi.file_wrapper', None)
        if file_wrapper and isinstance(result, file_wrapper):
            return FileWrapper1(result, self.__callback, environ)
        else:
            return Generator2(result, self.__callback, environ)

If you would like to play with this, then the code in the subversion repository for mod_wsgi 4.X has been modified to conform to these new requirements for wsgi.file_wrapper. Previously the mod_wsgi code had wsgi.file_wrapper be a function which returned the file wrapper iterable, rather than it being the class type. As a consequence, the above code should also be safe to use with older mod_wsgi versions, in which case it will fallback to the old way of doing things.

In summary, here is at least a solution for one of the problems with wsgi.file_wrapper. One which makes the use of performance optimisations for returning files more achievable.