Using mod_wsgi when developing Django sites.

If one were to ask on the Django forums or irc channels what server to use when developing a Django site, the advice generally given is to use Django's own internal development server and to only deploy it under mod_wsgi or another hosting mechanism when moving it into production. One of the main reasons given for doing this is that Django's development server will detect when you make code changes and automatically restart the Django process.

What seems to be not well known though is that mod_wsgi can also be setup to automatically restart the Django processes. Yes, some know about the ability when using mod_wsgi daemon mode to touch the WSGI script file and have the Django process restarted, but what I am talking about here is a completely automated system which will detect changes to any Python code files and restart the Django process, just like with the Django development server, and not just when a change is made to the WSGI script file. This automated system completely avoids the need to manually touch the WSGI script file after you have made any changes.

How this can be achieved is described in section 'Restarting Daemon Processes' of the mod_wsgi documentation about source code reloading. What I will do here though is summarise how that code recipe can be applied to a Django site.

Actual setup instructions for using Django with mod_wsgi are described in the Django integration guide on the mod_wsgi site.  Although they contain the important information you need, it is still best to have also read the quick configuration guide and configuration guidelines. 

The example configuration give in the Django integration guide for using daemon mode is:

WSGIDaemonProcess site-1 user=user-1 group=user-1 threads=25
WSGIProcessGroup site-1

Alias /media/ /usr/local/django/mysite/media/

<Directory /usr/local/django/mysite/media>
Order deny,allow
Allow from all
</Directory>

WSGIScriptAlias / /usr/local/django/mysite/apache/django.wsgi

<Directory /usr/local/django/mysite/apache>
Order deny,allow
Allow from all
</Directory>

The corresponding example WSGI script file is:

import os, sys
sys.path.append('/usr/local/django')
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()

It is this latter WSGI script file to which we need to make an addition. Rather than make all the changes to this file though, we will add a new Python code file into the Django site directory and put the bulk of the code in it.

So, in the same directory as where the 'settings.py' file is for your Django site, add a file called 'monitor.py'. In this file place the code recipe for implementing an automatic restart mechanism from the mod_wsgi documentation on reloading source code. Then go back to the WSGI script file and at the end of the file add:

import mysite.monitor
mysite.monitor.start(interval=1.0)

Obviously, the name of your site being different, you should substitute 'mysite' with the actual name of your site. Anyway, that is all that is required.

What will now happen is that when the WSGI script file is loaded, it will initialise the code which will monitor changes to any code files. This will have the side affect of creating a background thread which will periodically scan through all the loaded Python modules and try to determine if the corresponding file on disk has been changed since the last scan. If a change is detected, then the thread will signal the process that it needs to be shutdown. When shutdown is complete, mod_wsgi will detect this and startup a new process in its place. Upon a subsequent request for your Django site, the WSGI script file is loaded once more and we start over.

Now, why develop your Django site using mod_wsgi? There are a few reasons.

The first is that if mod_wsgi is going to be your production target anyway, you are guaranteed from the start that you will have an equivalent environment and not be hit with problems often seen whereby code works with the development server but not under mod_wsgi. Using mod_wsgi you will know about these issues sooner and be able to ensure mod_wsgi and your environment is configured correctly.

The second is that the Django development server is only single threaded. As a result, it isn't really a suitable test bed for ensuring your code is multithread safe through the whole development process.

A third is that Django development server is a single process web server. If using mod_wsgi it is quite likely you would deploy with a configuration which makes use of multiple process to handle requests. One cannot simulate this type of setup easily with the development server.

Finally, with mod_wsgi the setup would be close to if not the same as your deployment configuration. This means that you could do realistic tests of how well your application will perform all through the development process through running suitable benchmarks. This would allow you to identify much earlier any performance bottlenecks without having to go through a process of first having to install on a separate staging environment.

Just remember though when you do go to release to your production environment, or even just enable the development instance as the live site, that you disable the code monitor. The overhead may not be noticeable in the grand scheme of things, but better to avoid having processes restart when you least expect them to.

BTW, mod_wsgi daemon mode requires Apache 2.X on UNIX. The feature is not available on Apache 1.3 or Windows.

Python 3.0 and mod_wsgi.

Now that Python 3.0 has been released, it is probably worth while pointing out that mod_wsgi has been ported to Python 3.0 for quite a while now. It didn't always work, but that was actually because various versions of the Python 3.0 betas and release candidates managed to break support for sub interpreters. All these issues in Python 3.0 have been fixed now and mod_wsgi appears to be working fine for a basic WSGI test case. Of course, with Python 3.0 being so new and with the internal changes required to mod_wsgi to handle Unicode strings, there are no doubt some gremlins lurking in the mod_wsgi changes still. One can just hope that the WSGI folks out there don't ignore Python 3.0 for too long and start on porting some stuff to Python 3.0. This will then give me some decent code on which to test out mod_wsgi support for Python 3.0.

Note that if you want to play with mod_wsgi and Python 3.0, you will need to use the mod_wsgi source code directly from the subversion repository. You will also need to be aware of the proposed amendments to the WSGI 1.0 specification to make it compatible with Python 3.0.