9

Feb

Filed in Code, Django |

Today I’m going to talk (rant) about runserver, a great inclusion of the Django software package. It’s designed to help you easily run a Django project in a local (development) environment. Well, like many people, I have Rails in my background, and I miss certain things from it. One of those things, is the shiny SQL output in your terminal while running Mongrel.

Since I don’t have enough open source projects to maintain, I decided today that I needed this. It turned out to be extraordinarily easy even. I just ripped out my code from django-debug-toolbar (SQL and Cache tracking), flipped up a generic framework, and voila, django-devserver was born.

Now let’s get down to it. django-devserver provides a simple drop-in runserver replacement. It allows you to run a command, python manage.py rundevserver, and to get some additional information. As of writing, that additional information includes real-time SQL logging (aka mass query spam in your terminal), and a summary of cache calls.

Let’s take a look at some of our sample output:

While the syntax coloring still has a lot to be desired, it’s definitely a nice usable output. The SQL module, at the moment, simply outputs the query pre-execution, and post-execution outputs the time taken. The cache module outputs some generic stats, such as total time taken, # of calls, and hits vs misses.

The beautiful thing about the solution is how extensible it came out, for example, our cache module is only a few lines of code:

from django.template.loader import render_to_string
from django.shortcuts import render_to_response
from django.utils import simplejson
from django.core.cache import cache
 
from devserver.modules import DevServerModule
 
class CacheSummaryModule(DevServerModule):
    """
    Outputs a summary of cache events once a response is ready.
    """
 
    logger_name = 'cache'
 
    attrs_to_track = ['set', 'get', 'delete', 'add', 'get_many']
 
    def process_init(self):
        from devserver.utils.stats import track
 
        # save our current attributes
        self.old = dict((k, getattr(cache, k)) for k in self.attrs_to_track)
 
        for k in self.attrs_to_track:
            setattr(cache, k, track(getattr(cache, k), 'cache'))
 
    def process_complete(self):
        from devserver.utils.stats import stats
 
        self.logger.info('total time %(time)s - %(calls)s calls; %(hits)s hits; %(misses)s misses' % dict(
            calls = stats.get_total_calls('cache'),
            time = stats.get_total_time('cache'),
            hits = stats.get_total_hits('cache'),
            misses = stats.get_total_misses_for_function('cache', cache.get) + stats.get_total_misses_for_function('cache', cache.get_many),
            gets = stats.get_total_calls_for_function('cache', cache.get),
            sets = stats.get_total_calls_for_function('cache', cache.set),
            get_many = stats.get_total_calls_for_function('cache', cache.get_many),
            deletes = stats.get_total_calls_for_function('cache', cache.delete),
            #cache_calls_list = [(c['time'], c['func'].__name__, c['args'], c['kwargs'], simplejson.dumps(c['stack'])) for c in stats.get_calls('cache')],
        ))
 
        # set our attributes back to their defaults
        for k, v in self.old.iteritems():
            setattr(cache, k, v)

The entire thing supports all of the typical middleware processing, as well as the two additional methods: process_init() and process_complete().

I’m hoping to pull all of my usable work on the django-debug-toolbar into the devserver, but right now the two main additions I have planned are an SQLSummaryModule, and a ProcessTimeModule.

Hope you all enjoy, and get busy forking over at GitHub.

  • This is great stuff. Up until now, if I wanted to debug AJAX calls I had to build html views for them to use the toolbar. Thanks a lot.
  • David
    DEVSERVER_MODULES can now be applied properly in the settings :)

    I've also committed the fix for the utf8 characters.
  • See if you can integrate this work with the shipped runserver then... cloning management commands is a problem. E.g., I already use another management command for my particular environment that is a wrapper on the built-in one. I couldn't use yours in this case. Same goes for test_server. This is what I meant with backwards compatibility.
  • So good... great work !
    I´ll test this...
  • Nice project! But it seems that setting DEVSERVER_MODULES doesn't do anything?
  • This is great for debugging webservices, thanks!
  • pymind
    Don't read my previous comment:) Here is a solution:

    --- a/handlers.py
    +++ b/handlers.py
    - message = HTTP_INFO(message)
    + message = HTTP_INFO(message.encode('utf8'))
  • wow. great addon for built in dev server . thanx !
  • pymind
    David, I solved the problem by adding an encoding declaration "# -*- coding:utf-8 -*-" at the start of each file. Thanks for the shiny SQL output!
  • David
    You could use middleware, but a management command keeps it isolated. I'm also not sure what you mean regarding backwards compatibility. The management command is "rundevserver" rather than "runserver".

    I added a couple more modules and cleaned up the display tonight. It's making things a LOT nicer in our development environment, and I think it will do the same for a lot of people :)
  • Well... do you *really* need a new management command (and break backward compatibility)? I did this once just using logging on my project __init__.

    Anyway... good job on debug toolbar! I used it a lot, very useful.
  • This is such a useful middleware!
    Thanks for sharing with us.
  • Very neat! I forgot about this from the RoR days. I think I'll start tinkering with it :) Nice work, and thanks for the project!
  • David
    Good catch pymind -- I didn't test unicode behavior.

    I'll get some tests written and try to get that solved asap.
  • pymind
    It's great. But I get an exception:
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 111-119: ordinal not in range(128)

    File "/home/pymind/.venv/lib/python2.6/site-packages/devserver/modules/sql.py", line 74, in execute
    self.logger.debug('\n'.join(new_message), duration=duration, id='query')
    File "/home/pymind/.venv/lib/python2.6/site-packages/devserver/handlers.py", line 55, in
    debug = lambda x, *a, **k: x.log(level=logging.DEBUG, *a, **k)
    File "/home/pymind/.venv/lib/python2.6/site-packages/devserver/handlers.py", line 40, in log
    message = HTTP_INFO(message)
    File "/home/pymind/.venv/lib/python2.6/site-packages/django/utils/termcolors.py", line 68, in
    return lambda text: colorize(text, opts, **kwargs)
    File "/home/pymind/.venv/lib/python2.6/site-packages/django/utils/termcolors.py", line 42, in colorize
    text = str(text)
  • chris1610
    Very cool. This might be a good app to roll into django command extensions - http://code.google.com/p/django-command-extensi...
blog comments powered by Disqus