9

Feb

Filed in Code, Django |

Today I’m going to talk (rant) about runserver, a great inclusion of the Django software package. It’s designed to help you easily run a Django project in a local (development) environment. Well, like many people, I have Rails in my background, and I miss certain things from it. One of those things, is the shiny SQL output in your terminal while running Mongrel.

Since I don’t have enough open source projects to maintain, I decided today that I needed this. It turned out to be extraordinarily easy even. I just ripped out my code from django-debug-toolbar (SQL and Cache tracking), flipped up a generic framework, and voila, django-devserver was born.

Now let’s get down to it. django-devserver provides a simple drop-in runserver replacement. It allows you to run a command, python manage.py rundevserver, and to get some additional information. As of writing, that additional information includes real-time SQL logging (aka mass query spam in your terminal), and a summary of cache calls.

Let’s take a look at some of our sample output:

While the syntax coloring still has a lot to be desired, it’s definitely a nice usable output. The SQL module, at the moment, simply outputs the query pre-execution, and post-execution outputs the time taken. The cache module outputs some generic stats, such as total time taken, # of calls, and hits vs misses.

The beautiful thing about the solution is how extensible it came out, for example, our cache module is only a few lines of code:

from django.template.loader import render_to_string
from django.shortcuts import render_to_response
from django.utils import simplejson
from django.core.cache import cache
 
from devserver.modules import DevServerModule
 
class CacheSummaryModule(DevServerModule):
    """
    Outputs a summary of cache events once a response is ready.
    """
 
    logger_name = 'cache'
 
    attrs_to_track = ['set', 'get', 'delete', 'add', 'get_many']
 
    def process_init(self):
        from devserver.utils.stats import track
 
        # save our current attributes
        self.old = dict((k, getattr(cache, k)) for k in self.attrs_to_track)
 
        for k in self.attrs_to_track:
            setattr(cache, k, track(getattr(cache, k), 'cache'))
 
    def process_complete(self):
        from devserver.utils.stats import stats
 
        self.logger.info('total time %(time)s - %(calls)s calls; %(hits)s hits; %(misses)s misses' % dict(
            calls = stats.get_total_calls('cache'),
            time = stats.get_total_time('cache'),
            hits = stats.get_total_hits('cache'),
            misses = stats.get_total_misses_for_function('cache', cache.get) + stats.get_total_misses_for_function('cache', cache.get_many),
            gets = stats.get_total_calls_for_function('cache', cache.get),
            sets = stats.get_total_calls_for_function('cache', cache.set),
            get_many = stats.get_total_calls_for_function('cache', cache.get_many),
            deletes = stats.get_total_calls_for_function('cache', cache.delete),
            #cache_calls_list = [(c['time'], c['func'].__name__, c['args'], c['kwargs'], simplejson.dumps(c['stack'])) for c in stats.get_calls('cache')],
        ))
 
        # set our attributes back to their defaults
        for k, v in self.old.iteritems():
            setattr(cache, k, v)

The entire thing supports all of the typical middleware processing, as well as the two additional methods: process_init() and process_complete().

I’m hoping to pull all of my usable work on the django-debug-toolbar into the devserver, but right now the two main additions I have planned are an SQLSummaryModule, and a ProcessTimeModule.

Hope you all enjoy, and get busy forking over at GitHub.

  • chris1610

    Very cool. This might be a good app to roll into django command extensions – http://code.google.com/p/django-command-extensions/

  • pymind

    It’s great. But I get an exception:
    UnicodeEncodeError: ‘ascii’ codec can’t encode characters in position 111-119: ordinal not in range(128)

    File “/home/pymind/.venv/lib/python2.6/site-packages/devserver/modules/sql.py”, line 74, in execute
    self.logger.debug(‘\n’.join(new_message), duration=duration, id=’query’)
    File “/home/pymind/.venv/lib/python2.6/site-packages/devserver/handlers.py”, line 55, in
    debug = lambda x, *a, **k: x.log(level=logging.DEBUG, *a, **k)
    File “/home/pymind/.venv/lib/python2.6/site-packages/devserver/handlers.py”, line 40, in log
    message = HTTP_INFO(message)
    File “/home/pymind/.venv/lib/python2.6/site-packages/django/utils/termcolors.py”, line 68, in
    return lambda text: colorize(text, opts, **kwargs)
    File “/home/pymind/.venv/lib/python2.6/site-packages/django/utils/termcolors.py”, line 42, in colorize
    text = str(text)

  • David

    Good catch pymind — I didn’t test unicode behavior.

    I’ll get some tests written and try to get that solved asap.

  • http://www.codekoala.com Josh VanderLinden

    Very neat! I forgot about this from the RoR days. I think I’ll start tinkering with it :) Nice work, and thanks for the project!

  • http://www.kurs24.com/ kurs

    This is such a useful middleware!
    Thanks for sharing with us.

  • http://flavors.me/hcalves Henrique

    Well… do you *really* need a new management command (and break backward compatibility)? I did this once just using logging on my project __init__.

    Anyway… good job on debug toolbar! I used it a lot, very useful.

  • David

    You could use middleware, but a management command keeps it isolated. I’m also not sure what you mean regarding backwards compatibility. The management command is “rundevserver” rather than “runserver”.

    I added a couple more modules and cleaned up the display tonight. It’s making things a LOT nicer in our development environment, and I think it will do the same for a lot of people :)

  • pymind

    David, I solved the problem by adding an encoding declaration “# -*- coding:utf-8 -*-” at the start of each file. Thanks for the shiny SQL output!

  • http://blog.revolunet.com julien

    wow. great addon for built in dev server . thanx !

  • pymind

    Don’t read my previous comment:) Here is a solution:

    — a/handlers.py
    +++ b/handlers.py
    - message = HTTP_INFO(message)
    + message = HTTP_INFO(message.encode(‘utf8′))

  • http://mariz.org Nuno Mariz

    This is great for debugging webservices, thanks!

  • http://www.bolhoed.net Kevin Renskers

    Nice project! But it seems that setting DEVSERVER_MODULES doesn’t do anything?

  • http://www.berlotto.blog.br Sérgio Berlotto

    So good… great work !
    I´ll test this…

  • http://flavors.me/hcalves Henrique

    See if you can integrate this work with the shipped runserver then… cloning management commands is a problem. E.g., I already use another management command for my particular environment that is a wrapper on the built-in one. I couldn’t use yours in this case. Same goes for test_server. This is what I meant with backwards compatibility.

  • David

    DEVSERVER_MODULES can now be applied properly in the settings :)

    I’ve also committed the fix for the utf8 characters.

  • http://dvine.de dvine

    This is great stuff. Up until now, if I wanted to debug AJAX calls I had to build html views for them to use the toolbar. Thanks a lot.

  • http://www.wstearns.com/blog/2010/02/13/weekly-digest-for-february-13th/ Weekly Digest for February 13th | William Stearns

    [...] Shared David Cramer: Presenting django-devserver, a better runserver.. [...]

  • http://software.engineeringaddict.com/georges-software-engineering-log-diving-into-django/ George's Software Engineering Log: Diving Into Django | Software Engineering Addict

    [...] Presenting django-devserver, a better runserver. | David Cramer's Blog [...]

  • http://django.webglide.org/index.php/presenting-django-devserver-a-better-runserver/ Presenting django-devserver, a better runserver « WebGlide – Django

    [...] Presenting django-devserver, a better runserver. I really like this—it’s a Django management command (./manage.py rundevserver) which adds SQL logging and cache access summaries to the console output of Django’s local development server. It solves a similar set of problems to the debug toolbar, but requires slightly less setup and doesn’t inject additional HTML in to your pages. You can add your own custom modules to it as well. [...]

  • http://andrew.io/weblog/2010/03/presenting-django-devserver-a-better-runserver/ Presenting django-devserver, a better runserver.

    [...] most annoying thing to me about django’s runserver is lack of sql output in the terminal. django-devserver provides sql output, and does it nicely with syntax [...]

  • http://django.webglide.org/index.php/presenting-django-devserver-a-better-runserver-2/ Presenting django-devserver, a better runserver « WebGlide – Django

    [...] processor for django admin app_list »Presenting django-devserver, a better runserverPresenting django-devserver, a better runserver. I really like this—it’s a Django management command (./manage.py rundevserver) which adds SQL [...]

  • http://www.air-jordan-10.com/ air jordan 10

    Well , the view of the passage is totally correct ,your details is really reasonable and you guy give us valuable informative post, I totally agree the standpoint of upstairs. I often surfing on this forum when I m free and I find there are so much good information we can learn in this forum! http://spoon8.net/

  • http://www.nike-air-force-one.com air force 1 shoes

    I've been looking for a similar to this post. Not only extensively but also detailly. We can learn a lot from the post. I recommend to you , you can come communication in here.Let us grow up together.On the other hand ,I know some websites content is very well.you can go and see.Such as air force 1 kids

  • http://www.moncler-down-jackets.com air jordan 8

    The post of content is very interesting and exciting. I learned a lot from here.The content from simple to complex, air jordan 8 so all of you can come in .ugg boots for sale No matter you want to see what can be found.By the way ,there are some websites is also very wonderful,you can go and see.such asair force 1 shoes

  • http://yohan06.student.ipb.ac.id Yohani

    cool code thank you :)

  • http://ekos06.student.ipb.ac.id Kojeje

    Wowww… nice codes…
    thanks for the codes…

  • jordan shoes

    The air Jordan brand is a household name and people of all ages and social strata line up eagerly for the release of the latest model. Some of this success can be attributed to the fact that the shoes, from the Jordan III to the most recent model, have always started with their namesake, Michael Jordan. The designers take his ideas, hobbies, and life into account and incorporate these feelings into the air jordan shoes .The “jumpman” is the logo of jordan shoes.Now, we sell cheap jordan shoes. You can find wholesale jordan shoes.for example lebron james shoes ,kobe bryant shoes ,michael jordan shoes and so on, Welcome to our homepage.

  • nike air max

    air max shoes is an extremely comfortable shoe that uses a large air cushioning unit that can be seen from the side. This technology combined with the design of the shoe which is meant to compliment the spinal cord makes. Air Max shoes the ideal shoe for walking, running, playing basketball, or any other sport. nike air max ,nike air max 2009, nike air max 95, nike air max 90 and so on.Nike introduced its Air Cushioning Technology in 1979. By embedding Air Cushioning in the midsoles of its running shoes, it changed the way athletes approached and reacted to athletic performance forever.

  • china wholesale

    Excellent reading, I must say. http://www.cmonc.com You’ve researched the topic very well. thanks for sharing

  • http://twitter.com/nikeairmaxsale lolin

    Initial introduced in 1991

blog comments powered by Disqus