2

Dec

Filed in Code, Django, Python |

As a sort of challenge to myself, I once again attempted to recreate ticket #17 as a standalone component. After a few hours, and some previous code I had written as a good guide, it seems to be working. So I’d like to announce django-idmapper.

The goal of the project is a simple plug-and-play system for integrating the identity mapper into the ORM components. It allows you it simply define as a model with a different base class (via the SharedMemoryModel class), and all queries against this model (yes even from non-singleton models) are directed through the unique instance cache.

In many cases, this can provide a nice decrease of memory usage, and you may even see a nice speed bonus (as we did with Curse).

Here is a nice and dirty quick example of a situation which would gain from this benefit:

from idmapper import models
 
class Category(models.SharedMemoryModel):
    # We have 5 categories
    name = models.CharField(max_length=32)
 
class Article(models.SharedMemoryModel):
    # And 100 articles
    name = models.CharField(max_length=32)
    category = models.ForeignKey(Category)

Now in a (sort of) typical query, maybe we want to show every article (or 100, which isn’t too unreasonable). We also want to show the category they are in:

article_list = Article.objects.all().select_related('category')

What we have achieved here, is only having a maximum of N (where N is the number of categories) unique instances of category in memory here. Since we only have five, this means that we only created five category instances. This gives us a savings of the time it takes to create the additional 95 category objects, as well as the memory they would have consumed.

Please let me know if you run in to any problems, but so far, the tests are passing :)

Note: I’ve renamed the project due to the incorrect terminology. My apologies :)

  • tcdavis

    So, two questions.

    1. Per your explanation, it wasn't strictly necessary to define Article as a SingleModel, correct?

    2. Given the obvious benefits, are there any reasons not to use this class for all models? Am I missing something crucial here?

  • http://rob.cogit8.org Rob Hudson

    Could there be a way to enable this per query? In your example, it's obvious that creating only one instance per category is optimal, but are there cases where it may not be what you want?

  • Sam Toriel

    ^^ My questions exactly

  • http://www.davidcramer.net David Cramer

    I think per query is a bit out of scope for what I want to achieve. Doing this would require the queryset/manager to be aware (to some extent) of the singleton model. This wouldn't quite achieve the same performance benefit.

    The biggest thing about this, while it IS an obvious performance boost, its not the sole intention of the project. Having unique instances of an object in memory can solve a lot of other issues as well.

  • http://www.davidcramer.net David Cramer

    Personally, I plan to use it throughout the codebase to prevent some data issues. The one area this doesn't affect right now (due to complications we had with it at Curse) is serialization. It will ignore the instance caching when you unserialize objects (such as from the cache).

  • Steingrim

    I think calling this Singletons is quite misleading. The Singleton Pattern is used to enforce a single instance of a class; not a single instance of a business entity. Identity Map is the correct naming but note Fowler's emphasis on using one Identity Map per transaction.

  • http://www.davidcramer.net David Cramer

    Ah if you are correct then I will be renaming the project. I haven't done much research (in terminology), but had assumed it was correct via the ticket on Django.

  • http://dailycad.com/ dailycad

    Very useful info.. Thanks..

  • Tom.

    This sounds wicked! Thanks.

  • http://www.davidcramer.net David Cramer

    I think per query is a bit out of scope for what I want to achieve. Doing this would require the queryset/manager to be aware (to some extent) of the singleton model. This wouldn't quite achieve the same performance benefit.

    The biggest thing about this, while it IS an obvious performance boost, its not the sole intention of the project. Having unique instances of an object in memory can solve a lot of other issues as well.

  • http://www.davidcramer.net David Cramer

    Personally, I plan to use it throughout the codebase to prevent some data issues. The one area this doesn't affect right now (due to complications we had with it at Curse) is serialization. It will ignore the instance caching when you unserialize objects (such as from the cache).

  • Steingrim

    I think calling this Singletons is quite misleading. The Singleton Pattern is used to enforce a single instance of a class; not a single instance of a business entity. Identity Map is the correct naming but note Fowler's emphasis on using one Identity Map per transaction.

  • http://www.davidcramer.net David Cramer

    Ah if you are correct then I will be renaming the project. I haven't done much research (in terminology), but had assumed it was correct via the ticket on Django.

  • http://www.dailycad.com/ dailycad

    Very useful info.. Thanks..

  • Tom.

    This sounds wicked! Thanks.

blog comments powered by Disqus