2

Dec

Filed in Code, Django, Python |

As a sort of challenge to myself, I once again attempted to recreate ticket #17 as a standalone component. After a few hours, and some previous code I had written as a good guide, it seems to be working. So I’d like to announce django-idmapper.

The goal of the project is a simple plug-and-play system for integrating the identity mapper into the ORM components. It allows you it simply define as a model with a different base class (via the SharedMemoryModel class), and all queries against this model (yes even from non-singleton models) are directed through the unique instance cache.

In many cases, this can provide a nice decrease of memory usage, and you may even see a nice speed bonus (as we did with Curse).

Here is a nice and dirty quick example of a situation which would gain from this benefit:

from idmapper import models
 
class Category(models.SharedMemoryModel):
    # We have 5 categories
    name = models.CharField(max_length=32)
 
class Article(models.SharedMemoryModel):
    # And 100 articles
    name = models.CharField(max_length=32)
    category = models.ForeignKey(Category)

Now in a (sort of) typical query, maybe we want to show every article (or 100, which isn’t too unreasonable). We also want to show the category they are in:

article_list = Article.objects.all().select_related('category')

What we have achieved here, is only having a maximum of N (where N is the number of categories) unique instances of category in memory here. Since we only have five, this means that we only created five category instances. This gives us a savings of the time it takes to create the additional 95 category objects, as well as the memory they would have consumed.

Please let me know if you run in to any problems, but so far, the tests are passing :)

Note: I’ve renamed the project due to the incorrect terminology. My apologies :)

9 Responses to "An Identity Mapper in Django (formerly Django Singletons)"

Subscribe to this topic with RSS or get the Trackback URL
tcdavis (Dec 2nd):

So, two questions.

1. Per your explanation, it wasn't strictly necessary to define Article as a SingleModel, correct?

2. Given the obvious benefits, are there any reasons not to use this class for all models? Am I missing something crucial here?

Rob Hudson (Dec 2nd):

Could there be a way to enable this per query? In your example, it's obvious that creating only one instance per category is optimal, but are there cases where it may not be what you want?

Sam Toriel (Dec 2nd):

^^ My questions exactly

David Cramer (Dec 2nd):

I think per query is a bit out of scope for what I want to achieve. Doing this would require the queryset/manager to be aware (to some extent) of the singleton model. This wouldn't quite achieve the same performance benefit.

The biggest thing about this, while it IS an obvious performance boost, its not the sole intention of the project. Having unique instances of an object in memory can solve a lot of other issues as well.

David Cramer (Dec 2nd):

Personally, I plan to use it throughout the codebase to prevent some data issues. The one area this doesn't affect right now (due to complications we had with it at Curse) is serialization. It will ignore the instance caching when you unserialize objects (such as from the cache).

Steingrim (Dec 2nd):

I think calling this Singletons is quite misleading. The Singleton Pattern is used to enforce a single instance of a class; not a single instance of a business entity. Identity Map is the correct naming but note Fowler's emphasis on using one Identity Map per transaction.

David Cramer (Dec 2nd):

Ah if you are correct then I will be renaming the project. I haven't done much research (in terminology), but had assumed it was correct via the ticket on Django.

dailycad (Dec 2nd):

Very useful info.. Thanks..

Tom. (Dec 2nd):

This sounds wicked! Thanks.

Leave A Reply

 Username (*required)

 Email Address (*private)

 Website (*optional)

Note: Comments moderation may be active so there is no need to resubmit your comment.