How do you keep things quick for development and streamlined for high-load? Using a Django and LAMP setup that’s how. We had a pretty busy couple of days over at Curse (what powers curse) and managed to keep a consistent 500,000+ page views per hour with our web servers only be minimally loaded on CPU and memory.
For 6 months we had been struggling with the question “what happens next patch day”. That day was today. We managed to break 600,000 pages an hour last december, but immediately crashed afterwards due to the servers not being able to handle the load. This is partly due to optimizations on our end to Django’s ORM but also because of some major optimizations contributed by the community (thanks!).
So when you are thinking about that new website or application you’re about to build, think Python, and think Django. It has empowered us to deploy large-scale web applications in as little as a weekend, which would take a week or longer using Python or PHP by themselves.
15 Responses to "Rapid development serving 500,000 pages/hour"
How does your setup look like (the link over to curse-gaming was broken)? That’s mighty impressive, I must say!
Impressive! what are you using with Apache: mod_python or FastCGI?
http://www.davidcramer.net/curse/44/what-powers-curse.html
Cool. I remember you guys were having a lot of problems when you first deployed.
Unfortunately, I cant get the the technology page right now. I keep getting a 500 error.
[...] Rapid development serving 500,000 pages/hour (DavidCramer.net) How’s that for Django scalability! (tags: django webdev scalability performance) [...]
[...] Rapid development serving 500,000 pages/hour at DavidCramer.net (tags: performance development python scaling django availability lamp) [...]
Nice score…
Probably if you choice Smarty not Django, server could not hang too much visits/hits.
Smarty is the paint on a car, where as Django is the tight suspension, quick accelerating roadster
500.000 page views por hora…
Ótimo case em Django.
Em dezembro último, este site: [1] teve várias quedas por ter requisições demais. Tomaram a decisão de mudar de ferramenta, e escolheram o Django.
A decisão foi acertada e pode ser verificada aqui: [2] [3]
Links relaciona…
Let me first say that I use Django for every thing that I can. However, I’d like to mention that Smarty works quite well if used properly, allowing templates to be cached. I remember back in 2004 I had a site that was getting nearly 300 req/s using a Smarty-templated site.
Now, given the choice I would NEVER use Smarty again simply because I would hope to never use PHP again. Python/Django is just too… amazing… Words.. can’t describe. Should have sent… a poet.
[...] Rapid development serving 500,000 pages/hour at DavidCramer.net (tags: toread django performance scaling python) [...]
About PostgreSQL scaling better than Mysql:
http://tweakers.net/reviews/657/1/database-test-dual-intel-xeon-5160-pagina-1.html
Here it’s pretty clear what scales better and postgresql 8.3 is even faster (a lot) than the one used in the test.
And for alternative solutions to clustering and replication have a look at this page, it may help, otherwise… I just tried
http://developer.postgresql.org/index.php/Replication,_Clustering,_and_Connection_Pooling
[...] Rapid development serving 500,000 pages/hour | David Cramer.net [...]
Nice post!
Have you been thinking of optimizing the front end at Curse.com? CSS sprites and the likes?
Cheers
Leave A Reply