Curia

The great web technology shootout – Round 4: Pyramid vs Django vs TG vs Rails 2 & 3

by on Nov.18, 2010, under F/OSS, Web Development

[Note: Due to the popularity of these posts, I have decided to move all of the benchmarking information over to its own dedicated page. Please see the new framework shootout page for the latest information.]

[This post is the continuation of a series. Please read Round 1, Round 2, and Round 3 first if you are just now joining us.]

While I had originally intended for round 4 to showcase how microframeworks are changing the way we do “quick and dirty” web development (and how they make using PHP as “an extension to HTML” old hat), my current programming habits have kept me involved in the more “full-stack” framework solutions. So, rather than spitting out various benchmarks of frameworks that I have little or no interaction with, and since enough time has passed since the last “shootout” that the landscape has changed a bit (with the introduction of Pyramid and the release of Rails 3), I have instead decided to showcase the most recent data on the frameworks that I personally find myself in contact with on a regular basis.

 

Warning: Everything is different this time around!

These benchmarks were all run on a fresh Amazon EC2 instance in order to (hopefully) achieve a more isolated environment. Obviously, since these benchmarks have all been run on a completely different box than any of the previous rounds, no previous data should be compared with these numbers.

What you should know about Round 4:

  • The EC2 instance used was: ami-da0cf8b3 m1.large ubuntu-images-us/ubuntu-lucid-10.04-amd64-server-20101020.manifest.xml
  • As a “Large” instance, Amazon describes the resources as: 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of local instance storage, 64-bit platform.
  • The various system software used was whatever was current as of November 18, 2010 on Ubuntu 10.04′s repositories.
  • Apache 2.2.14 was used for all tests.
  • Python 2.6.5 and mod_wsgi 3.3 (embedded mode) were used for the Python tests.
  • Ruby 1.8.7 (except for the last Rails 3 test) and Phusion Passenger 3.0 were used for the Rails tests.
  • ApacheBench was run with -n 10000 and -c 10 about 5-10 times each, and the “best guess average” was chosen (yeah, I’m lazy).
  • The Pyramid/TG test apps used SQLAlchemy as the ORM and Jinja2 as the templating system (Jinja2 was consistently around 25-50r/s faster than Mako for me).

 

Remember, nothing here is really all that scientific and your mileage WILL vary. Now on to the results…

 

The “Hello World” string test

The “Hello World” string test simply spits out a string response. There’s no template or DB calls involved, so the level of processing should be minimal.

The return hello world test chart

The return hello world test chart

 

The template test

The template test simply spits out Lorem Ipsum via a template (thus engaging the framework’s templating systems).

The template test chart

The template test chart

 

The template + DB query test

The template/db test simply loads 5 rows of Lorem Ipsum from a SQLite DB and spits it out via a template (thus engaging both the framework’s ORM and templating system).

Template test with database query

Template test with database query

 

Closing thoughts

  • The Pyramid Alpha results excite me quite a bit with all the talk of the various frameworks looking to merge with it. Could this really become the “one Python framework to rule them all”?
  • Django is still going strong, and continues to be an exceptional choice if you’re in to its way of doing things.
  • The Rails results left me a little confused are interesting because Rails 2 apparently outperforms Rails 3 on Ruby 1.8. Of course, once you move to Ruby 1.9, Rails 3 runs quite nicely.
  • Don’t believe these numbers? Feel free to boot up your own EC2 instance and run one of the test apps:

 

(Please don’t leave benchmarking requests as comments. If you’d like me to take a look at your favorite framework, direct message me on twitter. Thanks!)

:, , , , ,

  • http://be.groovie.org Ben Bangert

    When you installed SQLAlchemy, did you run it with the C extension optimization? That would help speed it up a bit.

  • Seth

    @Ben: Yes, I did. However, the initial numbers (particularly with Pyramid and TG) were a bit off due to a misconfiguration. The new numbers should be a little more accurate.

  • http://alteroo.com David Bain

    I’d love to see Grok in these benchmarks. All I have to say about Pyramid is wow!!!

  • http://iamwithprograms.wordpress.com Craig

    I wonder what the templating results would look like for Django if you replaced the Django templating system with jinja2. I imagine the results for pyramid and django would be much closer. Just a thought though and don’t worry I’m not requesting a benchmark ^^

  • http://www.rasadacrea.com/en rasadacrea

    Thank you for this interesting benchmark. Pyramid rocks and Django is still very good for many types of projects. I’m keep going on using both of them for my customers

  • albazy

    Please rethink about your blog design , it is really hard to read.

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...