Comments on: The great web technology shootout – Round 3: Better, Faster, and Shinier

By: The great web technology shootout – Round 4: Pyramid vs Django vs TG vs Rails 2 & 3 - Curia

Fri, 19 Nov 2010 00:08:34 +0000

[...] Development [Note: This post is the continuation of a series. Please read Round 1, Round 2, and Round 3 first if you are just now joining [...]

By: jeo

jeo — Sun, 07 Mar 2010 08:54:22 +0000

Nice…not on twitter but sure would be cool to see a node.js test…

By: Chris Arndt

Chris Arndt — Fri, 09 Oct 2009 23:02:21 +0000

@Jorge I ran the TG2 tests with the 2.0.3 release.

By: Alex Dedul

Alex Dedul — Fri, 09 Oct 2009 08:51:17 +0000

Nice tests, thank you :)

By: Jorge Vargas

Jorge Vargas — Fri, 09 Oct 2009 05:01:42 +0000

@Chris Arndt

is that TG2 or TG2.1? as you can see from the tests 2.1 has been getting some performance improvements.

By: Jorge Vargas

Jorge Vargas — Fri, 09 Oct 2009 05:00:53 +0000

@Graham

That is a very good point. And I think that should be a really nice benchmark. And In fact I believe TG will come up really nice on that, mainly because TG is not affected by the “more routes more lookups”, in case you are not familiar TG builds a graph of objects ie:

class BlogController:
def index(self):
pass

class RootController:
blog = BlogController
def index(self):
pass

so /index and /blog/index are the urls. Since this is pure python traversal having as many methods/instances will not slow TG down. However it has one drawback in deep nested structures, that said I believe if you go beyond 5-6 elements in your PATH_INFO you are probably doing something else wrong.

By: Seth

Seth — Fri, 09 Oct 2009 04:13:40 +0000

Graham, your question is intriguing and perhaps in the coming weeks I will find the time to put such a test together.

Chris, as I believe CherryPy is expected to be faster than Paster, I put TG 1.1 through my mod_wsgi test and came up with the following results: http://bit.ly/1czNm3 . I was not able to get Mako to work for some reason.

By: Graham Dumpleton

Graham Dumpleton — Fri, 09 Oct 2009 00:12:56 +0000

Jens, a very basic hello world test may not say too much, but hello world tests can still be informative for developers of frameworks if they go to the extent of breaking down how much time is spent in each major component within their system. That is, routing, database, data processing and templating. And then varying the complexity of configured routing and templates to gauge how the growing complexity affects the throughput ability. To do this in a meaningful way you can’t just use any application as you need to control that complexity in a step wise fashion to get meaningful results.

This is why I was pushing for some benchmarks for increases in routing complexity as I have a feeling this is an area which is often neglected and where users may not even appreciate the implications on performance by adding more and more URLs within an application using whatever routing mechanism configuration has to be used.

By: Jens Diemer

Jens Diemer — Thu, 08 Oct 2009 06:34:25 +0000

IMHO a Benchmark make only sense if you test is near a real world example instead of a “Hello World”.
With a minimal Test app, you will only test the Overhead of this small part of the Framework.

By: Chris Arndt

Chris Arndt — Wed, 07 Oct 2009 23:52:15 +0000

Interesting comparison. I have ported the test code for TurboGears2 to TurboGears 1.1. On my hardware (a Mac mini 1.87 Ghz Core Duo with 2GB RAM), the results for TG 1.1 vs TG 2 are:

TG1 (using CherryPy built-in server):

Document Path: /
Requests per second: 284.03 [#/sec] (mean)
Document Path: /genshi_hello
Requests per second: 144.52 [#/sec] (mean)
Document Path: /mako_hello
Requests per second: 198.85 [#/sec] (mean)
Document Path: /jinja_hello
Requests per second: 208.91 [#/sec] (mean)
Document Path: /raw_sql
Requests per second: 121.98 [#/sec] (mean)
Document Path: /genshi_sql
Requests per second: 119.51 [#/sec] (mean)
Document Path: /mako_sql
Requests per second: 120.14 [#/sec] (mean)
Document Path: /jinja_sql
Requests per second: 120.77 [#/sec] (mean)

TG 2 (using paste built-in server:

Document Path: /
Requests per second: 199.22 [#/sec] (mean)
Document Path: /genshi_hello
Requests per second: 109.45 [#/sec] (mean)
Document Path: /mako_hello
Requests per second: 142.79 [#/sec] (mean)
Document Path: /jinja_hello
Requests per second: 153.79 [#/sec] (mean)
Document Path: /raw_sql
Requests per second: 90.98 [#/sec] (mean)
Document Path: /genshi_sql
Requests per second: 62.28 [#/sec] (mean)
Document Path: /mako_sql
Requests per second: 91.30 [#/sec] (mean)
Document Path: /jinja_sql
Requests per second: 95.47 [#/sec] (mean)

I’ll upload the code to http://chrisarndt.de/projects/wf-shootout/ shortly.