Comments on: The great web technology shootout – Round 3: Better, Faster, and Shinier

By: Chris Arndt

Chris Arndt — Fri, 09 Oct 2009 23:02:21 +0000

@Jorge I ran the TG2 tests with the 2.0.3 release.

By: Alex Dedul

Alex Dedul — Fri, 09 Oct 2009 08:51:17 +0000

Nice tests, thank you :)

By: Jorge Vargas

Jorge Vargas — Fri, 09 Oct 2009 05:01:42 +0000

@Chris Arndt

is that TG2 or TG2.1? as you can see from the tests 2.1 has been getting some performance improvements.

By: Jorge Vargas

Jorge Vargas — Fri, 09 Oct 2009 05:00:53 +0000

@Graham

That is a very good point. And I think that should be a really nice benchmark. And In fact I believe TG will come up really nice on that, mainly because TG is not affected by the “more routes more lookups”, in case you are not familiar TG builds a graph of objects ie:

class BlogController:
def index(self):
pass

class RootController:
blog = BlogController
def index(self):
pass

so /index and /blog/index are the urls. Since this is pure python traversal having as many methods/instances will not slow TG down. However it has one drawback in deep nested structures, that said I believe if you go beyond 5-6 elements in your PATH_INFO you are probably doing something else wrong.

By: Seth

Seth — Fri, 09 Oct 2009 04:13:40 +0000

Graham, your question is intriguing and perhaps in the coming weeks I will find the time to put such a test together.

Chris, as I believe CherryPy is expected to be faster than Paster, I put TG 1.1 through my mod_wsgi test and came up with the following results: http://bit.ly/1czNm3 . I was not able to get Mako to work for some reason.

By: Graham Dumpleton

Graham Dumpleton — Fri, 09 Oct 2009 00:12:56 +0000

Jens, a very basic hello world test may not say too much, but hello world tests can still be informative for developers of frameworks if they go to the extent of breaking down how much time is spent in each major component within their system. That is, routing, database, data processing and templating. And then varying the complexity of configured routing and templates to gauge how the growing complexity affects the throughput ability. To do this in a meaningful way you can’t just use any application as you need to control that complexity in a step wise fashion to get meaningful results.

This is why I was pushing for some benchmarks for increases in routing complexity as I have a feeling this is an area which is often neglected and where users may not even appreciate the implications on performance by adding more and more URLs within an application using whatever routing mechanism configuration has to be used.

By: Jens Diemer

Jens Diemer — Thu, 08 Oct 2009 06:34:25 +0000

IMHO a Benchmark make only sense if you test is near a real world example instead of a “Hello World”.
With a minimal Test app, you will only test the Overhead of this small part of the Framework.

By: Chris Arndt

Chris Arndt — Wed, 07 Oct 2009 23:52:15 +0000

Interesting comparison. I have ported the test code for TurboGears2 to TurboGears 1.1. On my hardware (a Mac mini 1.87 Ghz Core Duo with 2GB RAM), the results for TG 1.1 vs TG 2 are:

TG1 (using CherryPy built-in server):

Document Path: /
Requests per second: 284.03 [#/sec] (mean)
Document Path: /genshi_hello
Requests per second: 144.52 [#/sec] (mean)
Document Path: /mako_hello
Requests per second: 198.85 [#/sec] (mean)
Document Path: /jinja_hello
Requests per second: 208.91 [#/sec] (mean)
Document Path: /raw_sql
Requests per second: 121.98 [#/sec] (mean)
Document Path: /genshi_sql
Requests per second: 119.51 [#/sec] (mean)
Document Path: /mako_sql
Requests per second: 120.14 [#/sec] (mean)
Document Path: /jinja_sql
Requests per second: 120.77 [#/sec] (mean)

TG 2 (using paste built-in server:

Document Path: /
Requests per second: 199.22 [#/sec] (mean)
Document Path: /genshi_hello
Requests per second: 109.45 [#/sec] (mean)
Document Path: /mako_hello
Requests per second: 142.79 [#/sec] (mean)
Document Path: /jinja_hello
Requests per second: 153.79 [#/sec] (mean)
Document Path: /raw_sql
Requests per second: 90.98 [#/sec] (mean)
Document Path: /genshi_sql
Requests per second: 62.28 [#/sec] (mean)
Document Path: /mako_sql
Requests per second: 91.30 [#/sec] (mean)
Document Path: /jinja_sql
Requests per second: 95.47 [#/sec] (mean)

I’ll upload the code to http://chrisarndt.de/projects/wf-shootout/ shortly.

By: Marcel Hellkamp

Marcel Hellkamp — Wed, 07 Oct 2009 15:02:04 +0000

These new charts are interesting. Django seems to have problems with the templates-only test, almost loosing his second place to Pylons. Rails seems to handle databases very well, leaving TG2.1 behind in the database-query test.

By: Marcel Hellkamp

Marcel Hellkamp — Wed, 07 Oct 2009 12:18:23 +0000

Graham, I don’t. Route optimization is disabled by default. If you use it, you have to make sure that your routes don’t overlap each other. The default placeholder in dynamic routes matches anything but a slash, so they are save most of the time. I most cases, your scenario would rather look like this:

AliasMatch ^/suburl/[^/]*$ /some/script/suburl.cgi
AliasMatch ^/[^/]*$ /some/script/root.cgi

You can however add a ‘catchall’ handler (@bottle.default()) that will never be shifted in front of other routes.