« Should I change... | Main | AjaxAnywhere »
20050915 Thursday September 15, 2005

Jetty vs. Tomcat vs. Resin: A Performance Comparison This morning, I did some comparisons between Jetty 5.1.5rc1, Tomcat 5.5.9 and Resin 3.0.14 (OS version). I ran AppFuse's "test-canoo" target, which tests all the JSPs using Canoo WebTest. I did this as a Servlet 2.4 application, and had to tweak some stuff in my web.xml to make it work on Jetty and Resin. Nothing big, just stuff that Tomcat let pass through and these servers didn't. One interesting thing to note that Resin requires you to use "http://java.sun.com/jstl/fmt" for JSTL's "fmt" tag URI, while Jetty and Tomcat require "http://java.sun.com/jstl/fmt_rt". This is with Resin's "fast-jstl" turned off - b/c everything blows up if it's turned on (I don't feel like coding my JSTL to Resin's standards, that's why I turn it off).

Below is a list of the average time it took to run "test-canoo" after I ran it once to compile all the JSPs.

  • Jetty: 19 seconds
  • Tomcat: 19 seconds
  • Resin: 29 seconds

In addition, I tested how long it took for each server to startup - including the initialization of AppFuse.

  • Jetty: 7 seconds
  • Tomcat: 8 seconds
  • Resin: 13 seconds

So what does all this mean? A number of things:

  • I need to clean up AppFuse's web.xml a bit for 2.4 applications.
  • Putting the database connection pool configuration in a Spring context file (vs. JNDI) makes AppFuse much more portable.
  • Jetty isn't as fast as Jetty-lovers say it is (or maybe Tomcat just caught up).
  • The open source version of Resin is much slower than the other open source servlet containers.
  • I should restructure the build.xml to pull out Tomcat stuff and allow users to specify server deployment settings (i.e. in a ${servername}.xml file).
  • Orion still doesn't support the Servlet 2.4 or JSP 2.0 specifications.

Posted in Java at Sep 15 2005, 10:52:50 AM MDT 12 Comments

Comments:

I guessing all versions were run under the same JDK? 5.0?

Posted by Andy Hoffman on September 15, 2005 at 11:39 AM MDT #

JDK 1.4.2_04 on Windows XP (with Cygwin). These numbers are by no means scientific. IMO, the platform (2.6 GHz, 1.5 GB RAM) is not as important as the comparison between the different servers. Of course, if any of these servers are optimized for a particular JDK, let me know and I can retest.

FWIW, this isn't the first time I've done such a comparison.

Posted by Matt Raible on September 15, 2005 at 11:47 AM MDT #

Shouldn't you be using http://java.sun.com/jsp/jstl/fmt on all for a 2.4 app?

Posted by Stephen Duncan Jr on September 15, 2005 at 12:20 PM MDT #

Stephen, I believe this is true if you're using JSTL 1.1.x. I'm using JSTL 1.0.x so AppFuse can run on 2.3 containers.

Posted by Matt Raible on September 15, 2005 at 12:22 PM MDT #

Ok. Still, I wonder if (for this test) if all would run if you changed it to the JSTL 1.1.x URL. I can't think of any cases where it's not upwards compatible... I assume for JSTL 1.0.x the _rt version is correct for you. Not sure how Resin gets away with a) supporting the runtime expressions in the /fmt url, or how it manages not work WITH the _rt...

Posted by Stephen Duncan Jr on September 15, 2005 at 12:30 PM MDT #

Heh. Don't you hate when you take too long to write something and forget your structure. There should be a "b)" instead of "or how it manages" in that last sentence... :)

Posted by Stephen Duncan Jr on September 15, 2005 at 12:31 PM MDT #

hi matt,

thanks for sharing this. I was about to do this shortly -- saved me some good time.

RESIN : Although, wondering if this has to do with their JSTL/tag library implementation. Is there a way to know if the generic JSP/servlet implementation is also as bad ?

thank you,

BR,
~A

Posted by anjan bacchu on September 15, 2005 at 01:16 PM MDT #

Matt: Could you run your test on Tomcat 5.0.28 or thereabouts? It would be helpful to have a harder answer to the question: "Has Tomcat caught up?" Plus, *a lot* of people are still on 5.0.x for various reasons.

Posted by john on September 15, 2005 at 02:04 PM MDT #

P.S. By "Has Tomcat caught up?" I meant to imply: Was it actually slower before compared to the current versions of Jetty, Resin, etc., and by how much?

Posted by john on September 15, 2005 at 02:06 PM MDT #

John - as I said earlier, these tests aren't in a very controlled environment. I believe the first comparison is accurate, but testing Tomcat 5.0.x might not be b/c I didn't do it at the same time as the ones above. Nevertheless, here's the numbers for Tomcat 5.0.29:

<ul class="glassList">
  • <strong>ant test-canoo:</strong> 17 seconds
  • <strong>startup:</strong> 10 seconds

    • Posted by Matt Raible on September 15, 2005 at 02:45 PM MDT #

    How important is server startup, though, really? I know it can be to a certain extent but unless we're talking night and day here and 5 seconds just doesn't really matter it would seem. We have some large apps where the startup time does start to matter but it definitely started to swing back and forth as we used different tools that used the class loaders in different ways.

    The lessons learned about how to make it more portable are valuable, though. :)

    Posted by Mike Bosch on September 19, 2005 at 12:53 PM MDT #

    Was it actually slower before compared to the current versions of Jetty, Resin, etc., and by how much?

    It was dog-slow in the 3.x days (try anywhere up to an order of magnitude slower). 4.x was where Tomcat caught up some, but Resin was still 10-40% faster. Tomcat 5.x performance is quite good though.

    Posted by Trent on September 22, 2005 at 10:06 PM MDT #

    Post a Comment:
    Comments are closed for this entry.