« Pictures from the... | Main | XDoclet and Hibernat... »
20040702 Friday July 02, 2004

Jetty and WebSphere suck, JRun and Tomcat rule the pack I've been looking for an app server performance comparison for some time now. Maybe those folks that say Tomcat sucks and Jetty rules will change their tune. Via Lasse Koskela:

Web Performance Inc. has published a very interesting report, which included a couple of surprises. Jetty pretty much sucked, JRun was one of the top guns, and Tomcat wasn't too far behind JRun.

Anyone know of any similar (recent) comparisons? Posted in Java at Jul 02 2004, 02:59:21 PM MDT 9 Comments

Comments:

One thing to note in that comparison is that every Servlet engine was tested using its out-of-the-box (or tarball) configurations. As a Resin backer noted in the forum on this article, these settings have can have huge impact on performance, and a more accurate test would involve tuning each container's settings to that more befitting a "production" environment. Of course, then you get the problem of how (and who) to determine a container is "properly" tuned, but, if you could figure out how to get past that problem, that sort of test would yeild far more interesting results.

Posted by Anonymous Coward on July 02, 2004 at 05:00 PM MDT #

Thanks for the link. I do believe its an accurate test however. In my experience, there's a lot of folks that put these servers into production with their default settings. If the default settings not good for performance - I think they should be changed. If they're good for development, bad for production - that should be documented very visibly and preached over and over to the Java community. We all need to know more about our appservers.

Posted by Matt Raible on July 02, 2004 at 05:15 PM MDT #

The hardware spec seems pretty low: Dell PowerEdge 300 server (850MHz PIII, 512M RAM) on Windows 2000 Server. I don't know how it would affect the outcome, but I doubt that's a typical production environment for WebSphere installations. My laptop has more juice than that!

Posted by Steve Raeburn on July 02, 2004 at 06:34 PM MDT #

The reason I'm using Jetty is because it requires less memory. At least Jetty 4 required less memory than Tomcat 4. In my experience. And since I only have 128Mb of memory, 10 or 20Mb really makes a difference. Anyway I guess I'll have to run my own benchmark and see if Tomcat 5 requires less memory. Although I doubt it. So I'll probably stick with Jetty.

Posted by 213.243.165.14 on July 03, 2004 at 01:23 AM MDT #

For reference, Mortbay's commentary on this benchmark: http://www.mortbay.com/MB/log/gregw/?permalink=ServletPerformanceReport.html&page=comments

Posted by Damien Bonvillain on July 04, 2004 at 04:36 AM MDT #

Well, the Resin values are very clearly flagged as needing changing for production - they're set to get best effect in a development environment. Similarly, Jetty has a default max setting of 50 threads, whereas Tomcat has a default of 150 threads, so it's not suprising that it doesn't perform as well as Tomcat in the max loads tests! Given that, I'm afraid that it doesn't seem to be a study that I'd classify as useful...

Posted by Gwyn Evans on July 04, 2004 at 02:18 PM MDT #

I love Tomcat and use it constantly, but the second the requirements call for putting Tomcat behind IIS or Apache (basically always) I start looking for other options. In contrast to how professional the development of Tomcat has become over the years, the development of the Apache/IIS connector is horrible! Every new release includes revolving door bugs that were fixed two releases before and then somehow show their heads again in new distributions. The method for configuring the connector has gone from arcane to insane. It was never what I'd conisder intuitive, but now it's just obtuse. As long as the connector remains in its current state I don't see how Tomcat can be considered a viable production solution. -Matt Welch

Posted by Matt Welch on July 06, 2004 at 02:55 PM MDT #

Resin stood shoulder-to-shoulder with Tomcat. While not free for commercial use, it's reasonably priced and has great features and support. In my opinion it's easier to use than Tomcat, too, which counts for a lot in my book, and makes the purchase price a good value.

Posted by gerryg on July 06, 2004 at 05:18 PM MDT #

Some responses: To Anonymous Coward and Gwyn Evans: FYI, out of the box Tomcat is also configured for development. We include explicit documentation in the relevant sections (e.g. JSP compilation) for production tuning. So that claim that Resin is configured for devlepment and thus the benchmark is skewed is BS. And what Matt says about many people not knowing about production tuning and/or deploying production apps on out-of-the-box servers is true, making the benchmark extremely relevant. To Steve Raeburn: As for the hardware spec: so what if Websphere is a beast when it comes to resource consumption? I know of many production apps running on PCs with a similar configuration to the test machine in this benchmark. To Anonymous: We constantly profile Tomcat to minimize memory usage while not limiting performance, and we think we've achieved a pretty good tradeoff. I would contest that much of the out-of-the-box memory usage is for the manager/admin/examples webapps and associated resources, and that memory usage would be dwarfed by any signifcant production application very quickly. To gerryg: Your book is just that, your book, and there's nothing anyone can do to change your subjective "it's easier to use" opinion unless you post enhancement suggestions to the relevant forums, e.g. tomcat-user. I'd put the support on the tomcat-user list against most commercial support organizations (with which I have much experience) without any qualms. True, you can't contract the mailing list if you need to COA, and Resin is good for that, but so are other products.

Posted by Yoav Shapira on August 27, 2004 at 03:57 PM MDT #

Post a Comment:
  • HTML Syntax: Allowed