Raible Designs

Comparing Open Source Application Servers

With all the recent hubbub about GlassFish, I decided to do a quick performance test this morning. I downloaded all the most recent versions of the various open source application servers, deployed AppFuse 1.9.3 (Struts version) on them, and ran "ant test-canoo" to see if any of them were faster than the other. This was by no means a scientific, isolated test. It also didn't take into account any performance tuning you should do on these servers, I just used the out-of-the-box settings.

I ran these tests on my MacBook Pro (2.16 GHz Intel Core Duo, 2 GB DDR2 SDRAM) with my JAVA_OPTS set to:

-Xms768M -Xmx768M -XX:MaxPermSize=512m -Djava.awt.headless=true

When typing "java -version" at the command line, I got:

java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-103)
Java HotSpot(TM) Client VM (build 1.5.0_06-57, mixed mode, sharing)

Servers tested (in no particular order):

• JBoss 4.0.4
• GlassFish B48
• JOnAS 4.7.6
• Resin 3.0.21
• Geronimo 1.1

I'm pleased to note that all servers allowed me to deploy appfuse.war without using a console or command-line tool. They all support dropping the WAR in some sort of auto-deploy directory. Very cool! Secondly, I was able to successfully deploy AppFuse on all of them with no changes to AppFuse nor the server. Quite impressive.

My test consisted of the following:

• Copying appfuse.war into the appropriate directory
• Starting the server
• Running "ant test-canoo" from my $APPFUSE_HOME directory once
• Running "ant test-canoo" 3 times, recording the numbers for each run

Here's what I found:

Since I know you're going to ask about Jetty and Tomcat (the two main servlet-only containers), I ran the numbers on those too. First off, I tried Jetty 6 RC0. No dice - I got the following error when trying to start the server.

java.lang.IllegalStateException: Cannot initialize context because there is already a root application 
context present - check whether you have multiple ContextLoader* definitions in your web.xml!
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:173)

Since AppFuse deploys on all the above app servers, as well as Jetty 5.1.x, I'll chalk this up to a bug in Jetty 6. I used Jetty 5.1.11 for this test because I already had it installed on my machine.

I don't know that these numbers mean anything, but it was a fun experiment. For those of you who think these numbers might mean something, here's the rankings:

1. Tomcat 5.5.17
2. Resin 3.0.21
3. JBoss 4.0.4
4. Jetty 5.1.11/GlassFish B48 (tie)
5. Geronimo 1.1
6. JOnAS 4.7.6

Of course, the better test would be hammering each server with 1000 concurrent users (or a number higher than that) and comparing how each server holds up.

AppFuse Roadmap updated for 2.0

I've updated the AppFuse Roadmap for the upcoming 2.0 version. The current plan is to have a few milestone releases before the final 2.0 release. I can't promise any specific release date because of developer's schedules and current commitments. However, my hope is we can have it done by the end of this year.

Milestone 1

• Maven 2 integration
• Migrate from CVS to Subversion
• Remove XDoclet, putting generated files into source tree
• Get all flavors (WARs) deployed and running on Tomcat and Jetty Maven Plugin
• Spring 2.0: Upgrade XML to XSDs, annotations where appropriate
• Upgrade WebWork to Struts 2
• Documentation - move to new Confluence or DocBook-based system for 2.0 documentation

Milestone 2

• JDK 5, Annotations and JSP 2.0
• Archetypes for creating new AppFuse projects
• Documentation, including screencasts for tutorials

Milestone 3

• Integrate XFire by default (for existing classes and generated classes)
• Code Generation/AppGen - hopefully with the AppFuse Maven Plugin
• Documentation

2.0 Nice to haves

• TestNG replacing JUnit
• Hibernate Validation Annotations (hard part is hooking it into the web frameworks)
• Tomcat Plugin for Maven (that operates like Jetty's plugin)
• JPA Support
• Support for JDK 1.4 using something like Retroweaver
• Support for using Ant to build/test/deploy AppFuse-based projects
• Someone to do all the documentation ;-)

Let me know if there's anything that seems to be missing from this list.

If you can't wait until the end of this year, you may take a look at Patrick Lightbody's newly announced Project Able. He has a lot of good ideas in there that I hope to learn from.