With all the recent hubbub about GlassFish, I decided to do a quick performance test this morning. I downloaded all the most recent versions of the various open source application servers, deployed AppFuse 1.9.3 (Struts version) on them, and ran "ant test-canoo" to see if any of them were faster than the other. This was by no means a scientific, isolated test. It also didn't take into account any performance tuning you should do on these servers, I just used the out-of-the-box settings.
I ran these tests on my MacBook Pro (2.16 GHz Intel Core Duo, 2 GB DDR2 SDRAM) with my JAVA_OPTS set to:
-Xms768M -Xmx768M -XX:MaxPermSize=512m -Djava.awt.headless=true
When typing "java -version" at the command line, I got:
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-103)
Java HotSpot(TM) Client VM (build 1.5.0_06-57, mixed mode, sharing)
Servers tested (in no particular order):
- JBoss 4.0.4
- GlassFish B48
- JOnAS 4.7.6
- Resin 3.0.21
- Geronimo 1.1
I'm pleased to note that all servers allowed me to deploy appfuse.war without using a console or command-line tool. They all support dropping the WAR in some sort of auto-deploy directory. Very cool! Secondly, I was able to successfully deploy AppFuse on all of them with no changes to AppFuse nor the server. Quite impressive.
My test consisted of the following:
- Copying appfuse.war into the appropriate directory
- Starting the server
- Running "ant test-canoo" from my $APPFUSE_HOME directory once
- Running "ant test-canoo" 3 times, recording the numbers for each run
Here's what I found:
Server Name | 1st run (seconds) | 2nd run | 3rd run | Average |
JBoss 4.0.4 | 24 | 23 | 23 | 23.33 |
GlassFish B48 | 25 | 24 | 24 | 24.33 |
JOnAS 4.7.6 | 25 | 25 | 27 | 25.66 |
Resin 3.0.21 | 23 | 23 | 23 | 23 |
Geronimo 1.1 | 28 | 23 | 23 | 24.66 |
Since I know you're going to ask about Jetty and Tomcat (the two main servlet-only containers), I ran the numbers on those too. First off, I tried Jetty 6 RC0. No dice - I got the following error when trying to start the server.
java.lang.IllegalStateException: Cannot initialize context because there is already a root application
context present - check whether you have multiple ContextLoader* definitions in your web.xml!
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:173)
Since AppFuse deploys on all the above app servers, as well as Jetty 5.1.x, I'll chalk this up to a bug in Jetty 6. I used Jetty 5.1.11 for this test because I already had it installed on my machine.
Server Name | 1st run (seconds) | 2nd run | 3rd run | Average |
Jetty 5.1.11 | 24 | 25 | 24 | 24.33 |
Tomcat 5.5.17 | 23 | 23 | 22 | 22.66 |
I don't know that these numbers mean anything, but it was a fun experiment. For those of you who think these numbers might mean something, here's the rankings:
- Tomcat 5.5.17
- Resin 3.0.21
- JBoss 4.0.4
- Jetty 5.1.11/GlassFish B48 (tie)
- Geronimo 1.1
- JOnAS 4.7.6
Of course, the better test would be hammering each server with 1000 concurrent users (or a number higher than that) and comparing how each server holds up.
I've updated the AppFuse Roadmap for the upcoming 2.0 version. The current plan is to have a few milestone releases before the final 2.0 release. I can't promise any specific release date because of developer's schedules and current commitments. However, my hope is we can have it done by the end of this year.
Milestone 1
- Maven 2 integration
- Migrate from CVS to Subversion
- Remove XDoclet, putting generated files into source tree
- Get all flavors (WARs) deployed and running on Tomcat and Jetty Maven Plugin
- Spring 2.0: Upgrade XML to XSDs, annotations where appropriate
- Upgrade WebWork to Struts 2
- Documentation - move to new Confluence or DocBook-based system for 2.0 documentation
Milestone 2
Milestone 3
- Integrate XFire by default (for existing classes and generated classes)
- Code Generation/AppGen - hopefully with the AppFuse Maven Plugin
- Documentation
2.0 Nice to haves
- TestNG replacing JUnit
- Hibernate Validation Annotations (hard part is hooking it into the web frameworks)
- Tomcat Plugin for Maven (that operates like Jetty's plugin)
- JPA Support
- Support for JDK 1.4 using something like Retroweaver
- Support for using Ant to build/test/deploy AppFuse-based projects
- Someone to do all the documentation ;-)
Let me know if there's anything that seems to be missing from this list.
If you can't wait until the end of this year, you may take a look at Patrick Lightbody's newly announced Project Able. He has a lot of good ideas in there that I hope to learn from.