Matt RaibleMatt Raible is a Java Champion and Developer Advocate at Okta. developer.okta.com

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Comparing Open Source Application Servers

With all the recent hubbub about GlassFish, I decided to do a quick performance test this morning. I downloaded all the most recent versions of the various open source application servers, deployed AppFuse 1.9.3 (Struts version) on them, and ran "ant test-canoo" to see if any of them were faster than the other. This was by no means a scientific, isolated test. It also didn't take into account any performance tuning you should do on these servers, I just used the out-of-the-box settings.

I ran these tests on my MacBook Pro (2.16 GHz Intel Core Duo, 2 GB DDR2 SDRAM) with my JAVA_OPTS set to:

-Xms768M -Xmx768M -XX:MaxPermSize=512m -Djava.awt.headless=true

When typing "java -version" at the command line, I got:

java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-103)
Java HotSpot(TM) Client VM (build 1.5.0_06-57, mixed mode, sharing)

Servers tested (in no particular order):

  • JBoss 4.0.4
  • GlassFish B48
  • JOnAS 4.7.6
  • Resin 3.0.21
  • Geronimo 1.1

I'm pleased to note that all servers allowed me to deploy appfuse.war without using a console or command-line tool. They all support dropping the WAR in some sort of auto-deploy directory. Very cool! Secondly, I was able to successfully deploy AppFuse on all of them with no changes to AppFuse nor the server. Quite impressive.

My test consisted of the following:

  • Copying appfuse.war into the appropriate directory
  • Starting the server
  • Running "ant test-canoo" from my $APPFUSE_HOME directory once
  • Running "ant test-canoo" 3 times, recording the numbers for each run

Here's what I found:

Server Name1st run (seconds)2nd run3rd runAverage
JBoss 4.0.424232323.33
GlassFish B4825242424.33
JOnAS 4.7.625252725.66
Resin 3.0.2123232323
Geronimo 1.128232324.66

Since I know you're going to ask about Jetty and Tomcat (the two main servlet-only containers), I ran the numbers on those too. First off, I tried Jetty 6 RC0. No dice - I got the following error when trying to start the server.

java.lang.IllegalStateException: Cannot initialize context because there is already a root application 
context present - check whether you have multiple ContextLoader* definitions in your web.xml!
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:173)

Since AppFuse deploys on all the above app servers, as well as Jetty 5.1.x, I'll chalk this up to a bug in Jetty 6. I used Jetty 5.1.11 for this test because I already had it installed on my machine.

Server Name1st run (seconds)2nd run3rd runAverage
Jetty 5.1.1124252424.33
Tomcat 5.5.1723232222.66

I don't know that these numbers mean anything, but it was a fun experiment. For those of you who think these numbers might mean something, here's the rankings:

  1. Tomcat 5.5.17
  2. Resin 3.0.21
  3. JBoss 4.0.4
  4. Jetty 5.1.11/GlassFish B48 (tie)
  5. Geronimo 1.1
  6. JOnAS 4.7.6

Of course, the better test would be hammering each server with 1000 concurrent users (or a number higher than that) and comparing how each server holds up.

Posted in Java at Aug 15 2006, 10:00:56 AM MDT 11 Comments
Comments:

Last night and today I ported my apps from Jetty to Glassfish v2b12 to get rid of some extremely persistent JTA bugs in JOTM and XAPool. Compared to Jetty, Glassfish is a PermGen glutton, and its default JVM settings aren't very useful. The directory structure is clear and its configuration file format is easy to understand and there is a tool with which you can validate your hand-modified configuration files. I'm missing some bug fixes for JPA (likely to be included in the next release) before I can really start thinking of using the new JEE5 features.

Posted by Mikael Gueck on August 15, 2006 at 10:28 AM MDT #

Ditching Jetty just because it lacks JTA would be sad ! Have you tried integrating another JTA TM in Jetty ? In their documentation they explain how to integrate two other ones than JOTM: http://www.bitronix.be/Btm/Jetty6Howto.

Posted by Ludovic Orban on August 15, 2006 at 11:30 AM MDT #

What were you expecting / hoping to find with these tests? They complete in around 20s with a huge JVM heap size so unless something was really really screwed up in any of the containers, one would expect the the numbers to be pretty close for such a lightweight test which doesn't even bring GC into the picture to tell how well each container is managing resources. What would be more interesting to compare would probably be if all the functional tests run correctly and the ui displays well. (like the Jetty issue you discovered).

Posted by Sanjiv Jivan on August 15, 2006 at 12:38 PM MDT #

I don't see a Jetty bug report from you :) ... I can't reproduce the problem. I tried running appfuse 1.9.3 on both rc0 and SVN trunk. I don't have all the dependencies installed (specifically MySQL) but the app did appear to deploy OK, up until spring failed to build the App Context. Are you able to produce a test case for this? Or do you have more info you can offer, because at this stage I can't see a problem. Feel free to contact me by mail.

Posted by Tim Vernum on August 16, 2006 at 01:15 AM MDT #

It's encouraging to see that it was so easy to deploy AppFuse so easily on such a broad range of containers. As for the benchmark - I'd be surprised if you did see any variance given what you're testing and the pretty low injection rate. I'd love to see what happens when you crank it right up (ie. see what breaks first) - that's alway more fun. Jean Francois has some other comparative numbers for Tomcat and GlassFish which demonstrates the advantages of GlassFish's NIO architecture. http://weblogs.java.net/blog/jfarcand/archive/2006/03/can_a_grizzly_r.html - Rich

Posted by Rich Sharples on August 16, 2006 at 09:45 AM MDT #

Tim - I figured out the problem. Basically, the first issue (stack trace above) is caused by the fact that AppFuse's .tld has the listener classes defined in it (as well as in web.xml).

Because of this, Jetty 6 seems to try and load the listeners twice. This issue is caused by XDoclet (which we used to generate the .tld). Even though I don't have @web.listener in my source file, XDoclet adds it anyway. If I remove it manually, and re-war, everything starts fine.

However, when I try to run the app, I get the following error:

org.apache.jasper.JasperException: /login.jsp(7,61) According to TLD or 
attribute directive in tag file, attribute value does not accept any expressions
I've seen this happen on Resin too, and it's usually b/c the server's JSTL JAR is being used instead of AppFuse's. Creating the WAR with -Djsp.2=true (with changes to a servlet 2.4 web.xml and newer JSTL JARs) fixes the problem. So it looks like Jetty 6 can't run 2.3-based applications.

After getting everything running, I ran the same test. Here's the results:

1st run: 26 seconds
2nd run: 26
3rd run: 25
Average: 25.66 seconds

Posted by Matt Raible on August 16, 2006 at 10:48 AM MDT #

So the conclusion is there is essentially no difference in performance between these containers? It would be really nice to see what the actual memory usage is, especially over a longer test that actually stresses them.

Posted by Guy Mac on August 16, 2006 at 12:26 PM MDT #

Is there any nice benchmark report can be referred to ? I want to know the performance difference when there are more than 10k concurrent users. Any help is appreciate.

Posted by cuper on August 16, 2006 at 09:44 PM MDT #

Hi Matt, > So it looks like Jetty 6 can't run 2.3-based applications. I think the problem you had is that Jetty decides which JSP version to use according to your Java SE version - as you were running Java 5, it picked up JSP 2.1. Try to run the same test using JDK 1.4; if it works, you can try to manually set JSP to be 2.0 (there is a way to do it when using the M2 Jetty 6 plugin, so there might be someway to do it using a plain Jetty server...). -- Felipe

Posted by Felipe Leme on August 20, 2006 at 04:29 PM MDT #


Damn it, I didn't realize I should have used HTML tags :-(
So, here is the post again...

Hi Matt,

>So it looks like Jetty 6 can't run 2.3-based applications.

I think the problem you had is that Jetty 6 decides which JSP version to use according to your Java SE version - as you were running Java 5, it picked up JSP 2.1.

Try to run the same test using JDK 1.4; if it works, you can try to manually set JSP to be 2.0 (there is a way to do it when using the M2 Jetty 6 plugin, so there might be someway to do it using a plain Jetty server...).

-- Felipe

Posted by Felipe Leme on August 20, 2006 at 04:34 PM MDT #

Anybody reading this as a app server comparison, please realize this is by no means a performance comparison. Unless I'm completely missing something, there is only a single thread doing the requests, while in a live environment there will be lots of concurrent request. App servers usually perform quite different under different loads. So DO NOT use this comparison to base a decision for deployment.

Posted by Mattias J on October 18, 2007 at 02:12 AM MDT #

Post a Comment:
  • HTML Syntax: Allowed