Raible Designs

Improving AppFuse's PageSpeed with Apache

One of the most important things when developing webapps is to make them fast. With AppFuse, we've tried to incorporate many of the 14 rules for faster-loading websites. We had a gzip filter before it was cool (2003) and replaced it with the one from EhCache. However, users experienced issues with both of these, both with XFire/CXF and WebWork/Struts 2 and JSPs. Because of these issues, we disabled gzipping a few releases ago.

This article is designed to show you how you can make your AppFuse webapp faster, without modifying any code. The good news is this applies to any webapp that you can deploy behind Apache.

Last Friday, I sent an email to the good folks at Contegix to see if they could install mod_pagespeed on the Apache server that sits in front of *.appfuse.org. My goal was to improve the YSlow and PageSpeed scores of the apps hosted on demo.appfuse.org. I discovered they were getting a dismal score of 24 and figured we could do a lot better. mod_pagespeed speeds up your site and reduces page load time by automatically applying web performance best practices. It seemed like an easy solution.

Unfortunately, we were unable to use mod_pagespeed. From the guys at Contegix:

Attempting to install mod_pagespeed as you requested, we find that it requires Apache httpd 2.2 and libstdc++ 4.1.2, both of which are unsupported in RHEL4. To get mod_pagespeed to work on your present operating system basically means re-rolling the core components, which would make them unsupported. I'm afraid mod_pagespeed is simply not an option on your present configuration.

Since I still wanted to improve performance, I opted for another route instead: using mod_deflate (for gzipping) and mod_expires (for expires headers). I also turned on KeepAlive as recommended by PageSpeed Insights.

mod_deflate
mod_deflate was already installed in Apache (version 2.0.52), so all I had to do was configure it. On RHEL4, Apache is installed at /etc/httpd and there's a conf.d directory that contains all the configuration files. I created a file at /etc/httpd/conf.d/deflate.conf and populated it with the following:

#
# mod_deflate configuration
#
<IfModule mod_deflate.c>
    SetOutputFilter DEFLATE
    
    AddOutputFilterByType DEFLATE text/plain text/html text/xml text/css application/xml application/xhtml+xml application/rss+xml application/javascript application/x-javascript
    
    DeflateCompressionLevel 9
    
    BrowserMatch ^Mozilla/4 gzip-only-text/html
    BrowserMatch ^Mozilla/4\.0[678] no-gzip
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
    
    DeflateFilterNote Input instream
    DeflateFilterNote Output outstream
    DeflateFilterNote Ratio ratio
    
    LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate
    
</IfModule>

At first, I had separate lines for all the different content types (as recommended by this article). The Contegix support crew figured out the solution (everything needed to be on one line) in 14 minutes, updated the config and verified it worked using an http compression testing page.

mod_expires
mod_expires was already installed, so I added a config file at /etc/httpd/conf.d/expires.conf. I used this howto and asked Contegix for help when it didn't work. Their response took quite a bit longer this time (49 minutes), but they once again figured it out:

It appears that FilesMatch does not like to play will JkMount. It does work using content type.

My final config for expires.conf:

<IfModule mod_expires.c>
    ExpiresActive On
    
    <FilesMatch "\.(jpe?g|png|gif|js|css)$">
        ExpiresDefault "access plus 1 week"
    </FilesMatch>
    
    ExpiresByType image/jpeg "access plus 1 week"
    ExpiresByType image/png "access plus 1 week"
    ExpiresByType image/gif "access plus 1 week"
    ExpiresByType text/css "access plus 1 week"
    ExpiresByType application/javascript "access plus 1 week"
    ExpiresByType application/x-javascript "access plus 1 week"
</IfModule>

I used "1 week" because we're changing things quite a bit right now and we haven't integrated resource fingerprinting yet.

KeepAlive
The last thing I did to improve performance was to turn on KeepAlive by editing /etc/httpd/conf/httpd.conf and changing Off to On.

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

Summary
As a result of these changes, our PageSpeed score went from 24 to 96 and YSlow went from a 90 to a 98. When I started this experiment, I was only trying to fix demo.appfuse.org. However, it also improved the speed of all the other *.appfuse.org sites, including Confluence, Bamboo, JIRA and FishEye. Thanks for all the help Contegix! There's a good chance you've given me back a few minutes in each day.

Originally posted on the AppFuse Blog.

What have I been working on at Taleo?

2011 has been a year of great clients for me. I started working with O.co and very much enjoyed my time there, especially on powder days in Utah. The people were great, the contract was great (no end date), but the work was not my forte. I was on a project to modularize the main shopping site's codebase, which involved mostly refactoring. By refactoring, I mean creating new Maven projects, modifying lots of pom.xml files and literally moving files from one directory to another. IntelliJ made this easy, the hard part was refactoring tests, moving from EasyMock to Mockito and splitting classes into interfaces and implementations where appropriate. As a developer who likes developing UIs and visually seeing my accomplishments, the project wasn't that exciting. However, I knew that it was strategically important to O.co, so I didn't complain much.

In mid-May, I received a LinkedIn message from the Director of Software Engineering at Taleo.

This is OB, I am the Director of Software Engineering at Taleo. We are the 2nd largest Software as a Service company. I am building a new specialist UI team that will take the product to the next level. I am looking for someone to lead this initiative. If you are interested to have a chat about it, please let me know.

At that time, I'd never heard of Taleo and quickly recommended they not hire me.

This probably isn't the best position for me. While I am a good leader, I'm not willing to relocate from Denver. I've found that leaders usually do best when face-to-face with their developers.

This conversation continued back-and-forth where I explained how I wasn't willing to go full-time and I didn't want to leave Overstock. In the end, OB was persistent and explained how the position would entail lots of UI work and wouldn't require me to travel much. Our negotiations trailed off in June and resumed in July after I returned from vacation in Montana. Shortly after, we met each other's expectations, agreed on a start date and I started working at Taleo in early September.

When I started, there were three features they wanted to add to to Taleo Business Edition: Profile Pictures, Talent Cards and Org Charts. They knew the schedule was tight (8 weeks), but I was confident I could make it happen. At first, I groaned at the fact that they were using Ant to build the project. Then I smiled when I learned they'd standardized on IntelliJ and set things up so you could do everything from the IDE. After using Maven for many years, this setup has actually become refreshing and I rarely have to restart or long for something like JRebel. Of course, a new kick-ass laptop and awesome IDE make it so I rarely wait for anything to happen.

To give you a taste of how I implemented each of these new features in 8 weeks, I've broken them into sections below.

Profile Pictures
Adding profile pictures was a pretty simple concept, one you see on my social networking sites today. I needed to give users the ability to upload a JPEG or PNG and crop it so it looked good. The uploading was a pretty straightforward process and I used a lot of internal APIs to grab the file from the request and save it to disk. The more difficult part was scaling the image to certain dimensions on upload (to save space) and allowing users to crop it after.

Most of Taleo Business Edition (TBE) is written in good ol' servlets and JSPs, with lots of scriptlets in their JSPs. When I saw the amount of HTML produced from Java, I laughed out loud and cringed. Soon after, I breathed a sigh of relief when I learned that any new features could be written using FreeMarker templates, which IntelliJ has excellent support for.

For image resizing on upload, I used Chris Campbell's Image.getScaledInstance() tutorial. For creating thumbnails, I used a combination of scaling, getSubimage() and the Java Image I/O API. I made sure to write to BufferedOutputStream for scalability. For cropping images client-side, I used jQuery UI's Dialog and Jcrop, the jQuery image cropping plugin. Below is a screenshot of what the cropping UI looks like:

Talent Cards
Talent Cards were a whole different beast. Not only did they need to display profile pictures, they also needed to contain contact information, work history and a number of other data points. Not just for employees, but for candidates as well. They also needed to be rendered with tabs at the bottom that allowed you to navigate between different data sections.

I'll admit, most of the hard work for this feature was done by the server-side developers (Harish and Vlad) on my team. Vlad built the tabbed interface and Harish built the administrative section that allows you to add/remove/sort fields, as well as show and hide certain tabs. I performed most of my magic with jQuery, its clueTip plugin and good ol' CSS. I was particularly thankful for CSS3 and its border-radius, box-shadow and Justin Maxwell's tutorial on CSS String Truncation with Ellipsis. I used DWR to fetch all the data from the server using Ajax.

Talent Cards are a slick feature in TBE 11.5 and I think they're a great way to see a lot of information about someone very quickly. If you enable them for your company, you'll be able to mouse over any employee or candidate's names and see their information.

Org Chart
The last feature I completed in this 8-week sprint was creating an organization chart. For this, I was given a rough prototype based on Caprica Software's JQuery/CSS Organisation Chart. When I received it, it had all kinds of cool CSS 3 transformations (like this one), but they only worked in Safari and Chrome. I ended up removing the transformations and adding the ability to navigate up and down the org tree with Ajax (we currently only show three levels at a time).

The Org Chart feature also allows you to see how many direct/indirect reports an employee has, as well as access their Talent Card by hovering over their name. It's one of my favorite features because it's so visual and because it builds upon all the other features we've built.

Summary
As you might've guessed by now, I've been having a lot of fun doing UI development over the last few months. While I seem to have a knack for backend Java development, I enjoy developing UIs a lot more. The smile you see on people's faces during demos is priceless. I can't help but think this kind of thing contributes greatly to my developer happiness. All these features will be in next week's release of TBE and I couldn't be happier.

If you'd like to work on my team at Taleo (or even take over my current role as UI Architect), please drop me a line. If you live near their headquarters (Dublin, CA), it'd also be great to see you at the next Silicon Valley Spring User Group meetup. I'll be speaking about What's New in Spring 3.1 on February 1st.

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.

Making Code Generation Smarter with Maven

As you might've read in my last entry, I recently started a new gig with Overstock.com. On my first day, I was quickly immersed into the development process by joining the Conversion Team. The Conversion Team is responsible for developing the checkout UI and handling payments from customers. I quickly discovered Overstock was mostly a Linux + Eclipse Shop and did my best to get my favorite Mac + IntelliJ + JRebel installed and configured. Thanks to my new Team Lead, I was able to get everything up and running the first day, as well as checkin my first contribution: making mvn jetty:run work so I didn't have to use my IDE to deploy to Tomcat.

In setting up my environment, I couldn't help but notice running jetty:run took quite a while to run each time. Specifically, the build process took 45 seconds to start executing the Jetty plugin, then another 23 seconds to startup after that. The first suspicious thing I noticed was that the UI templates were being re-generated and compiled on each execution. The UI Templating Framework at Overstock is Jamon, and is described as follows:

Jamon is a text template engine for Java, useful for generating dynamic HTML, XML, or any text-based content. In a typical Model-View-Controller architecture, Jamon clearly is aimed at the View (or presentation) layer.

Because it is compiled to non-reflective Java code, and statically type-checked, Jamon is ideally suited to support refactoring of template-based UI applications. Using mock objects -like functionality, Jamon also facilitates unit testing of the controller and view.

To generate .java files from .jamon templates, we use the Jamon Plugin for Maven. Remembering that the Maven Compiler Plugin has an incremental-compile feature, I turned to its source code to find out how to implement this in the Jamon plugin. I was pleasantly surprised to find the StaleSourceScanner. This class allows you to easily compare two files to see if the source needs to re-examined for generation or compilation.

I noticed the Jamon Plugin had the following code to figure out which files it should generate into .java files:

private List<File> accumulateSources(File p_templateSourceDir)
{
  final List<File> result = new ArrayList<File>();
  if (p_templateSourceDir == null)
  {
    return result;
  }
  for (File f : p_templateSourceDir.listFiles())
  {
    if (f.isDirectory())
    {
      result.addAll(accumulateSources(f));
    }
    else if (f.getName().toLowerCase(Locale.US).endsWith(".jamon"))
    {
      String filePath = f.getPath();
       // FIXME !?
      String basePath = templateSourceDir().getAbsoluteFile().toString();
      result.add(new File(filePath.substring(basePath.length() + 1)));
    }
  }
  return result;
}

I changed it to be smarter and only generate changed templates with the following code:

private List<File> accumulateSources(File p_templateSourceDir) throws MojoExecutionException
{
  final List<File> result = new ArrayList<File>();
  if (p_templateSourceDir == null)
  {
    return result;
  }
  SourceInclusionScanner scanner = getSourceInclusionScanner( staleMillis );
  SourceMapping mapping = new SuffixMapping( ".jamon", ".java");

  scanner.addSourceMapping( mapping );

  final Set<File> staleFiles = new LinkedHashSet<File>();

  for (File f : p_templateSourceDir.listFiles())
  {
    if (!f.isDirectory())
    {
      continue;
    }

    try
    {
      staleFiles.addAll( scanner.getIncludedSources(f.getParentFile(), templateOutputDir()));
    }
    catch ( InclusionScanException e )
    {
      throw new MojoExecutionException(
        "Error scanning source root: \'" + p_templateSourceDir.getPath()
          + "\' " + "for stale files to recompile.", e );
    }
  }

  // Trim root path from file paths
  for (File file : staleFiles) {
    String filePath = file.getPath();
    String basePath = templateSourceDir().getAbsoluteFile().toString();
    result.add(new File(filePath.substring(basePath.length() + 1)));
  }
}

This method references a getSourceInclusionScanner() method, which is implemented as follows:

protected SourceInclusionScanner getSourceInclusionScanner( int staleMillis )
{
  SourceInclusionScanner scanner;

  if ( includes.isEmpty() && excludes.isEmpty() )
  {
      scanner = new StaleSourceScanner( staleMillis );
  }
  else
  {
      if ( includes.isEmpty() )
      {
          includes.add( "**/*.jamon" );
      }
      scanner = new StaleSourceScanner( staleMillis, includes, excludes );
  }

  return scanner;
}

If you're using Jamon and its Maven Plugin, you can view my patch at SourceForge. If you're looking to include this functionality in your project, I invite you to look at the code I learned from in the Maven Compiler's AbstractCompilerMojo class.

After making this change, I was able to reduce the build execution time by over 50%. Now it takes 20 seconds to hit the Jetty plugin and 42 seconds to finishing starting. Of course, in an ideal world, I'd like to get this down to 20 seconds or less. Strangely enough, the easiest way to do this seems to be simple: use Linux.

On the Linux desktop they provided me, it takes 12 seconds to hit the Jetty plugin and 23 seconds to finish starting. I'd like to think this is a hardware thing, but it only get 20% faster on OS X when using an 8GB RAM + SSD machine (vs. a 4GB + 5400 drive). Since Overstock has provided me with a 4GB MacBook Pro, I'm considering installing Ubuntu on it, just to see what the difference is.

In related news, Overstock.com is looking to hire a whole bunch of Java Developers this year. The pictures of the new Provo office look pretty sweet. Of course, you can also work at HQ, which is a mere 25 minutes from some of the best skiing in the world. Personally, I think Colorado's powder is better, but I can't argue with the convenience of no traffic. In addition to full-time gigs, they've started hiring more remote contractors like myself, so they pretty much have something for everyone. So if you love Java, like to get some turns in before work, and aren't an asshole - you should and I'll try to hook you up.

Update: After writing this post, I received an email from Neil Hartner questioning these numbers. Basically, he was able to get his MacBook Pro to run just as fast as Linux. Turns out, the reason my Mac was so much slower was because JRebel was configured in my MAVEN_OPTS. JRebel's FAQ does state the following:

Does JRebel make the server start up slower?
JRebel needs to do more work on startup (search more places for classes and resources, instrument classes, etc), so some slowdown can be expected. If it's larger than 50% please contact [email protected].

Since it's right around 50% slower, I guess there's no reason to call them. My guess is the best thing to do is remove JRebel from MAVEN_OPTS, but have an alias that can enable it, or simply run it from your IDE.

Dojo/Comet support in Java Web Frameworks

This week I'm doing a research project for a client. The main purpose of the project is to find out which Java-based web framework works best with Dojo and Comet. Here's the key requirement from the client:

It's all about Comet, we want Comet everywhere we can put it, but we want to isolate the icky bits of fiddling with pages with JavaScript. We're kind of wed to the Dojo implementation of the client-side bit, so we may as well use more of the Dojo widgets for a richer UI. For us, "works best with" needs to pay a certain amount of consideration to "fits naturally with", if you understand what I mean. I know that any framework that lets you spit out raw HTML will let you hand code in your Dojo / Comet, but that's certain to become very tiresome very quickly.

The candidate frameworks they asked me to look at are Wicket and Tapestry 5. They're willing to upgrade to Struts 2 since they're already using Struts 1. However, they don't feel that action-based frameworks naturally lead to rich UIs, so they'd prefer a component-based framework. They're currently using Seam for an administration-type application and feel it's too heavy for their customer-facing application.

Here's what I've found so far in my research. Please let me know if anything is incorrect.

• Tapestry 5 doesn't have Dojo or Comet support (Prototype and Scriptaculous are the baked-in Ajax frameworks).
• Struts 2 has old (version 0.4.3) and somewhat deprecated Dojo support. The developers seem to be in favor of removing it and promoting people hand-code Dojo instead. Struts 2 doesn't have support for Comet.
• Wicket has support for Dojo 1.1 that includes Comet support. This was written by Stefan Fußenegger and posted to the mailing list last month. I e-mailed Stefan and asked him about documentation. His response: "I lost my ambition to document it properly since I didn't receive any feedback on the mailing list. :)"

At this point, it seems that if the client really wants to use Dojo, they should use Wicket, and possibly pay Stefan to document it properly. However, they're willing to consider other options, as long as they have Comet support.

One option I thought of is to use DWR and its Reverse Ajax/Comet support. Another option would be to add better Dojo support to Tapestry 5. However, I don't think this is possible since the Prototype/Scriptaculous code is generated by the framework and would likely require a changes to switch it to Dojo.

Are there any other Java-based web frameworks that support easily creating Dojo widgets and working with Comet? Keith Donald tweeted that Spring MVC has Dojo support. However, I believe it's only for widgets and it still requires you to write JavaScript. If your framework doesn't have Dojo/Comet support, how hard would it be to add it?

Update: I also posted this question on LinkedIn. Make sure and check my question for additional thoughts from folks.

Upgrading to Spring Security 2.0

This evening I spent a few hours and upgraded AppFuse to use Acegi Spring Security 2.0. The upgrade was fairly straightforward:

• %s/org.acegisecurity/org.springframework.security/g
• Upgraded dependencies (exclusions are necessary if you're using Spring 2.5.x and don't want 2.0.x dependencies pulled in):

  <dependency>
      <groupId>org.springframework.security</groupId>
      <artifactId>spring-security-core-tiger</artifactId>
      <version>${spring.security.version}</version>
      <exclusions>
          <exclusion>
              <groupId>org.springframework</groupId>
              <artifactId>spring-core</artifactId>
          </exclusion>
          <exclusion>
              <groupId>org.springframework</groupId>
              <artifactId>spring-support</artifactId>
          </exclusion>
      </exclusions>
  </dependency>
  ...
  <dependency>
      <groupId>org.springframework.security</groupId>
      <artifactId>spring-security-taglibs</artifactId>
      <version>${spring.security.version}</version>
      <exclusions>
          <exclusion>
              <groupId>org.springframework</groupId>
              <artifactId>spring-web</artifactId>
          </exclusion>
      </exclusions>
  </dependency>
  

• Changed taglib prefix from "authz" to "security" and change the associated taglib declaration to:

  <%@ taglib uri="http://www.springframework.org/security/tags" 
      prefix="security" %>
  

• In web.xml, I changed <filter-class> to org.springframework.web.filter.DelegatingFilterProxy. Since I didn't name my filter springSecurityFilterChain, I also had to add the following <init-param>:

      <init-param>
          <param-name>targetBeanName</param-name>
          <param-value>springSecurityFilterChain</param-value>
      </init-param>
  

• Lastly, I modified security.xml to use the new syntax. AppFuse's security.xml went from 175 lines to 33 with the new security namespace configuration!

It's hard to believe I first looked at Acegi almost 4 years ago. At that time, I said it contained too much XML for my needs. Ben's reaction:

Seriously, the "whole lotta XML" gives you exponentially more power and flexibility than a method such as this could ever hope to provide you.

It's nice to see that Spring Security 2.0 gives you exponentially more power and flexibility without all the XML. Thanks guys!

P.S. You can also view the full changelog for this upgrade.

Update: If you're using <authz:authentication property="fullName"/> in your JSPs, you'll need to change it to <security:authentication property="principal.fullName"/>.

FreeMarker vs. JSP 2

I've been doing quite a bit of prototyping with Spring MVC and Struts 2 with both JSP and FreeMarker in the last few months. I'm trying to migrate a proprietary servlet-based framework with a proprietary JSP compiler to something that's open source. There's a couple of important features that the proprietary view framework has:

1. It's expression language allows methods to be called with arguments.
2. Templates can be loaded from a JAR on a remote server.
3. XML in variables is escaped by default.

For #1, I've found this to be impossible with JSP EL or JSTL. I've created JSP functions that allow argument passing, but they don't allow overloading of functions. FreeMarker solves #1.

For #2, JSPs again fail because the templates have to be on the file system or in a WAR. FreeMarker solves this problem as well.

For #3, neither JSP or FreeMarker solve this problem. I realize it can be fixed in FreeMarker by hacking the code - I've done the same with Tomcat and solved it for JSP as well.

So based on the requirements in this project, FreeMarker is the clear winner. Here's some problems that I see with using it:

• No XML escaping of expressions by default
• No compile-time checking of expressions
• IDE support is limited to Eclipse (meaning very little in the way of code-completion)

FreeMarker users - are there other problems you've experienced when using FreeMarker in your applications?

GWT Presentation with Bob Vawter

Tonight I'm in Mountain View attending the inaugural meeting for the new Silicon Valley Google Technology User Group (SV-GTUG). The good news about this meeting is Bob plans to discuss some of the new features in the GWT 1.5 release. Although it is not available yet, it sounds like they are putting the finishing touches on the 1.5 release and that it will be available soon.

The first ting I've learned is that GWT is actually pronounced "gwit". Bob is a member of the GWT team and he's the only one that's not based out of Atlanta. GWT has a very Swing-like programming model. It has widgets, panels and windows and operates very much like a traditional UI programming API. To create an application, you create a class that implements EntryPoint. Below is an example Hello World application.

One of the really nice things about GWT is that you can easily debug your application in your favorite Java IDE. GWT doesn't prevent you from writing JavaScript - you can insert native JavaScript into your code using a special /*-{ script here }-*/.

When you compile a GWT application, you get a whole bunch of stuff. When trying to develop cross-browser JavaScript code, you often end up with "if soup" to handle the different browsers. GWT does the same thing, but it creates different versions of the application for the different browsers. The naming of these files is an MD5 hash that's bootstrapped by a simple JS file. GWT files are all static and the file sizes are ~40KB/each for the Hello World application.

If you have to write fewer than 100 lines of code, it's likely that GWT will be overkill - you can do it with JavaScript easier. Once you get more lines of code and larger teams, Java's tools do all the heavy lifting. Yes, you can create large applications with large teams with scripting languages, but Java is the clear winner when you have large teams of developers working on projects.

GWT provides a RemoteService for client code and a RemoteServiceServlet for server-side code that you can use to implement RPC calls. For more information, see the Remote Procedure Calls documentation.

At this point, we're 45 minutes into the presentation and it's pretty disappointing. This is likely because there's no presentation and Bob doesn't seem to have any sort of agenda. I think this type of presentation would work well if folks had experience with GWT and were asking lots of questions. However, it seems that most folks haven't used GWT - so the questions (and his resulting answers) aren't very meaty.

One attendee asked about making GWT applications SEO-able. Bob's suggested solution is to continue to use Servlet or JSPs to render the page, but then use GWT to for the user interaction that needs to happen within the page. The RootPanel class has a get(String id) method that can be used to put the GWT application into a particular div or other placeholder.

Google Base and Google Checkout are currently using GWT. There's also a lot of Google internal development that uses GWT. Apparently, there's quite a few applications built with GWT that haven't launched yet.

Has anyone used the Maven Gwt Plugin? It sounds like it might be pretty nice as it contains the GWT compiler with no OS-dependent Google tooling.

The main new feature in GWT 1.5 is Java 5 support. There's also a GWT Incubator that's used to host GWT features that may go into GWT's core. You can view the demos online.

GWT's mission statement:

GWT's mission is to radically improve the web experience for users by enabling developers to use existing Java tools to build no-compromise AJAX for any modern browser.

GWT's competition according to Bob - Visual Studio.

Comparing Web Frameworks: Time for a Change?

I first came up with the idea to do a "Comparing Web Frameworks" talk in 2004. I submitted a talk to ApacheCon and it got accepted. From there, I outlined, created sample apps and practiced this talk before ApacheCon. Believe it or not, that was my first time speaking in front of a large audience.

Historical note: October 2004 was a pretty cool month - I discovered Rails and Roller had a 1.0 release candidate.

When I created the presentation, it was in large part due to all the WebWork and Tapestry folks harassing me on this very blog. I started using Struts in June 2001 (the same month 1.0 was released) and had used it successfully on many projects. Part of the reason this blog became so popular was I posted lots of tips and tricks that I learned about Struts (and its related project) while using it. After a while, the noise became too heavy to ignore it - especially after I'd tried Spring MVC. So in an effort to learn more about the the other frameworks, I submitted a talk and forced myself to learn them. It seems to have worked out pretty well.

With that being said, I think it's time for a change. The reason I originally wrote this was to educate developers on how the top Java web frameworks differed and encourage developers to try more than one. A while later, I realized there's different tools for different jobs and it's not a one-size-fits-all web framework world. It's not a component vs. request-based framework world either. There's lots of options now. When I've delivered this talk earlier this year, I've always felt like I've left quite a few frameworks out. The solution could be to add more and more frameworks. However, I don't think that's a good idea. The talk is already difficult to squeeze into 90 minutes and it's unlikely that adding more frameworks is going to help.

The change I'd like to do is to reduce the number of frameworks down to (what I consider) the top web frameworks for deploying to the JVM. What are those frameworks? IMHO, they are as follows, in no particular order:

• GWT-Ext
• Wicket
• Grails
• Flex/OpenLaszlo
• Seam
• Struts 2

The RIFE, Tapestry and ZK folks can start bitching now. Sorry - less frameworks make for a more interesting talk. Maybe I'll add you in the future and I can ask the audience which ones they want compared then we can choose four and go from there. Why don't I mention Spring MVC? Because I think Struts 2 is easier to learn and be productive with and I also like it's more open and active community. I've written applications with both and I like Struts 2 better. As for Flex vs. OpenLaszlo, I'm somewhat torn. It seems like learning Flex is going to be better for your career, but it's likely useless without the Flex Builder - which is not open source. However, at $250, it's likely worth its price. I know the Picnik folks used Flex for their UI - I wonder how much they used Flex Builder in the process?

What do you think? Are these the top web frameworks for JVM deployment today? The next time I give this talk is this Thursday at ApacheCon. I may try to re-write my talk and then give the audience a choice of old vs. new. The downside of doing the new talk is I won't have time to write apps with GWT, Flex or Seam. Anyone care to post their top three pros and cons for any of these frameworks?

Upgrading AppFuse to Spring 2.5

Last night, I spent a few minutes upgrading AppFuse to Spring 2.5 RC1. According to InfoQ, Spring 2.5 is a drop-in upgrade for Spring 2.0. However, if you're using Maven, it's not quite that easy. The good news is it is easy, you just need to change your pom.xml a bit. The steps I used to upgrade AppFuse are listed below:

• Add a repository for Spring's milestone releases:

  <repository>
      <id>spring-milestone</id>
      <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
  </repository>
  

• Change artifactId of "spring-mock" to be "spring-test".
• Change version to be 2.5-rc1.

At this point, if you're using "spring" as your artifactId (instead of the smaller fine-grained dependencies), you'll likely get the following error in a Spring MVC application:

java.lang.NoClassDefFoundError: 
org/springframework/web/servlet/handler/AbstractUrlHandlerMapping

This happens because Spring MVC is no longer included in the uber spring.jar. You'll need to add a dependency on "spring-webmvc" to solve this problem. Unfortunately, this JAR is dependent on the fine-grained modules, so you may have to modify your pom.xml to depend on the fine-grained modules - or exclude them all from spring-webmvc.

The good news is Spring has excluded all the invalid commons-logging dependencies for you so you don't have to anymore.

After getting all the dependencies straightened out - I ran the integration tests:

org.springframework.beans.NotReadablePropertyException: Invalid property 
'fileUpload' of bean class [org.appfuse.webapp.controller.FileUpload]: Bean 
property 'fileUpload' is not readable or has an invalid getter method: Does the 
return type of the getter match the parameter type of the setter?

Looking at uploadForm.jsp, I'm guessing the problem happens because of the following code:

<spring:bind path="fileUpload.file">
<input type="file" name="file" id="file" class="file medium" value="<c:out value="${status.value}"/>"/>
</spring:bind>

Confirmed - changing the "path" attribute to "file" fixes the problem. I also found out that setting the "value" on an <input type="file"> doesn't work, so wrapping the field with <spring:bind> doesn't make a whole lot of sense anyway.

To conclude, it doesn't look like the first release candidate of Spring 2.5 is exactly a drop-in upgrade for Spring 2.0, but it's pretty darn close. I'm sure by the time it's released, it will be. I'd encourage you to try 2.5 in your Spring-dependent projects to see if you find any issues.

Update: I was successfully able to migrate AppFuse from using the uber JAR to fine-grained JARs. However, I ran into a couple issues in the process. The first is that even though I'm including spring-aop in the appfuse-service module, it's not pulled in for the web frameworks (which depend on appfuse-service). Explicitly declaring spring-aop as a dependency for the appfuse-web module fixes this. Secondly, I had to modify my Acegi Security exclusions so it wouldn't include dependencies that no longer exist in 2.5.

<dependency>
    <groupId>org.acegisecurity</groupId>
    <artifactId>acegi-security-tiger</artifactId>
    <version>${acegi.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-dao</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jdbc</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-remoting</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>