Raible Designs

Java Web Application Security - Part III: Apache Shiro Login Demo

A couple weeks ago, I wrote a tutorial on how to implement security with Spring Security. The week prior, I wrote a similar tutorial for Java EE 6. This week, I'd like to show you how to implement the same features using Apache Shiro. As I mentioned in previous articles, I'm writing this because I told the audience at April's UJUG that I would publish screencasts of the demos.

Today, I've finished the third screencast showing how to implement security with Apache Shiro. Below is the presentation (with the screencast embedded on slide 22) as well as a step-by-step tutorial.

Apache Shiro Login Tutorial

• Download and Run the Application
• Implement Basic Authentication
• Force SSL
• Implement Form-based Authentication
• Store Users in a Database
• Summary

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a shiro.ini file Shiro's configuration. Create src/main/resources/shiro.ini and populate it with the contents below:

[main]

[users]
admin = admin, ROLE_ADMIN

[roles]
ROLE_ADMIN = *

[urls]
/app/users = authcBasic

You can see this file has four sections and is pretty simple to read and understand. For more information about what each section is for, check out Shiro's configuration documentation.

Next, open src/main/webapp/WEB-INF/web.xml and add Shiro's IniShiroFilter:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.apache.shiro.web.servlet.IniShiroFilter</filter-class>
    <!-- no init-param means load the INI config from classpath:shiro.ini -->
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Then add Shiro's core and web dependencies to your pom.xml:

<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-web</artifactId>
    <version>1.1.0</version>
</dependency>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Apache Shiro is easier to configure than Spring Security out-of-the-box, mostly because it doesn't require XML.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Shiro doesn't currently have a way to logout with its API. However, it will be added in the 1.2 release.

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Before you implement form-based authentication, I'd like to show you how easy it is to force SSL with Apache Shiro.

Force SSL
Apache Shiro allows you to force SSL on a URL by simply adding "ssl[port]" to a URL in the [urls] section. If you don't specify the port, it will use the default port (443). I'm not sure if it allows you to switch back to http like Spring Security's requires-channel, but I don't think it does. Modify the URLs section of your shiro.ini to have the following:

[urls]
/app/users = ssl[8443],authc

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll be prompted to accept the Untrusted Certificate and then redirected to https://localhost:8443/users after logging in.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a few lines to shiro.ini. First of all, since I'd rather not change the name of the input elements in login.jsp, override the default names in the [main] section:

# name of request parameter with username; if not present filter assumes 'username'
authc.usernameParam = j_username
# name of request parameter with password; if not present filter assumes 'password'
authc.passwordParam = j_password
authc.failureKeyAttribute = shiroLoginFailure

Then change the [urls] section to filter on login.jsp and use "authc" instead of "authcBasic":

[urls]
# The /login.jsp is not restricted to authenticated users (otherwise no one could log in!), but
# the 'authc' filter must still be specified for it so it can process that url's
# login submissions. It is 'smart' enough to allow those requests through as specified by the
# shiro.loginUrl above.
/login.jsp = authc
/app/users = ssl[8443],authc

Then change login.jsp so the form's action is blank (causing it to submit to itself) instead of j_security_check:

<form action="" id="loginForm" method="post">

Now, restart Jetty and you should be prompted to login with this JSP instead of the basic authentication dialog.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a few settings to shiro.ini to define your database and tables to use. Open src/main/resources/shiro.ini and add the following lines under the [main] section.

jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
#jdbcRealm.permissionsLookupEnabled=false
# If not filled, subclasses of JdbcRealm assume "select password from users where username = ?"
jdbcRealm.authenticationQuery = select user_pass from users where user_name = ?
# If not filled, subclasses of JdbcRealm assume "select role_name from user_roles where username = ?"
jdbcRealm.userRolesQuery = select role_name from users_roles where user_name = ?

ds = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds.serverName = localhost
ds.user = root
ds.databaseName = appfuse
jdbcRealm.dataSource = $ds

This configuration is similar to what I did with the Java EE 6 tutorial where I'm pointing to a database other than the H2 instance that's used by the application. I believe Shiro can talk to a DAO like Spring Security, but I have yet to explore that option.

While you're at it, add the following lines to enable password encryption.

sha256Matcher = org.apache.shiro.authc.credential.Sha256CredentialsMatcher
jdbcRealm.credentialsMatcher = $sha256Matcher

You'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database using the following command:

mysql -u root -p -e 'create database appfuse'

Then create the tables necessary and populate it with an 'admin' user. Login using "mysql -u root -p appfuse" and execute the following SQL statements:

create table users (
  user_name         varchar(30) not null primary key,
  user_pass         varchar(100) not null
);

create table user_roles (
  user_name         varchar(30) not null,
  role_name         varchar(30) not null,
  primary key (user_name, role_name)
);

insert into users values ('admin', '22f256eca1f336a97eef2b260773cb0d81d900c208ff26e94410d292d605fed8');
insert into user_roles values ('admin', 'ROLE_ADMIN');

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Apache Shiro 1.1.0. I don't have a lot of experience with Apache Shiro, but I was able to get the basics working without too much effort. This tutorial doesn't show how to do Remember Me because I couldn't figure it out in 5 minutes, which means I have 5 more minutes before it fails the 10-minute test.

Shiro was formerly named JSecurity and has been an Apache project for less than a year. It seems to be more targeted towards non-web use, so its certainly something to look at if you're more interested in cryptography or non-web apps. I think there's a good chance this project will continue to grow and be used more as more developers learn about it. The Apache brand certainly doesn't hurt.

I didn't include a slide about the limitations I found with Shiro, mostly because I haven't used it much. I've used Java EE and Spring Security for several years. The main limitation I found was the lack of documentation, but I've heard it's improving rapidly.

In the next couple weeks, I'll post a Part IV on implementing programmatic login using the APIs of Java EE 6, Spring Security and Apache Shiro. I'll be presenting this topic at Jazoon as well as the long-form version (with hacking) at ÜberConf. Hopefully I'll see you at one of those conferences.

Update: Thanks to help from Les Hazlewood, I've figured out how to implement Remember Me with Apache Shiro. In the [urls] section of shiro.ini, the second url (shown below) says to Shiro "In order to visit the /app/users URL, you must be connecting via SSL on port 8443 and you must also be authenticated."

/app/users = ssl[8443],authc

Remembered users are not authenticated because their identity hasn't been proven during the current session. What I want Shiro to say is "In order to visit the /app/users URL, you must be connecting via SSL on 8443 and you must also be a known user. If you're not, you should login first." Where a known user is someone who has a recognized identity and has either authenticated during the current session or is known via RememberMe from a previous session. The documentation gives a good example with Amazon.com for why Shiro makes this distinction. It allows more control (usually necessary), but you can relax the control as you see fit.

So, to relax my configuration a bit to match what I want (known users), I updated shiro.ini's [urls] section to be as follows:

/app/users = ssl[8443],user

The key is that the /app/users url is now protected with the more relaxed user filter instead of the authc filter. However, you would typically want an account profile page (or credit card information page, or similar) protected with the authc filter instead to guarantee proof of identity for those sensitive operations.

Java Web Application Security - Part II: Spring Security Login Demo

Last week, I wrote a tutorial on how to implement Security in Java EE 6. This week, I'd like to show you how to implement the same features using Spring Security. Before I begin, I'd like to explain my reason for writing this article.

Last month, I presented a talk on Java Web Application Security at the Utah JUG (UJUG). As part of that presentation, I did a number of demos about how to implement security with Java EE 6, Spring Security and Apache Shiro. I told the audience that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense.

Today, I've finished the second screencast showing how to implement security with Spring Security. Below is the presentation (with the screencast embedded on slide 16) as well as a step-by-step tutorial.

Spring Security Login Tutorial

• Download and Run the Application
• Implement Basic Authentication
• Force SSL
• Implement Form-based Authentication
• Add Remember Me
• Store Users in a Database
• Summary

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a Spring context file that contains Spring Security's configuration. Create src/main/webapp/WEB-INF/security.xml and populate it with the contents below:

  <?xml version="1.0" encoding="UTF-8"?>
  <beans:beans xmlns="http://www.springframework.org/schema/security"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:beans="http://www.springframework.org/schema/beans"
               xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">

      <!-- New in Spring Security 3.1 -->
      <!-- <http pattern="/css/**" security="none"/> -->

      <http auto-config="true">
          <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN"/>
          <http-basic/>
      </http>

      <authentication-manager alias="authenticationManager">
          <authentication-provider>
              <password-encoder hash="sha"/>
              <user-service>
                  <user name="user" password="12dea96fec20593566ab75692c9949596833adc9" authorities="ROLE_USER"/>
                  <user name="admin" password="d033e22ae348aeb5660fc2140aec35850c4da997" authorities="ROLE_ADMIN"/>
              </user-service>
          </authentication-provider>
      </authentication-manager>

      <!-- Override userSecurityAdvice bean in appfuse-service to allow any role to update a user. -->
      <beans:bean id="userSecurityAdvice" class="org.appfuse.examples.webapp.security.UserSecurityAdvice"/>
  </beans:beans>

The last bean, userSecurityAdvice, is an aspect that's needed to override some behavior in AppFuse. You won't need this normally when implementing Spring Security.

Next, open src/main/webapp/WEB-INF/web.xml and add Spring's DelegatingFilterProxy:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
    <init-param>
        <param-name>targetBeanName</param-name>
        <param-value>springSecurityFilterChain</param-value>
    </init-param>
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

You don't need to add any dependencies in your pom.xml is because this project depends on AppFuse, which already contains these dependencies.

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Spring Security is a bit easier to configure than Java EE 6 out-of-the-box, mostly because it doesn't require you to configure your container.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Spring Security has a way to configure "logout" to match a URL and get rid of a class like LogoutController. Since it was already in the project, I don't cover that in this tutorial.

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Before you implement form-based authentication, I'd like to show you how easy it is to force SSL with Spring Security.

Force SSL
Spring Security allows you to switch between secure (https) and non-secure (http) protocols using a simple requires-channel attribute on the <intercept-url> element. Possible values are "http", "https" and "any". Add requires-channel="https" to your security.xml file:

<intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll be prompted to accept the Untrusted Certificate and then redirected to https://localhost:8443/users after logging in. This is an improvement on Java EE's user-data-constraint for two reasons:

• You can switch between http and https protocols. With Java EE, you can only force https. You have to write a custom filter to switch back to http.
• Redirecting to https actually works. With Java EE (on Jetty at least), a 403 is returned instead of redirecting the request.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a <form-login> element in security.xml's <http> element:

<http auto-config="true">
    <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>
    <form-login login-page="/login" authentication-failure-url="/login?error=true"
                login-processing-url="/j_security_check"/>
    <http-basic/>
</http>

You can leave the <http-basic> element since Spring Security is smart enough to serve up the form for browsers and use Basic Authentication for clients such as web services. The login.jsp page (that /login forwards to) already exists in the project, in the src/main/webapp directory. The forwarding is done by the UrlRewriteFilter with the following configuration in src/main/webapp/WEB-INF/urlrewrite.xml.

<rule>
    <from>/login</from>
    <to>/login.jsp</to>
</rule>

This JSP has 3 important elements: 1) a form that submits to "/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Add Remember Me
Remember Me is a feature you see in many web applications today. It's usually a checkbox on the login form that allows you to auto-login the next time you visit a site. This feature doesn't exist in Java EE security, but it does exist in Spring Security. To enable it, add the following just below <form-login> in security.xml:

<remember-me user-service-ref="userDao" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"/>

Next, open src/main/webapp/login.jsp and change the name of the "remember me" checkbox to be _spring_security_remember_me:

<input type="checkbox" name="_spring_security_remember_me" id="rememberMe"/>

After making these changes, you should be able to restart Jetty, go to http://localhost:8080/users, enter admin/adminjdbc, check the Remember Me checkbox and login. Then close your browser, and repeat the process. This time, you won't be prompted to login. For more information on this feature, see Spring Security's Remember Me documentation.

While storing usernames and passwords in a file is convenient for demos, it's not very real-world-ish. The next section shows you how to configure Spring Security to use a database for its user store.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a user-service-ref attribute to the <authentication-provider> element. You can also delete the <user-service> element.

<authentication-manager alias="authenticationManager">
    <authentication-provider user-service-ref="userDao">
        <password-encoder hash="sha"/>
    </authentication-provider>
</authentication-manager>

The "userDao" bean is provided by AppFuse and its UserDaoHibernate.java class. This class implements Spring Security's UserDetailsService interface. With Java EE, I had to configure a database connection and make sure the JDBC Driver was in my container's classpath. With Spring Security, you can talk to the database you already have configured in your application.

2011-05-13 08:47:29.081:WARN::UserRealm Java EE Login could not connect to database; will try later
org.h2.jdbc.JdbcSQLException: Database may be already in use: "Locked by another process". 
        Possible solutions: close all other connection(s); use the server mode [90020-154]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
	at org.h2.message.DbException.get(DbException.java:167)
	at org.h2.message.DbException.get(DbException.java:144)
	at org.h2.store.FileLock.getExceptionAlreadyInUse(FileLock.java:443)
	at org.h2.store.FileLock.lockFile(FileLock.java:338)
	at org.h2.store.FileLock.lock(FileLock.java:134)
	at org.h2.engine.Database.open(Database.java:535)
	at org.h2.engine.Database.openDatabase(Database.java:218)

The password for the "admin" user is configured in src/test/resources/sample-data.xml and it's loaded by DbUnit before the application starts. You can view your pom.xml and the dbunit-maven-plugin's configuration if you're interested in learning how this is done. The password is currently configured to "adminjdbc", but you can reset it by generating a new password and modifying sample-data.xml.

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Spring Security 3.0.5. In addition to the basic XML configuration, Spring Security also provides a AOP support and annotations you can use to secure methods. It also has many more features than standard Java EE Security. In my opinion, it's the most mature security framework we have in Java today. Currently, I think its reference documentation is the best place to learn more.

• The authentication mechanism (file, database, ldap, etc.) is contained in the WAR
• Securing methods only works on Spring beans
• Remember Me doesn't work in my screencast (because I forgot to rename the checkbox in login.jsp)

Of course, you can configure Spring to load its configuration from outside the WAR (e.g. a file or JNDI), but it's not as easy as including the configuration in your app.

In the next couple weeks, I'll post Part III of this series, where I'll show you how to implement this same set of features using Apache Shiro. In the meantime, please let me know if you have any questions.

I created the screencasts with Camtasia. For small screens, and embedding in the presentation, I created it at 50% and used the SmartFocus feature to zoom in and out during the demo. For larger screens, I published another screencast at 100%, in HD. If you have a preference for which screencast is better, I'd love to hear about it.

AppFuse 2.1 Released!

The AppFuse Team is pleased to announce the release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are JPA 2, JSF 2, Tapestry 5 and Spring 3. In addition, we've migrated from XFire to CXF and enabled REST for web services. There's even a new appfuse-ws archetype that leverages Enunciate to generate web service endpoints, documentation and downloadable clients. This release fixes many issues with archetypes, improving startup time and allowing jetty:run to be used for quick turnaround while developing. For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is an open source project and application that uses open source frameworks to help you develop Web applications with Java quickly and efficiently. It was originally developed to eliminate the ramp-up time when building new web applications. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project. If you use JRebel with IntelliJ, you can achieve zero-turnaround in your project and develop features without restarting the server.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose with persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run "appfuse:full-source".

AppFuse comes in a number of different flavors. It offers "light", "basic" and "modular" and archetypes. Light archetypes use an embedded H2 database and contain a simple CRUD example. Light archetypes allow code generation and full-source features, but do not currently support Stripes or Wicket. Basic archetypes have web services using CXF, authentication from Spring Security and features including signup, login, file upload and CSS theming. Modular archetypes are similar to basic archetypes, except they have multiple modules which allows you to separate your services from your web project.

AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket. You can see demos of these archetypes at http://demo.appfuse.org.

For information on creating a new project, please see the QuickStart Guide.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the mailing list or create an issue in JIRA.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

Livin' it up in Vegas at TSSJS 2011

Last Wednesday, Trish and I traveled to Las Vegas for TheServerSide Java Symposium 2011 conference. We had a free room from TechTarget, but opted to upgrade to a suite with a view over the Bellagio Fountains. Trish won a trip to Vegas as a sales award earlier in the year and cleverly exchanged it for cash, so our upgrade was sort of free.

After I finished speaking, we headed to happy hour and met up with some friends that happened to be in town. We had dinner at the Todd English Pub and headed to the Penn & Teller show at the Rio. We closed the night after Trish had a 45-minute roll at the craps table at O'Sheas.

We slept in on Thursday and I gave my Comparing JVM Web Frameworks talk that afternoon. I made sure to mention some other methods to choosing web frameworks: doing performance comparisons like Peter Thomas has done or choosing Lift because one of its developers says it's the best. While Vaadin did sneak into the #5 spot, I made sure and mentioned that Wicket and Tapestry seem to belong there moreso (based on stats, mailing list traffic, etc.).

Trish took a bunch of pictures during my talk, which had a great turnout and lots of participation.

That evening, we celebrated St. Patty's Day with some college buddies of mine, ate great sushi at Mizuya and experienced the joys of three card poker. Thanks to TechTarget for inviting me to TSSJS 2011; we had an awesome time. You can find all the pictures we took on Flickr.

P.S. If you can't see the presentations in this post (a.k.a. you don't have Flash), you can view them on on Slideshare or download the PDFs.

Adding Search to AppFuse with Compass

Over 5 years ago, I recognized that AppFuse needed to have a search feature and entered an issue in JIRA. Almost 4 years later, a Compass Tutorial was created and shortly after Shay Banon (Compass Founder), sent in a patch. From the message he sent me:

Fast forward 2 years and I finally found the time/desire to put a UI on the backend Compass implementation that Shay provided. Yes, I realize that Compass is being replaced by ElasticSearch. I may change to use ElasticSearch in the future; now that the search feature exists, I hope to see it evolve and improve.

Since Shay's patch integrated the necessary Spring beans for indexing and searching, the only thing I had to do was to implement the UI. Rather than having an "all objects" results page, I elected to implement it so you could search on an entity's list screen. I started with Spring MVC and added a search() method to the UserController:

@RequestMapping(method = RequestMethod.GET)
public ModelAndView handleRequest(@RequestParam(required = false, value = "q") String query) throws Exception {
    if (query != null && !"".equals(query.trim())) {
        return new ModelAndView("admin/userList", Constants.USER_LIST, search(query));
    } else {
        return new ModelAndView("admin/userList", Constants.USER_LIST, mgr.getUsers());
    }
}

public List<User> search(String query) {
    List<User> results = new ArrayList<User>();
    CompassDetachedHits hits = compassTemplate.findWithDetach(query);
    log.debug("No. of results for '" + query + "': " + hits.length());
    for (int i = 0; i < hits.length(); i++) {
        results.add((User) hits.data(i));
    }
    return results;
}

At first, I used compassTemplate.find(), but got an error because I wasn't using an OpenSessionInViewFilter. I decided to go with findWithDetach() and added the following search form to the top of the userList.jsp page:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms"/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

NOTE: I tried using HTML5's <input type="search">, but found Canoo WebTest doesn't support it.

Next, I wrote a unit test to verify everything worked as expected. I found I had to call compassGps.index() as part of my test to make sure my index was created and up-to-date.

public class UserControllerTest extends BaseControllerTestCase {
    @Autowired
    private CompassGps compassGps;
    @Autowired
    private UserController controller;

    public void testSearch() throws Exception {
        compassGps.index();
        ModelAndView mav = controller.handleRequest("admin");
        Map m = mav.getModel();
        List results = (List) m.get(Constants.USER_LIST);
        assertNotNull(results);
        assertTrue(results.size() >= 1);
        assertEquals("admin/userList", mav.getViewName());
    }
}

After getting this working, I started integrating similar code into AppFuse's other web framework modules (Struts, JSF and Tapestry). When I was finished, they all looked pretty similar from a UI perspective.

Struts:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms..."/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

JSF:

<div id="search">
<h:form id="searchForm">
    <h:inputText id="q" name="q" size="20" value="#{userList.query}"/>
    <h:commandButton value="#{text['button.search']}" action="#{userList.search}"/>
</h:form>
</div>

Tapestry:

<div id="search">
<t:form method="get" t:id="searchForm">
    <t:textfield size="20" name="q" t:id="q"/>
    <input t:type="submit" value="${message:button.search}"/>
</t:form>
</div>

One frustrating thing I found was that Tapestry doesn't support method="get" and AFAICT, neither does JSF 2. With JSF, I had to make my UserList bean session-scoped or the query parameter would be null when it listed the results. Tapestry took me the longest to implement, mainly because I had issues figuring out how it's easy-to-understand-once-you-know onSubmit() handlers worked and I had the proper @Property and @Persist annotations on my "q" property. This tutorial was the greatest help for me. Of course, now that it's all finished, the code looks pretty intuitive.

Feeling proud of myself for getting this working, I started integrating this feature into AppFuse's code generation and found I had to add quite a bit of code to the generated list pages/controllers.

So I went on a bike ride...

While riding, I thought of a much better solution and added the following search method to AppFuse's GenericManagerImpl.java. In the code I added to pages/controllers previously, I'd already refactored to use CompassSearchHelper and I continued to do so in the service layer implementation.

@Autowired
private CompassSearchHelper compass;

public List<T> search(String q, Class clazz) {
    if (q == null || "".equals(q.trim())) {
        return getAll();
    }

    List<T> results = new ArrayList<T>();

    CompassSearchCommand command = new CompassSearchCommand(q);
    CompassSearchResults compassResults = compass.search(command);
    CompassHit[] hits = compassResults.getHits();

    if (log.isDebugEnabled() && clazz != null) {
        log.debug("Filtering by type: " + clazz.getName());
    }

    for (CompassHit hit : hits) {
        if (clazz != null) {
            if (hit.data().getClass().equals(clazz)) {
                results.add((T) hit.data());
            }
        } else {
            results.add((T) hit.data());
        }
    }

    if (log.isDebugEnabled()) {
        log.debug("Number of results for '" + q + "': " + results.size());
    }

    return results;
}

This greatly simplified my page/controller logic because now all I had to do was call manager.search(query, User.class) instead of doing the Compass login in the controller. Of course, it'd be great if I didn't have to pass in the Class to filter by object, but that's the nature of generics and type erasure.

Other things I learned along the way:

• To index on startup, I added compassGps.index() to the StartupListener..
• In unit tests that leveraged transactions around methods, I had to call compassGps.index() before any transactions started.
• To scan multiple packages for searchable classes, I had to add a LocalCompassBeanPostProcessor.

But more than anything, I was reminded it always helps to take a bike ride when you don't like the design of your code.

This feature and many more will be in AppFuse 2.1, which I hope to finish by the end of the month. In the meantime, please feel free to try out the latest snapshot.

WebSockets with Johnny Wey at Denver JUG

This evening, I attended Denver JUG to hear Johnny Wey talk about WebSockets. This month, the location moved and even though I had a nice bike ride to the meeting, I showed up about 20 minutes late. Johnny's talk lasted about 40 minutes, so I missed the first half.

When I arrived, he was talking about workarounds for implementing push applications in browsers. He had a slide that talked about Comet and iframes as the common implementation, and the other major option being ActionScript's XMLSocket. The biggest issues with XMLSocket (according to Johnny) are:

• Not available on many modern mobile platforms.
• Flash and managing / detecting plugin versions can add unwanted complexity.
• Many would consider Flash solutions deprecated.

The biggest issue with implementing push on a client is managing it all, especially if you need to support older browsers. Socket.IO is one possible solution. It rides on the coattails of node.js. Features of Socket.IO include:

• Abstracts socket methods into a unified API.
• Open source (MIT) with active community.
• Multiple server implementations (including Java) with the "reference" implementation developed in node.js.

The client API looks as follows:

var socket = new io.Socket(); 
socket.on('connect', function(){ 
  socket.send('hi!'); 
}) 
socket.on('message', function(data){ 
  alert(data);
})
socket.on('disconnect', function(){}) 

jWebSocket is another solution and it's where a lot of the Java WebSocket development is ending up right now. Highlights about the project include:

• Open source (LGPL) with relatively active community.
• Servlet-like API.
• More "enterprisey" than Socket.IO.

Other options include CometD, which is a Dojo-driven Comet implementation that uses a specification called Bayeux. Jetty and GlassFish both support WebSockets in various forms of functionality and stability. Finally, there's Pusher (a SaaS implementation of push with a RESTful API) and Atmosphere (a container-agnostic framework).

How do you scale web sockets? The same way you make a webapp scale:

• Go stateless
• Use short request / response cycle
• Use the smallest payload possible
• Cache as much as possible

Scaling challenges with web sockets:

• Connections have intrinsic state (they never close!)
• Communications pipeline to your app server
• Some sort of introspection on LB side (JMX)

There's also some existing controversy in the WebSockets Community, mostly around using Upgrade vs. CONNECT with HTTP. An (IETF) experiment found Upgrade portion of HTTP protocol was often improperly implemented by proxy servers and other network hardware. This seems to have caused Google Chrome to deprecate using Upgrade in favor of CONNECT. CONNECT used in this manner is seen by many as an abuse of the web.

Other useful links that Johnny provided were What can I use… to find out native support across browsers. For example, you can see which browsers support websockets. He also pointed out that websocket.org provides a good intro to WebSockets.

I'm glad I attended Johnny's talk. I've been a little leery of using WebSockets in my applications because of older browsers. Now that I'm aware of frameworks (like Socket.IO) that solve this problem, I'm eager to try it when the need arises.

Related: Dojo/Comet support in Java Web Frameworks

JSR 303 and JVM Web Framework Support

Emmanuel Bernard recently sent an email to the JSR 303 Experts Group about the next revision of the Bean Validation JSR (303). Rather than sending the proposed changes privately, he blogged about them. I left a comment with what I'd like to see:

+1 for Client-side validation. I'd love to see an API that web frameworks can hook into to add "required" to their tags for HTML5. Or some service that can be registered so the client can make Ajax requests to an API to see if an object is valid.

Emmanuel replied that most of the necessary API already exists for this, but frameworks have been slow to adopt it.

Hi Matt,

The sad thing is that the API is present on the Bean Validation side but presentation frameworks are slow to adopt it and use it :(

RichFaces 4 now has support for it but I wished more presentation frameworks had worked on the integration. If you can convince a few people or have access to a few people, feel free to send them by me :)

The integration API is described here. Let me know if you think some parts are missing or should be improved. We should definitely do some more buzz around it.

In the interest of generating more buzz around it, I decided to do some research and see what JVM Frameworks support JSR 303. Here's what I've come up with so far (in no particular order):

• RichFaces 4 - Client Side Validation
• Tapestry's JSR 303 - Bean Validation Library
• Wicket JSR-303 Validators
• Configuring a JSR-303 Validator for use by Spring MVC
• GWT Validation
• Vaadin Bean Validation JSR 303 support
• Inline field validation in Scala/Lift using JPA and JSR 303

Struts 2 has an open issue, but doesn't seem to support JSR 303. Since I did a quick-n-dirty google search for most of these, I'm not sure if they support client-side JavaScript or HTML5's required. If you know of other JVM-based web frameworks that support JSR 303, please let me know in the comments.

Upcoming Conferences: TSSJS in Las Vegas and 33rd Degree in Kraków, Poland

It's that time of year again - the beginning of Conference Season. I generally like to speak at a few conferences a year and 2011 is no different. For March Madness, I'll be heading to Las Vegas to speak at TheServerSide Java Symposium. I'll be giving updated talks similar to the ones I gave at last year's Rich Web Experience in Fort Lauderdale:

• Everything You Ever Wanted To Know About Online Video
• Comparing JVM Web Frameworks

You might remember my Comparing JVM Web Frameworks talk from Devoxx 2010 and some of the interesting debate it caused. I've done some minor updates to my video presentation and some updates to my JVM Web Frameworks presentation as well. Most notably, I'll be including some findings from Peter Thomas's perfbench project. I also hope to update AppFuse to JSF 2 and integrate extensionless URLs in AppFuse Light. Marcin Zajaczkowski was nice enough to provide an upgrade to Wicket 1.4.15, so it'll be interesting to see how well Wicket supports extensionless URLs.

In April, I'll be presenting Comparing JVM Web Frameworks at the 33rd Degree Conference in Kraków, Poland. While I studied in Russia a couple summers in college, I've never been to Poland, so I'm really looking forward to this trip. With any luck, I'll have AppFuse 2.1 released by then and my knowledge of all its web frameworks' latest versions will be update-to-date. As you know, it's unlikely I'll recommend a best web framework (because there isn't one), but I hope to provide some techniques you can use to decide the best framework for your particular needs.

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and Über Conf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

If you're planning on attending TSSJS or 33rd Degree, hopefully I'll see you there.

Integration Testing with HTTP, HTTPS and Maven

Earlier this week, I was tasked with getting automated integration tests working in my project at Overstock.com. By automated, I mean that ability to run "mvn install" and have the following process cycled through:

• Start a container
• Deploy the application
• Run all integration tests
• Stop the container

Since it makes sense for integration tests to run in Maven's integration-test phase, I first configured the maven-surefire-plugin to skip tests in the test phase and execute them in the integration-test phase. I used the <id>default-phase</id> syntax to override the plugins' usual behavior.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <executions>
    <execution>
      <id>default-test</id>
      <configuration>
        <excludes>
          <exclude>**/*Test*.java</exclude>
        </excludes>
      </configuration>
    </execution>
    <execution>
      <id>default-integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/*Test.java</include>
        </includes>
        <excludes>
          <exclude>none</exclude>
          <exclude>**/TestCase.java</exclude>
        </excludes>
      </configuration>
    </execution>
  </executions>
</plugin>

After I had this working, I moved onto getting the container started and stopped properly. In the past, I've done this using Cargo and it's always worked well for me. Apart from the usual setup I use in AppFuse archetypes (example pom.xml), I added a couple additional items:

• Added <timeout>180000</timeout> so the container would wait up to 3 minutes for the WAR to deploy.
• In configuration/properties, specified <context.path>ROOT</context.path> so the app would deploy at the / context path.
• In configuration/properties, specified <cargo.protocol>https</cargo.protocol> since many existing unit tests made requests to secure resources.

I started by using Cargo with Tomcat and had to create certificate keystore in order to get Tomcat to start with SSL enabled. After getting it to start, I found the tests failed with the following errors in the logs:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: 
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1649)

Co-workers told me this was easily solved by adding my 'untrusted' cert to my JVM keystore. Once all this was working, I thought I was good to go, but found that some tests were still failing. The failures turned out to be because they were talking to http and https was the only protocol enabled. After doing some research, I discovered that Cargo doesn't support starting on both http and https ports.

So back to the drawing board I went. I ended up turning to the maven-jetty-plugin and the tomcat-maven-plugin to get the functionality I was looking for. I also automated the certificate keystore generation using the keytool-maven-plugin. Below is the extremely-verbose 95-line profiles section of my pom.xml that allows either container to be used.

Sidenote: I wonder how this same setup would look using Gradle?

<profiles>
  <profile>
    <id>jetty</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <build>
      <plugins>
        <plugin>
          <groupId>org.mortbay.jetty</groupId>
          <artifactId>maven-jetty-plugin</artifactId>
          <version>6.1.26</version>
          <configuration>
            <contextPath>/</contextPath>
            <connectors>
              <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <!-- forwarded == true interprets x-forwarded-* headers -->
                <!-- http://docs.codehaus.org/display/JETTY/Configuring+mod_proxy -->
                <forwarded>true</forwarded>
                <port>8080</port>
                <maxIdleTime>60000</maxIdleTime>
              </connector>
              <connector implementation="org.mortbay.jetty.security.SslSocketConnector">
                <forwarded>true</forwarded>
                <port>8443</port>
                <maxIdleTime>60000</maxIdleTime>
                <keystore>${project.build.directory}/ssl.keystore</keystore>
                <password>overstock</password>
                <keyPassword>overstock</keyPassword>
              </connector>
            </connectors>
            <stopKey>overstock</stopKey>
            <stopPort>9999</stopPort>
          </configuration>
          <executions>
            <execution>
              <id>start-jetty</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
              <configuration>
                <daemon>true</daemon>
              </configuration>
            </execution>
            <execution>
              <id>stop-jetty</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>stop</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
  <profile>
    <id>tomcat</id>
    <build>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>tomcat-maven-plugin</artifactId>
          <version>1.1</version>
          <configuration>
            <addContextWarDependencies>true</addContextWarDependencies>
            <fork>true</fork>
            <path>/</path>
            <port>8080</port>
            <httpsPort>8443</httpsPort>
            <keystoreFile>${project.build.directory}/ssl.keystore</keystoreFile>
            <keystorePass>overstock</keystorePass>
          </configuration>
          <executions>
            <execution>
              <id>start-tomcat</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
            </execution>
            <execution>
              <id>stop-tomcat</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>shutdown</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

With this setup in place, I was able to automate running our integration tests by simply typing "mvn install" (for Jetty) or "mvn install -Ptomcat" (for Tomcat). For running in Hudson, it's possible I'll have to further enhance things to randomize the port and pass that into tests as a system property. The build-helper-maven-plugin and its reserve-network-port goal is a nice way to do this. Note: if you want to run more than one instance of Tomcat at a time, you might have to randomize the ajp and rmi ports to avoid collisions.

The final thing I encountered was our app didn't shutdown gracefully. Luckily, this was fixed in a newer version of our core framework and upgrading fixed the problem. Here's the explanation from an architect on the core framework team.

The hanging problem was caused by the way the framework internally aggregated statistics related to database connection usage and page response times. The aggregation runs on a separate thread but not as a daemon thread. Previously, the aggregation threads weren't being terminated on shutdown so the JVM would hang waiting for them to finish. In the new frameworks, the aggregation threads are terminated on shutdown.

Hopefully this post helps you test your secure and unsecure applications at the same time. At the same time, I'm hoping it motivates the Cargo developers to add simultaneous http and https support.

Update: In the comments, Ron Piterman recommended I use the Maven Failsafe Plugin because its designed to run integration tests while Surefire Plugin is for unit tests. I changed my configuration to the following and everything still passes. Thanks Ron!

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <skipTests>true</skipTests>
  </configuration>
</plugin>
<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <includes>
      <include>**/*Test.java</include>
    </includes>
    <excludes>
      <exclude>**/TestCase.java</exclude>
    </excludes>
  </configuration>
  <executions>
    <execution>
      <id>integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <phase>verify</phase>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Update 2: In addition to application changes to solve hanging issues, I also had to change my Jetty Plugin configuration to use a different SSL connector implementation. This also required adding the jetty-sslengine dependency, which has been renamed to jetty-ssl for Jetty 7.

<connector implementation="org.mortbay.jetty.security.SslSelectChannelConnector">
...
<dependencies>
  <dependency>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-sslengine</artifactId>
    <version>6.1.26</version>
  </dependency>
</dependencies>

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.