Matt RaibleMatt Raible is a Web Architecture Consultant specializing in open source frameworks.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Fixing XSS in JSP 2

Way back in 2007, I wrote about Java Web Frameworks and XSS. My main point was that JSP EL doesn't bother to handle XSS.

Of course, the whole problem with JSP EL could be solved if Tomcat (and other containers) would allow a flag to turn on XML escaping by default. IMO, it's badly needed to make JSP-based webapps safe from XSS.

A couple months later, I proposed a Tomcat enhancement to escape JSP's EL by default. I also entered an enhancement request for this feature and attached a patch. That issue has remained open and unfixed for 3 and 1/2 years.

Yesterday, Chin Huang posted a handy-dandy ELResolver that XML-escapes EL values.

I tried Chin's resolver in AppFuse today and it works as well as advertised. To do this, I copied his EscapeXML*.java files into my project, changed the JSP API's Maven coordinates from javax.servlet:jsp-api:2.0 to javax.servlet.jsp:jsp-api:2.1 and added the listener to web.xml.

With Struts 2 and Spring MVC, I was previously able to have ${param.xss} and pass in ?xss=<script>alert('gotcha')</script> and it would show a JavaScript alert. After using Chin's ELResolver, it prints the string on the page instead of displaying an alert.

Thanks to Chin Huang for this patch! If you're using JSP, I highly recommend you add this to your projects as well.

Posted in Java at Feb 28 2011, 02:08:46 PM MST 7 Comments

Upcoming Conferences: TSSJS in Las Vegas and 33rd Degree in Kraków, Poland

It's that time of year again - the beginning of Conference Season. I generally like to speak at a few conferences a year and 2011 is no different. For March Madness, I'll be heading to Las Vegas to speak at TheServerSide Java Symposium. I'll be giving updated talks similar to the ones I gave at last year's Rich Web Experience in Fort Lauderdale:

You might remember my Comparing JVM Web Frameworks talk from Devoxx 2010 and some of the interesting debate it caused. I've done some minor updates to my video presentation and some updates to my JVM Web Frameworks presentation as well. Most notably, I'll be including some findings from Peter Thomas's perfbench project. I also hope to update AppFuse to JSF 2 and integrate extensionless URLs in AppFuse Light. Marcin Zajaczkowski was nice enough to provide an upgrade to Wicket 1.4.15, so it'll be interesting to see how well Wicket supports extensionless URLs.

In April, I'll be presenting Comparing JVM Web Frameworks at the 33rd Degree Conference in Kraków, Poland. While I studied in Russia a couple summers in college, I've never been to Poland, so I'm really looking forward to this trip. With any luck, I'll have AppFuse 2.1 released by then and my knowledge of all its web frameworks' latest versions will be update-to-date. As you know, it's unlikely I'll recommend a best web framework (because there isn't one), but I hope to provide some techniques you can use to decide the best framework for your particular needs.

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and Über Conf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

If you're planning on attending TSSJS or 33rd Degree, hopefully I'll see you there.

Posted in Java at Feb 25 2011, 03:14:59 PM MST 1 Comment

Implementing Ajax Authentication using jQuery, Spring Security and HTTPS

I've always had a keen interest in implementing security in webapps. I implemented container-managed authentication (CMA) in AppFuse in 2002, watched Tomcat improve it's implementation in 2003 and implemented Remember Me with CMA in 2004. In 2005, I switched from CMA to Acegi Security (now Spring Security) and never looked back. I've been very happy with Spring Security over the years, but also hope to learn more about Apache Shiro and implementing OAuth to protect JavaScript APIs in the near future.

I was recently re-inspired to learn more about security when working on a new feature at Overstock.com. The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience. User experience aside, I think it's interesting to see what's required to implement such a feature.

To demonstrate how we did it, I whipped up an example using AppFuse Light, jQuery and Spring Security. The source is available in my ajax-login project on GitHub. To begin, I wanted to accomplish a number of things to replicate the Overstock environment:

  1. Force HTTPS for authentication.
  2. Allow testing HTTPS without installing a certificate locally.
  3. Implement a RESTful LoginService that allows users to login.
  4. Implement login with Ajax, with the request coming from an insecure page.

Forcing HTTPS with Spring Security
The first feature was fairly easy to implement thanks to Spring Security. Its configuration supports a requires-channel attribute that can be used for this. I used this to force HTTPS on the "users" page and it subsequently causes the login to be secure.

<intercept-url pattern="/app/users" access="ROLE_ADMIN" requires-channel="https"/>

Testing HTTPS without adding a certificate locally
After making the above change in security.xml, I had to modify my jWebUnit test to work with SSL. In reality, I didn't have to modify the test, I just had to modify the configuration that ran the test. In my last post, I wrote about adding my 'untrusted' cert to my JVM keystore. For some reason, this works for HttpClient, but not for jWebUnit/HtmlUnit. The good news is I figured out an easier solution - adding the trustStore and trustStore password as system properties to the maven-failsafe-plugin configuration.

<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7.2</version>
<configuration>
    <includes>
        <include>**/*WebTest.java</include>
    </includes>
    <systemPropertyVariables>
      <javax.net.ssl.trustStore>${project.build.directory}/ssl.keystore</javax.net.ssl.trustStore>
      <javax.net.ssl.trustStorePassword>appfuse</javax.net.ssl.trustStorePassword>
    </systemPropertyVariables>
</configuration>

The disadvantage to doing things this way is you'll have to pass these in as arguments when running unit tests in your IDE.

Implementing a LoginService
Next, I set about implementing a LoginService as a Spring MVC Controller that returns JSON thanks to the @ResponseBody annotation and Jackson.

package org.appfuse.examples.web;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
@RequestMapping("/api/login.json")
public class LoginService {

  @Autowired
  @Qualifier("authenticationManager")
  AuthenticationManager authenticationManager;

  @RequestMapping(method = RequestMethod.GET)
  @ResponseBody
  public LoginStatus getStatus() {
    Authentication auth = SecurityContextHolder.getContext().getAuthentication();
    if (auth != null && !auth.getName().equals("anonymousUser") && auth.isAuthenticated()) {
      return new LoginStatus(true, auth.getName());
    } else {
      return new LoginStatus(false, null);
    }
  }

  @RequestMapping(method = RequestMethod.POST)
  @ResponseBody
  public LoginStatus login(@RequestParam("j_username") String username,
                           @RequestParam("j_password") String password) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    User details = new User(username);
    token.setDetails(details);

    try {
      Authentication auth = authenticationManager.authenticate(token);
      SecurityContextHolder.getContext().setAuthentication(auth);
      return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
      return new LoginStatus(false, null);
    }
  }

  public class LoginStatus {

    private final boolean loggedIn;
    private final String username;

    public LoginStatus(boolean loggedIn, String username) {
      this.loggedIn = loggedIn;
      this.username = username;
    }

    public boolean isLoggedIn() {
      return loggedIn;
    }

    public String getUsername() {
      return username;
    }
  }
}

To verify this class worked as expected, I wrote a unit test using JUnit and Mockito. I used Mockito because Overstock is transitioning to it from EasyMock and I've found it very simple to use.

package org.appfuse.examples.web;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Matchers;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.TestingAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.context.SecurityContextImpl;

import static org.junit.Assert.*;
import static org.mockito.Mockito.*;

public class LoginServiceTest {

  LoginService loginService;
  AuthenticationManager authenticationManager;

  @Before
  public void before() {
    loginService = new LoginService();
    authenticationManager = mock(AuthenticationManager.class);
    loginService.authenticationManager = authenticationManager;
  }

  @After
  public void after() {
    SecurityContextHolder.clearContext();
  }

  @Test
  public void testLoginStatusSuccess() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    SecurityContext context = new SecurityContextImpl();
    context.setAuthentication(auth);
    SecurityContextHolder.setContext(context);

    LoginService.LoginStatus status = loginService.getStatus();
    assertTrue(status.isLoggedIn());
  }

  @Test
  public void testLoginStatusFailure() {
    LoginService.LoginStatus status = loginService.getStatus();
    assertFalse(status.isLoggedIn());
  }

  @Test
  public void testGoodLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    when(authenticationManager.authenticate(Matchers.<Authentication>anyObject())).thenReturn(auth);
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertTrue(status.isLoggedIn());
    assertEquals("foo", status.getUsername());
  }

  @Test
  public void testBadLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(false);
    when(authenticationManager.authenticate(Matchers.anyObject()))
        .thenThrow(new BadCredentialsException("Bad Credentials"));
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertFalse(status.isLoggedIn());
    assertEquals(null, status.getUsername());
  }
}

Implement login with Ajax
The last feature was the hardest to implement and still isn't fully working as I'd hoped. I used jQuery and jQuery UI to implement a dialog that opens the login page on the same page rather than redirecting to the login page. The "#demo" locator refers to a button in the page.

Passing in the "ajax=true" parameter disables SiteMesh decoration on the login page, something that's described in my Ajaxified Body article.

var dialog = $('<div></div>');

$(document).ready(function() {
    $.get('/login?ajax=true', function(data) {
        dialog.html(data);
        dialog.dialog({
            autoOpen: false,
	       title: 'Authentication Required'
        });
    });

    $('#demo').click(function() {
      dialog.dialog('open');
      // prevent the default action, e.g., following a link
      return false;
    });
});

Instead of adding a click handler to a specific id, it's probably better to use a CSS class that indicates authentication is required for a link, or -- even better -- use Ajax to see if the link is secured.

The login page then has the following JavaScript to add a click handler to the "login" button that submits the request securely to the LoginService.

var getHost = function() {
    var port = (window.location.port == "8080") ? ":8443" : "";
    return ((secure) ? 'https://' : 'http://') + window.location.hostname + port;
};

var loginFailed = function(data, status) {
    $(".error").remove();
    $('#username-label').before('<div class="error">Login failed, please try again.</div>');
};

$("#login").live('click', function(e) {
    e.preventDefault();
    $.ajax({url: getHost() + "/api/login.json",
        type: "POST",
        data: $("#loginForm").serialize(),
        success: function(data, status) {
            if (data.loggedIn) {
                // success
                dialog.dialog('close');
                location.href= getHost() + '/users';
            } else {
                loginFailed(data);
            }
        },
        error: loginFailed
    });
});

The biggest secret to making this all work (the HTTP -> HTTPS communication, which is considered cross-domain), is the window.name Transport and the jQuery plugin that implements it. To make this plugin work with Firefox 3.6, I had to implement a Filter that adds Access-Control headers. A question on Stackoverflow helped me figure this out.

public class OptionsHeadersFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;

        response.setHeader("Access-Control-Allow-Origin", "*");
        response.setHeader("Access-Control-Allow-Methods", "GET,POST");
        response.setHeader("Access-Control-Max-Age", "360");
        response.setHeader("Access-Control-Allow-Headers", "x-requested-with");

        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {
    }

    public void destroy() {
    }
}

Issues
I encountered a number of issues when implementing this in the ajax-login project.

  • If you try to run this with ports (e.g. 8080 and 8443) in your URLs, you'll get a 501 (Not Implemented) response. Removing the ports by fronting with Apache and mod_proxy solves this problem.
  • If you haven't accepted the certificate in your browser, the Ajax request will fail. In the example, I solved this by clicking on the "Users" tab to make a secure request, then going back to the homepage to try and login.
  • The jQuery window.name version 0.9.1 doesn't work with jQuery 1.5.0. The error is "$.httpSuccess function not found."
  • Finally, even though I was able to authenticate successfully, I was unable to make the authentication persist. I tried adding the following to persist the updated SecurityContext to the session, but it doesn't work. I expect the solution is to create a secure JSESSIONID cookie somehow.
    @Autowired
    SecurityContextRepository repository;
    
    @RequestMapping(method = RequestMethod.POST)
    @ResponseBody
    public LoginStatus login(@RequestParam("j_username") String username,
                             @RequestParam("j_password") String password,
                             HttpServletRequest request, HttpServletResponse response) {
    
        UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
        ...
    
        try {
            Authentication auth = authenticationManager.authenticate(token);
            SecurityContextHolder.getContext().setAuthentication(auth);
            // save the updated context to the session
            repository.saveContext(SecurityContextHolder.getContext(), request, response);
            return new LoginStatus(auth.isAuthenticated(), auth.getName());
        } catch (BadCredentialsException e) {
            return new LoginStatus(false, null);
        }
    }
    

Conclusion
This article has shown you how to force HTTPS for login, how to do integration testing with a self-generated certificate, how to implement a LoginService with Spring MVC and Spring Security, as well as how to use jQuery to talk to a service cross-domain with the window.name Transport. While I don't have everything working as much as I'd like, I hope this helps you implement a similar feature in your applications.

One thing to be aware of is with lightbox/dialog logins and HTTP -> HTTPS is that users won't see a secure icon in their address bar. If your app has sensitive data, you might want to force https for your entire app. OWASP's Secure Login Pages has a lot of good tips in this area.

Update: I've posted a demo of the ajax-login webapp. Thanks to Contegix for hosting the demo and helping obtain/install an SSL certificate so quickly.

Posted in Java at Feb 23 2011, 04:55:55 PM MST 12 Comments

Integration Testing with HTTP, HTTPS and Maven

Earlier this week, I was tasked with getting automated integration tests working in my project at Overstock.com. By automated, I mean that ability to run "mvn install" and have the following process cycled through:

  • Start a container
  • Deploy the application
  • Run all integration tests
  • Stop the container

Since it makes sense for integration tests to run in Maven's integration-test phase, I first configured the maven-surefire-plugin to skip tests in the test phase and execute them in the integration-test phase. I used the <id>default-phase</id> syntax to override the plugins' usual behavior.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <executions>
    <execution>
      <id>default-test</id>
      <configuration>
        <excludes>
          <exclude>**/*Test*.java</exclude>
        </excludes>
      </configuration>
    </execution>
    <execution>
      <id>default-integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/*Test.java</include>
        </includes>
        <excludes>
          <exclude>none</exclude>
          <exclude>**/TestCase.java</exclude>
        </excludes>
      </configuration>
    </execution>
  </executions>
</plugin>

After I had this working, I moved onto getting the container started and stopped properly. In the past, I've done this using Cargo and it's always worked well for me. Apart from the usual setup I use in AppFuse archetypes (example pom.xml), I added a couple additional items:

  • Added <timeout>180000</timeout> so the container would wait up to 3 minutes for the WAR to deploy.
  • In configuration/properties, specified <context.path>ROOT</context.path> so the app would deploy at the / context path.
  • In configuration/properties, specified <cargo.protocol>https</cargo.protocol> since many existing unit tests made requests to secure resources.

I started by using Cargo with Tomcat and had to create certificate keystore in order to get Tomcat to start with SSL enabled. After getting it to start, I found the tests failed with the following errors in the logs:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: 
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1649)

Co-workers told me this was easily solved by adding my 'untrusted' cert to my JVM keystore. Once all this was working, I thought I was good to go, but found that some tests were still failing. The failures turned out to be because they were talking to http and https was the only protocol enabled. After doing some research, I discovered that Cargo doesn't support starting on both http and https ports.

So back to the drawing board I went. I ended up turning to the maven-jetty-plugin and the tomcat-maven-plugin to get the functionality I was looking for. I also automated the certificate keystore generation using the keytool-maven-plugin. Below is the extremely-verbose 95-line profiles section of my pom.xml that allows either container to be used.

Sidenote: I wonder how this same setup would look using Gradle?

<profiles>
  <profile>
    <id>jetty</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <build>
      <plugins>
        <plugin>
          <groupId>org.mortbay.jetty</groupId>
          <artifactId>maven-jetty-plugin</artifactId>
          <version>6.1.26</version>
          <configuration>
            <contextPath>/</contextPath>
            <connectors>
              <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <!-- forwarded == true interprets x-forwarded-* headers -->
                <!-- http://docs.codehaus.org/display/JETTY/Configuring+mod_proxy -->
                <forwarded>true</forwarded>
                <port>8080</port>
                <maxIdleTime>60000</maxIdleTime>
              </connector>
              <connector implementation="org.mortbay.jetty.security.SslSocketConnector">
                <forwarded>true</forwarded>
                <port>8443</port>
                <maxIdleTime>60000</maxIdleTime>
                <keystore>${project.build.directory}/ssl.keystore</keystore>
                <password>overstock</password>
                <keyPassword>overstock</keyPassword>
              </connector>
            </connectors>
            <stopKey>overstock</stopKey>
            <stopPort>9999</stopPort>
          </configuration>
          <executions>
            <execution>
              <id>start-jetty</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
              <configuration>
                <daemon>true</daemon>
              </configuration>
            </execution>
            <execution>
              <id>stop-jetty</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>stop</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
  <profile>
    <id>tomcat</id>
    <build>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>tomcat-maven-plugin</artifactId>
          <version>1.1</version>
          <configuration>
            <addContextWarDependencies>true</addContextWarDependencies>
            <fork>true</fork>
            <path>/</path>
            <port>8080</port>
            <httpsPort>8443</httpsPort>
            <keystoreFile>${project.build.directory}/ssl.keystore</keystoreFile>
            <keystorePass>overstock</keystorePass>
          </configuration>
          <executions>
            <execution>
              <id>start-tomcat</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
            </execution>
            <execution>
              <id>stop-tomcat</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>shutdown</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

With this setup in place, I was able to automate running our integration tests by simply typing "mvn install" (for Jetty) or "mvn install -Ptomcat" (for Tomcat). For running in Hudson, it's possible I'll have to further enhance things to randomize the port and pass that into tests as a system property. The build-helper-maven-plugin and its reserve-network-port goal is a nice way to do this. Note: if you want to run more than one instance of Tomcat at a time, you might have to randomize the ajp and rmi ports to avoid collisions.

The final thing I encountered was our app didn't shutdown gracefully. Luckily, this was fixed in a newer version of our core framework and upgrading fixed the problem. Here's the explanation from an architect on the core framework team.

The hanging problem was caused by the way the framework internally aggregated statistics related to database connection usage and page response times. The aggregation runs on a separate thread but not as a daemon thread. Previously, the aggregation threads weren't being terminated on shutdown so the JVM would hang waiting for them to finish. In the new frameworks, the aggregation threads are terminated on shutdown.

Hopefully this post helps you test your secure and unsecure applications at the same time. At the same time, I'm hoping it motivates the Cargo developers to add simultaneous http and https support. ;)

Update: In the comments, Ron Piterman recommended I use the Maven Failsafe Plugin because its designed to run integration tests while Surefire Plugin is for unit tests. I changed my configuration to the following and everything still passes. Thanks Ron!

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <skipTests>true</skipTests>
  </configuration>
</plugin>
<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <includes>
      <include>**/*Test.java</include>
    </includes>
    <excludes>
      <exclude>**/TestCase.java</exclude>
    </excludes>
  </configuration>
  <executions>
    <execution>
      <id>integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <phase>verify</phase>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Update 2: In addition to application changes to solve hanging issues, I also had to change my Jetty Plugin configuration to use a different SSL connector implementation. This also required adding the jetty-sslengine dependency, which has been renamed to jetty-ssl for Jetty 7.

<connector implementation="org.mortbay.jetty.security.SslSelectChannelConnector">
...
<dependencies>
  <dependency>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-sslengine</artifactId>
    <version>6.1.26</version>
  </dependency>
</dependencies>

Posted in Java at Feb 11 2011, 03:54:16 PM MST 9 Comments

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Filter NameDispatchers
securityFilter request
rewriteFilter request, forward
struts-prepare request
sitemesh request, forward, include
staticFilter request, forward
struts request

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

Filter NameDispatchers
rewriteFilter request
securityFilter request, forward, include
struts-prepare request, forward
sitemesh request, forward, include
struts forward

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.

Posted in Java at Feb 10 2011, 04:53:27 PM MST 10 Comments

Crested Butte

Last week was a great one for my ski season and happiness meter. First of all, I had the pleasure of skiing Alta for the first time while traveling to Overstock.com. I started working early on an overcast day and was delighted to drive into sunny blue skies above the clouds of Salt Lake City. I never expected it to be so nice and was super impressed with the mountain and its powder stashes.

I flew back to Denver on Thursday night, had a gorgeous lunch ride in 70°F weather on Friday afternoon, then picked up my lovely girlfriend Trish and drove to Crested Butte for a weekend of skiing with James Ward and friends. It was my first time visiting Crested Butte and I can see why people call it Paradise.

We stayed at a sweet location right in town, ate some exquisite cuisine and had some of the most beautiful days I've ever had skiing. Hopefully the pictures below convey how beautiful it was. You can also checkout the full set on Flickr.

Crested Butte, The Town Top of Crested Butte

The Steep Stuff Blue Sky

Trish and I are up to 17 ski days for the year so far, and hoping to hit 20 by Valentine's Day. Not bad for a couple Flatlanders. ;)

Posted in General at Feb 01 2011, 05:57:41 PM MST 1 Comment