Matt RaibleMatt Raible is a Web Developer and Java Champion. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Livin' it up in Vegas at TSSJS 2011

Last Wednesday, Trish and I traveled to Las Vegas for TheServerSide Java Symposium 2011 conference. We had a free room from TechTarget, but opted to upgrade to a suite with a view over the Bellagio Fountains. Trish won a trip to Vegas as a sales award earlier in the year and cleverly exchanged it for cash, so our upgrade was sort of free.

Caesars Pool The Bellagio Fountains

My first talk was on Online Video and my experience at Time Warner Cable. With my former team's iPad app releasing the day before, it was a fun session. The attendance was kind of sparse, but I had some good competition so wasn't surprised.

After I finished speaking, we headed to happy hour and met up with some friends that happened to be in town. We had dinner at the Todd English Pub and headed to the Penn & Teller show at the Rio. We closed the night after Trish had a 45-minute roll at the craps table at O'Sheas.

We slept in on Thursday and I gave my Comparing JVM Web Frameworks talk that afternoon. I made sure to mention some other methods to choosing web frameworks: doing performance comparisons like Peter Thomas has done or choosing Lift because one of its developers says it's the best. While Vaadin did sneak into the #5 spot, I made sure and mentioned that Wicket and Tapestry seem to belong there moreso (based on stats, mailing list traffic, etc.).

Trish took a bunch of pictures during my talk, which had a great turnout and lots of participation.

Getting Intro'd My Intro My Dream on Display

The Problem How do you choose? Choosing a Framework

That evening, we celebrated St. Patty's Day with some college buddies of mine, ate great sushi at Mizuya and experienced the joys of three card poker. Thanks to TechTarget for inviting me to TSSJS 2011; we had an awesome time. You can find all the pictures we took on Flickr.

P.S. If you can't see the presentations in this post (a.k.a. you don't have Flash), you can view them on on Slideshare or download the PDFs.

Posted in Java at Mar 22 2011, 09:04:17 AM MDT Add a Comment

Adding Search to AppFuse with Compass

Over 5 years ago, I recognized that AppFuse needed to have a search feature and entered an issue in JIRA. Almost 4 years later, a Compass Tutorial was created and shortly after Shay Banon (Compass Founder), sent in a patch. From the message he sent me:

A quick breakdown of enabling search:

  1. Added Searchable annotations to the User and Address.
  2. Defined Compass bean, automatically scanning the model package for mapped searchable classes. It also automatically integrates with Spring transaction manager, and stores the index on the file system ([work dir]/target/test-index).
  3. Defined CompassTemplate (similar in concept to HibernateTemplate).
  4. Defined CompassSearchHelper. Really helps to perform search since it does pagination and so on.
  5. Defined CompassGps, basically it allows for index operation allowing to completely reindex the data from the database. JPA and Hiberante also automatically mirror changes done through their API to the index. iBatis uses AOP.

Fast forward 2 years and I finally found the time/desire to put a UI on the backend Compass implementation that Shay provided. Yes, I realize that Compass is being replaced by ElasticSearch. I may change to use ElasticSearch in the future; now that the search feature exists, I hope to see it evolve and improve.

Since Shay's patch integrated the necessary Spring beans for indexing and searching, the only thing I had to do was to implement the UI. Rather than having an "all objects" results page, I elected to implement it so you could search on an entity's list screen. I started with Spring MVC and added a search() method to the UserController:

@RequestMapping(method = RequestMethod.GET)
public ModelAndView handleRequest(@RequestParam(required = false, value = "q") String query) throws Exception {
    if (query != null && !"".equals(query.trim())) {
        return new ModelAndView("admin/userList", Constants.USER_LIST, search(query));
    } else {
        return new ModelAndView("admin/userList", Constants.USER_LIST, mgr.getUsers());
    }
}

public List<User> search(String query) {
    List<User> results = new ArrayList<User>();
    CompassDetachedHits hits = compassTemplate.findWithDetach(query);
    log.debug("No. of results for '" + query + "': " + hits.length());
    for (int i = 0; i < hits.length(); i++) {
        results.add((User) hits.data(i));
    }
    return results;
}

At first, I used compassTemplate.find(), but got an error because I wasn't using an OpenSessionInViewFilter. I decided to go with findWithDetach() and added the following search form to the top of the userList.jsp page:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms"/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

NOTE: I tried using HTML5's <input type="search">, but found Canoo WebTest doesn't support it.

Next, I wrote a unit test to verify everything worked as expected. I found I had to call compassGps.index() as part of my test to make sure my index was created and up-to-date.

public class UserControllerTest extends BaseControllerTestCase {
    @Autowired
    private CompassGps compassGps;
    @Autowired
    private UserController controller;

    public void testSearch() throws Exception {
        compassGps.index();
        ModelAndView mav = controller.handleRequest("admin");
        Map m = mav.getModel();
        List results = (List) m.get(Constants.USER_LIST);
        assertNotNull(results);
        assertTrue(results.size() >= 1);
        assertEquals("admin/userList", mav.getViewName());
    }
}

After getting this working, I started integrating similar code into AppFuse's other web framework modules (Struts, JSF and Tapestry). When I was finished, they all looked pretty similar from a UI perspective.

Struts:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms..."/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

JSF:

<div id="search">
<h:form id="searchForm">
    <h:inputText id="q" name="q" size="20" value="#{userList.query}"/>
    <h:commandButton value="#{text['button.search']}" action="#{userList.search}"/>
</h:form>
</div>

Tapestry:

<div id="search">
<t:form method="get" t:id="searchForm">
    <t:textfield size="20" name="q" t:id="q"/>
    <input t:type="submit" value="${message:button.search}"/>
</t:form>
</div>

One frustrating thing I found was that Tapestry doesn't support method="get" and AFAICT, neither does JSF 2. With JSF, I had to make my UserList bean session-scoped or the query parameter would be null when it listed the results. Tapestry took me the longest to implement, mainly because I had issues figuring out how it's easy-to-understand-once-you-know onSubmit() handlers worked and I had the proper @Property and @Persist annotations on my "q" property. This tutorial was the greatest help for me. Of course, now that it's all finished, the code looks pretty intuitive.

Feeling proud of myself for getting this working, I started integrating this feature into AppFuse's code generation and found I had to add quite a bit of code to the generated list pages/controllers.

So I went on a bike ride...

While riding, I thought of a much better solution and added the following search method to AppFuse's GenericManagerImpl.java. In the code I added to pages/controllers previously, I'd already refactored to use CompassSearchHelper and I continued to do so in the service layer implementation.

@Autowired
private CompassSearchHelper compass;

public List<T> search(String q, Class clazz) {
    if (q == null || "".equals(q.trim())) {
        return getAll();
    }

    List<T> results = new ArrayList<T>();

    CompassSearchCommand command = new CompassSearchCommand(q);
    CompassSearchResults compassResults = compass.search(command);
    CompassHit[] hits = compassResults.getHits();

    if (log.isDebugEnabled() && clazz != null) {
        log.debug("Filtering by type: " + clazz.getName());
    }

    for (CompassHit hit : hits) {
        if (clazz != null) {
            if (hit.data().getClass().equals(clazz)) {
                results.add((T) hit.data());
            }
        } else {
            results.add((T) hit.data());
        }
    }

    if (log.isDebugEnabled()) {
        log.debug("Number of results for '" + q + "': " + results.size());
    }

    return results;
}

This greatly simplified my page/controller logic because now all I had to do was call manager.search(query, User.class) instead of doing the Compass login in the controller. Of course, it'd be great if I didn't have to pass in the Class to filter by object, but that's the nature of generics and type erasure.

Other things I learned along the way:

  • To index on startup, I added compassGps.index() to the StartupListener..
  • In unit tests that leveraged transactions around methods, I had to call compassGps.index() before any transactions started.
  • To scan multiple packages for searchable classes, I had to add a LocalCompassBeanPostProcessor.

But more than anything, I was reminded it always helps to take a bike ride when you don't like the design of your code. ;-)

This feature and many more will be in AppFuse 2.1, which I hope to finish by the end of the month. In the meantime, please feel free to try out the latest snapshot.

Posted in Java at Mar 15 2011, 05:11:12 PM MDT 1 Comment

WebSockets with Johnny Wey at Denver JUG

This evening, I attended Denver JUG to hear Johnny Wey talk about WebSockets. This month, the location moved and even though I had a nice bike ride to the meeting, I showed up about 20 minutes late. Johnny's talk lasted about 40 minutes, so I missed the first half.

When I arrived, he was talking about workarounds for implementing push applications in browsers. He had a slide that talked about Comet and iframes as the common implementation, and the other major option being ActionScript's XMLSocket. The biggest issues with XMLSocket (according to Johnny) are:

  • Not available on many modern mobile platforms.
  • Flash and managing / detecting plugin versions can add unwanted complexity.
  • Many would consider Flash solutions deprecated.

The biggest issue with implementing push on a client is managing it all, especially if you need to support older browsers. Socket.IO is one possible solution. It rides on the coattails of node.js. Features of Socket.IO include:

  • Abstracts socket methods into a unified API.
  • Open source (MIT) with active community.
  • Multiple server implementations (including Java) with the "reference" implementation developed in node.js.

The client API looks as follows:

var socket = new io.Socket(); 
socket.on('connect', function(){ 
  socket.send('hi!'); 
}) 
socket.on('message', function(data){ 
  alert(data);
})
socket.on('disconnect', function(){}) 

jWebSocket is another solution and it's where a lot of the Java WebSocket development is ending up right now. Highlights about the project include:

  • Open source (LGPL) with relatively active community.
  • Servlet-like API.
  • More "enterprisey" than Socket.IO.

Other options include CometD, which is a Dojo-driven Comet implementation that uses a specification called Bayeux. Jetty and GlassFish both support WebSockets in various forms of functionality and stability. Finally, there's Pusher (a SaaS implementation of push with a RESTful API) and Atmosphere (a container-agnostic framework).

How do you scale web sockets? The same way you make a webapp scale:

  • Go stateless
  • Use short request / response cycle
  • Use the smallest payload possible
  • Cache as much as possible

Scaling challenges with web sockets:

  • Connections have intrinsic state (they never close!)
  • Communications pipeline to your app server
  • Some sort of introspection on LB side (JMX)

There's also some existing controversy in the WebSockets Community, mostly around using Upgrade vs. CONNECT with HTTP. An (IETF) experiment found Upgrade portion of HTTP protocol was often improperly implemented by proxy servers and other network hardware. This seems to have caused Google Chrome to deprecate using Upgrade in favor of CONNECT. CONNECT used in this manner is seen by many as an abuse of the web.

Other useful links that Johnny provided were What can I use… to find out native support across browsers. For example, you can see which browsers support websockets. He also pointed out that websocket.org provides a good intro to WebSockets.

I'm glad I attended Johnny's talk. I've been a little leery of using WebSockets in my applications because of older browsers. Now that I'm aware of frameworks (like Socket.IO) that solve this problem, I'm eager to try it when the need arises.

Related: Dojo/Comet support in Java Web Frameworks

Posted in Java at Mar 09 2011, 07:10:12 PM MST Add a Comment

JSR 303 and JVM Web Framework Support

Emmanuel Bernard recently sent an email to the JSR 303 Experts Group about the next revision of the Bean Validation JSR (303). Rather than sending the proposed changes privately, he blogged about them. I left a comment with what I'd like to see:

+1 for Client-side validation. I'd love to see an API that web frameworks can hook into to add "required" to their tags for HTML5. Or some service that can be registered so the client can make Ajax requests to an API to see if an object is valid.

Emmanuel replied that most of the necessary API already exists for this, but frameworks have been slow to adopt it.

Hi Matt,

The sad thing is that the API is present on the Bean Validation side but presentation frameworks are slow to adopt it and use it :(

RichFaces 4 now has support for it but I wished more presentation frameworks had worked on the integration. If you can convince a few people or have access to a few people, feel free to send them by me :)

The integration API is described here. Let me know if you think some parts are missing or should be improved. We should definitely do some more buzz around it.

In the interest of generating more buzz around it, I decided to do some research and see what JVM Frameworks support JSR 303. Here's what I've come up with so far (in no particular order):

Struts 2 has an open issue, but doesn't seem to support JSR 303. Since I did a quick-n-dirty google search for most of these, I'm not sure if they support client-side JavaScript or HTML5's required. If you know of other JVM-based web frameworks that support JSR 303, please let me know in the comments.

Posted in Java at Mar 08 2011, 11:33:24 AM MST 4 Comments

Upgrading to JSF 2

Last week, I spent a few hours upgrading AppFuse from JSF 1.2 to JSF 2.0. In reality, I upgraded from MyFaces 1.2.7 to 2.0.4, but all JSF implementations should be the same, right? All in all, it was a pretty easy upgrade with a few minor AppFuse-specific things. My goal in upgrading was to do the bare minimum to get things working and to leave integration of JSF 2 features for a later date.

In addition to upgrading MyFaces, I had to upgrade Tomahawk by changing the dependency's artifactId to tomahawk20. I was also able to remove the following listener from my web.xml:

<listener>
    <listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
<listener>

After that, I discovered that MyFaces uses a new URI (/javax.faces.resource/) for serving up some of its resource files. I kindly asked Spring Security to ignore these requests by adding the following to my security.xml file.

<intercept-url pattern="/javax.faces.resource/**" filters="none"/>

Since JSF 2 includes Facelets by default, I tried removing Facelets as a dependency. After doing this, I received the following error:

ERROR [308855416@qtp-120902214-7] ViewHandlerWrapper.fillChain(158) | Error instantiation parent Faces ViewHandler
java.lang.ClassNotFoundException: com.sun.facelets.FaceletViewHandler
        at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:401)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at org.ajax4jsf.framework.ViewHandlerWrapper.fillChain(ViewHandlerWrapper.java:144)
        at org.ajax4jsf.framework.ViewHandlerWrapper.calculateRenderKitId(ViewHandlerWrapper.java:68)
        at org.apache.myfaces.lifecycle.DefaultRestoreViewSupport.isPostback(DefaultRestoreViewSupport.java:179)
        at org.apache.myfaces.lifecycle.RestoreViewExecutor.execute(RestoreViewExecutor.java:113)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:171)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Figuring this was caused by the following element in my web.xml ...

<context-param>
    <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
    <param-value>com.sun.facelets.FaceletViewHandler</param-value>
</context-param>

... I removed it and tried again. This time I received a NoClassDefFoundError:

java.lang.NoClassDefFoundError: com/sun/facelets/tag/TagHandler
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:392)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.myfaces.shared_impl.util.ClassUtils.classForName(ClassUtils.java:184)
        at org.apache.myfaces.view.facelets.util.ReflectionUtil.forName(ReflectionUtil.java:67)

Since everything seemed to work with Facelets in the classpath, I decided to save this headache for a later date. I entered two issues in AppFuse's JIRA, one for removing Facelets and one for replacing Ajax4JSF with RichFaces.

The next issue I encountered was redirecting from AppFuse's password hint page. The navigation-rule for this page is as follows:

<navigation-rule>
    <from-view-id>/passwordHint.xhtml</from-view-id>
    <navigation-case>
        <from-outcome>success</from-outcome>
        <to-view-id>/login</to-view-id>
        <redirect/>
    </navigation-case>
</navigation-rule>

With JSF 2.0, the rule changes the URL to /login.xhtml when redirecting (where it was left as /login with 1.2) and it was caught by the security setting in my web.xml that prevents users from viewing raw templates.

<security-constraint>
    <web-resource-collection>
        <web-resource-name>Protect XHTML Templates</web-resource-name>
        <url-pattern>*.xhtml</url-pattern>
    </web-resource-collection>
    <auth-constraint/>
</security-constraint>

To solve this issue, I had to make a couple of changes:

  • Comment out the security-constraint in web.xml and move it to Spring Security's security.xml file.
    <intercept-url pattern="/**/*.xhtml" access="ROLE_NOBODY"/>
    
  • Add a rule to urlrewrite.xml that redirects back to login (since login.xhtml doesn't exist and I'm using extensionless URLs).
    <rule match-type="regex">
        <from>^/login.xhtml$</from>
        <to type="redirect">%{context-path}/login</to>
    </rule>
    

After getting the Password Hint feature passing in the browser, I tried running the integration tests (powered by Canoo WebTest). The Password Hint test kept failing with the following error:

[ERROR] /Users/mraible/dev/appfuse/web/jsf/src/test/resources/web-tests.xml:51: JavaScript error loading
page http://localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/passwordHint?username=admin: syntax error (http://
localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/javax.faces.resource/oamSubmit.js.jsf?ln=org.apache.myfaces#122)

Figuring this was caused by my hack to submit the form when the page was loaded, I turned to Pretty Faces, which allows you to call a method directly from a URL. After adding the Pretty Faces dependencies to my pom.xml, I created a src/main/webapp/WEB-INF/pretty-config.xml file with the following XML:

<url-mapping>
    <pattern value="/editProfile"/>
    <view-id value="/userForm.jsf"/>
    <action>#{userForm.edit}</action>
</url-mapping>

<url-mapping>
    <pattern value="/passwordHint/#{username}"/>
    <view-id value="/passwordHint.jsf"/>
    <action>#{passwordHint.execute}</action>
</url-mapping>

This allowed me to remove both editProfile.xhtml and passwordHint.xhtml, both of which simply auto-submitted forms.

At this point, I figured I'd be good to go and ran my integration tests again. The first thing I discovered was that ".jsf" was being tacked onto my pretty URL, most likely by the UrlRewriteFilter. Adding the following to my PasswordHint.java class solved this.

if (username.endsWith(".jsf")) {
    username = username.substring(0, username.indexOf(".jsf"));
}

The next thing was a cryptic error that took me a while to figure out.

DEBUG [1152467051@qtp-144702232-0] PasswordHint.execute(38) | Processing Password Hint...
2011-03-05 05:48:52.471:WARN::/passwordHint/admin
com.ocpsoft.pretty.PrettyException: Exception occurred while processing <:#{passwordHint.execute}> null
        at com.ocpsoft.pretty.faces.beans.ActionExecutor.executeActions(ActionExecutor.java:71)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.processEvent(PrettyPhaseListener.java:214)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.afterPhase(PrettyPhaseListener.java:108)
        at org.apache.myfaces.lifecycle.PhaseListenerManager.informPhaseListenersAfter(PhaseListenerManager.java:111)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:185)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Digging into the bowels of MyFaces, I discovered a class was looking for a viewId with an extension and no view-id was being set. Adding the following to the top of my execute() method solved this.

getFacesContext().getViewRoot().setViewId("/passwordHint.xhtml");

After making this change, all AppFuse's integration tests are passing and the upgrade seems complete. The only other issues I encountered were logging-related. The first is an error about Tomahawk that doesn't seem to affect anything.

Mar 5, 2011 6:44:01 AM com.sun.facelets.compiler.TagLibraryConfig loadImplicit
SEVERE: Error Loading Library: jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml
java.io.IOException: Error parsing [jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml]: 
        at com.sun.facelets.compiler.TagLibraryConfig.create(TagLibraryConfig.java:410)
        at com.sun.facelets.compiler.TagLibraryConfig.loadImplicit(TagLibraryConfig.java:431)
        at com.sun.facelets.compiler.Compiler.initialize(Compiler.java:87)
        at com.sun.facelets.compiler.Compiler.compile(Compiler.java:104)

The second is excessive logging from MyFaces. As far as I can tell, this is because MyFaces switched to java.util.logging instead of commons logging. With all the frameworks that AppFuse leverages, I think it has all the logging frameworks in its classpath now. I was hoping to fix this by posting a message to the mailing list, but haven't received a reply yet.

[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider newInstance
[WARNING] [talledLocalContainer] INFO: Creating instance of org.appfuse.webapp.action.BasePage
[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider destroyInstance
[WARNING] [talledLocalContainer] INFO: Destroy instance of org.appfuse.webapp.action.BasePage

After successfully upgrading AppFuse, I turned to AppFuse Light, where things were much easier.

Now that AppFuse uses JSF 2, I hope to start leveraging some of its new features. If you're yearning to get started with them today, I invite you to grab the source and start integrating them yourself.

Posted in Java at Mar 07 2011, 01:24:53 PM MST 3 Comments

Fixing XSS in JSP 2

Way back in 2007, I wrote about Java Web Frameworks and XSS. My main point was that JSP EL doesn't bother to handle XSS.

Of course, the whole problem with JSP EL could be solved if Tomcat (and other containers) would allow a flag to turn on XML escaping by default. IMO, it's badly needed to make JSP-based webapps safe from XSS.

A couple months later, I proposed a Tomcat enhancement to escape JSP's EL by default. I also entered an enhancement request for this feature and attached a patch. That issue has remained open and unfixed for 3 and 1/2 years.

Yesterday, Chin Huang posted a handy-dandy ELResolver that XML-escapes EL values.

I tried Chin's resolver in AppFuse today and it works as well as advertised. To do this, I copied his EscapeXML*.java files into my project, changed the JSP API's Maven coordinates from javax.servlet:jsp-api:2.0 to javax.servlet.jsp:jsp-api:2.1 and added the listener to web.xml.

With Struts 2 and Spring MVC, I was previously able to have ${param.xss} and pass in ?xss=<script>alert('gotcha')</script> and it would show a JavaScript alert. After using Chin's ELResolver, it prints the string on the page instead of displaying an alert.

Thanks to Chin Huang for this patch! If you're using JSP, I highly recommend you add this to your projects as well.

Posted in Java at Feb 28 2011, 02:08:46 PM MST 7 Comments

Upcoming Conferences: TSSJS in Las Vegas and 33rd Degree in Kraków, Poland

It's that time of year again - the beginning of Conference Season. I generally like to speak at a few conferences a year and 2011 is no different. For March Madness, I'll be heading to Las Vegas to speak at TheServerSide Java Symposium. I'll be giving updated talks similar to the ones I gave at last year's Rich Web Experience in Fort Lauderdale:

You might remember my Comparing JVM Web Frameworks talk from Devoxx 2010 and some of the interesting debate it caused. I've done some minor updates to my video presentation and some updates to my JVM Web Frameworks presentation as well. Most notably, I'll be including some findings from Peter Thomas's perfbench project. I also hope to update AppFuse to JSF 2 and integrate extensionless URLs in AppFuse Light. Marcin Zajaczkowski was nice enough to provide an upgrade to Wicket 1.4.15, so it'll be interesting to see how well Wicket supports extensionless URLs.

In April, I'll be presenting Comparing JVM Web Frameworks at the 33rd Degree Conference in Kraków, Poland. While I studied in Russia a couple summers in college, I've never been to Poland, so I'm really looking forward to this trip. With any luck, I'll have AppFuse 2.1 released by then and my knowledge of all its web frameworks' latest versions will be update-to-date. As you know, it's unlikely I'll recommend a best web framework (because there isn't one), but I hope to provide some techniques you can use to decide the best framework for your particular needs.

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and Über Conf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

If you're planning on attending TSSJS or 33rd Degree, hopefully I'll see you there.

Posted in Java at Feb 25 2011, 03:14:59 PM MST 1 Comment

Implementing Ajax Authentication using jQuery, Spring Security and HTTPS

I've always had a keen interest in implementing security in webapps. I implemented container-managed authentication (CMA) in AppFuse in 2002, watched Tomcat improve it's implementation in 2003 and implemented Remember Me with CMA in 2004. In 2005, I switched from CMA to Acegi Security (now Spring Security) and never looked back. I've been very happy with Spring Security over the years, but also hope to learn more about Apache Shiro and implementing OAuth to protect JavaScript APIs in the near future.

I was recently re-inspired to learn more about security when working on a new feature at Overstock.com. The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience. User experience aside, I think it's interesting to see what's required to implement such a feature.

To demonstrate how we did it, I whipped up an example using AppFuse Light, jQuery and Spring Security. The source is available in my ajax-login project on GitHub. To begin, I wanted to accomplish a number of things to replicate the Overstock environment:

  1. Force HTTPS for authentication.
  2. Allow testing HTTPS without installing a certificate locally.
  3. Implement a RESTful LoginService that allows users to login.
  4. Implement login with Ajax, with the request coming from an insecure page.

Forcing HTTPS with Spring Security
The first feature was fairly easy to implement thanks to Spring Security. Its configuration supports a requires-channel attribute that can be used for this. I used this to force HTTPS on the "users" page and it subsequently causes the login to be secure.

<intercept-url pattern="/app/users" access="ROLE_ADMIN" requires-channel="https"/>

Testing HTTPS without adding a certificate locally
After making the above change in security.xml, I had to modify my jWebUnit test to work with SSL. In reality, I didn't have to modify the test, I just had to modify the configuration that ran the test. In my last post, I wrote about adding my 'untrusted' cert to my JVM keystore. For some reason, this works for HttpClient, but not for jWebUnit/HtmlUnit. The good news is I figured out an easier solution - adding the trustStore and trustStore password as system properties to the maven-failsafe-plugin configuration.

<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7.2</version>
<configuration>
    <includes>
        <include>**/*WebTest.java</include>
    </includes>
    <systemPropertyVariables>
      <javax.net.ssl.trustStore>${project.build.directory}/ssl.keystore</javax.net.ssl.trustStore>
      <javax.net.ssl.trustStorePassword>appfuse</javax.net.ssl.trustStorePassword>
    </systemPropertyVariables>
</configuration>

The disadvantage to doing things this way is you'll have to pass these in as arguments when running unit tests in your IDE.

Implementing a LoginService
Next, I set about implementing a LoginService as a Spring MVC Controller that returns JSON thanks to the @ResponseBody annotation and Jackson.

package org.appfuse.examples.web;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
@RequestMapping("/api/login.json")
public class LoginService {

  @Autowired
  @Qualifier("authenticationManager")
  AuthenticationManager authenticationManager;

  @RequestMapping(method = RequestMethod.GET)
  @ResponseBody
  public LoginStatus getStatus() {
    Authentication auth = SecurityContextHolder.getContext().getAuthentication();
    if (auth != null && !auth.getName().equals("anonymousUser") && auth.isAuthenticated()) {
      return new LoginStatus(true, auth.getName());
    } else {
      return new LoginStatus(false, null);
    }
  }

  @RequestMapping(method = RequestMethod.POST)
  @ResponseBody
  public LoginStatus login(@RequestParam("j_username") String username,
                           @RequestParam("j_password") String password) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    User details = new User(username);
    token.setDetails(details);

    try {
      Authentication auth = authenticationManager.authenticate(token);
      SecurityContextHolder.getContext().setAuthentication(auth);
      return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
      return new LoginStatus(false, null);
    }
  }

  public class LoginStatus {

    private final boolean loggedIn;
    private final String username;

    public LoginStatus(boolean loggedIn, String username) {
      this.loggedIn = loggedIn;
      this.username = username;
    }

    public boolean isLoggedIn() {
      return loggedIn;
    }

    public String getUsername() {
      return username;
    }
  }
}

To verify this class worked as expected, I wrote a unit test using JUnit and Mockito. I used Mockito because Overstock is transitioning to it from EasyMock and I've found it very simple to use.

package org.appfuse.examples.web;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Matchers;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.TestingAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.context.SecurityContextImpl;

import static org.junit.Assert.*;
import static org.mockito.Mockito.*;

public class LoginServiceTest {

  LoginService loginService;
  AuthenticationManager authenticationManager;

  @Before
  public void before() {
    loginService = new LoginService();
    authenticationManager = mock(AuthenticationManager.class);
    loginService.authenticationManager = authenticationManager;
  }

  @After
  public void after() {
    SecurityContextHolder.clearContext();
  }

  @Test
  public void testLoginStatusSuccess() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    SecurityContext context = new SecurityContextImpl();
    context.setAuthentication(auth);
    SecurityContextHolder.setContext(context);

    LoginService.LoginStatus status = loginService.getStatus();
    assertTrue(status.isLoggedIn());
  }

  @Test
  public void testLoginStatusFailure() {
    LoginService.LoginStatus status = loginService.getStatus();
    assertFalse(status.isLoggedIn());
  }

  @Test
  public void testGoodLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    when(authenticationManager.authenticate(Matchers.<Authentication>anyObject())).thenReturn(auth);
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertTrue(status.isLoggedIn());
    assertEquals("foo", status.getUsername());
  }

  @Test
  public void testBadLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(false);
    when(authenticationManager.authenticate(Matchers.anyObject()))
        .thenThrow(new BadCredentialsException("Bad Credentials"));
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertFalse(status.isLoggedIn());
    assertEquals(null, status.getUsername());
  }
}

Implement login with Ajax
The last feature was the hardest to implement and still isn't fully working as I'd hoped. I used jQuery and jQuery UI to implement a dialog that opens the login page on the same page rather than redirecting to the login page. The "#demo" locator refers to a button in the page.

Passing in the "ajax=true" parameter disables SiteMesh decoration on the login page, something that's described in my Ajaxified Body article.

var dialog = $('<div></div>');

$(document).ready(function() {
    $.get('/login?ajax=true', function(data) {
        dialog.html(data);
        dialog.dialog({
            autoOpen: false,
	       title: 'Authentication Required'
        });
    });

    $('#demo').click(function() {
      dialog.dialog('open');
      // prevent the default action, e.g., following a link
      return false;
    });
});

Instead of adding a click handler to a specific id, it's probably better to use a CSS class that indicates authentication is required for a link, or -- even better -- use Ajax to see if the link is secured.

The login page then has the following JavaScript to add a click handler to the "login" button that submits the request securely to the LoginService.

var getHost = function() {
    var port = (window.location.port == "8080") ? ":8443" : "";
    return ((secure) ? 'https://' : 'http://') + window.location.hostname + port;
};

var loginFailed = function(data, status) {
    $(".error").remove();
    $('#username-label').before('<div class="error">Login failed, please try again.</div>');
};

$("#login").live('click', function(e) {
    e.preventDefault();
    $.ajax({url: getHost() + "/api/login.json",
        type: "POST",
        data: $("#loginForm").serialize(),
        success: function(data, status) {
            if (data.loggedIn) {
                // success
                dialog.dialog('close');
                location.href= getHost() + '/users';
            } else {
                loginFailed(data);
            }
        },
        error: loginFailed
    });
});

The biggest secret to making this all work (the HTTP -> HTTPS communication, which is considered cross-domain), is the window.name Transport and the jQuery plugin that implements it. To make this plugin work with Firefox 3.6, I had to implement a Filter that adds Access-Control headers. A question on Stackoverflow helped me figure this out.

public class OptionsHeadersFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;

        response.setHeader("Access-Control-Allow-Origin", "*");
        response.setHeader("Access-Control-Allow-Methods", "GET,POST");
        response.setHeader("Access-Control-Max-Age", "360");
        response.setHeader("Access-Control-Allow-Headers", "x-requested-with");

        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {
    }

    public void destroy() {
    }
}

Issues
I encountered a number of issues when implementing this in the ajax-login project.

  • If you try to run this with ports (e.g. 8080 and 8443) in your URLs, you'll get a 501 (Not Implemented) response. Removing the ports by fronting with Apache and mod_proxy solves this problem.
  • If you haven't accepted the certificate in your browser, the Ajax request will fail. In the example, I solved this by clicking on the "Users" tab to make a secure request, then going back to the homepage to try and login.
  • The jQuery window.name version 0.9.1 doesn't work with jQuery 1.5.0. The error is "$.httpSuccess function not found."
  • Finally, even though I was able to authenticate successfully, I was unable to make the authentication persist. I tried adding the following to persist the updated SecurityContext to the session, but it doesn't work. I expect the solution is to create a secure JSESSIONID cookie somehow.
    @Autowired
    SecurityContextRepository repository;
    
    @RequestMapping(method = RequestMethod.POST)
    @ResponseBody
    public LoginStatus login(@RequestParam("j_username") String username,
                             @RequestParam("j_password") String password,
                             HttpServletRequest request, HttpServletResponse response) {
    
        UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
        ...
    
        try {
            Authentication auth = authenticationManager.authenticate(token);
            SecurityContextHolder.getContext().setAuthentication(auth);
            // save the updated context to the session
            repository.saveContext(SecurityContextHolder.getContext(), request, response);
            return new LoginStatus(auth.isAuthenticated(), auth.getName());
        } catch (BadCredentialsException e) {
            return new LoginStatus(false, null);
        }
    }
    

Conclusion
This article has shown you how to force HTTPS for login, how to do integration testing with a self-generated certificate, how to implement a LoginService with Spring MVC and Spring Security, as well as how to use jQuery to talk to a service cross-domain with the window.name Transport. While I don't have everything working as much as I'd like, I hope this helps you implement a similar feature in your applications.

One thing to be aware of is with lightbox/dialog logins and HTTP -> HTTPS is that users won't see a secure icon in their address bar. If your app has sensitive data, you might want to force https for your entire app. OWASP's Secure Login Pages has a lot of good tips in this area.

Update: I've posted a demo of the ajax-login webapp. Thanks to Contegix for hosting the demo and helping obtain/install an SSL certificate so quickly.

Posted in Java at Feb 23 2011, 04:55:55 PM MST 13 Comments

Integration Testing with HTTP, HTTPS and Maven

Earlier this week, I was tasked with getting automated integration tests working in my project at Overstock.com. By automated, I mean that ability to run "mvn install" and have the following process cycled through:

  • Start a container
  • Deploy the application
  • Run all integration tests
  • Stop the container

Since it makes sense for integration tests to run in Maven's integration-test phase, I first configured the maven-surefire-plugin to skip tests in the test phase and execute them in the integration-test phase. I used the <id>default-phase</id> syntax to override the plugins' usual behavior.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <executions>
    <execution>
      <id>default-test</id>
      <configuration>
        <excludes>
          <exclude>**/*Test*.java</exclude>
        </excludes>
      </configuration>
    </execution>
    <execution>
      <id>default-integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/*Test.java</include>
        </includes>
        <excludes>
          <exclude>none</exclude>
          <exclude>**/TestCase.java</exclude>
        </excludes>
      </configuration>
    </execution>
  </executions>
</plugin>

After I had this working, I moved onto getting the container started and stopped properly. In the past, I've done this using Cargo and it's always worked well for me. Apart from the usual setup I use in AppFuse archetypes (example pom.xml), I added a couple additional items:

  • Added <timeout>180000</timeout> so the container would wait up to 3 minutes for the WAR to deploy.
  • In configuration/properties, specified <context.path>ROOT</context.path> so the app would deploy at the / context path.
  • In configuration/properties, specified <cargo.protocol>https</cargo.protocol> since many existing unit tests made requests to secure resources.

I started by using Cargo with Tomcat and had to create certificate keystore in order to get Tomcat to start with SSL enabled. After getting it to start, I found the tests failed with the following errors in the logs:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: 
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1649)

Co-workers told me this was easily solved by adding my 'untrusted' cert to my JVM keystore. Once all this was working, I thought I was good to go, but found that some tests were still failing. The failures turned out to be because they were talking to http and https was the only protocol enabled. After doing some research, I discovered that Cargo doesn't support starting on both http and https ports.

So back to the drawing board I went. I ended up turning to the maven-jetty-plugin and the tomcat-maven-plugin to get the functionality I was looking for. I also automated the certificate keystore generation using the keytool-maven-plugin. Below is the extremely-verbose 95-line profiles section of my pom.xml that allows either container to be used.

Sidenote: I wonder how this same setup would look using Gradle?

<profiles>
  <profile>
    <id>jetty</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <build>
      <plugins>
        <plugin>
          <groupId>org.mortbay.jetty</groupId>
          <artifactId>maven-jetty-plugin</artifactId>
          <version>6.1.26</version>
          <configuration>
            <contextPath>/</contextPath>
            <connectors>
              <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <!-- forwarded == true interprets x-forwarded-* headers -->
                <!-- http://docs.codehaus.org/display/JETTY/Configuring+mod_proxy -->
                <forwarded>true</forwarded>
                <port>8080</port>
                <maxIdleTime>60000</maxIdleTime>
              </connector>
              <connector implementation="org.mortbay.jetty.security.SslSocketConnector">
                <forwarded>true</forwarded>
                <port>8443</port>
                <maxIdleTime>60000</maxIdleTime>
                <keystore>${project.build.directory}/ssl.keystore</keystore>
                <password>overstock</password>
                <keyPassword>overstock</keyPassword>
              </connector>
            </connectors>
            <stopKey>overstock</stopKey>
            <stopPort>9999</stopPort>
          </configuration>
          <executions>
            <execution>
              <id>start-jetty</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
              <configuration>
                <daemon>true</daemon>
              </configuration>
            </execution>
            <execution>
              <id>stop-jetty</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>stop</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
  <profile>
    <id>tomcat</id>
    <build>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>tomcat-maven-plugin</artifactId>
          <version>1.1</version>
          <configuration>
            <addContextWarDependencies>true</addContextWarDependencies>
            <fork>true</fork>
            <path>/</path>
            <port>8080</port>
            <httpsPort>8443</httpsPort>
            <keystoreFile>${project.build.directory}/ssl.keystore</keystoreFile>
            <keystorePass>overstock</keystorePass>
          </configuration>
          <executions>
            <execution>
              <id>start-tomcat</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
            </execution>
            <execution>
              <id>stop-tomcat</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>shutdown</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

With this setup in place, I was able to automate running our integration tests by simply typing "mvn install" (for Jetty) or "mvn install -Ptomcat" (for Tomcat). For running in Hudson, it's possible I'll have to further enhance things to randomize the port and pass that into tests as a system property. The build-helper-maven-plugin and its reserve-network-port goal is a nice way to do this. Note: if you want to run more than one instance of Tomcat at a time, you might have to randomize the ajp and rmi ports to avoid collisions.

The final thing I encountered was our app didn't shutdown gracefully. Luckily, this was fixed in a newer version of our core framework and upgrading fixed the problem. Here's the explanation from an architect on the core framework team.

The hanging problem was caused by the way the framework internally aggregated statistics related to database connection usage and page response times. The aggregation runs on a separate thread but not as a daemon thread. Previously, the aggregation threads weren't being terminated on shutdown so the JVM would hang waiting for them to finish. In the new frameworks, the aggregation threads are terminated on shutdown.

Hopefully this post helps you test your secure and unsecure applications at the same time. At the same time, I'm hoping it motivates the Cargo developers to add simultaneous http and https support. ;)

Update: In the comments, Ron Piterman recommended I use the Maven Failsafe Plugin because its designed to run integration tests while Surefire Plugin is for unit tests. I changed my configuration to the following and everything still passes. Thanks Ron!

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <skipTests>true</skipTests>
  </configuration>
</plugin>
<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <includes>
      <include>**/*Test.java</include>
    </includes>
    <excludes>
      <exclude>**/TestCase.java</exclude>
    </excludes>
  </configuration>
  <executions>
    <execution>
      <id>integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <phase>verify</phase>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Update 2: In addition to application changes to solve hanging issues, I also had to change my Jetty Plugin configuration to use a different SSL connector implementation. This also required adding the jetty-sslengine dependency, which has been renamed to jetty-ssl for Jetty 7.

<connector implementation="org.mortbay.jetty.security.SslSelectChannelConnector">
...
<dependencies>
  <dependency>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-sslengine</artifactId>
    <version>6.1.26</version>
  </dependency>
</dependencies>

Posted in Java at Feb 11 2011, 03:54:16 PM MST 9 Comments

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Filter NameDispatchers
securityFilter request
rewriteFilter request, forward
struts-prepare request
sitemesh request, forward, include
staticFilter request, forward
struts request

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

Filter NameDispatchers
rewriteFilter request
securityFilter request, forward, include
struts-prepare request, forward
sitemesh request, forward, include
struts forward

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.

Posted in Java at Feb 10 2011, 04:53:27 PM MST 10 Comments