Raible Designs

AppFuse 3.5 Released!

Matt Raible — Fri, 20 Feb 2015 09:08:53 -0700

The AppFuse Team is pleased to announce the release of AppFuse 3.5. This release contains a number of improvements.

XML reduced by 8x in projects generated with AppFuse
CRUD generation support for Wicket, as well as AppFuse Light archetypes (Spring Security, Spring FreeMarker and Stripes)
Upgraded Tapestry to 5.4
Integrated Spring IO Platform for dependency management
Refactored unit tests to use JUnit 4
Renamed maven-warpath-plugin to warpath-maven-plugin
Upgraded to jWebUnit 3 for AppFuse Light integration tests
Updated all AppFuse Light modules to be up-to-date

For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is a full-stack framework for building web applications on the JVM. It was originally developed to eliminate the ramp-up time when building new web applications. Over the years, it has matured into a very testable and secure system for creating Java-based webapps.

Demos for this release can be viewed at http://demo.appfuse.org. Please see the QuickStart Guide to get started with this release.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the users mailing list. You can also post them to Stack Overflow with the "appfuse" tag.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix, and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

Integrating GWT into AppFuse

Matt Raible — Thu, 7 Mar 2013 18:49:28 -0700

I've been interested in integrating GWT into AppFuse ever since I blogged about it 4 years ago. A few months after that post, I wrote about Enhancing Evite.com with GWT and Grails. After Evite, I had a gig near Boston where I developed with GXT for the remainder of the year. When all was said and done, I ended up spending a year with GWT and really enjoyed my experience. I haven't used it much since.

GWT is scheduled to be integrated into AppFuse in version 4.0. That's quite a ways off. The good news is you might not have to wait that long, thanks to Iván García Sainz-Aja. Iván let us know about his work a couple weeks ago in an email to the appfuse-dev mailing list.

It's still work in progress but it has already most of AppFuse functionality..

If you want to give it a try

https://github.com/ivangsa/appfuse.git

the quickest way to have a go would be

web/gwt> mvn -P gwtDebug -Dgwt.inplace=true gwt:compile jetty:run

at the moment it still requires this fork of gwt-bootstrap to be compiled first

https://github.com/ivangsa/gwt-bootstrap.git

It needs a lot of testing yet but it's getting quite there

As you can imagine, I was very excited to hear about Iván's work. So I cloned his repo, built gwt-bootstrap locally and checked it out. Functionality wise, it was great! However, when I dug into the source code, I found a whole lotta code.

To see how the GWT flavor compared to the other implementations in AppFuse, I created a cloc report on the various web frameworks in AppFuse. I'm sure these reports could be adjusted to be more accurate, but I believe they give a good general overview. I posted some graphs that displays my findings in visual form.

When I sent this to the mailing list, Ivan responded that it was a lot of code and estimated 12 new files would be needed to CRUD an entity. This sure seems like a lot to me, but he defended this yesterday and noted that his implementation follows many of GWT's latest best practices: MVP pattern, Activities and Places, EventBus, Gin and Guice. He also shared a wiki page with explanations and diagrams of how things work.

The reason I'm writing this post is to get more feedback on this implementation. First of all, does GWT really require this much code? Secondly, are there other GWT implementations that reduce a lot of the boilerplate? SmartGWT, Vaadin* and Errai come to mind.

If you were starting a new GWT project and using AppFuse, how would you want it implemented?

* Vaadin 7 claims it can be used as a drop-in replacement for GWT. I tried replacing the gwt-servlet and gwt-user dependencies with Vaadin's, but it didn't work.

Livin' it up in Vegas at TSSJS 2011

Matt Raible — Tue, 22 Mar 2011 09:04:17 -0600

Last Wednesday, Trish and I traveled to Las Vegas for TheServerSide Java Symposium 2011 conference. We had a free room from TechTarget, but opted to upgrade to a suite with a view over the Bellagio Fountains. Trish won a trip to Vegas as a sales award earlier in the year and cleverly exchanged it for cash, so our upgrade was sort of free.

My first talk was on Online Video and my experience at Time Warner Cable. With my former team's iPad app releasing the day before, it was a fun session. The attendance was kind of sparse, but I had some good competition so wasn't surprised.

After I finished speaking, we headed to happy hour and met up with some friends that happened to be in town. We had dinner at the Todd English Pub and headed to the Penn & Teller show at the Rio. We closed the night after Trish had a 45-minute roll at the craps table at O'Sheas.

We slept in on Thursday and I gave my Comparing JVM Web Frameworks talk that afternoon. I made sure to mention some other methods to choosing web frameworks: doing performance comparisons like Peter Thomas has done or choosing Lift because one of its developers says it's the best. While Vaadin did sneak into the #5 spot, I made sure and mentioned that Wicket and Tapestry seem to belong there moreso (based on stats, mailing list traffic, etc.).

Trish took a bunch of pictures during my talk, which had a great turnout and lots of participation.

That evening, we celebrated St. Patty's Day with some college buddies of mine, ate great sushi at Mizuya and experienced the joys of three card poker. Thanks to TechTarget for inviting me to TSSJS 2011; we had an awesome time. You can find all the pictures we took on Flickr.

P.S. If you can't see the presentations in this post (a.k.a. you don't have Flash), you can view them on on Slideshare or download the PDFs.

JSR 303 and JVM Web Framework Support

Matt Raible — Tue, 8 Mar 2011 11:33:24 -0700

Emmanuel Bernard recently sent an email to the JSR 303 Experts Group about the next revision of the Bean Validation JSR (303). Rather than sending the proposed changes privately, he blogged about them. I left a comment with what I'd like to see:

+1 for Client-side validation. I'd love to see an API that web frameworks can hook into to add "required" to their tags for HTML5. Or some service that can be registered so the client can make Ajax requests to an API to see if an object is valid.

Emmanuel replied that most of the necessary API already exists for this, but frameworks have been slow to adopt it.

Hi Matt,

The sad thing is that the API is present on the Bean Validation side but presentation frameworks are slow to adopt it and use it :(

RichFaces 4 now has support for it but I wished more presentation frameworks had worked on the integration. If you can convince a few people or have access to a few people, feel free to send them by me :)

The integration API is described here. Let me know if you think some parts are missing or should be improved. We should definitely do some more buzz around it.

In the interest of generating more buzz around it, I decided to do some research and see what JVM Frameworks support JSR 303. Here's what I've come up with so far (in no particular order):

Struts 2 has an open issue, but doesn't seem to support JSR 303. Since I did a quick-n-dirty google search for most of these, I'm not sure if they support client-side JavaScript or HTML5's required. If you know of other JVM-based web frameworks that support JSR 303, please let me know in the comments.

How I Calculated Ratings for My JVM Web Frameworks Comparison

Matt Raible — Mon, 6 Dec 2010 11:55:18 -0700

When I re-wrote my Comparing JVM Web Frameworks presentation from scratch, I decided to add a matrix that allows you to rate a framework based on 20 different criteria. The reason I did this was because I'd used this method when choosing an Ajax framework for Evite last year. The matrix seemed to work well for selecting the top 5 frameworks, but it also inspired a lot of discussion in the community that my ratings were wrong.

I expected this, as I certainly don't know every framework as well as I'd like. The mistake I made was asking for the community to provide feedback on my ratings without describing how I arrived at them. From Peter Thomas's blog:

What you are doing is adjusting ratings based on who in the community shouts the loudest. I can't help saying that this approach comes across as highly arrogant and condescending, you seem to expect framework developers and proponents to rush over and fawn over you to get better ratings, like waiters in a restaurant trying to impress a food-critic for Michelin stars.

I apologize for giving this impression. It certainly wasn't my intent. By having simple numbers (1.0 == framework does well, 0.5 == framework is OK and 0 == framework not good at criteria) with no rationalization, I can see how the matrix can be interpreted as useless (or to put it bluntly, as something you should wipe your ass with). I don't blame folks for getting angry.

For my Rich Web Experience presentation, I documented why I gave each framework the rating I did. Hopefully this will allow folks to critique my ratings more constructively and I can make the numbers more accurate. You can view this document below or on Google Docs.

In the end, what I was hoping to do with this matrix was to simply highlight a technique for choosing a web framework. Furthermore, I think adding a "weight" to each criteria is important because things like books often aren't as important as REST support. To show how this might be done, I added a second sheet to the matrix and made up some weighting numbers. I'd expect anyone that wants to use this to downloaded the matrix, verify the ratings are accurate for your beliefs and weight the criteria accordingly.

Of course, as I and many others have said, the best way to choose a web framework is to try them yourself. I emphasized this at the end of my presentation with the following two slides.

My Comparing JVM Web Frameworks Presentation from Devoxx 2010

Matt Raible — Thu, 18 Nov 2010 05:23:10 -0700

This week, I've been having a great time in Antwerp, Belgium at the Devoxx Conference. This morning, I had the pleasure of delivering my Comparing JVM Web Frameworks talk. I thoroughly enjoyed giving this presentation, especially to such a large audience. You can view the presentation below (if you have Flash installed) or download it here.

Unlike previous years, I chose to come up with a spreadsheet matrix that shows why I chose the 5 I did. This spreadsheet and rankings given to each framework are likely to be debated, as I don't know all the frameworks as well as I'd like to. Also, the missing column on this spreadsheet is a "weighting" column where you can prioritize certain criteria like I've done in the past when Comparing Ajax Frameworks. If you believe there are incorrect numbers, please let me know and I'll try to get those fixed before I do this talk again at The Rich Web Experience.

One thing that doesn't come across in this presentation is that I believe anyone can use this matrix, and weightings, to make any of these frameworks come out on top. I also believe web frameworks are like spaghetti sauce in The Ketchup Conundrum. That is, the only way to make more happy spaghetti sauce lovers was to make more types of spaghetti sauce. You can read more about this in my There is no "best" web framework article.

Update: If you disagree with the various ratings I gave to web frameworks in this presentation, please provide your opinions by filling out this survey. Thanks to Sebastien Arbogast for setting this up.

Update: Sebastien has posted his survey results at JVM Web Framework Survey, First Results.

Update 12/6: A video of this presentation is now available on Parleys.com.

P.S. My current gig is ending in mid-December. If you're looking for a UI Architect with a passion for open source frameworks, please let me know.

My Presentations from The Irish Software Show 2010

Matt Raible — Thu, 10 Jun 2010 07:11:35 -0600

This week I've been enjoying Dublin, Ireland thanks to the 2nd Annual Irish Software Show. On Wednesday night, I spoke about The Future of Web Frameworks and participated in a panel with Grails, Rails, ASP.NET MVC and Seaside developers. It was a fun night with lots of lively discussion. Below is my presentation from this event.

This morning, I delivered my Comparing Kick-Ass Web Frameworks talk. This presentation contains updated statistics for various metrics comparing Rails vs. Grails and Flex vs. GWT.

Thanks to all who attended my talks this week!

P.S. I believe audio was recorded on Wednesday night, but I'm unsure how it turned out. I'm pretty sure no recordings were done on this morning's session.

My TSSJS 2010 Presentations and Summary

Matt Raible — Fri, 19 Mar 2010 17:29:08 -0600

This afternoon, I delivered my last talk at TSSJS 2010 on The Future of Web Frameworks. It's true that I made some bold statements, but please remember that this is my personal opinion, based on my experience. For the most part, I've been involved in super high-traffic websites for the last few years and this has influenced my opinion on web frameworks. Just because I don't recommend your favorite framework doesn't mean it won't work for you. In fact, many of the best web applications today were built without an open source (or commercial) web framework. In the end, it's not as much about the web framework you're using as it is about hiring smart people. Below is my slide deck from this talk.

Yesterday, I did a GWT vs. Flex Smackdown with James Ward. While there wasn't as much trash talking as I'd hoped, I enjoyed delivering it and disputing the greatness of Flex. Below is the presentation that James and I delivered.

The show itself was great this year. It had more attendees than I've seen in a long time. There were a lot of really interesting sessions and and an often humorous Twitter back-channel. I attended quite a few talks and jotted down my notes from several of them. Please see the links below if you're interested in the sessions I attended. You can view all of the presentations from TSSJS 2010 on SlideShare.

Thanks to everyone who came to Vegas and to TheServerSide for an excellent conference.

Comparing Kick-Ass Web Frameworks at The Rich Web Experience

Matt Raible — Fri, 4 Dec 2009 08:16:48 -0700

Yesterday, I delivered my Comparing Kick-Ass Web Frameworks talk at the Rich Web Experience in Orlando, Florida. Below are the slides I used:

Although it's difficult to convey a presentation in a slide deck, I can offer you my conclusion: there is no "best" web framework. I believe web frameworks are like spaghetti sauce in that everyone has different tastes and having so many choices is necessary to satisfy everyone. You can read more about the plural nature of perfection in Malcolm Gladwell's The Ketchup Conundrum (a written version of What we can learn from spaghetti sauce). Even though there is no "best" web framework, I believe GWT, Flex, Rails and Grails are frameworks that every web developer should try. They really do make it fun to develop web applications.

You can find the slides for my other RWE talk at Building SOFEA Applications with GWT and Grails.

Kudos to Jay Zimmerman for putting on a great show in Orlando this year. I had a great time talking with folks and learning in the sessions I attended. I particularly enjoyed bringing my parents and kids and staying at such a nice resort. Disney World (Magic Kingdom) and Universal Studios was very enjoyable due to the short lines. Also, the weather was perfect - especially considering the freezing cold in Denver this week.

GWT OAuth and LinkedIn APIs

Matt Raible — Tue, 24 Nov 2009 15:46:05 -0700

When I worked at LinkedIn last year, I received a lot of inquiries from friends and developers about LinkedIn's APIs. After a while, I started sending the following canned response:

For API access to build LinkedIn features into your application, fill out the following form:

http://www.linkedin.com/static?key=developers_apis

For requests to build an application, go to:

http://www.linkedin.com/static?key=developers_opensocial

I talked with the API team and they did say they look at every request that's sent via these forms. They don't respond to all of them b/c they know that many people would be angry if they told them "no", so they'd rather not have that headache.

Yesterday, I was pumped to see that they've finally decided to open up their API to Developers.

Starting today, developers worldwide can integrate LinkedIn into their business applications and Web sites. Developer.linkedin.com is now live and open for business.

First of all, congratulations to the API team on finally making this happen! I know it's no small feat. Secondly, it's great to see them using Jive SBS for their API documentation and developer community. My current client uses this to facilitate development and I love how it integrates a wiki, JIRA, FishEye, Crucible and Bamboo into one central jumping off point.

I've always been a fan of LinkedIn, ever since I joined way back in May 2003. However, I've longed for a way to access my data. LinkedIn Widgets are nice, but there's something to be said for the full power of an API. Last night, I sat down for a couple hours and enhanced my Implementing OAuth with GWT example to support LinkedIn's API.

I'm happy to report my experiment was a success and you can download GWT OAuth 1.2 or view it online. For now, I'm simply authenticating with OAuth and accessing the Profile API.

In the process, I learned a couple things:

LinkedIn's OAuth implementation returns an oauth_verifier parameter after authenticating, whereas Google and Twitter do not. This parameter needs to be included when calling the Access token path.
The Profile API example I implemented gets the current user's profile with http://api.linkedin.com/v1/people/~. This returns a "light" version of your profile. To get a more detailed version, you need to use Field Selectors. For example: http://api.linkedin.com/v1/people/~:(id,first-name,last-name,picture-url,headline,summary,positions,educations)
LinkedIn's API only supports passing OAuth parameters in a header, rather than query parameters. To make this work, I modified my ProxyServlet to convert query parameters to an "Authorization" header at the end of the setProxyRequestHeaders() method.

// For LinkedIn's OAuth API, convert request parameters to an AuthorizationHeader
if (httpServletRequest.getRequestURL().toString().contains("linkedin-api")) {
    String[] parameters = httpServletRequest.getQueryString().split("&");
    StringBuilder sb = new StringBuilder("OAuth realm=\"http://api.linkedin.com/\",");
    for (int i = 0; i < parameters.length; i++) {
        sb.append(parameters[i]);
        if (i < parameters.length - 1) {
            sb.append(",");
        }
    }

    Header authorization = new Header("Authorization", sb.toString());
    httpMethodProxyRequest.setRequestHeader(authorization);
}

You might recall that my previous example had issues authenticating with Google, but worked well with Twitter. LinkedIn's authentication seems to work flawlessly. This leads me to believe that Twitter and LinkedIn have a much more mature OAuth implementation than Google.

Related OAuth News: Apache Roller 5 will be shipping with OAuth support. See Dave Johnson's What's New in Roller 5 presentation for more information.

Update December 6, 2009: I modified the gwt-oauth project to use GWT 1.7.1 and changed to the Maven GWT Plugin from Codehaus. Download GWT OAuth 1.3 or view it online.

Building SOFEA Applications with GWT and Grails

Matt Raible — Thu, 12 Nov 2009 09:30:09 -0700

Last night, I spoke at the Denver Java User Group meeting. The consulting panel with Matthew, Tim and Jim a lot of fun and I enjoyed delivering my Building SOFEA Applications with GWT and Grails presentation for the first time. The talk was mostly a story about how we enhanced Evite.com with GWT and Grails and what we did to make both frameworks scale. I don't believe the presentation reflects the story format that well, but it's not about the presentation, it's about the delivery of it.

If you'd like to hear the story about this successful SOFEA implementation at a high-volume site, I'd recommend attending the Rich Web Experience next month. If you attended last night's meeting and have any feedback on how this talk can be improved, I'd love to hear it.

Consulting, SOFEA, Grails and GWT at next week's Denver JUG

Matt Raible — Thu, 5 Nov 2009 22:52:37 -0700

Next Wednesday, I'll be at Denver's JUG meeting to talk about Independent Consulting and Building SOFEA Applications with Grails and GWT. The first talk will be a a panel discussion among local independent consultants, including James Goodwill, Matthew McCullough, Tim Berglund and myself.

This session explores the trials and tribulations of an independent consultant. How do you find contracts? Should you setup an LLC, an S-Corp or just be a sole proprietorship? What about health insurance and benefits? Are recruiters helpful or hurtful? Learn lots of tips and tricks to get your dream job and your ideal lifestyle.

The Grails and GWT talk is a preview of a talk I'll be doing at the Rich Web Experience in December. Below is a rewrite of the abstract in first-person.

Earlier this year, I participated in a major enhancement of a high-traffic well-known internet site. The company wanted us to quickly re-architect their site and use a modern Ajax framework to do it with. An Ajax Framework evaluation was done to help the team choose the best framework for their skillset. The application was built with a SOFEA architecture using GWT on the frontend and Grails/REST on the backend.

This talk will cover how Bryan Noll, Scott Nicholls, James Goodwill and I came to choose GWT and Grails, as well as stumbling blocks we encountered along the way. In addition, we'll explore many topics such as raw GWT vs. GXT/SmartGWT, the Maven GWT Plugin, modularizing your code, multiple EntryPoints, MVP, integration testing and JSON parsing with Overlay Types.

If you're in Denver next Wednesday night (November 11th), you should stop by the Denver JUG meeting. It'll be a fun night and there's sure to be a few beers afterward.

Testing GWT Libraries with Selenium and Maven

Matt Raible — Wed, 4 Nov 2009 22:09:27 -0700

On Tuesday, I wrote about Running Hosted Mode in GWT Libraries. Today I added an additional module to our project to run Selenium tests against our GWT library. In the process, I discovered some things I needed to modify in my GWT library's pom.xml. I'm writing this post so others can use this setup to write GWT libraries and package them for testing with Selenium.

First of all, I noticed that when you're using the GWT Maven Plugin with a JAR project, it doesn't automatically run gwt:compile or gwt:test in the compile and test phases. I had to explicitly configure the compile goal to run in the compile phase. I also had to add <webappDirectory> to the configuration to compile the JavaScript files into the war directory.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>gwt-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <module>org.appfuse.gwt.core.CoreUI</module>
        <runTarget>index.html</runTarget>
        <webappDirectory>war</webappDirectory>
    </configuration>
    <executions>
        <execution>
            <phase>compile</phase>
            <goals>
                <goal>compile</goal>
            </goals>
        </execution>
    </executions>
</plugin>

To package the generated JavaScript and index.html in the JAR, I added the following <resources> section to the maven-resources-plugin configuration I mentioned in my previous post.

<resource>
    <directory>war</directory>
    <includes>
        <include>core.ui/**</include>
        <include>index.html</include>
    </includes>
</resource>

In addition, I discovered some javax.servlet.* classes in my JAR after running "mvn package". I believe this is caused by the GWT plugin sucking these in when it compiles my ProxyServlet. I excluded them by adding the maven-jar-plugin.

<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
        <excludes>
            <exclude>javax/servlet/**</exclude>
        </excludes>
    </configuration>
</plugin>

After doing this, I was able to publish my JAR with all the contents I needed to run Selenium tests against it.

Testing the GWT Library with Selenium
The module that contains the Selenium tests is a WAR project that uses war overlays, Cargo and Selenium RC. You can read about the Maven setup I use for running Selenium tests in Packaging a SOFEA Application for Distribution.

The major difference when testing a JAR (vs. a WAR), is I had to use the maven-dependency-plugin to unpack the JAR so its contents would get included in the WAR for testing. Below is the configuration I used to accomplish this:

<plugin>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <id>unpack</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>unpack</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>org.appfuse</groupId>
                        <artifactId>gwt-core</artifactId>
                        <version>1.0-SNAPSHOT</version>
                        <type>jar</type>
                        <overWrite>false</overWrite>
                        <excludes>META-INF/**,org/**,javax/**</excludes>
                    </artifactItem>
                </artifactItems>
                <outputDirectory>
                    ${project.build.directory}/${project.build.finalName}
                </outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

Hopefully this will help you develop GWT libraries and run Selenium tests against them. If you have any suggestions for simplifying this configuration, please let me know.

NOTE: I did considering a couple of other options for running Selenium tests against our GWT library:

Add something to the existing project that 1) creates a WAR and 2) fires up Cargo/Selenium in a profile to test it.
Create the tests in a GWT (war) project that includes widgets from the library.

I decided on the solution documented above because it seemed like the best option.

Running Hosted Mode in GWT Libraries (when using Maven)

Matt Raible — Tue, 3 Nov 2009 09:37:07 -0700

Earlier this year, I wrote about Modularizing GWT Applications with GWT-Maven. Fast forward 8 months and I'm still working with GWT and using this same technique. However, this time I'm working with the Maven GWT Plugin from Codehaus. In my last post, I wrote:

The results of modularizing your application are beneficial (shared code) and detrimental (you have to mvn install gwt-core whenever you make changes in shared classes). If you know of a way to configure the gwt-maven plugin to read sources from both gwt-core and gwt-webapp in hosted mode, I'd love to hear about it.

The good news is I found a solution for this, using the Builder Helper Maven Plugin. The GWT Maven Plugin's Productivity tip for multi-project setup has more information on how to configure this (note: we use IntelliJ and Eclipse on my project and did not need to configure this in a profile).

All was fine and dandy with this configuration until I wanted to be able to run hosted mode to develop/test everything in my library before including it in my main project. Luckily, you can still run mvn gwt:run on a JAR project. However, when you configure your pom.xml so sources are included in your JAR, you run into an issue: your *.java files will be copied to war/WEB-INF/classes and hosted mode will use these files as source rather than the ones you're editing in src/main/java.

To solve this, I changed my pom.xml to do two things:

Only copy resources right before packaging (in the test phase).
When packaging is complete, delete the *.java files from war/WEB-INF/classes (using Ant).

Below is the XML I used to make this possible. Please let me know if you have a way to simplify this configuration.

<plugin>
    <artifactId>maven-resources-plugin</artifactId>
    <version>2.4.1</version>
    <executions>
        <execution>
            <phase>test</phase>
            <goals>
                <goal>copy-resources</goal>
            </goals>
            <configuration>
                <outputDirectory>${project.build.outputDirectory}</outputDirectory>
                <resources>
                    <resource>
                        <directory>src/main/java</directory>
                    </resource>
                    <resource>
                        <directory>src/main/resources</directory>
                    </resource>
                </resources>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.3</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>run</goal>
            </goals>
            <configuration>
                <tasks>
                    <delete>
                        <fileset dir="${project.build.outputDirectory}" includes="**/*.java"/>
                    </delete>
                </tasks>
            </configuration>
        </execution>
    </executions>
</plugin>

This solution seems to work pretty well. As far as developing your library in hosted mode, you'll need to configure two *.gwt.xml files, one that doesn't have an <entry-point> defined and one that does. Configure the one with the entry point as the <module> in your gwt-maven-plugin configuration.

As a side note, I found a few issues with the 1.1 version of the Maven GWT Archetype. Below are the steps I used to fix these issues and upgrade to GWT 1.7.0 (I realize 1.7.1 is out, but gwt-dev-1.7.1-mac.jar doesn't exist in Maven central).

First, create a new project by running the following from the command line:

mvn archetype:generate \
  -DarchetypeGroupId=org.codehaus.mojo \
  -DarchetypeArtifactId=gwt-maven-plugin \
  -DarchetypeVersion=1.1 \
  -DgroupId=com.yourcompany \
  -DartifactId=gwt-project -Dversion=1.0-SNAPSHOT -B

After creating the project, you'll need to modify the pom.xml as follows:

Change the gwt-maven-plugin's version to 1.1.
Change the ${gwtVersion} property to 1.7.0.
Add <runTarget>Application.html</runTarget> to the <configuration> element of the plugin.
Move Application.html and web.xml so they're under the "war" directory.
Update Application.html to prepend the GWT module name in the <script> tag.

I hope these instructions help you create modular GWT projects with Maven. This setup is working great on my current project.

Developing and Testing GWT Client Services

Matt Raible — Wed, 21 Oct 2009 06:55:17 -0600

Earlier this week, Hiram Chirino released RestyGWT, a GWT generator for REST services and JSON encoded data transfer objects. You can read more about it in Hiram's post RestyGWT, a Better GWT RPC??. First of all, I'm impressed with RestyGWT because provides something I've always wanted with GWT: the ability to call RESTful services and get a populated POJO in your callback, much like AsyncCallback provides for RPC services.

RestyGWT also allows you to easily create services using only interfaces and JAX-RS annotations. For example:

import javax.ws.rs.POST;
...
public interface PizzaService extends RestService {
    @POST
    public void order(PizzaOrder request, MethodCallback<OrderConfirmation> callback);
}

After taking a brief look at RestyGWT, I thought it'd be interesting to share how I develop and test GWT client services.

Developing GWT Client Services
Writing services in a GWT application can be helpful when you're using MVP, especially since you can EasyMock them in a test. On my GWT projects, I've often used overlay types because they allow me to write less code and they make parsing JSON super simple. I've had issues testing my presenters when using overlay types. The good news is I think I've figured out a reasonable solution, but it does require using GWTTestCase. If RestyGWT supported overlay types, there's a good chance I'd use it, especially since its integration tests seem to require GWTTestCase too.

Rather than using callbacks in my presenters, I try to only use them in my service implementations. That way, my presenters don't have to worry about overlay types and can be tested in a JUnit-only fashion. The callbacks in my services handle JSON parsing/object population and fire events with the populated objects.

GWT's RequestBuilder is one option for communicating with RESTful services. The Development Guide for HTTP Requests explains how to use this class. To simplify REST requests and allow multiple callbacks, I'm using a RestRequest class, and a number of other utility classes that make up a small GWT REST framework (created by a former colleague). RestRequest wraps RequestBuilder and provides a Fluent API for executing HTTP requests. Another class, Deferred, is a GWT implementation of Twisted's Deferred.

As part of my service implementation, I inject an EventBus (with GIN) into the constructor and then proceed to implement callbacks that fire Events to indicate loading, saving and deleting has succeeded. Here's an example service:

public class ConversationServiceImpl implements ConversationService {
    private EventBus eventBus;

    @Inject
    public ConversationServiceImpl(EventBus eventBus) {
        this.eventBus = eventBus;
    }

    public void getConversation(String name) {
        Deferred<Representation> d =
                RestRequest.get(URLs.CONVERSATION + "/" + URL.encode(name)).build();

        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                Conversation conversation = convertResultToConversation(result);
                eventBus.fireEvent(new ResourceLoadedEvent<Conversation>(conversation));
            }
        });

        d.run();
    }

    public void saveConversation(Conversation conversation) {
        Deferred<Representation> d = RestRequest.post(URLs.CONVERSATION)
                .setRequestData(conversation.toJson()).build();
        
        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                Conversation conversation = convertResultToConversation(result);
                eventBus.fireEvent(new ResourceSavedEvent<Conversation>(conversation));
            }
        });

        d.run();
    }

    public void deleteConversation(Long id) {
        Deferred<Representation> d =
                RestRequest.post(URLs.CONVERSATION + "/" + id).build();

        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                eventBus.fireEvent(new ResourceDeletedEvent());
            }
        });

        d.run();
    }

    /**
     * Convenience method to populate object in one location
     *
     * @param result the result of a resource request.
     * @return the populated object.
     */
    private Conversation convertResultToConversation(Representation result) {
        JSOModel model = JSOModel.fromJson(result.getData());
        return new Conversation(model);
    }
}

In the saveConversation() method you'll notice the conversation.toJson() method call. This method uses a JSON class that loops through an objects properties and constructs a JSON String.

public JSON toJson() {
    return new JSON(getMap());
}

Testing Services
In my experience, the hardest part about using overlay types is writing your objects so they get populated correctly. I've found that writing tests which read JSON from a file can be a great productivity boost. However, because of overlay types, you have to write a test that extends GWTTestCase. When using GWTTestCase, you can't simply read from the filesystem. The good news is there is a workaround where you can subclass GWTShellServlet and overwrite GWT's web.xml to have your own servlet that can read from the filesystem. A detailed explanation of how to do this was written by Alex Moffat in Implementing a -noserver flag for GWTTestCase.

Once this class is in place, I've found you can easily write services using TDD and the server doesn't even have to exist. When constructing services, I've found the following workflow to be the most productive:

Create a file with the expected JSON in src/test/resources/resource.json where resource matches the last part of the URL for your service.
Create a *ServiceGwtTest.java and write tests.
Run tests to make sure they fail.
Implement the service and run tests to ensure JSON is getting consumed/produced properly to/from model objects.

Below is the code for my JsonReaderServlet.java:

public class JsonReaderServlet extends GWTShellServlet {

    public void service(ServletRequest servletRequest, ServletResponse servletResponse)
            throws ServletException, IOException {

        HttpServletRequest req = (HttpServletRequest) servletRequest;
        HttpServletResponse resp = (HttpServletResponse) servletResponse;

        String uri = req.getRequestURI();
        if (req.getQueryString() != null) {
            uri += "?" + req.getQueryString();
        }

        if (uri.contains("/services")) {
            String method = req.getMethod();
            String output;

            if (method.equalsIgnoreCase("get")) {
                // use the part after the last slash as the filename
                String filename = uri.substring(uri.lastIndexOf("/") + 1, uri.length()) + ".json";
                System.out.println("loading: " + filename);
                String json = readFileAsString("/" + filename);
                System.out.println("loaded json: " + json);
                output = json;
            } else {
                // for posts, return the same body content
                output = getBody(req);
            }

            PrintWriter out = resp.getWriter();
            out.write(output);
            out.close();

            resp.setStatus(HttpServletResponse.SC_OK);
        } else {
            super.service(servletRequest, servletResponse);
        }
    }

    private String readFileAsString(String filePath) throws IOException {
        filePath = getClass().getResource(filePath).getFile();
        BufferedReader reader = new BufferedReader(new FileReader(filePath));
        return getStringFromReader(reader);
    }

    private String getBody(ServletRequest request) throws IOException {
        BufferedReader reader = new BufferedReader(new InputStreamReader(request.getInputStream()));
        return getStringFromReader(reader);
    }

    private String getStringFromReader(Reader reader) throws IOException {
        StringBuilder sb = new StringBuilder();
        char[] buf = new char[1024];
        int numRead;
        while ((numRead = reader.read(buf)) != -1) {
            sb.append(buf, 0, numRead);
        }
        reader.close();
        return sb.toString();
    }
}

This servlet is mapped to <url-pattern>/*</url-pattern> in a web.xml file in src/test/resources/com/google/gwt/dev/etc/tomcat/webapps/ROOT/WEB-INF.

My Service Test starts by getting an EventBus from GIN and registering itself to handle the fired events.

public class ConversationServiceGwtTest extends AbstractGwtTestCase
        implements ResourceLoadedEvent.Handler, ResourceSavedEvent.Handler, ResourceDeletedEvent.Handler {
    ConversationService service;
    ResourceLoadedEvent<Conversation> loadedEvent;
    ResourceSavedEvent<Conversation> savedEvent;
    ResourceDeletedEvent deletedEvent;

    @Override
    public void gwtSetUp() throws Exception {
        super.gwtSetUp();
        DesigntimeGinjector injector = GWT.create(MyGinjector.class);
        EventBus eventBus = injector.getEventBus();
        service = new ConversationServiceImpl(eventBus);
        eventBus.addHandler(ResourceLoadedEvent.ENGINE, this);
        eventBus.addHandler(ResourceSavedEvent.ENGINE, this);
        eventBus.addHandler(ResourceDeletedEvent.ENGINE, this);
    }

    @SuppressWarnings("unchecked")
    public void onLoad(ResourceLoadedEvent event) {
        this.loadedEvent = event;
    }

    @SuppressWarnings("unchecked")
    public void onSave(ResourceSavedEvent event) {
        this.savedEvent = event;
    }

    public void onDelete(ResourceDeletedEvent event) {
        this.deletedEvent = event;
    }
}

After this groundwork has been done, a test can be written that loads up the JSON file and verifies the objects are populated correctly.

public void testGetConversation() {

    service.getConversation("test-conversation");

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceLoadedEvent not received", loadedEvent);
            Conversation conversation = loadedEvent.getResource();
            assertEquals("Conversation name is incorrect","Test Conversation", conversation.getName());

            assertNotNull("Conversation has no channel", conversation.getChannel());
            assertEquals("Conversation has incorrect task size", 3, conversation.getTasks().size());

            convertToAndFromJson(conversation);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}

private void convertToAndFromJson(Conversation fromJsonModel) {
    Representation json = fromJsonModel.toJson();
    assertNotNull("Cannot convert empty JSON", json.getData());

    // change back into model
    JSOModel data = JSOModel.fromJson(json.getData());
    Conversation toJsonModel = new Conversation(data);
    verifyModelBuiltCorrectly(toJsonModel);
}

private void verifyModelBuiltCorrectly(Conversation model) {
    assertEquals("Conversation name is incorrect", "Test Conversation", model.getString("name"));
    assertEquals("Conversation has incorrect task size", 3, model.getTasks().size());
    assertEquals("Conversation channel is incorrect", "Web", model.getChannel().getString("type"));
}

For more information on the usage of the Timer, finishTest() and delayTestFinish(), see GWTTestCase's javadoc.

The tests for saving and deleting a resource look as follows:

public void testSaveConversation() {
    Conversation conversation = new Conversation().setName("Test").setId("1");

    List<Task> tasks = new ArrayList<Task>();
    for (int i = 1; i < 4; i++) {
        tasks.add(new Task().setName("Task " + i));
    }
    conversation.setTasks(tasks);

    System.out.println("conversation.toJson(): " + conversation.toJson());

    assertTrue(conversation.toJson().toString().contains("Task 1"));

    service.saveConversation(conversation);

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceSavedEvent not received", savedEvent);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}
  
public void testDeleteConversation() {
    service.deleteConversation(1L);

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceDeletedEvent not received", deletedEvent);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}

Summary
This article has shown you how I develop and test GWT Client Services. If RestyGWT supported overlay types, there's a good chance I could change my service implementation to use it and I wouldn't have to change my test. Robert Cooper, author of GWT in Practice, claims he has a framework that does this. Here's to hoping this article stimulates the GWT ecosystem and we get a GWT REST framework that's as easy to use as GWT RPC.

Update: Today I enhanced this code to use Generics-based classes (inspired by Don't repeat the DAO!) for the boiler-plate CRUD code in a service. In a nutshell, a service interface can now be written as:

public interface FooService extends GenericService<Foo, String> {
 
}

The implementation class is responsible for the URL and converting the JSON result to an object:

public class FooServiceImpl extends GenericServiceImpl<Foo, String> implements FooService {

    @Inject
    public FooServiceImpl(EventBus eventBus) {
        super(eventBus, "/services/foo");
    }

    @Override
    protected Foo convertResultToModel(Representation result) {
        return new Foo(JSOModel.fromJson(result.getData()));
    }
}

I'm sure this can be further enhanced to get rid of the need to create classes altogether, possibly leveraging GIN or some sort of factory. The parent classes referenced in this code can be viewed at the following URLs:

There's also a GenericServiceGwtTest.java that proves it all works as expected.

What would you like to see at TSSJS 2010?

Matt Raible — Mon, 12 Oct 2009 11:28:21 -0600

A couple months ago, I was asked by TheServerSide to speak at next year's TheServerSide Java Symposium in Las Vegas. In addition, they asked me to help them evaluate presentation proposals and suggest topics/speakers.

First of all, I think the biggest thing that TSSJS could do to improve is to host more networking events. With the JavaOne Party being over, I think there's a tremendous opportunity to fill a gap in the networking needs of the Java Community. When I first attended TSSJS in 2006, there were a fair amount of parties and everyone got to interact quite a bit. In 2008, there were no networking events. I believe having a strong networking story would attract a lot more attendees, companies and sponsors.

Secondly, I think it's possible that TSSJS has too many server-side related sessions. IMO, the server-side (and middleware in general) isn't that exciting. TechTarget appears to own TheClientSide, so why not add some more client-side stuff to the mix? For example, I'd love to see a Struts 1 app-makeover using different technologies (for example, Flex, GWT and jQuery). I think HTML5 and Google Wave's Architecture sessions would be interesting too. If adding client-side sessions is too far away from TheServerSide, maybe it should be renamed to TheServerSide JVM Symposium and there can be all kinds of sessions on JVM languages (e.g. Scala, JRuby, Groovy) and all the great things those languages can accomplish.

Lastly, I've been asked to send a couple session proposals. Currently, I'm thinking about a doing GWT vs. Flex Smackdown with James Ward, but I'm open to other ideas. It's been quite awhile since I did a "Comparing Web Frameworks" talk. Maybe "Hot Web Frameworks for 2010" is more appropriate? I also think it'd be interesting to do a somewhat philosophical talk on "The State of Web Frameworks" and where we're headed in the next year.

What would make you want to attend TSSJS next year? Let me know your thoughts and I'll do my best to make them a reality.

Update October 22, 2009: Whoo hoo! It looks like TheClientSide will be a part of TSSJS Vegas next year. Should be a great show.

Packaging a SOFEA Application for Distribution

Matt Raible — Tue, 6 Oct 2009 01:17:38 -0600

The project I'm working on is a bit different from those I'm used to. I'm used to working on web applications that are hosted on servers and customers access with their browser. SaaS if you will. My current client is different. They're a product company that sells applications and distributes them to customers via download and CD. Their customers install these applications on internal servers (supported servers include WebSphere, WebLogic and Tomcat).

The product I'm currently working on is structured as a SOFEA application and therefore consists of two separate modules - a backend and a frontend. Since it's installed in a servlet container, both modules are WARs and can be installed separately.

Building the backend and frontend as separate projects makes a lot of sense for two reasons:

In development, different teams can work on the frontend and backend projects.
Having them as separate projects allows them to be versioned separately.

However, having them as two separate projects does make it a bit more difficult for distribution. I'm writing this post to show you how I recently added support for distributing our application as 2 WARs or 1 WAR using the power of Maven, war overlays and the UrlRewriteFilter.

Project Setup
First of all, we have several different Maven modules, but the most important ones are as follows:

product-services
product-client
product-integration-tests

Of course, our modules aren't really named "product", but you get the point. The services project is really just a WAR project with Spring Security configured. It depends on other JAR modules that the services exist in. The client project is a GWT WAR that has a proxy servlet defined in its web.xml that makes it easier to develop. It also contains some UrlRewrite configuration that allows GWT Log's Remote Logging feature to work. The proxy servlet is something we don't want to ship with our product, so we have a separate web.xml for production vs. development. We do the substitution using the maven-war-plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.0.2</version>
    <configuration>
        <!-- Production web.xml -->
        <webXml>src/main/resources/web.xml</webXml>
        <warSourceDirectory>war</warSourceDirectory>
        <!-- Exclude everything but urlrewrite JAR -->
        <warSourceExcludes>
            WEB-INF/lib/aop**,WEB-INF/lib/commons-**,WEB-INF/lib/gin-**,
            WEB-INF/lib/guice-**,WEB-INF/lib/gwt-**,WEB-INF/lib/gxt-**,
            WEB-INF/lib/junit-**
        </warSourceExcludes>
    </configuration>
</plugin>

I could exclude WEB-INF/lib/** and WEB-INF/classes/**, but in my particular project, we still want UrlRewrite in standalone mode, and we have some i18n properties files in WEB-INF/classes that are served up for Selenium tests.

With this configuration, we have a services WAR and a client WAR that can be installed and used by clients. To collapse them into one and make it possible to ship a single war, I turned to our product-integration-tests module. This module contains Selenium tests that test both types of distributions.

Merging 2 WARs into 1
The most important thing in the product-integration-tests module is that it creates a single WAR. First of all, it uses <packaging>war</packaging> to make this possible. The rest is done using the following 3 steps.

1. Its dependencies include the client and servlet WARs (and Selenium RC for testing).

<dependencies>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-services</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-client</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium.client-drivers</groupId>
        <artifactId>selenium-java-client-driver</artifactId>
        <version>1.0.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

2. The WAR created excludes the "integration-tests" part of the name:

<build>
    <finalName>product-${project.version}</finalName>
    ...
</build>

3. WAR overlays are configured so the everything in the client's WEB-INF directory is excluded from the merged WAR.

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <!-- http://maven.apache.org/plugins/maven-war-plugin/overlays.html -->
        <overlays>
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-services</artifactId>
                <excludes>
                    <!-- TODO: Rename to api.html (this is the Enunciate-generated documentation) -->
                    <exclude>index.html</exclude>
                </excludes>
            </overlay>
            <!-- No server needed in product-client -->
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-client</artifactId>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
            <!-- Only include META-INF/context.xml to set the ROOT path -->
            <overlay>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
        </overlays>
    </configuration>
</plugin>

That's it! Using this configuration, it's possible to distribute a Maven-based SOFEA project as single or multiple WARs. However, there are some nuances.

One thing you might notice is the reference to META-INF/context.xml in the overlays configuration. This subtly highlights one issue I experienced when merging the WARs. In our GWT client, we're using URLs that point to our services at /product-services/*. This works in development (via a proxy servlet) and when the WARs are installed separately - as long as the services WAR is installed at /product-services. However, when they're merged, a little URL rewriting needs to happen. To do this, I added the UrlRewriteFilter to the product-services module and configured a simple rule.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCENGINE urlrewrite PUBLIC "-//tuckey.org//DTD UrlRewrite 3.0//EN"
        "http://tuckey.org/res/dtds/urlrewrite3.0.dtd">

<urlrewrite use-query-string="true">
    <!-- Used when services are merged into WAR with GWT client -->
    <rule>
        <from>^/product-services/(.*)$</from>
        <to type="forward">/$1</to>
    </rule>
</urlrewrite>

Because the services URLs point to the root (/product-services), the merged WAR has to be installed as the ROOT application. When you're using Cargo with Tomcat and want to deploy to ROOT, you have to have a META-INF/context.xml with a path="" reference (ref: CARGO-516).

<Context path=""/>

It is possible to change the URLs in the client to be relative, but this gets seems to get messy when you're using separate WARs. When using relative URLs, I found I had to do solution using cross-context forwarding to get the results I wanted. Using a redirect instead of a forward worked, but resulted in the client talking to the server twice (once to get redirected, a second time for the actual call). Cross-context forwarding is supported by the UrlRewriteFilter and Tomcat, but I'm not sure WebSphere or WebLogic support it. The best solution is probably to change the URLs dynamically at runtime, possibly using some sort of deferred binding technique.

Testing with Cargo and Selenium
Once I had everything merged, I wanted to configure Cargo and Selenium to allow testing both distribution types. If I installed all 3 wars at the same time, the "product-services" WAR would be used by both the product-client.war and the product.war, so I had to use profiles to allow installing the single merged WAR or both WARs. Below is the profile I used for starting Cargo, deploying the merged WAR, starting Selenium RC and running Selenium tests.

<properties>
    <cargo.container>tomcat6x</cargo.container>
    <cargo.container.url>
        http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.zip
    </cargo.container.url>
    <cargo.host>localhost</cargo.host>
    <cargo.port>23433</cargo.port>
    <cargo.wait>false</cargo.wait>
    <cargo.version>1.0</cargo.version>

    <!-- *safari and *iexplore are additional options -->
    <selenium.browser>*firefox</selenium.browser>
</properties>
...
<profile>
    <id>itest-bamboo</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <wait>${cargo.wait}</wait>
                    <container>
                        <containerId>${cargo.container}</containerId>
                        <log>${project.build.directory}/${cargo.container}/cargo.log</log>
                        <zipUrlInstaller>
                            <url>${cargo.container.url}</url>
                            <installDir>${installDir}</installDir>
                        </zipUrlInstaller>
                    </container>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <!-- Deploy as ROOT since XHR requests are made to /product-services -->
                        <deployables>
                            <deployable>
                                <properties>
                                    <context>ROOT</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
                <executions>
                    <execution>
                        <id>start-container</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-container</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>selenium-maven-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start-server</goal>
                        </goals>
                        <configuration>
                            <background>true</background>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <excludes>
                                <exclude>none</exclude>
                            </excludes>
                            <includes>
                                <include>**/*SeleniumTest.java</include>
                            </includes>
                            <systemProperties>
                                <property>
                                    <name>selenium.browser</name>
                                    <value>${selenium.browser}</value>
                                </property>
                                <property>
                                    <name>cargo.port</name>
                                    <value>${cargo.port}</value>
                                </property>
                            </systemProperties>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

This profile is run by our Bamboo nightly tests with mvn install -Pitest-bamboo. The 2nd profile I added doesn't install the project's WAR, but instead installs the two separate WARs. Running mvn install -Pitest-bamboo,multiple-wars executes the Selenium tests against the multi-WAR distribution.

<profile>
    <id>multiple-wars</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <deployables>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-client</artifactId>
                                <pingURL>http://${cargo.host}:${cargo.port}/product-client/index.html</pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-client</context>
                                </properties>
                            </deployable>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-services</artifactId>
                                <pingURL>
                                    http://${cargo.host}:${cargo.port}/project-services/index.jspx
                                </pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-services</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

I won't be including any information on authoring Selenium tests because there's already many good references. I encourage you to checkout the following if you're looking for Selenium testing techniques.

Summary
This article has shown you how I used Maven, war overlays and the UrlRewriteFilter to allow create different distributions of a SOFEA application. I'm still not sure which packaging (1 WAR vs. 2) mechanism is best, but it's nice to know there's options. If you package and distribute SOFEA applications, I'd love to hear about your experience in this area.

Building GWT Applications with MVP and Issues with Overlay Types

Matt Raible — Tue, 22 Sep 2009 13:41:36 -0600

MVP has recently become a popular strategy for structuring GWT applications. This is largely due to its testability and Ray Ryan's Best Practices For Architecting Your GWT App from this year's Google I/O. GWT, by itself, is simply a widget toolkit and doesn't ship with any sort of MVC (or MVP) framework.

On my current project, we're using GXT, a GWT implementation based on ExtJS. It has its own MVC framework, but it has very little documentation and can be confusing when using it with GWT's History management. At one point, I attempted to make it more understandable by writing a blog entry on GXT's MVC Framework.

One of my initial assignments was to decide if we should use MVP or MVC. Regardless of which one was chosen, I was also tasked with deciding if we should use an existing framework or write our own. After watching Ray Ryan's session on YouTube and recalling my frustration with GXT MVC on my last project, I quickly became convinced MVP was the answer.

To test my "MVP is best for our project" theory, I did a spike to implement it. I used the GWT MVP Example tutorial as a starting point and added the following libraries to my project.

GWT-Presenter: An implementation of the MVP pattern.
Google Gin: Dependency Injection based on Google's Guice.
GWT-Log: A log4j-style logger for GWT.

Implementing the MVP pattern itself was relatively straightforward, but I did encounter a few issues. I'm writing this post to see if anyone has solved these issues.

MVP Implementation Issues
The first issue I ran across was GXT's widgets don't implement standard GWT interfaces. To try and figure out a solution, I posted the following on Twitter:

"Wondering if it's possible to do MVP with GXT since it's buttons don't implement standard GWT interfaces."

The best response I received was from Simon Stewart (founder of the WebDriver project, works for Google):

"Put the GXT buttons in the View. Let that turn DOM events into semantic events."

He also pointed me to his tdd-gwt-gae project which shows many techniques for unit testing (with jMock) and integration testing (with GWTTestCase). Using Simon's examples, I was able to determine an initial strategy for implementing MVP with GXT.

My strategy is instead of getting widgets from the view and adding handlers, you add handlers to to the view and it takes care of adding them to the widgets that should listen for them. This seems to work, but my View interface has a lot of void methods, which is a bit different than standard MVP patterns I've seen.

The 2nd issue I encountered is with unit testing. Unit testing can be be performed on MVP applications by mocking out any dependencies that use JSNI. Classes that use JSNI are not testable with plain ol' JUnit and typically requires you to use GWTTestCase, which can be slow and cumbersome. This isn't to say that GWTTestCase isn't useful, just that it has its place.

When unit testing MVP applications, the recommended practice seems to be you should test presenters and not views. Services fall into a similar "don't unit test" category because they'll need to connect to the server-side, which won't be running in a unit testing environment.

This is where I ran into a major issue that I don't have a solution for.

I'm using Overlay Types (as described in JSON Parsing with JavaScript Overlay Types) to do JSON parsing in callbacks. Since Overlay Types use JSNI, it's not possible to do any JSON parsing in unit tests. The problem with not being able to do any JSON parsing is the callbacks will often call eventBus.fireEvent(GwtEvent) after the JSON parsing has happened. This means I can't fully test the flow of a presenter if event firing happens in a callback.

In attempt to try different mocking techniques for callbacks, I created a test that uses two recommended EasyMock-ing strategies. The first is a "CallbackSuccessMatcher" and is described in more detail in Testing GWT without GwtTestCase. The second technique uses a "CallbackMockSupport" class to allow EasyMock expectations such as expectLastCallAsync() and expectLastCallAsyncSuccess(T). You can read more about this technique in Test driven development for GWT UI code with asynchronous RPC.

Both of these examples use RPC, which typically has callbacks that have an onSuccess(T type) method. It's easy to use these callbacks in unit tests since T is a POJO and the onSuccess() method contains no JSNI code.

Currently, I see a few possible solutions to this problem:

Figure out a way to detect when unit tests are running and add if/else logic to callbacks.
Modify presenters and services so a callback can be set that is unit test-friendly.
Make JSOModel an interface that can be replaced/mocked in tests.

The last solution seems like best one, but I'm also curious to know what others are doing. My hunch is that most GWT apps use RPC and haven't run into this issue.

My Experience with Java REST Frameworks (specifically Jersey and CXF)

Matt Raible — Thu, 27 Aug 2009 14:20:51 -0600

Recently I was tasked with developing a server-side REST strategy for a client. When I started working with them, they were using GWT RPC for exposing services. They wanted to move to RESTful services to allow for a more open platform, that multiple types of clients could talk to. There were interested in supporting SOAP and GWT RPC as well, but it wasn't required. They are using Spring, a custom namespace (for easily creating remote services) and an HTML documentation generator to expose their API in human readable form.

When I first starting developing, I chose to try Enunciate. From Enunciate's homepage:

Enunciate is an engine for creating, maintaining, and deploying your rich Web service API on the Java platform. If that sounds complicated, it's not. All you have to do is define your service interfaces in Java source code. ... Then invoke Enunciate.

Sounds pretty sweet, eh? At first glance, the things I liked about Enunciate were:

The ability to generate multiple endpoints (and clients).
Generates nice-looking documentation.
Allows selecting different frameworks (for example, CXF instead of JAX-WS RI).

Initially, the hardest part of using Enunciate was integrating it into my project. This was somewhat related to having a multi-module project, but moreso related to getting the settings right in my enunciate.xml file. After getting everything working, I encountered a few Spring wiring issues with the GWT Endpoints and with Jersey's Spring support.

The good news is I believe most of these issues were related to my project and how it proxies Spring beans that Jersey couldn't find. Jersey's Spring support only supports annotations at this time, so I was unable to tell it the proxied bean names via XML (or even its own annotations). I'm sure this problem is solvable, but after struggling with it for a day or two, I decided to give up on Enunciate. I had to get something working, and fast.

At this point, I was back to the drawing board. I knew there were plenty of good Java REST frameworks available, but Spring and CXF (for SOAP) were already available dependencies in my project, so I chose that route. I was able to get something up and running fairly quickly with this tutorial and CXF's JAX-RS documentation. I ran into an Invalid JSON Namespace issue, but was able to solve it by adding a custom namespace.

It's not all rosy though, there are still a couple CXF issues I haven't solved:

I'd like to remove the namespace prefixes on returned JSON. I've never seen JSON with "prefix." notation before and it doesn't seem right.
I'm unable to return String[] from services.

While CXF does take a bit of XML for each service, it's kinda slick in that it only requires you to annotate your service interfaces. It also generates documentation for your services (WSDL and WADL), but it's not as pretty as Enunciate's. To solve this, I added Enunciate for documentation only. To make Enunciate work, I did have to add some annotations to my service implementation classes and parse the generated HTML to fix some links to CXF's services, but ultimately it works pretty well.

In the end, I recommended my client not use Enunciate for generating endpoints. This was primarily related to their unique Spring configuration, but also because I was able to easily use the same classes for REST, SOAP and GWT Endpoints. However, I will continue to keep my eye on Enunciate. It could be very useful for the upcoming appfuse-ws archetype. I'm also eager to see better GWT support and the ability to generate Overlay types in future releases.

The 2009 Rich Web Experience

Matt Raible — Tue, 11 Aug 2009 22:16:24 -0600

Late last year, I decided to take a year off from speaking at conferences. My reason was simple: I wanted to hunker down and do some learning. Ever since November 2007, I'd been interested in getting into developing SOFEA applications. This year, I was able to accomplish that and I've learned a lot about GWT and RESTful architectures in the process. Now I'm happy to announce I'll be sharing my experiences at this year's Rich Web Experience.

I look forward to speaking on the following topics:

The first one might look familiar since I used the same title at OSCON 2008. The difference is this time I plan to dive much deeper into the frameworks and help the audience learn more. I also plan to re-visit developing with Flex and Rails in the near future. If there's anything in particular you'd like to see covered in the above talks, please leave a comment.

This is sure to be a fun conference. First of all, it's pretty close to Disney World. A few years back, TSS hosted the "Java In Action" conference in Disney World and while the conference didn't do too well, it was awesome being there with my family. I plan on taking my kids and having some fun when I'm not speaking. Secondly, this conference is the first to offer all-inclusive pricing for attendees - registration, flight (continental U.S.) and 3 nights lodging. This should allow you to get approval without having to jump through the traditional travel hoops. Lastly, the JSF Summit will be held concurrently and you get access to it at no additional cost.

So come on down to Florida in December, listen to all the great speakers, have some fun with your family and look me up for a beer or two. You know it'll be a good time!

My attempt at browser-based username/password autocomplete with GWT

Matt Raible — Fri, 7 Aug 2009 08:09:11 -0600

Last night, I did a quick spike to try and implement username/password autocomplete in my GXT application. By "autocomplete", I don't mean Ajax-style autocomplete, but rather browser-based autocomplete. The best information I found on doing this is in the following post:

http://osdir.com/ml/GoogleWebToolkit/2009-04/msg01838.html

I didn't use this technique (wrapping an existing form with a FormPanel) because I'm using GXT and didn't want to lose the look-and-feel of my login form.

I was successful in getting everything to work in Firefox (it populates both the username and password). In IE, it only populates the username, not the password. In Safari/Chrome, it doesn't work at all. Here's how I did it:

1. Created a hidden HTML form on my HTML page that embeds GWT.

<form method="post" action="javascript:void(0)" style="display: none">
    <input type="text" id="username" name="username" value=""/>
    <input type="password" id="password" name="password" value=""/>
    <input type="submit" value="Login" id="login"/>
</form>

2. When a user clicks on the "Login" button in my GWT application, populate the fields in this hidden form and "click" on the Login button (which will do nothing since the action="javascript:void(0)").

// Set the hidden fields to trigger the browser to remember
DOM.getElementById("username").setAttribute("value", username.getValue());
DOM.getElementById("password").setAttribute("value", password.getValue());
clickFormLogin();

...

public static native void clickFormLogin() /*-{
$doc.getElementById("login").click();
}-*/;

This works in Firefox 3.5 and prompts me to save the user/pass at the top of the screen. Why doesn't this work in Safari/Chrome? My guess is because the form's action doesn't go anywhere and the form is not submitted. If I change the action to be an actual URL and show the form, clicking on the form's Login button will save it in those browsers.

Grabbing the actual populated values when the screen loads did turn out to be a bit tricky. With IE, I was able to grab the username, but not the password. It seems that keyboard or mouse interaction (tabbing off or focusing/blurring) with the username field is necessary to get the password field populated. In Firefox, I was unable to grab the values if I did it straight away. The solution turned out to be wrapping everything in a Timer and waiting a 1/2 second.

// grab user/pass from browser and hidden login form (if user has stored them)
// runs 1/2 second after page loads so cursor goes at end of username and works in Firefox
Timer t = new Timer() {
    public void run() {
        username.setValue(getElementValue("username"));
        password.setValue(getElementValue("password"));
        if (username.getValue() != null) {
            password.focus();
        }
        validate();
    }
};

t.schedule(500);

...

public static native String getElementValue(String domId) /*-{
    return $doc.getElementById(domId).value;
}-*/;

While I'm glad I got it working in Firefox, I'm disappointed with IE's lack of password autocompletion. More than anything, I can't help but think there's a way to make this work in WebKit-based browsers. If you've successfully implemented cross-browser username/password autocomplete in GWT, please share your experience. If you share your experience about successfully doing it in Safari/Chrome, I might even buy you a beer or two.

Integrating GWT with Spring Security

Matt Raible — Thu, 6 Aug 2009 08:50:15 -0600

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

How to do cross-domain GWT RPC with a ProxyServlet

Matt Raible — Wed, 5 Aug 2009 16:06:12 -0600

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Map;
import java.util.zip.GZIPInputStream;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileUploadException;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.StringRequestEntity;
import org.apache.commons.httpclient.methods.multipart.ByteArrayPartSource;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.StringPart;

/**
 * ProxyServlet from http://edwardstx.net/wiki/attach/HttpProxyServlet/ProxyServlet.java
 * (This seems to be a derivative of Noodle -- http://noodle.tigris.org/)
 * 
 * Patched to skip "Transfer-Encoding: chunked" headers, avoid double slashes
 * in proxied URLs, handle GZip and allow GWT RPC.
 */
public class ProxyServlet extends HttpServlet {

    private static final int FOUR_KB = 4196;

    /**
     * Serialization UID.
     */
    private static final long serialVersionUID = 1L;
    /**
     * Key for redirect location header.
     */
    private static final String STRING_LOCATION_HEADER = "Location";
    /**
     * Key for content type header.
     */
    private static final String STRING_CONTENT_ENGINE_HEADER_NAME = "Content-Type";
    /**
     * Key for content length header.
     */
    private static final String STRING_CONTENT_LENGTH_HEADER_NAME = "Content-Length";
    /**
     * Key for host header
     */
    private static final String STRING_HOST_HEADER_NAME = "Host";
    /**
     * The directory to use to temporarily store uploaded files
     */
    private static final File FILE_UPLOAD_TEMP_DIRECTORY = new File(System.getProperty("java.io.tmpdir"));

    // Proxy host params
    /**
     * The host to which we are proxying requests. Default value is "localhost".
     */
    private String stringProxyHost = "localhost";
    /**
     * The port on the proxy host to wihch we are proxying requests. Default value is 80.
     */
    private int intProxyPort = 80;
    /**
     * The (optional) path on the proxy host to wihch we are proxying requests. Default value is "".
     */
    private String stringProxyPath = "";
    /**
     * Setting that allows removing the initial path from client. Allows specifying /twitter/* as synonym for twitter.com.
     */
    private boolean removePrefix;
    /**
     * The maximum size for uploaded files in bytes. Default value is 5MB.
     */
    private int intMaxFileUploadSize = 5 * 1024 * 1024;
    private boolean isSecure;
    private boolean followRedirects;

    /**
     * Initialize the <code>ProxyServlet</code>
     * @param servletConfig The Servlet configuration passed in by the servlet container
     */
    public void init(ServletConfig servletConfig) {
        // Get the proxy host
        String stringProxyHostNew = servletConfig.getInitParameter("proxyHost");
        if (stringProxyHostNew == null || stringProxyHostNew.length() == 0) {
            throw new IllegalArgumentException("Proxy host not set, please set init-param 'proxyHost' in web.xml");
        }
        this.setProxyHost(stringProxyHostNew);
        // Get the proxy port if specified
        String stringProxyPortNew = servletConfig.getInitParameter("proxyPort");
        if (stringProxyPortNew != null && stringProxyPortNew.length() > 0) {
            this.setProxyPort(Integer.parseInt(stringProxyPortNew));
        }
        // Get the proxy path if specified
        String stringProxyPathNew = servletConfig.getInitParameter("proxyPath");
        if (stringProxyPathNew != null && stringProxyPathNew.length() > 0) {
            this.setProxyPath(stringProxyPathNew);
        }
        // Get the maximum file upload size if specified
        String stringMaxFileUploadSize = servletConfig.getInitParameter("maxFileUploadSize");
        if (stringMaxFileUploadSize != null && stringMaxFileUploadSize.length() > 0) {
            this.setMaxFileUploadSize(Integer.parseInt(stringMaxFileUploadSize));
        }
    }

    /**
     * Performs an HTTP GET request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a GET request
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("GET Request URL: " + httpServletRequest.getRequestURL(),
              "Destination URL: " + destinationUrl);
        GetMethod getMethodProxyRequest = new GetMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, getMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
        // Execute the proxy request
        this.executeProxyRequest(getMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Performs an HTTP POST request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doPost(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a standard POST request
        String contentType = httpServletRequest.getContentType();
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("POST Request URL: " + httpServletRequest.getRequestURL(),
              "    Content Type: " + contentType,
              " Destination URL: " + destinationUrl);
        PostMethod postMethodProxyRequest = new PostMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, postMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, postMethodProxyRequest);
        // Check if this is a mulitpart (file upload) POST
        if (ServletFileUpload.isMultipartContent(httpServletRequest)) {
            this.handleMultipartPost(postMethodProxyRequest, httpServletRequest);
        } else {
            if (contentType == null || PostMethod.FORM_URL_ENCODED_CONTENT_ENGINE.equals(contentType)) {
                this.handleStandardPost(postMethodProxyRequest, httpServletRequest);
            } else {
                this.handleContentPost(postMethodProxyRequest, httpServletRequest);
            }
        }
        // Execute the proxy request
        this.executeProxyRequest(postMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Sets up the given {@link PostMethod} to send the same multipart POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a multipart POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the mutlipart POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleMultipartPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest)
            throws ServletException {
        // Create a factory for disk-based file items
        DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
        // Set factory constraints
        diskFileItemFactory.setSizeThreshold(this.getMaxFileUploadSize());
        diskFileItemFactory.setRepository(FILE_UPLOAD_TEMP_DIRECTORY);
        // Create a new file upload handler
        ServletFileUpload servletFileUpload = new ServletFileUpload(diskFileItemFactory);
        // Parse the request
        try {
            // Get the multipart items as a list
            List<FileItem> listFileItems = (List<FileItem>) servletFileUpload.parseRequest(httpServletRequest);
            // Create a list to hold all of the parts
            List<Part> listParts = new ArrayList<Part>();
            // Iterate the multipart items list
            for (FileItem fileItemCurrent : listFileItems) {
                // If the current item is a form field, then create a string part
                if (fileItemCurrent.isFormField()) {
                    StringPart stringPart = new StringPart(
                            fileItemCurrent.getFieldName(), // The field name
                            fileItemCurrent.getString() // The field value
                            );
                    // Add the part to the list
                    listParts.add(stringPart);
                } else {
                    // The item is a file upload, so we create a FilePart
                    FilePart filePart = new FilePart(
                            fileItemCurrent.getFieldName(), // The field name
                            new ByteArrayPartSource(
                            fileItemCurrent.getName(), // The uploaded file name
                            fileItemCurrent.get() // The uploaded file contents
                            ));
                    // Add the part to the list
                    listParts.add(filePart);
                }
            }
            MultipartRequestEntity multipartRequestEntity = new MultipartRequestEntity(
                    listParts.toArray(new Part[]{}),
                    postMethodProxyRequest.getParams());
            postMethodProxyRequest.setRequestEntity(multipartRequestEntity);
            // The current content-type header (received from the client) IS of
            // type "multipart/form-data", but the content-type header also
            // contains the chunk boundary string of the chunks. Currently, this
            // header is using the boundary of the client request, since we
            // blindly copied all headers from the client request to the proxy
            // request. However, we are creating a new request with a new chunk
            // boundary string, so it is necessary that we re-set the
            // content-type string to reflect the new chunk boundary string
            postMethodProxyRequest.setRequestHeader(STRING_CONTENT_ENGINE_HEADER_NAME, multipartRequestEntity.getContentType());
        } catch (FileUploadException fileUploadException) {
            throw new ServletException(fileUploadException);
        }
    }

    /**
     * Sets up the given {@link PostMethod} to send the same standard POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleStandardPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) {
        // Get the client POST data as a Map
        Map<String, String[]> mapPostParameters = (Map<String, String[]>) httpServletRequest.getParameterMap();
        // Create a List to hold the NameValuePairs to be passed to the PostMethod
        List<NameValuePair> listNameValuePairs = new ArrayList<NameValuePair>();
        // Iterate the parameter names
        for (String stringParameterName : mapPostParameters.keySet()) {
            // Iterate the values for each parameter name
            String[] stringArrayParameterValues = mapPostParameters.get(stringParameterName);
            for (String stringParamterValue : stringArrayParameterValues) {
                // Create a NameValuePair and store in list
                NameValuePair nameValuePair = new NameValuePair(stringParameterName, stringParamterValue);
                listNameValuePairs.add(nameValuePair);
            }
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestBody(listNameValuePairs.toArray(new NameValuePair[]{}));
    }

    /**
     * Sets up the given {@link PostMethod} to send the same content POST
     * data (JSON, XML, etc.) as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    private void handleContentPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) throws IOException, ServletException {
        StringBuilder content = new StringBuilder();
        BufferedReader reader = httpServletRequest.getReader();
        for (;;) {
            String line = reader.readLine();
            if (line == null) break;
            content.append(line);
        }

        String contentType = httpServletRequest.getContentType();
        String postContent = content.toString();

        if (contentType.startsWith("text/x-gwt-rpc")) {
            String clientHost = httpServletRequest.getLocalName();
            if (clientHost.equals("127.0.0.1")) {
                clientHost = "localhost";
            }

            int clientPort = httpServletRequest.getLocalPort();
            String clientUrl = clientHost + ((clientPort != 80) ? ":" + clientPort : "");
            String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + intProxyPort : "") + httpServletRequest.getServletPath();
            //debug("Replacing client (" + clientUrl + ") with server (" + serverUrl + ")");
            postContent = postContent.replace(clientUrl , serverUrl);
        }

        String encoding = httpServletRequest.getCharacterEncoding();
        debug("POST Content Type: " + contentType + " Encoding: " + encoding,
              "Content: " + postContent);
        StringRequestEntity entity;
        try {
            entity = new StringRequestEntity(postContent, contentType, encoding);
        } catch (UnsupportedEncodingException e) {
            throw new ServletException(e);
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestEntity(entity);
    }

    /**
     * Executes the {@link HttpMethod} passed in and sends the proxy response
     * back to the client via the given {@link HttpServletResponse}
     * @param httpMethodProxyRequest An object representing the proxy request to be made
     * @param httpServletResponse An object by which we can send the proxied
     *                             response back to the client
     * @throws IOException Can be thrown by the {@link HttpClient}.executeMethod
     * @throws ServletException Can be thrown to indicate that another error has occurred
     */
    private void executeProxyRequest(
            HttpMethod httpMethodProxyRequest,
            HttpServletRequest httpServletRequest,
            HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a default HttpClient
        HttpClient httpClient = new HttpClient();
        httpMethodProxyRequest.setFollowRedirects(false);
        // Execute the request
        int intProxyResponseCode = httpClient.executeMethod(httpMethodProxyRequest);
        String response = httpMethodProxyRequest.getResponseBodyAsString();

        // Check if the proxy response is a redirect
        // The following code is adapted from org.tigris.noodle.filters.CheckForRedirect
        // Hooray for open source software
        if (intProxyResponseCode >= HttpServletResponse.SC_MULTIPLE_CHOICES /* 300 */ && intProxyResponseCode < HttpServletResponse.SC_NOT_MODIFIED /* 304 */) {
            String stringStatusCode = Integer.toString(intProxyResponseCode);
            String stringLocation = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
            if (stringLocation == null) {
                throw new ServletException("Received status code: " + stringStatusCode + " but no " + STRING_LOCATION_HEADER + " header was found in the response");
            }
            // Modify the redirect to go to this proxy servlet rather that the proxied host
            String stringMyHostName = httpServletRequest.getServerName();
            if (httpServletRequest.getServerPort() != 80) {
                stringMyHostName += ":" + httpServletRequest.getServerPort();
            }
            stringMyHostName += httpServletRequest.getContextPath();
            if (followRedirects) {
                if (stringLocation.contains("jsessionid")) {
                    Cookie cookie = new Cookie("JSESSIONID", stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
                    cookie.setPath("/");
                    httpServletResponse.addCookie(cookie);
                    //debug("redirecting: set jessionid (" + cookie.getValue() + ") cookie from URL");
                } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
	                Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
                    String[] cookieDetails = header.getValue().split(";");
					String[] nameValue = cookieDetails[0].split("=");

					Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
					cookie.setPath("/");
					//debug("redirecting: setting cookie: " + cookie.getName() + ":" + cookie.getValue() + " on " + cookie.getPath());
					httpServletResponse.addCookie(cookie);
                }
                httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() + this.getProxyPath(), stringMyHostName));
                return;
            }
        } else if (intProxyResponseCode == HttpServletResponse.SC_NOT_MODIFIED) {
            // 304 needs special handling.  See:
            // http://www.ics.uci.edu/pub/ietf/http/rfc1945.html#Code304
            // We get a 304 whenever passed an 'If-Modified-Since'
            // header and the data on disk has not changed; server
            // responds w/ a 304 saying I'm not going to send the
            // body because the file has not changed.
            httpServletResponse.setIntHeader(STRING_CONTENT_LENGTH_HEADER_NAME, 0);
            httpServletResponse.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
            return;
        }

        // Pass the response code back to the client
        httpServletResponse.setStatus(intProxyResponseCode);

        // Pass response headers back to the client
        Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
        for (Header header : headerArrayResponse) {
            if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
                    header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") || // don't copy gzip header
                    header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header so browser doesn't prompt on failed basic auth
                // proxy servlet does not support chunked encoding
            } else {
                httpServletResponse.setHeader(header.getName(), header.getValue());
            }
        }

        List<Header> responseHeaders = Arrays.asList(headerArrayResponse);

        if (isBodyParameterGzipped(responseHeaders)) {
            debug("GZipped: true");
            if (!followRedirects && intProxyResponseCode == HttpServletResponse.SC_MOVED_TEMPORARILY) {
                response = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
                httpServletResponse.setStatus(HttpServletResponse.SC_OK);
                intProxyResponseCode = HttpServletResponse.SC_OK;
                httpServletResponse.setHeader(STRING_LOCATION_HEADER, response);
            } else {
                response = new String(ungzip(httpMethodProxyRequest.getResponseBody()));
            }
            httpServletResponse.setContentLength(response.length());
        }

        // Send the content to the client
        debug("Received status code: " + intProxyResponseCode,
              "Response: " + response);

        httpServletResponse.getWriter().write(response);
    }


    /**
     * The response body will be assumed to be gzipped if the GZIP header has been set.
     *
     * @param responseHeaders of response headers
     * @return true if the body is gzipped
     */
    private boolean isBodyParameterGzipped(List<Header> responseHeaders) {
        for (Header header : responseHeaders) {
            if (header.getValue().equals("gzip")) {
                return true;
            }
        }
        return false;
    }

    /**
     * A highly performant ungzip implementation. Do not refactor this without taking new timings.
     * See ElementTest in ehcache for timings
     *
     * @param gzipped the gzipped content
     * @return an ungzipped byte[]
     * @throws java.io.IOException when something bad happens
     */
    private byte[] ungzip(final byte[] gzipped) throws IOException {
        final GZIPInputStream inputStream = new GZIPInputStream(new ByteArrayInputStream(gzipped));
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(gzipped.length);
        final byte[] buffer = new byte[FOUR_KB];
        int bytesRead = 0;
        while (bytesRead != -1) {
            bytesRead = inputStream.read(buffer, 0, FOUR_KB);
            if (bytesRead != -1) {
                byteArrayOutputStream.write(buffer, 0, bytesRead);
            }
        }
        byte[] ungzipped = byteArrayOutputStream.toByteArray();
        inputStream.close();
        byteArrayOutputStream.close();
        return ungzipped;
    }

    public String getServletInfo() {
        return "GWT Proxy Servlet";
    }

    /**
     * Retrieves all of the headers from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestHeaders(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an Enumeration of all of the header names sent by the client
        Enumeration enumerationOfHeaderNames = httpServletRequest.getHeaderNames();
        while (enumerationOfHeaderNames.hasMoreElements()) {
            String stringHeaderName = (String) enumerationOfHeaderNames.nextElement();
            if (stringHeaderName.equalsIgnoreCase(STRING_CONTENT_LENGTH_HEADER_NAME)) {
                continue;
            }
            // As per the Java Servlet API 2.5 documentation:
            //		Some headers, such as Accept-Language can be sent by clients
            //		as several headers each with a different value rather than
            //		sending the header as a comma separated list.
            // Thus, we get an Enumeration of the header values sent by the client
            Enumeration enumerationOfHeaderValues = httpServletRequest.getHeaders(stringHeaderName);
            while (enumerationOfHeaderValues.hasMoreElements()) {
                String stringHeaderValue = (String) enumerationOfHeaderValues.nextElement();
                // In case the proxy host is running multiple virtual servers,
                // rewrite the Host header to ensure that we get content from
                // the correct virtual server
                if (stringHeaderName.equalsIgnoreCase(STRING_HOST_HEADER_NAME)) {
                    stringHeaderValue = getProxyHostAndPort();
                }
                Header header = new Header(stringHeaderName, stringHeaderValue);
                // Set the same header on the proxy request
                httpMethodProxyRequest.setRequestHeader(header);
            }
        }
    }

    /**
     * Retrieves all of the cookies from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestCookies(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an array of all of all the cookies sent by the client
        Cookie[] cookies = httpServletRequest.getCookies();
        if (cookies == null) {
            return;
        }

        for (Cookie cookie : cookies) {
            cookie.setDomain(stringProxyHost);
            cookie.setPath(httpServletRequest.getServletPath());
            httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() + "=" + cookie.getValue() + "; Path=" + cookie.getPath());
        }
    }

    // Accessors
    private String getProxyURL(HttpServletRequest httpServletRequest) {
        // Set the protocol to HTTP
        String protocol = (isSecure) ? "https://" : "http://";
        String stringProxyURL = protocol + this.getProxyHostAndPort();

        // simply use whatever servlet path that was part of the request as opposed to getting a preset/configurable proxy path
        if (!removePrefix) {
            stringProxyURL += httpServletRequest.getServletPath();
        }
        stringProxyURL += "/";
        
        // Handle the path given to the servlet
        String pathInfo = httpServletRequest.getPathInfo();
        if (pathInfo != null && pathInfo.startsWith("/")) {
            if (stringProxyURL != null && stringProxyURL.endsWith("/")) {
                // avoid double '/'
                stringProxyURL += pathInfo.substring(1);
            }
        } else {
            stringProxyURL += httpServletRequest.getPathInfo();
        }
        // Handle the query string
        if (httpServletRequest.getQueryString() != null) {
            stringProxyURL += "?" + httpServletRequest.getQueryString();
        }
        
        return stringProxyURL;
    }

    private String getProxyHostAndPort() {
        if (this.getProxyPort() == 80) {
            return this.getProxyHost();
        } else {
            return this.getProxyHost() + ":" + this.getProxyPort();
        }
    }

    protected String getProxyHost() {
        return this.stringProxyHost;
    }

    protected void setProxyHost(String stringProxyHostNew) {
        this.stringProxyHost = stringProxyHostNew;
    }

    protected int getProxyPort() {
        return this.intProxyPort;
    }

    protected void setSecure(boolean secure) {
        this.isSecure = secure;
    }
    
    protected void setFollowRedirects(boolean followRedirects) {
        this.followRedirects = followRedirects;
    }

    protected void setProxyPort(int intProxyPortNew) {
        this.intProxyPort = intProxyPortNew;
    }

    protected String getProxyPath() {
        return this.stringProxyPath;
    }

    protected void setProxyPath(String stringProxyPathNew) {
        this.stringProxyPath = stringProxyPathNew;
    }

    protected void setRemovePrefix(boolean removePrefix) {
        this.removePrefix = removePrefix;
    }

    protected int getMaxFileUploadSize() {
        return this.intMaxFileUploadSize;
    }

    protected void setMaxFileUploadSize(int intMaxFileUploadSizeNew) {
        this.intMaxFileUploadSize = intMaxFileUploadSizeNew;
    }

    private void debug(String ... msg) {
        for (String m : msg) {
            System.out.println("[DEBUG] " + m);
        }
    }
}

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

How to use GWT 2.0 with Maven and Generate SOYC Reports

Matt Raible — Thu, 25 Jun 2009 23:45:04 -0600

One of the most interesting features coming in GWT 2.0 is code splitting and the ability to use GWT.runAsync() to reduce the size of your application's initial download. This week, I learned how to use GWT 2.0 with my GWT 1.6/Maven project. Below are instructions on how to build and use the latest GWT with Maven.

Checkout GWT and setup GWT_TOOLS.
Set a GWT_VERSION environment variable to 2.0.0-SNAPSHOT (export GWT_VERSION=2.0.0-SNAPSHOT).
Build GWT with the ant command.

After building completes, install the GWT artifacts into your local Maven repository using the following commands:

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-user -Dversion=2.0.0-SNAPSHOT \
-Dpackaging=jar -Dfile=build/lib/gwt-user.jar

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-servlet -Dversion=2.0.0-SNAPSHOT \
-Dpackaging=jar -Dfile=build/lib/gwt-servlet.jar

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
-Dclassifier=mac -Dpackaging=jar -Dfile=build/lib/gwt-dev-mac.jar

mkdir temp
tar -zxf build/dist/gwt-mac-2.0.0-SNAPSHOT.tar.gz -C temp
cd temp/gwt-mac-2.0.0-SNAPSHOT
zip -0 gwt-mac-2.0.0-SNAPSHOT.zip lib*.jnilib
cd ../..

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
-Dclassifier=mac-libs -Dpackaging=zip \
-Dfile=temp/gwt-mac-2.0.0-SNAPSHOT/gwt-mac-2.0.0-SNAPSHOT.zip

Thanks to Jason for his help with this script.

Modify the pom.xml of your GWT project to use the the gwt-maven-plugin from Codehaus. Of course, you'll need to modify the <runTarget> to fit your project.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>gwt-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <runTarget>org.appfuse.gwt.mvc.${entry.point}/${entry.point}.html</runTarget>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>compile</goal>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Modify your dependencies to match the ones below. With the Codehaus plugin, dependencies are much more concise.

<dependency>
    <groupId>com.google.gwt</groupId>
    <artifactId>gwt-servlet</artifactId>
    <version>${gwt.version}</version>
    <scope>compile</scope>
</dependency>
<dependency>
    <groupId>com.google.gwt</groupId>
    <artifactId>gwt-user</artifactId>
    <version>${gwt.version}</version>
    <scope>provided</scope>
</dependency>

Add <gwt.version>2.0.0-SNAPSHOT</gwt.version> to the properties section of your pom.xml.
At this point, you should be able to compile your project with mvn gwt:compile and run it in hosted mode using mvn gwt:run.

Generate SOYC Reports
In Google's code splitting documentation, it mentions The Story of Your Compile (SOYC). From the documentation:

To obtain a SOYC report for your application, there are two steps necessary. First, add -soyc to the compilation options that are passed to the GWT compiler. This will cause the compiler to emit raw information about the compile to XML files in an -aux directory beside the rest of the compiled output. In that directory, you will see an XML file for each permutation and a manifest.xml file that describes the contents of all the others.

The second step is to convert that raw information into viewable HTML. This is done with the SoycDashboard tool.

The first step is not currently possible with the gwt-maven-plugin, so I created a patch for it.

If you patch the gwt-maven-plugin and install it locally, make sure and change the version in your pom.xml to 1.2-SNAPSHOT.

To use the SoycDashboard tool, you'll need to install the gwt-soyc-vis.jar.

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-soyc-vis -Dversion=2.0.0-SNAPSHOT \
-Dpackaging=jar -Dfile=build/lib/gwt-soyc-vis.jar

Now you can generate SOYC reports with mvn gwt:compile -Dgwt.compiler.soyc=true. You can also add <soyc>true</soyc> to the <configuration> section of the gwt-maven-plugin.

The second step (converting the raw information into viewable HTML) is possible using java from the command-line, or by using the exec-maven-plugin. Here's the (lengthy) command-line version:

java -Xmx1024m -cp /Users/mraible/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:/Users/mraible/.m2/repository/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-mac.jar com.google.gwt.soyc.SoycDashboard -resources ~/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar -out target/soyc-report target/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz

In this example, I'm using the files from stories0.xml.gz, dependencies0.xml.gz, splitPoints0.xml.gz. In the soycReport output directory, there's 5 of each these files and I'm not sure what the difference between reports is. Hopefully someone on the GWT team can elaborate. The exec-maven-plugin version is as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <executable>java</executable>
        <arguments>
            <argument>-cp</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:${settings.localRepository}/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-${platform}.jar
            </argument>
            <argument>com.google.gwt.soyc.SoycDashboard</argument>
            <argument>-out</argument>
            <argument>target/soyc-report</argument>
            <argument>-resources</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar
            </argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz</argument>
        </arguments>
    </configuration>
</plugin>

After configuring this plugin in your project, you should be able to run mvn gwt:compile exec:exec and open the generated report (at target/soyc-report/SoycDashboard-index.html). Currently, there doesn't seem to be much documentation on SOYC. Fred Sauer's recent presentation talks a bit about SOYC and GWT.runAsync(), but that's about it.

To figure out how to use GWT 2.0 with Maven, I used my GWT MVC Example project. The first SOYC report I generated said the initial download was 108,967 KB. To integrate GWT.runAsync(), I modified all the project's controllers so their handleEvent() methods changed from this:

public void handleEvent(AppEvent event) {
    onViewHome(event);
}

To this:

public void handleEvent(final AppEvent event) {
    GWT.runAsync(new RunAsyncCallback() {
        public void onFailure(Throwable throwable) {
            Window.alert(throwable.getMessage());
        }

        public void onSuccess() {
            onViewHome(event);
        }
    });
}

When I generated a new SOYC report, the initial download size was reduced to 56,718 KB. Furthermore, I was able to see that my "Leftovers code" consisted of 63,175 KB. I'm sure there's better ways to split my project using GWT.runAsync(), but I'm happy to see I was able to reduce the initial download by 50%.

If you'd like to try GWT 2.0, you can can download my gwt-mvc example project. To build/run this project, you'll need to 1) build and install GWT, 2) patch gwt-maven-plugin and 3) run mvn gwt:compile exec:exec to generate the SOYC report. In an ideal world, the gwt-maven-plugin can be enhanced to generate the SOYC report (rather than using the exec-maven-plugin). In the meantime, I think it's pretty cool that you can try out GWT 2.0 features while they're still being developed.

JSON Parsing with JavaScript Overlay Types in GWT

Matt Raible — Wed, 24 Jun 2009 09:52:49 -0600

A reader recently asked:

I would love to see a snippet of how to eval the JSON coming from RequestBuilder into the OverlayTypes. What is the mapping like? I used OverlayTypes to read in static data that I render into the head section of the hosted page, which is pretty easy and fast, but I don't know how to do this "reading" dynamically at runtime.

If you're not familiar with GWT's Overlay Types (added in 1.5), see Getting to really know GWT, Part 2: JavaScript Overlay Types. In our project, we're using Overlay Types to simplify JSON parsing and make our application lean-and-mean as possible.

First of all, we have a JSOModel class that acts as our overlay type:

import java.util.HashSet;
import java.util.Set;

import com.google.gwt.core.client.JavaScriptObject;
import com.google.gwt.core.client.JsArray;
import com.google.gwt.core.client.JsArrayString;

/**
 * Java overlay of a JavaScriptObject.
 */
public abstract class JSOModel extends JavaScriptObject {

    // Overlay types always have protected, zero-arg constructors
    protected JSOModel() {
    }

    /**
     * Create an empty instance.
     * 
     * @return new Object
     */
    public static native JSOModel create() /*-{
        return new Object();
    }-*/;

    /**
     * Convert a JSON encoded string into a JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '{"foo":"bar","number":123}'
     *
     * @return a populated JSOModel object
     */
    public static native JSOModel fromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    /**
     * Convert a JSON encoded string into an array of JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '[{"foo":"bar","number":123}, {...}]'
     *
     * @return a populated JsArray
     */
    public static native JsArray<JSOModel> arrayFromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    public final native boolean hasKey(String key) /*-{
        return this[key] != undefined;
    }-*/;

    public final native JsArrayString keys() /*-{
        var a = new Array();
        for (var p in this) { a.push(p); }
        return a;
    }-*/;

    @Deprecated
    public final Set<String> keySet() {
        JsArrayString array = keys();
        Set<String> set = new HashSet<String>();
        for (int i = 0; i < array.length(); i++) {
            set.add(array.get(i));
        }
        return set;
    }

    public final native String get(String key) /*-{
        return "" + this[key];
    }-*/;

    public final native String get(String key, String defaultValue) /*-{
        return this[key] ? ("" + this[key]) : defaultValue;
    }-*/;

    public final native void set(String key, String value) /*-{
        this[key] = value;
    }-*/;

    public final int getInt(String key) {
        return Integer.parseInt(get(key));
    }

    public final boolean getBoolean(String key) {
        return Boolean.parseBoolean(get(key));
    }

    public final native JSOModel getObject(String key) /*-{
        return this[key];
    }-*/;

    public final native JsArray<JSOModel> getArray(String key) /*-{
        return this[key] ? this[key] : new Array();
    }-*/;
}

This class alone allows you to easily parse JSON returned in a callback. For example, here's an example of parsing Twitter's User Timeline in my OAuth with GWT application.

private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<JSOModel> statuses = new ArrayList<JSOModel>();
            for (int i = 0; i < data.length(); i++) {
                statuses.add(data.get(i));
            }

            // populate textarea with returned statuses
            for (JSOModel status : statuses) {
                payload.setValue(payload.getValue() + status.get("text") + "\n\n");
            }
            
            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

To simply things even more, we created a BaseModel class that can be extended.

import java.util.Map;
import java.util.HashMap;

import com.google.gwt.core.client.JsArrayString;
import com.google.gwt.user.client.Element;
import com.google.gwt.user.client.DOM;

public abstract class BaseModel {

    protected JSOModel data;

    public BaseModel(JSOModel data) {
        this.data = data;
    }

    public String get(String field) {
        String val = this.data.get(field);
        if (val != null && "null".equals(val) || "undefined".equals(val)) {
            return null;
        } else {
            return escapeHtml(val);
        }
    }

    public Map<String, String> getFields() {
        Map<String, String> fieldMap = new HashMap<String, String>();

        if (data != null) {
            JsArrayString array = data.keys();

            for (int i = 0; i < array.length(); i++) {
                fieldMap.put(array.get(i), data.get(array.get(i)));
            }
        }
        return fieldMap;
    }

    private static String escapeHtml(String maybeHtml) {
        final Element div = DOM.createDiv();
        DOM.setInnerText(div, maybeHtml);
        return DOM.getInnerHTML(div);
    }
}

You can extend this class and create model objects that represent a more Java-like view of your data. For example, I could create a Status class with the following code:

public class Status extends BaseModel {
    
    public Status(JSOModel data) {
        super(data);
    }

    public String getText() {
        return get("text");
    }
}

Then I could change my JSON parsing in TwitterApiCallback to be:

    private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<Status> statuses = new ArrayList<Status>();
            for (int i = 0; i < data.length(); i++) {
                Status s = new Status(data.get(i));
                statuses.add(s);
            }

            // populate textarea with returned statuses
            for (Status status : statuses) {
                payload.setValue(payload.getValue() + status.getText() + "\n\n");
            }

            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

That's how we're doing lightweight JSON parsing with GWT. I've updated my GWT with OAuth demo with this code. You can also download the source. Please let me know if you have any questions.

Update October 20, 2009: I recently had to enhance the JSOModel and BaseModel classes in my project to handle nested objects and arrays. In my project, I have a Conversation object that has a Channel and a List of Task objects. These objects are available in the JSOModel of my BaseModel, I just needed to grab them a bit differently.

public Channel getChannel() {
    return new Channel(data.getObject("channel"));
}

public List<Task> getTasks() {
    JsArray<JSOModel> array = data.getArray("tasks");
    List<Task> tasks = new ArrayList<Task>(array.length());

    for (int i = 0; i < array.length(); i++) {
        Task task = new Task(array.get(i));
        tasks.add(task);
    }
    
    return tasks;
}

To set a Channel, it's as simple as:

data.set("channel", channel.toJson().toString());

To allow setting Lists, I had to enhance JSOModel by adding the following two methods:

public final void set(String key, List<JSOModel> values) {
    JsArray<JSOModel> array = JavaScriptObject.createArray().cast();
    for (int i=0; i < values.size(); i++) {
        array.set(i, values.get(i));
    }
    setArray(key, array);
}

protected final native void setArray(String key, JsArray<JSOModel> values) /*-{
    this[key] = values;
}-*/;

After making this change, I was able to convert my List to List and set it on the underlying JSOModel.

public void setTasks(List<Task> tasks) {
    List<JSOModel> values = new ArrayList<JSOModel>();
    for (Task task : tasks) {
        values.add(task.getModel());
    }

    data.set("tasks", values);
}

To allow the task.getModel() method to work, I added a getter to BaseModel to allow retrieving the underlying JSOModel. Currently, I'm using a homegrown JSON.java class to produce JSON from my BaseModel objects. It all seems to work great and I'm pumped I can receive and send all my JSON using overlay types.

Implementing OAuth with GWT

Matt Raible — Thu, 18 Jun 2009 13:59:13 -0600

I've heard about OAuth for quite some time, but never had an opportunity to implement it on a project. For a good explanation of what OAuth is, see its Introduction. Here's an excerpt:

...it allows you the User to grant access to your private resources on one site (which is called the Service Provider), to another site (called Consumer, not to be confused with you, the User). While OpenID is all about using a single identity to sign into many sites, OAuth is about giving access to your stuff without sharing your identity at all (or its secret parts).

The reason I needed OAuth was to interact with the Google Contacts API. I've always hated how sites make you import all your contacts from Gmail. I wanted to develop a system that'd let you simply read your contacts from Google in real-time.

Since the application I'm working on uses GWT, I chose to implement an OAuth client in GWT. After googling for "gwt oauth", I found two examples. Unfortunately, neither worked out-of-the-box.

The good news is I did manage to create a working solution. The bad news is it only seems to work at random. That's right folks, I created a solution that only works 50% of the time. I'm somewhat embarrassed to post it here, but I also realize the power of open source and community. By sharing, I hope we can find the flaws in my logic and come up with a solution for all GWT applications.

The best project for OAuth libraries seems to be oauth on Google Code. However, you'll notice that there is no JavaScript implementation listed on the homepage. I did look at the Java implementation, but quickly realized it wouldn't be usable in GWT. Therefore, I opted for the JavaScript implementation.

OAuth consists of several steps. The following diagram explains the authentication flow nicely.

In a nutshell, you have to complete the following steps:

Get a token from the service provider.
Redirect user to service provider to grant access and redirect back to application.
Request access token to access protected resources.
Access protected resources and pull/push data.

To access a service provider's OAuth service, you'll likely need to start by registering your application. For Google, OAuth Authentication for Web Applications is an excellent resource. Google's OAuth Playground is a great way to with the Google Data APIs after you've registered.

Now that you know how OAuth works, let's look at how I implemented it with GWT. I started by adding the necessary JavaScript references to my *.gwt.xml file.

<script src="//oauth.googlecode.com/svn/code/javascript/oauth.js"/>
<script src="//oauth.googlecode.com/svn/code/javascript/sha1.js"/>

Next, I needed a way to sign the request. I tried to use Sergi Mansilla's OAuth.java for this, but discovered issues with how the parameters were being written with GWT 1.6. I opted for Paul Donnelly's makeSignedRequest function instead. By adding this to my application's HTML page, I'm able to call it using the following JSNI method:

public native static String signRequest(String key, String secret, String tokenSecret, String url) /*-{
    return $wnd.makeSignedRequest(key, secret, tokenSecret, url);
}-*/;

After the URL is signed, it needs to be sent to the provider to get a request token. To do this, I used GWT's RequestBuilder and created a send() method:

protected void send(RequestCallback cb, String URL) {
    RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, URL);
    builder.setTimeoutMillis(10000);
    builder.setCallback(cb);
    
    Request req = null;
    try {
        req = builder.send();
    } catch (RequestException e) {
        cb.onError(req, e);
    }
}

If you try this with Google's Request Token URL in GWT's hosted mode, nothing will happen. Compile/browse to Safari and you'll still see nothing. Try it in Firefox and you'll see the following.

To workaround browsers' Same Origin Policy, I added a proxy servlet to send the requests. I started with Jason Edwards's ProxyServlet and modified it to fit my needs. I then registered it in both *.gwt.xml and web.xml.

<servlet path="/google/" class="org.appfuse.gwt.servlet.AlternateHostProxyServlet"/>

Now, before calling the send() method, I replace the start of the URL so the request would be routed through the servlet.

public void getToken(RequestCallback cb) {
    String url = signRequest(provider.getConsumerKey(), 
                             provider.getConsumerSecret(), 
                             "", provider.getRequestTokenURL());
    url = url.replace("https://www.google.com/", "/google/");
    send(cb, url);
}

When the request returns, I create two cookies by calling a createOAuthCookies() method with the payload returned:

public static String[] createOAuthCookies(String data) {
    String oauth_token = data.substring(data.indexOf("oauth_token=") + 12);
    oauth_token = oauth_token.substring(0, oauth_token.indexOf("&"));

    String oauth_token_secret = data.substring(data.indexOf("oauth_token_secret=") + 19);

    Cookies.setCookie("oauth_token", URL.decode(oauth_token));
    Cookies.setCookie("oauth_token_secret", URL.decode(oauth_token_secret));
    return new String[]{oauth_token, oauth_token_secret};
}

The next step is to authorize the token. This is where things got tricky with my proxy servlet and I had to add some special logic for GWT. Google was sending back a 302 with a Location header, but it wasn't hitting the onResponseReceived() method in my callback. For this reason, I had to change it to a 200 status code and add the redirect location to the body. I also discovered that sometimes they'd return an HTML page with a <meta http-equiv="refresh" ...> tag. When using Twitter, I discovered the full HTML for the allow/deny page was returned. Below is the callback I'm using. WindowUtils is a class I got from Robert Hanson and the gwt-widget project.

public void onResponseReceived(Request request, Response response) {
    String text = response.getText();
    if (response.getStatusCode() == 200 && response.getText().startsWith("http")) {
        WindowUtils.changeLocation(response.getText());
    } else {
        // look for meta-tag that refreshes and grab its URL
        if (text.contains("";
            String url = text.substring(text.indexOf(tokenToStartWith) + tokenToStartWith.length());
            url = url.substring(0, url.indexOf(tokenToEndWith) + tokenToEndWith.length());
            WindowUtils.changeLocation(url);
        } else {
            // Twitter returns a full HTML page, so redirect to the authorize URL manually
            if (provider instanceof Twitter) {
                String url = provider.getAuthorizeTokenURL();
                url = url.replace("$1", OAuthRequest.getAuthToken());
                url = url.replace("$2", DefaultRequest.getCurrentLocation());
                WindowUtils.changeLocation(url);
            } else {
                onError(request, new RequestException(text));
            }
        }
    }
}

public void onError(Request request, Throwable caught) {
    Window.alert("Calling authorize token failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + caught.getMessage());
}

The 3rd step is to get an access token. The most important thing to remember when you do this is to include the "oauth_token_secret" value when signing the request.

signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
            getAuthTokenSecret(), url);

After this completes with a 200, I create the cookies again (since oauth_token and oauth_token_secret are returned in the body), then call the API to get a list of contacts. The ContactsRequests class is responsible for making the call. The DefaultRequest class contains the send() method as well as utility methods to get the cookie values of the oauth tokens.

public class ContactsRequest extends DefaultRequest {
    private static final String GOOGLE_CONTACTS_URL = 
        "http://www.google.com/m8/feeds/contacts/default/thin?oauth_token=$1";
    private OAuthProvider provider;

    public ContactsRequest(OAuthProvider provider) {
        this.provider = provider;
    }

    public void getContacts(RequestCallback cb) {
        String url = GOOGLE_CONTACTS_URL.replace("$1", getAuthToken());
        url = signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
                          getAuthTokenSecret(), url);

        String proxiedURLPrefix = "/contacts/";
        // allow for deploying at /gwt-oauth context
        if (WindowUtils.getLocation().getPath().contains("gwt-oauth")) {
            proxiedURLPrefix = "/gwt-oauth" + proxiedURLPrefix;
        }

        url = url.replace("http://www.google.com/", proxiedURLPrefix);

        send(cb, url);
    }
}

If all goes well, the response contains the data you requested and it's used to populate a textarea (at least in this demo application). Of course, additional processing needs to occur to parse/format this data into something useful.

This all sounds pretty useful for GWT applications, right? I believe it does - but only if it works consistently. I sent a message to the OAuth Google Group explaining the issues I've had.

I'm trying to use the JavaScript API to authenticate with OAuth from a GWT application. I've got it working with both Google and Twitter's OAuth implementations. However, it seems to fail to sign the URL at random. In other words, it works 1 out of 3 times. ... Any idea why this could be happening?

I received a response with a cleaner makeSignedRequest() function. I tried it and, unfortunately, it seems to be equally unreliable. I suspect the problem is with the OAuth JavaScript implementation, GWT's interpretation of it, or that OAuth isn't as mature as it appears to be. I'd like to think one of the first two causes the problem.

To make it easier to create a robust example of GWT and OAuth, I created a gwt-oauth project you can download or view online. Please keep in mind the demo is likely to be flakey. If you're persistent and try enough times, it's likely to work. Firefox seems to succeed moreso than Safari or Chrome. If you have any suggestions for improving this example, please let me know.

Enhancing Evite.com with GWT and Grails

Matt Raible — Mon, 15 Jun 2009 07:41:37 -0600

On my LinkedIn Profile, it says my current gig is a SOFEA consultant at a stealth-mode startup.

SOFEA Consultant, Stealth Mode Startup, Los Angeles, CA. December 2008 -- Present.

OK, I lied. It's not a startup, it's a well-known company that helps you plan parties. For the last 5+ months, my UI team from LinkedIn has been working with Evite.com to enhance portions of their site with a SOFEA architecture.

In January, we started evaluating Ajax Frameworks and came to the conclusion that GWT was right for us. After we chose the UI framework, other team members chose Grails and memcached to develop scalable RESTful services. The architecture we implemented involves using GWT's RequestBuilder to talk to Grails' services, which cache almost all their JSON output in memcached.

To see an example of a feature we developed with GWT, see Evite's Design Gallery. I personally worked on this feature and very much enjoyed becoming a GWT aficionado in the process. GWT's zero-turnaround feature made doing pure client-side work a lot of fun. It's definitely something I'd like to continuing doing at my next gig.

Everyone from Evite is very happy with what we've been able to do with GWT and Grails. We have a stateless architecture and are quickly able to develop both client-side and server-side features. We've learned to scale the client by using out-of-the-box GWT components. We've scaled Grails by caching as much as possible. We serve up Ads and Analytics using the same JavaScript mechanisms that traditional server-side frameworks use.

At the end of this month, my gig with Evite comes to an end. I'll be spending a few weeks at my family's cabin in Montana and then it's on to the next big thing. What's the next big thing? I'm not sure yet, hence the reason for writing this. If you're looking to develop a GWT application, introduce a SOFEA architecture at your company, or simply adopt some open source frameworks, I'd love to help out. Drop me a line and let's start a conversation.

Creating a Facebook-style Autocomplete with GWT

Matt Raible — Fri, 5 Jun 2009 07:05:10 -0600

Have you used the "To:" widget on on Facebook or LinkedIn when composing a message? It's an autocompleter that looks up contact names and displays them as you type. It looks like a normal textbox (a.k.a. <input type="text">), but wraps the contact name to allow you to easily delete it. Here's a screenshot of what Facebook's widget looks like.

Last week, I was asked to create a similar widget with GWT. After searching the web and not finding much, I decided to try writing my own. The best example I found on how to create this widget was from James Smith's Tokenizing Autocomplete jQuery Plugin. I used its demo to help me learn how the DOM changed after you selected a contact.

GWT's SelectBox allows you to easily create an autocompleter. However, it doesn't have support for multiple values (for example, a comma-delimited list). The good news is it's not difficult to add this functionality using Viktor Zaprudnev's HowTo. Another feature you might want in a SelectBox is to populate it with POJOs. GWT SuggestBox backed by DTO Model is a good blog post that shows how to do this.

Back to the Facebook Autocompleter. To demonstrate how to create this widget in GWT, I put together a simple application. You can view the demo or download it. The meat of this example is in an InputListWidget. After looking at the jQuery example, I learned the widget was a <div> with a unordered list (<ul>). It starts out looking like this:

<ul class="token-input-list-facebook">
    <li class="token-input-input-token-facebook">
        <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
    </li>
</ul>

I did this in GWT using custom BulletList and ListItem widgets (contained in the download).

final BulletList list = new BulletList();
list.setStyleName("token-input-list-facebook");

final ListItem item = new ListItem();
item.setStyleName("token-input-input-token-facebook");

final TextBox itemBox = new TextBox();
itemBox.getElement().setAttribute("style", 
        "outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;");

final SuggestBox box = new SuggestBox(getSuggestions(), itemBox);
box.getElement().setId("suggestion_box");

item.add(box);
list.add(item);

After tabbing off the input, I noticed that it was removed and replaced with a <p> around the value and a <span> to show the "x" to delete it. After adding a couple items, the HTML is as follows:

<ul class="token-input-list-facebook">
    <li class="token-input-token-facebook">
        <p>What's New Scooby-Doo?</p>
        <span class="token-input-delete-token-facebook">x</span>
    </li>
    <li class="token-input-token-facebook">
        <p>Fear Factor</p>
        <span class="token-input-delete-token-facebook">x</span>
     </li>
     <li class="token-input-input-token-facebook">
         <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
     </li>
</ul>

To do this, I created a deselectItem() method that triggers the DOM transformation.

private void deselectItem(final TextBox itemBox, final BulletList list) {
    if (itemBox.getValue() != null && !"".equals(itemBox.getValue().trim())) {
        /** Change to the following structure:
         * <li class="token-input-token-facebook">
         * <p>What's New Scooby-Doo?</p>
         * <span class="token-input-delete-token-facebook">x</span>
         * </li>
         */

        final ListItem displayItem = new ListItem();
        displayItem.setStyleName("token-input-token-facebook");
        Paragraph p = new Paragraph(itemBox.getValue());

        displayItem.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                displayItem.addStyleName("token-input-selected-token-facebook");
            }
        });

        Span span = new Span("x");
        span.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                list.remove(displayItem);
            }
        });

        displayItem.add(p);
        displayItem.add(span);
        
        list.insert(displayItem, list.getWidgetCount() - 1);
        itemBox.setValue("");
        itemBox.setFocus(true);
    }
}

This method is called after selecting a new item from the SuggestBox:

box.addSelectionHandler(new SelectionHandler<SuggestOracle.Suggestion>() {
    public void onSelection(SelectionEvent selectionEvent) {
        deselectItem(itemBox, list);
    }
});

I also added the ability for you to type in an e-mail address manually and to delete the previous item when you backspace from the input field. Here's the handler that calls deselectItem() and allows deleting with backspace:

// this needs to be on the itemBox rather than box, or backspace will get executed twice
itemBox.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
            // only allow manual entries with @ signs (assumed email addresses)
            if (itemBox.getValue().contains("@"))
                deselectItem(itemBox, list);
        }
        // handle backspace
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            if ("".equals(itemBox.getValue().trim())) {
                ListItem li = (ListItem) list.getWidget(list.getWidgetCount() - 2);
                Paragraph p = (Paragraph) li.getWidget(0);

                list.remove(li);
                itemBox.setFocus(true);
            }
        }
    }
});

I'm happy with the results, and grateful for the jQuery plugin's CSS. However, it still has one issue that I haven't been able to solve: I'm unable to click on a list item (to select it) and then delete it (with the backspace key). I believe this is because I'm unable to give focus to the list item. Here's the code that highlights the item and you can see the commented-out code that doesn't work.

displayItem.addClickHandler(new ClickHandler() {
    public void onClick(ClickEvent clickEvent) {
        displayItem.addStyleName("token-input-selected-token-facebook");
    }
});

/** TODO: Figure out how to select item and allow deleting with backspace key
displayItem.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            list.remove(displayItem);
        }
    }
});
displayItem.addBlurHandler(new BlurHandler() {
    public void onBlur(BlurEvent blurEvent) {
        displayItem.removeStyleName("token-input-selected-token-facebook");
    }
});
*/

If you know of a solution to this issue, please let me know. Feel free to use this widget and improve it as you see fit. I'd love to see this as a native widget in GWT. In the meantime, here's the GWT Facebook-style Autocomplete demo and code.

Ajax Framework Analysis Results

Matt Raible — Thu, 23 Apr 2009 20:34:44 -0600

Way back in January, I wrote about how my colleagues and I were evaluating Ajax frameworks to build a SOFEA-style architecture. To make our choice, we used the following process:

Choose a short list of frameworks to prototype with.
Create an application prototype with each framework.
Document findings and create a matrix with important criteria.
Create presentation to summarize document.
Deliver document, presentation and recommendation.

When I wrote that entry, we had just finished step 2 and were starting step 3. I first wrote this blog post a week later, when we delivered step 5. Here is the comparison and conclusion sections of the analysis document we composed.

Framework Comparison
In order to evaluate the different frameworks against important criteria, we created a matrix with weights and ranks for each framework. This matrix shows how our weighting and rankings lead us to the winner for our project. You can view this matrix online or see below for a summary.

Note: Criteria whose values were identical across all candidates were weighted at zero. Charting capability was weighted at zero b/c we decided to use Flash for this.

This matrix indicates that GWT is the best candidate for our team to develop SOFEA-style applications with. In addition to the matrix, below are graphs that illustrate interesting (and possibly meaningless) statistics about each project.

Conclusion
After working with the various frameworks, we believe that all the frameworks were very good and could be used to write applications with. If all weights are equal, these frameworks were almost even when compared against our evaluation criteria. The graph below illustrates this.

Even after applying the weighted criteria, the evenness doesn't change a whole lot.

Without considering the even or weighted criteria, we believe the decision all comes down to what the developers on the project feel they will be most comfortable with. If you're developing with Dojo or YUI, chances are you're dressing up existing HTML and possibly using progressive enhancement to add more rich functionality. On the other hand, Ext JS and GWT are similar to Swing programming where you build the UI with code (JavaScript for Ext JS, Java for GWT).

The tools available for JavaScript development have gotten increasingly better in recent years. IntelliJ IDEA has a JavaScript Editor that provides many of the same features as its Java editor. Aptana Studio also has excellent support for authoring and debugging JavaScript. However, we believe the Java debugging and authoring support in IDEs is much better. Furthermore, we are more familiar with organizing code in Java projects and feel more comfortable in this development environment.

Based on this evaluation, we believe that GWT is the best framework for our team to develop SOFEA-style applications with.

Flash Forward to Today...
The core GWT library from Google doesn't have a whole lot of widgets, nor do they look good out-of-the-box. So early on, we experimented with two alternative implementations that continue to leverage GWT concepts and tools:

GXT: a GWT version of Ext JS
SmartGWT: a GWT version of SmartClient

Unfortunately, over the past few months, we've found that both of these implementations are too heavy for our requirements, mostly because of the file size of the generated JavaScript code. For example, a feature I wrote generated a 275K *.cache.html file using GXT. After determining that was too slow to give users the initial "pop", I re-wrote it without GXT. After a day, we had an application with *.cache.html files of 133K. Yes, that's over a 50% reduction in size!*

Because of these findings, we are proceeding with the core GWT library from Google and adding in new components as needed. It is cool to know you can make a UI "pop" with GWT, as long as you stick to the core - close-to-the-metal - components. For those applications that can afford an initial "loading..." state, I'd definitely recommend looking at GXT and SmartGWT.

* To make refactoring easier, I copied GXT MVC into our source tree and modified all imports.

Optimizing a GWT Application with Multiple EntryPoints

Matt Raible — Wed, 25 Mar 2009 16:00:37 -0600

Building a GWT application is an easy way for Java Developers to write Ajax applications. However, it can be difficult to release a GWT application to production before it's finished. One of the most important things I've learned in Software Development is to get a new application into production as soon as possible. Not only does getting it from dev → qa → prod verify your process works, it also can do a lot to test the viability of the new application.

One of the biggest issues with GWT applications is size. The project I'm working on compiles Java to JavaScript and creates ~570K *.cache.html files (one for each modern browser). These files end up being around 180K gzipped. I believe this is an OK size for an entire application. However, if you're going to release early, release often with GWT, chances are you'll just want to release one feature at a time.

When the first feature was completed on my project, the *.cache.html files were around 300K. Rather than using branches to release to QA and UAT, bug fixes and new features were developed on trunk. Unfortunately, the QA and UAT process took several weeks longer than expected so by the time the feature was ready to release, the *.cache.html files had grown to around ~570K. The reason the file had grown so much was because it included all of the other features.

Earlier this week, while running to a dentist appointment, I thought of a solution to this problem. The basic idea was to optimize the compilation process so only the to-be-released feature was included. Even better, the solution didn't require more modularization. The results:

Before: *.cache.html -> 569K, gzipped 175K
After: *.cache.html -> 314K, gzipped 100K

According to my calculations, that's a 56% reduction in size. How did I do it?

Created a new FeatureName.java EntryPoint with only the to-be-released features imported.
Created a new FeatureName.gwt.xml that references the new EntryPoint.
Copied old (kitchen-sink) EntryPoint.html to FeatureName.html and changed the reference to the nocache.js file.
Created a Maven profile that allows using -PFeatureName to build a FeatureName-only module.

One downside to doing things this way is it's possible to create a WAR that has the same name and different features. Surely the Maven Overlords would frown upon this. Since this is just a temporary solution to release features incrementally, I'm not too worried about it. A possible workaround is to create different WAR names when a feature's profile is activated. I believe the true "Maven way" would be to make the "kitchen sink" application into a JAR and have several WAR modules with the different EntryPoints. Seems a bit complicated to me.

Other than this Maven publishing issue, the only other issue I can foresee is keeping the two EntryPoints and HTML files in synch. Then again, the separate files allow a feature to be customized for the release and can be deleted when its no longer needed.

What do you think? Do you know of a better way to compile a GWT application so it only contains certain features?

Modularizing GWT Applications with GWT-Maven

Matt Raible — Mon, 23 Mar 2009 10:36:08 -0600

Last week, I spent some time modularizing the GWT application I'm working on. By modularizing, I mean splitting the code from one GWT module into a "core" and "webapp" module. The reason for doing this was so the "core" module could be used by another GWT application. Creating GWT Modules is fairly straightforward, but it wasn't as intuitive as expected when using the gwt-maven-plugin.

The hardest part of moving the code was figuring out how to run tests in the new "core" module. After getting it all working, it seems easy enough. Hopefully this post will make it easy for others. Here's the steps I'd recommend:

Convert your GWT project into a multi-module project where you have a top-level pom.xml and two sub-modules (e.g. gwt-core and gwt-webapp).
Do the normal single-to-multi-project Maven stuff like declaring the <parent> element in the modules and moving plugins/dependencies to the top-level pom.xml.
Refactor your gwt-webapp project to push down all shared classes (and their tests) to gwt-core.

In the gwt-core project, include *.xml and *.java in your JAR so GWT can extract/compile the source code when building gwt-webapp.

<resources>
    <resource>
        <directory>src/main/java</directory>
        <includes>
            <include>**/*.java</include>
            <include>**/*.xml</include>
        </includes>
    </resource>
</resources>

In gwt-core/src/main/java, create a Core.gwt.xml that references the modules you'd like to use in all your applications. For example:

<module>
    <inherits name="com.google.gwt.user.User"/>
    <inherits name="com.google.gwt.i18n.I18N"/>
    <inherits name="com.extjs.gxt.ui.GXT"/>
    <inherits name="pl.rmalinowski.gwt2swf.GWT2SWF"/>
</module>

Now the tricky part begins, mostly because of how the gwt-maven plugin currently works. In src/test/java, create a NoOpEntryPoint.gwt.xml that inherits your Core module and defines an EntryPoint.
```
<module>
    <inherits name="com.company.app.Core"/>
    <entry-point class="com.company.app.NoOpEntryPoint"/>
</module>
```

Create a NoOpEntryPoint.java class in the same directory as NoOpEntryPoint.gwt.xml.

public class NoOpEntryPoint implements EntryPoint {
    
    public void onModuleLoad() {
        // do nothing
    }
}

In any class that extends GWTTestCase (I usually create a parent class for all tests), reference the NoOpEntryPoint in the getModuleName() method.
```
    @Override
    public String getModuleName() {
        return "com.company.app.NoOpEntryPoint";
    }
```

Lastly, in the gwt-maven plugin's configuration (in gwt-core/pom.xml), reference the NoOpEntryPoint in <compileTargets>, a non-existent file in <runTarget> and only the "test" goal in the executions.

<plugin>
    <groupId>com.totsp.gwt</groupId>
    <artifactId>maven-googlewebtoolkit2-plugin</artifactId>
    <version>2.0-beta26</version>
    <configuration>
        <compileTargets>
            <value>com.company.app.NoOpEntryPoint</value>
        </compileTargets>
        <runTarget>com.company.app.NoOpEntryPoint/doesntexist.html</runTarget>
        <logLevel>INFO</logLevel>
        <style>OBF</style>
        <noServer>false</noServer>
        <extraJvmArgs>-Xmx512m</extraJvmArgs>
        <gwtVersion>${gwtVersion}</gwtVersion>
        <testFilter>*GwtTestSuite.java</testFilter>
        <testSkip>${skipTests}</testSkip>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

GXT's MVC Framework

Matt Raible — Fri, 13 Mar 2009 11:48:41 -0600

For the past couple of months, I've been developing a GWT application using a mix of plain ol' GWT and GXT widgets. When I first started developing it, I didn't know how to best organize my code and separate the logic. The solution I came up with was to adopt some sort of MVC framework. Since I was already using GXT, I opted for GXT's lightweight MVC implementation.

As mentioned in Testing GWT Applications, GXT's MVC doesn't have much documentation. The best reference documentation seems to be Christian's Getting started with Ext-GWT: The Mail reference application.

Page Transitioning with Dispatcher
After working with GXT MVC for a couple months, I'm still not sure I fully understand how navigation and event dispatching works. The biggest point of confusion for me is how to best use GXT's Dispatcher class.

The problem with Dispatcher is it has a two methods that seem to do the same thing.

forwardEvent (4 variations)
dispatch (3 variations)

In addition to these methods in Dispatcher, there's two fireEvent methods in GXT's View class. According to my calculations, that means there's 9 different options for transitioning from one view to the next. Which one is best to use?

From what I've learned, I think it's best to use fireEvent in Views and forwardEvent in Controllers and other widgets. IMO, dispatcher should never be used except in your HistoryListener's implementation onHistoryChanged method. The important thing to realize about this method is it should only work if the View's Controller is registered for the event.

  protected void fireEvent(AppEvent event) {
    Controller c = controller;
    while (c != null) {
      if (c.canHandle(event)) {
        c.handleEvent(event);
      }
      c = c.parent;
    }
  }

However, fireEvent seems to work even when the View's Controller isn't registered for that event. This is because onHistoryChanged gets called in the EntryPoint. For experienced GXT MVC users, does this navigation handling mesh with your findings?

The most important thing for navigation to work successfully is enabling History support. The next section talks about how to do this effectively.

Enabling History Support
To help explain things better, I created a simple GWT MVC Example application and used Maven to create an archetype with it. You can create a project from the archetype using the following command:

mvn archetype:create -DarchetypeGroupId=org.appfuse.archetypes \
-DarchetypeArtifactId=gwt-mvc -DarchetypeVersion=1.0-SNAPSHOT \
-DgroupId=com.mycompany.app -DartifactId=myproject \
-DremoteRepositories=http://oss.sonatype.org/content/repositories/appfuse-snapshots

To enable history support in this application, I implemented HistoryListener in my EntryPoint (Application.java) and added the following logic to initialize:

// If the application starts with no history token, redirect to 'home' state
String initToken = History.getToken();
if (initToken.length() == 0) {
    History.newItem(HistoryTokens.HOME);
}

// Add history listener
History.addHistoryListener(this);

// Now that we've setup our listener, fire the initial history state.
History.fireCurrentHistoryState();

In this example, HistoryTokens is a class that contains all the URLs of the "views" in the application.

public class HistoryTokens {
    public static final String HOME = "home";
    public static final String CALENDAR = "calendar";
    public static final String NOTES = "notes";
    public static final String SEARCH = "search";
}

In order to make URLs like http://localhost:8080/#calendar go to the calendar view, the following logic exists in the onHistoryChanged method.

        Dispatcher dispatcher = Dispatcher.get();

        if (historyToken != null) {
            if (historyToken.equals(HistoryTokens.HOME)) {
                dispatcher.dispatch(AppEvents.GoHome);
            } else if (historyToken.equals(HistoryTokens.CALENDAR)) {
                dispatcher.dispatch(AppEvents.Calendar);
            } else if (historyToken.equals(HistoryTokens.NOTES)) {
                dispatcher.dispatch(AppEvents.Notes);
            } else if (historyToken.equals(HistoryTokens.SEARCH)) {
                dispatcher.dispatch(AppEvents.Search);
            } else {
                GWT.log("HistoryToken '" + historyToken + "' not found!", null);
            }
        }

Controllers are registered in the EntryPoint as follows:

        final Dispatcher dispatcher = Dispatcher.get();
        dispatcher.addController(new CalendarController());
        dispatcher.addController(new HomeController());
        dispatcher.addController(new NotesController());
        dispatcher.addController(new SearchController());

Controllers respond to events they're registered for. This is done in their constructor:

    public CalendarController() {
        registerEventTypes(AppEvents.Calendar);
    }

In order for navigation to work, you have to create links with history tokens¹. For example, here's a link from the HomeView class:

	Hyperlink notesLink = new Hyperlink("Notes", HistoryTokens.NOTES);
	notesLink.addClickListener(new ClickListener() {
	    public void onClick(Widget widget) {
	        Dispatcher.get().fireEvent(AppEvents.Notes);
	    }
	});

You'll notice in this example, I'm using Dispatcher's fireEvent method. If I wanted to pass some data with your event, you'll need to use forwardEvent. Here's an example from CalendarView:

    Button submit = new Button("Submit");

    submit.addSelectionListener(new SelectionListener<ButtonEvent>() {
        public void componentSelected(ButtonEvent ce) {
            AppEvent<Date> event = 
                new AppEvent<Date>(AppEvents.GoHome, date.getValue(), HistoryTokens.HOME);
            Dispatcher.forwardEvent(event);
        }
    });

In this example, you could also use Dispatcher.dispatcher(), but I believe this will cause the transition to happen twice because the onHistoryChanged method gets called too. This doesn't matter for the most part, except when you start to use DispatcherListeners.

Hopefully this article has helped you understand how GXT's MVC framework works. I'm interested in learning how other GWT MVC frameworks work. If you've used one, I'd love to hear about your experience.

Friday Fun Test
Here's a test for those interested in digging into the GXT MVC example. There's a bug in this application that prevents something from happening. I'll buy a drink for the person that finds the bug and I'll buy two drinks for the person that comes up with a solution.

1. If you use the default constructor on Hyperlink and use setText(), make sure to call setTargetHistoryToken() too. If you don't, a blank history token will be used and # causes the browser to scroll to the top before a page transition happens.

GWT and AppFuse

Matt Raible — Wed, 4 Mar 2009 22:50:26 -0700

Someone recently sent me the following e-mail asking about GWT integration in AppFuse.

I see from your blog that you're spending some time with GWT at the moment. What's your plan, are you going to integrate GWT as another UI Option for AppFuse?

The reason I'm asking is that I actually checked out all of the AppFuse code from the svn repository yesterday, with the intention of starting off adding some GWT stuff in there. My intention was to start by getting a basic Maven archetype together for GWT as an AppFuse UI.

However, if you're planning on doing this yourself in the near future, then there's no point in me starting doing it, I'd have to learn how to write archetype's for a start (not that it looks too difficult) but you'd obviously do it much quicker.

Being a good open-source developer, I moved the discussion to the developer mailing list and replied there:

It's likely I'll create a version of AppFuse Light with GWT, but I doubt I'll do it in the near future. I hope to release AppFuse 2.1 first (which will include "light" archetypes). I wouldn't get your hopes up in waiting for me to do the work. However, I'd be happy to assist you in doing it. AppFuse Light is now modular and uses the AppFuse backend.

http://raibledesigns.com/rd/entry/appfuse_light_converted_to_maven

Here's how I believe GWT should be integrated:

Create an appfuse-ws archetype that serves up RESTful services (http://issues.appfuse.org/browse/APF-897).
Create an appfuse-gwt archetype that consumes those services. This archetype would contain a proxy servlet that allows #1 to be on a separate host/port.

In addition to #1, I hope to convert the Struts 2 and Spring MVC archetypes to use those frameworks' REST support.

For #2, we could use SmartGWT or GXT. SmartGWT might be better since Sanjiv is a committer on this project.

I know I've been slacking on AppFuse development, but it is ski season and running to work seems to drain my late-night coding ambitions. With that being said, I'm committed to getting AppFuse 2.1 released by JavaOne (hopefully sooner). I figure it's a good week's worth of work and I'll probably have to do it late at night to find the time. That's OK though, I usually really start to enjoy it once I get into it.

GWTTestSuite makes builds faster, but requires JUnit 4.1

Matt Raible — Fri, 27 Feb 2009 11:58:12 -0700

Earlier this week, I spent some time implementing GWTTestSuite to speed up my project's build process. In Hudson, the project was taking around 15 minutes to build, locally it was only taking 5 minutes for mvn test. In IDEA, I could run all the tests in under a minute. While 15 minutes isn't a long time for a build to execute, a co-worker expressed some concern:

Does Maven have to run GWT test and individual Java processes? (See target/gwtTest/*.sh) This arrangement and the overhead of JVM launches is another reason why builds take so long. As we add more GWT tests we are going to test that LinkedIn record for the slowest build ever.

After this comment, I started looking into GWTTestSuite using Olivier Modica's blog entry as a guide. It was very easy to get things working in IDEA. However, when I'd run mvn test, I'd get the following error:

Error: java.lang.ClassCastException

No line numbers. No class information. Zilch. After comparing my project's pom.xml with the one from the default gwt-maven archetype, I noticed the default used JUnit 4.1, while I had the latest-and-supposedly-greatest JUnit 4.4. Reverting to JUnit 4.1 fixed the problem. Now Hudson takes 3:15 to execute the build instead of 15 minutes.

The reason for this blog post is this doesn't seem to be documented anywhere. Hopefully other developers will find this entry when googling for this issue.

Related to making GWT faster, I also added the following line to my Application.gwt.xml file:

<set-property name="user.agent" value="safari" />

This dropped the gwt:compile time from 1 minute to 25 seconds. As explained in the documentation, you can use the "user.agent" setting to only generate one JS file for your app instead of 4. The strange thing about adding this setting was I pretty much forgot about it since everything seemed to work fine on both Safari and Firefox. When I started testing things in IE6, I started seeing a lot of JavaScript errors. After debugging for an hour or so, I realized this setting was there, removed it, and everything started working great in all browsers.

Now if I could just figure out how to use safari-only for development, but remove the line when building the WAR. Suggestions welcome.

Enhancing your GWT Application with the UrlRewriteFilter

Matt Raible — Mon, 23 Feb 2009 17:02:29 -0700

Last week, I spent some time trying to change the location of my cache/nocache HTML files in my GWT project. I started the project with the gwt-maven-plugin's archetype. The message I posted to the gwt-maven Google Group is below.

Rather than having my application's HTML file in src/main/java/com/mycompany/Application.html, I'd like to move it to src/main/webapp/index.html. I tried copying the HTML and adding the following to my index.html, but no dice:

<meta name="gwt:module" content="com.mycompany.Application"/>

Is this possible with the gwt-maven-plugin? I'd like to have my main HTML and CSS at the root of my application.

The good news is I figured out a solution using the UrlRewriteFilter that 1) allows hosted mode to work as usual and 2) allows your app to be served up from the root URL (/ instead of /com.company.Module/Application.html). Here's the urlrewrite.xml that makes it all possible.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCENGINE urlrewrite PUBLIC "-//tuckey.org//DTD UrlRewrite 3.0//EN"
        "http://tuckey.org/res/dtds/urlrewrite3.0.dtd">

<urlrewrite>
    <rule>
        <from>^/$</from>
        <to type="forward" last="true">/com.mycompany.app.Application/Application.html</to>
    </rule>
    <rule>
        <from>/index.html</from>
        <to type="forward" last="true">/com.mycompany.app.Application/Application.html</to>
    </rule>
    <-- This last rule is necessary for JS and CSS files -->
    <rule>
        <from>^/(.*)\.(.*)$</from>
        <to type="forward">/com.mycompany.app.Application/$1.$2</to>
    </rule>
</urlrewrite>

If you're using the gwt-maven plugin, this file goes in src/main/webapp/WEB-INF. In addition, you'll need to add the following to your web.xml.

    <filter>
        <filter-name>rewriteFilter</filter-name>
        <filter-class>org.tuckey.web.filters.urlrewrite.UrlRewriteFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>rewriteFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

Finally, add the UrlRewriteFilter dependency in your pom.xml:

    <dependency>
        <groupId>org.tuckey</groupId>
        <artifactId>urlrewritefilter</artifactId>
        <version>3.1.0</version>
    </dependency>

Please let me know if you have any questions.

Update: Jeff posted an alternative configuration that allows you to eliminate the last rule in urlrewrite.xml, as well as use the beloved mvn jetty:run command. To use cleaner WAR packaging and the Jetty plugin, add the following to your pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <webappDirectory>
            ${project.build.directory}/${project.build.finalName}/com.mycompany.app.Application
        </webappDirectory>
    </configuration>
</plugin>
<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.14</version>
    <configuration>
        <webAppConfig>
            <contextPath>/</contextPath>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}/com.mycompany.app.Application
                </resourcesAsCSV>
            </baseResource>
        </webAppConfig>
        <scanIntervalSeconds>3</scanIntervalSeconds>
        <scanTargets>
            <scanTarget>${basedir}/src/main/resources</scanTarget>
            <scanTarget>${basedir}/src/main/webapp/WEB-INF</scanTarget>
            <scanTarget>
                ${project.build.directory}/${project.build.finalName}/com.mycompany.app.Application
            </scanTarget>
        </scanTargets>
    </configuration>
</plugin>

Then you can trim your urlrewrite.xml down to:

<urlrewrite>
    <rule>
        <from>^/$</from>
        <to type="forward" last="true">/Application.html</to>
    </rule>
    <rule>
        <from>/index.html</from>
        <to type="forward" last="true">/Application.html</to>
    </rule>
</urlrewrite>

Of course, you could also change the welcome-file in your web.xml or use index.html and the <meta http-equiv="REFRESH"> option. Personally, I have so much affection for the UrlRewriteFilter that I like having it in my project. I'm sure I'll need it someday.

Thanks Jeff!

Testing GWT Applications

Matt Raible — Mon, 9 Feb 2009 15:27:36 -0700

Last week, I did some research on GWT, how to test it and code coverage options for tests that extend GWTTestCase. The reason I did this is because I've found that most of the GWT tests I write have to extend GWTTestCase and I'd like to have code coverage reports. Read below for more information on my findings for testing GWT classes.

There are quite a few articles about testing GWT applications. Here are a few samples:

Test-First GWT
Test driven development for GWT UI code
Test driven development for GWT UI code with asynchronous RPC
Enabling Test Driven Development in GWT client code
JUnit tests for widgets - thread about which components to test.

The main gist of these articles is that you should structure your code to make the core functionality of your application testable without having to depend on GWTTestCase.

All of them also advocate using an MVC or MVP (Model View Presenter) pattern. Currently, I'm using GXT and its MVC Framework. Unfortunately, GXT's MVC doesn't have much documentation. The good news is there is a good article that explained enough that I was able to refactor my project to use it.

The unfortunate side of this refactoring was I discovered that classes that extend GXT's MVC Framework have to be tested with GWTTestCase. The downside to extending GWTTestCase is there is it's difficult to create code coverage reports.

GWT's issue 799 has some patches that should make code coverage possible. I tried to implement code coverage with Eclipse and EclEmma using this README, but failed. In the process, I discovered an issue with Eclipse 3.4 and JUnit on OS X. Reverting to Eclipse 3.3 solved this problem, but I was still unable to make EclEmma work with GWT.

After failing with Eclipse, I tried to use the emma-maven-plugin. I was also unable to get this to work, with my findings documented in this thread.

Finally, I did have some luck with getting IDEA's built-in code coverage feature working. However, after getting it to work once, it failed to work for the rest of the day and I haven't had success since.

Code Coverage and GWT
Because of these issues with GWT 1.5 and code coverage, I think I'll wait until GWT 1.6 to worry about it. The good news is 1.6 M1 was released last Friday. If continuing to use GWTTestCase becomes an issue, I may write my own MVC Framework that doesn't use classes that call native JavaScript. Hopefully GXT MVC's framework will provide a good example.

In addition to trying to get code coverage working, I used the internets to figure out how run GWT tests inside of Eclipse and IDEA. I don't remember the resources I used, but hopefully this up-to-date documentation will help others. The nice thing about using an IDE to run these tests is they typically execute much faster.

Running GWT Tests in Eclipse
You should be able to run most of your GWT tests from within Eclipse using the following steps.

Right-click on a test that extends GWTTestCase and go to Run As > JUnit Test. It's likely you will see the error message below.

Invalid launch configuration: -XstartOnFirstThread not specified.

On Mac OS X, GWT requires that the Java virtual machine be invoked with the
-XstartOnFirstThread VM argument.

Example:
  java -XstartOnFirstThread -cp gwt-dev-mac.jar com.google.gwt.dev.GWTShell

To fix this error, go to Run > Open Run Dialog. Click on the Arguments tab and add the following values. The 2nd value is to increase the amount of memory available to the test and avoid an OOM error.
```
-XstartOnFirstThread -Xmx512M
```

When you re-run the test, you will probably see the following error:

com.google.gwt.junit.JUnitFatalLaunchException: The test class 'org.richresume.client.home.HomeControllerGwtTest' 
was not found in module 'org.richresume.client.Application'; no compilation unit for that type was seen
  at com.google.gwt.junit.JUnitShell.checkTestClassInCurrentModule(JUnitShell.java:193)
  at com.google.gwt.junit.JUnitShell.runTestImpl(JUnitShell.java:628)
  at com.google.gwt.junit.JUnitShell.runTest(JUnitShell.java:150)
  at com.google.gwt.junit.client.GWTTestCase.runTest(GWTTestCase.java:219)

To fix this, open the Run Dialog again, click on the Classpath tab and click on User Entries. Click on the Advanced button and select Add Folders. In the Folder Selection dialog, select your source and test directories (e.g. src/main/java and src/test/java).
Run the test again and you should see a green bar in your JUnit tab.
To create a JUnit configuration that runs all tests, duplicate the previously mentioned run configuration. Then change the name to "All Tests" and select the 2nd radio button to run all tests in the project.
Click Run to execute all the tests in the project.

Running GWT Tests in IDEA
You should be able to run your GWT tests from within IDEA using the following steps.

Right-click on a test that extends GWTTestCase and go to Run "TestNameGwtTes...". It's likely you will see the error message below.

Invalid launch configuration: -XstartOnFirstThread not specified.

On Mac OS X, GWT requires that the Java virtual machine be invoked with the
-XstartOnFirstThread VM argument.

Example:
  java -XstartOnFirstThread -cp gwt-dev-mac.jar com.google.gwt.dev.GWTShell

If you get a compiler error instead, you may need to add the GWT Facet to your project. To do this, right-click on your project's top-most folder in the left pane. Select Module Settings > Facets and enable GWT for your module.
To fix the -XstartOnFirstThread issue, go to Run > Edit Configurations. Add the following values to the VM Arguments field. The 2nd value is to increase the amount of memory available to the test and avoid an OOM error.
```
-XstartOnFirstThread -Xmx512M
```
NOTE: If you still get a compiler error, see this page for a possible solution.
Run the test again and you should see a green bar in your Run tab.
To create a JUnit configuration that runs all tests, duplicate the previously mentioned run configuration. Then change the name to "All Tests" and change the Test configuration to search for tests in the whole project.
Run the new configuration to execute all the tests in the project.

Testing GWT applications isn't as straightforward as writing JUnit tests, but I do believe it's getting better. If you have any additional tips and tricks, please let me know.

Choosing an Ajax Framework

Matt Raible — Thu, 8 Jan 2009 21:36:22 -0700

This past week, my colleagues and I have been researching Ajax Frameworks. We're working on a project that's following SOFEA-style architecture principles and we want the best framework for our needs. I'm writing this post to see 1) if you, the community, agree with our selection process and 2) to learn about your experiences with the frameworks we're evaluating. Below is the process we're following to make our choice.

Choose a short list of frameworks to prototype with.
Create an application prototype with each framework.
Document findings and create a matrix with important criteria.
Create presentation to summarize document.
Deliver document, presentation (with demos) and recommendation.

For #1, we chose Ext JS, Dojo, YUI and GWT because we feel these Ajax libraries offer the most UI widgets. We also considered Prototype/Scriptaculous, jQuery and MooTools, but decided against them because of their lack of UI widgets.

For #2, we time-boxed ourselves to 3 days of development. In addition to basic functionality, we added several features (i.e. edit in place, drag and drop, calendar widgets, transitions, charts, grid) that might be used in the production application. We all were able to complete most of the functionality of the application. Of course, there's still some code cleanup as well as styling to make each app look good for the demo. The nice thing about doing this is we're able to look at each others code and see how the same thing is done in each framework. None of us are experts in any of the frameworks, so it's possible we could do things better. However, I think it's good we all started somewhat green because it shows what's possible for someone relatively new to the frameworks.

For #3, we're creating a document with the following outline:

Introduction

Ajax Framework Candidates
(intro and explanation)

  Project Information
  (history)
  (license / cost)
  (number of committers)
  (support options)
  (mailing list traffic (nov/dec 2008))

Matrix and Notes

Conclusion

For the Matrix referenced in the outline above, we're using a table with weights and ranks:

Weight	Criteria	Dojo	YUI	GWT	Ext JS	Notes
#	Important Criteria for Customer	0..1	0..1	0..1	0..1	Notes about rankings

Our strategy for filling in this matrix:

Customer adjusts the weight for each criteria (removing/adding as needed) so all weights add up to 1.
We rank each framework with 0, .5 or 1 where 0 = doesn't satisfy criteria, .5 = partially satisfies, 1 = satisfies.

The list of criteria provided to us by our client is as follows (in no particular order).

Quality of Documentation/Tutorials/Self Help
Browser support (most important browsers/versions based on web stats)
Testability (esp. Selenium compatibility)
Licensing
Project health/adoption
Performance
Scalability
Flexibility/extensibility
Productivity (app dev, web dev)
Richness of widget/component library
Charting capability
Ability to create new widgets
Match to existing Java team skill-set
Ease of deployment (on Ops, QA, Users)
Degree of risk generally
Ability to integrate with existing site (which includes Prototype)
Easy to style with CSS
Validation (esp. marking form elements invalid)
Component Theme-ing/Decoration
CDN Availability (i.e. Google's Ajax Libraries API or Ext CDN)

What do you think? How could this process be improved? Of course, if you have framework answers (0, .5 or 1) for our matrix, we'd love to hear your opinions.

RESTful Web Applications with Subbu Allamaraju

Matt Raible — Fri, 24 Oct 2008 09:52:02 -0600

Subbu works at Yahoo! developing standards, patterns and practices for HTTP web languages. In the past, he was a web service and Java developer. He was also a standards contributor at BEA and an author of books on Java EE. His current passion is HTTP and REST. Subbu confesses that he's not a web developer, has no interest in the internals of programming models used for web frameworks and he's only interested in the visible aspects of the architecture.

"The Web is Mostly Restful"

Being RESTful in an abstract sense means:

Resources are named by URIs
Resources have representations (Atom, HTML, JSON, XML)
Resources contain contextual links to allow navigation of state
There's a Uniform Interface

In the web today, some resources and URIs are personalized, but most are not. Some depend on sessions, but most do not. The consequence of a personalized UI with a non-unique URI is you cannot rely on browser caching.

The web is full of different representations (HTML, XML, JS, PDF, CSS, Flash). The problem with HTML is you can't tell links that you want a particular representation based of a link. The links are hard-coded to be a particular content-type. However, some media types on responses are ignored. This is often a problem with browsers and whether the user has plugins installed.

The Uniform Interface for the web is HTML and primarily links and forms (GET and POST). There's still some misconceptions (e.g. POST is secure). However, it's not about security, it's about idempotency and safety. You need to make sure you only use POST when you're changing data. POSTs are not repeatable. GET URIs are not always refreshable, which is quite unfortunate. Users shouldn't have to fight the back button.

Caching is a fundamental aspect of the web. Even in a personalized site, most of the content can be cached. The web is read-only for the most part. However, many enterprise web applications don't take advantage of caching. This is fine when there's not that many users, but it's bad when you want to scale to thousands of users. There's several frameworks that use cache-busting and prefer backend caching over HTTP caching. These frameworks are not using the web like they should.

Backend caching (e.g. Memcached) uses a non-uniform interface and you need to explicitly program to it. Frontend/HTTP caching has a uniform interface that's pluggable. Backend caching is generally more expensive to develop and deploy. There are cases where data should be cached on the backend, but you shouldn't focus all on backend caching w/o doing some frontend caching.

With Ajax, you get more opportunities to be RESTful. XMLHttpRequest is another HTTP client that can be programmed to. It has full support for the uniform interface, which allows content negotiation, optimistic concurrency and caching. Cross-domain hacks can be done with <script> and <iframe> to tunnel requests over GET. The W3C has been working for the last two years on how to do cross-domain Ajax w/o using hacks. The problem with current cross-domain implementations is they often use GET for everything, which isn't very RESTful. Subbu recommends using a proxy on the same domain if you do need to talk to other domains. This will allow your Ajax code to remain RESTful.

Web Frameworks
Web development is hard because of all the moving pieces that exist. Because of this, many web frameworks have been created to solve the various problems. In 1997, there were servlets. They provided basic plumbing and closely reflected HTTP/1.1. Servlets provided a poor programming model, but it allowed a lot of frameworks to be built on top of it. We don't use servlets to write applications, only to write application frameworks. The second era came about in 2001 when Action-oriented frameworks became popular. In 2004, JSF and friends came to play. JSF is a component-based framework with known limitations (complex, slow, uses POST for almost everything, Ajax is difficult). These limitations have resulted in a number of third-party patches being developed to solve these issues.

JSF was designed to use the request to create a component tree that maintains state. Unfortunately, the state is not something the developer has control over. It's not the state of the application, it's the state of the components. The client's knowledge of the state is mentioned with a cookie and the server keeps the state in the session. The problem with JSF is you don't have a choice of state in your application - you can't write stateless applications like you can with servlets.

JSF uses overloaded URIs for its resources. When you have one URI with multiple representations, there's no way to tell how a representation was chosen. JSF's compromise is to allow client-side state saving. However, they do this by putting hidden field in the form and requiring POST for navigation.

JSF vs. REST
Basically, these two are at opposite extremes. JSF is focused heavily on a UI component model. The people that developed it misinterpreted how the web works and made some fundamental questionable choices. You can patch it, but you can not fix it.

Web 2.0 Frameworks
GWT is a cross-compilation based framework. You write Java to generate JavaScript (b/c everyone hates writing JavaScript). It mashes client and server code into a single source. These layers communicate using GWT-RPC. Typical RPC concerns do not apply since code generation handles coupling and the client is downloaded from the same application. GWT-PRC does POSTs to the server and uses HTTP like a transport layer. To be fair, GWT does allow you to use a RequestBuilder to use the web like it should be used. This class allows more control over HTTP requests, it supports GET and POST and it allows so-called RESTful layers (GWT-REST and GET-Restlet). GWT is focused heavily on ease-of-use, which is good. It's modeled after RPC and breaks the uniform interface and focuses on backend caching. Unlike JSF, GWT is fixable, but the community tends to use RPC instead of RequestBuilder.

SOFEA has a central promise of SOA. Business logic is a reusable service that changes less often. The presentation application calls those services and changes more often. The nice thing about this type of architecture is it allows a separation of concerns and loose coupling. However, it doesn't embrace REST like it should. Appcelerator is an implementation of SOFEA that has a Ruby on Rails-like usability. However, it uses a SOAP/HTTP style with messaging and POSTs to a single URI. Appcelerator is interesting, but it introduces a different style of coupling. It breaks URI opacity and client deals with POX instead of links.

Conclusion
Don't fight the architecture of the web. Innovate and enhance instead of breaking. If nothing else, break judiciously. As developers, we should demand more from our frameworks and make sure they use the web and HTTP like it should be used.

Building Rich Applications with Appcelerator

Matt Raible — Wed, 22 Oct 2008 16:18:42 -0600

This afternoon, I delivered my Building Rich Applications with Appcelerator talk for the 3rd time at Colorado Software Summit. When I first proposed this topic, I hadn't used Appcelerator and saw this as a good opportunity to learn more about it. I'm glad I did.

IMO, Appcelerator is a lot like Dojo in how it parses pages and turns HTML with special attributes into JavaScript widgets. I can't help but think a pre-compilation step would be nice to speed things up. I like Appcelerator's extensive Widget Library, and I especially like that they re-use many widgets rather than re-creating their own. Finally, I really dig the "SOA in a browser" approach where everything is a message and you can easily publish and subscribe to events - on the client and server. Below is my presentation, please let me know if you have any questions.

View on Slideshare

Xebia RIA Framework Contest

Matt Raible — Tue, 7 Oct 2008 08:06:48 -0600

Last year, I blogged about the Xebia Web Framework Contest where Struts 2, GWT, Wicket and MyFaces (JSF) were all used to develop the same applications. It seems they've done it again this year, comparing RIA frameworks (English translation) this time.

Five teams were formed, five frameworks have been selected:

Flex 3 Flex 3
Silverlight 2.0 beta 2 Silverlight 2.0 beta 2
Google GWT 1.5 Google GWT 1.5
Echo3 Echo3
JavaFX Preview SDK JavaFX SDK Preview

The number of teams is limited, the list of frameworks is also selected, and we have therefore necessarily set aside some frameworks that would have certainly had their place in the contest. Examples Ext JS, Yahoo! UI, Curl, XUL, ZK or OpenLaszlo. If you have experience on one of these frameworks, feel free to share in the comments on this article! [Read More »]

The winner? Flex - which doesn't surprise me a whole lot.

After this day, the Flex team, has clearly demarcated from its competitors. After two sprints of development and ownership, it could devote the third and final sprint to get rich quick and easy application, focusing only on the features and user experience. It is thanks to the wealth of high-level components, the wealth of documentation available and a maturity framework that the team has made the Flex application's most successful.

Good stuff - thanks Xebia!

Presenting Web Frameworks of the Future Tomorrow in Denver

Matt Raible — Wed, 30 Jul 2008 21:56:17 -0600

Tomorrow (Thursday) night, I'll be doing an encore presentation of my Web Frameworks of the Future at DeRailed. If you're in Denver and would like to hear me ramble while drinking a beer, join us at Forestroom 5 at 6:30.

After the last few days, I'm happy to report I should be in good enough condition to pull this off. If you're curious to learn more about my experience at OSCON and this presentation, please see my writeup on the LinkedIn Blog.

[OSCON 2008] Web Frameworks of the Future: Flex, GWT, Grails and Rails

Matt Raible — Wed, 23 Jul 2008 16:25:23 -0600

Below is the presentation I'm delivering at OSCON today. Unfortunately, I had to remove slides on GWT and Flex to fit w/in the 45 minute time limit. I hope to expand this presentation in the future, as well as continue to develop the side project I'm working on using these technologies.

view on slideshare | download

GWT and REST

Matt Raible — Mon, 21 Jul 2008 10:31:58 -0600

I've posted two message to the GWT Google Group in the last couple of days. However, new member messages are moderated and neither has shown up yet. I'm reposting my questions here in hopes of getting some answers.

Is there a way to easily use a REST backend with GWT? I tried GWT-REST. It works, but it seems to be centered towards Rails (I'm using Grails) and it suffers from the SOP issue.

JSONRequest looks promising for cross-domain support, but I can't get it to work either. The provided examples work, but not my simple hello world that returns:

{"response":"Hello World!"}

Also, the example implementation only has GET support, not PUT, DELETE or POST. I can post my REST backend on the public internet if anyone is interested in seeing the issues I'm having.

Thanks in advance for any advice.

RE: What's a good RIA to develop in 20 hours?

Matt Raible — Wed, 11 Jun 2008 11:02:23 -0600

Thanks to everyone who commented on my previous post and offered recommendations for RIAs to develop in 20 hours or less. In order to narrow down my choices, I've created a survey on SurveyMonkey.com. Here's a list of the application ideas I received from comments and e-mails:

Lightweight CMS
MP3 Player
Resume Editor/Publisher
Meal/Calorie Tracker
Contact Management
Planning Application
Timesheet Application
DB/SQL Client
Status Updater/Aggregator (LinkedIn, Twitter and Facebook)
Online File Explorer (browser-based FTP interface)

I like #3 (Resume) and #9 (Status) because I may be able to tie those into LinkedIn's RESTful API.

Click here to vote for the application you'd like me to develop »

Voting ends at noon on Friday (Mountain Time).

Update: My co-workers had a good suggestion at lunch today: pitchersacrossamerica.com. It seems it's kinda difficult to find bars that serve pitchers these days (at least in Denver). Create an app that allows people to enter in bars and restaurants that serve pitchers and show them on a map. Seems simple and fun. If enough people like the idea, I'll restart the survey with this as an option. In the meantime, the current (Wednesday night) numbers are here.

Update 2: Here's the results as of Thursday night. Only 15.5 more hours to vote!

Update 3: Final Results. Thanks to everyone who voted! I'm traveling a lot next week (Mountain View followed by Boston), but I'll try to write an entry on next steps.

What's a good RIA to develop in 20 hours?

Matt Raible — Mon, 9 Jun 2008 21:56:30 -0600

In less than two months, I'm making my annual trek to Portland, Oregon to speak at OSCON. To prepare for my talk, I'd like to develop the same application with two different combinations: Flex + Rails and GWT + Grails.

As luck would have it, I'm having a hard time coming up with a good application to write. I'd like to time-box it so I only spend 10 hours on the backend (for each) and 10 hours on the front-end, for a total of 40 hours for both applications.

Can you think of any good applications that would warrant a rich front-end and wouldn't take too long to create? I'd like to put both applications in production and generate enough traffic to be faced with scalability issues.

Over the next several weeks, I hope to start creating the applications and blog about what I've learned along the way. At some point, I hope to post an outline and a rough draft. With your help, I believe this can be an excellent presentation. If the presentation and applications are as good as I hope they'll be, it's likely I'll open source them for everyone to use.

Thanks in advance for any advice.

Update: Thanks for all the great feedback. I've posted a survey to narrow the choices.

Proposal accepted for OSCON 2008

Matt Raible — Mon, 17 Mar 2008 19:21:10 -0600

From an e-mail I received earlier this afternoon:

We are pleased to accept the following proposal for OSCON 2008.

* Web Frameworks of the Future: Flex, GWT, Grails and Rails

It has been scheduled for 16:30 on 23 Jul 2008.

My Abstract:

What if the choices in web framework was reduced to 4? If RIA are the way of the future, it's possible that these 4 frameworks are the best choices for this development paradigm. This session will explore these frameworks, as well as entertain many other's opinions on the future of web development.

RESTful backends are easy to create with both Rails and Grails. Ajax frontends are simple to create and maintain with GWT. Flex gives you flash and a pretty UI. If you're an HTML developer, Rails allows you to quickly develop MVC applications. If you're a Java Developer, GWT + Grails might be a match made in heaven. This session is designed to help you learn more about each framework and decide which combination is best for your project.

I'm really looking forward to learning about GWT and Flex in the coming months. If you have any experience (or opinions) about the abstract above, I'd love to hear it. The louder the better.

For those who haven't been, OSCON is one of those truly special conferences. Possible reasons:

It's only an hour from my parent's house
It's a beautiful time of year in Portland
It's always the same weekend as the Oregon Brewers Festival
It's a kickass conference with the greatest diversity of Open Source Committers

I'm going for all 4 reasons and even made a reservation to stay at The Kennedy School. Should be a fun show.

GWT Presentation with Bob Vawter

Matt Raible — Tue, 15 Jan 2008 21:26:09 -0700

Tonight I'm in Mountain View attending the inaugural meeting for the new Silicon Valley Google Technology User Group (SV-GTUG). The good news about this meeting is Bob plans to discuss some of the new features in the GWT 1.5 release. Although it is not available yet, it sounds like they are putting the finishing touches on the 1.5 release and that it will be available soon.

The first ting I've learned is that GWT is actually pronounced "gwit". Bob is a member of the GWT team and he's the only one that's not based out of Atlanta. GWT has a very Swing-like programming model. It has widgets, panels and windows and operates very much like a traditional UI programming API. To create an application, you create a class that implements EntryPoint. Below is an example Hello World application.


/**

 * HelloWorld application.

 */

public class Hello implements EntryPoint {



  public void onModuleLoad() {

    Button b = new Button("Click me", new ClickListener() {

      public void onClick(Widget sender) {

        Window.alert("Hello, AJAX");

      }

    });



    RootPanel.get().add(b);

  }

}

One of the really nice things about GWT is that you can easily debug your application in your favorite Java IDE. GWT doesn't prevent you from writing JavaScript - you can insert native JavaScript into your code using a special /*-{ script here }-*/.

When you compile a GWT application, you get a whole bunch of stuff. When trying to develop cross-browser JavaScript code, you often end up with "if soup" to handle the different browsers. GWT does the same thing, but it creates different versions of the application for the different browsers. The naming of these files is an MD5 hash that's bootstrapped by a simple JS file. GWT files are all static and the file sizes are ~40KB/each for the Hello World application.

If you have to write fewer than 100 lines of code, it's likely that GWT will be overkill - you can do it with JavaScript easier. Once you get more lines of code and larger teams, Java's tools do all the heavy lifting. Yes, you can create large applications with large teams with scripting languages, but Java is the clear winner when you have large teams of developers working on projects.

GWT provides a RemoteService for client code and a RemoteServiceServlet for server-side code that you can use to implement RPC calls. For more information, see the Remote Procedure Calls documentation.

At this point, we're 45 minutes into the presentation and it's pretty disappointing. This is likely because there's no presentation and Bob doesn't seem to have any sort of agenda. I think this type of presentation would work well if folks had experience with GWT and were asking lots of questions. However, it seems that most folks haven't used GWT - so the questions (and his resulting answers) aren't very meaty.

One attendee asked about making GWT applications SEO-able. Bob's suggested solution is to continue to use Servlet or JSPs to render the page, but then use GWT to for the user interaction that needs to happen within the page. The RootPanel class has a get(String id) method that can be used to put the GWT application into a particular div or other placeholder.

Google Base and Google Checkout are currently using GWT. There's also a lot of Google internal development that uses GWT. Apparently, there's quite a few applications built with GWT that haven't launched yet.

Has anyone used the Maven Gwt Plugin? It sounds like it might be pretty nice as it contains the GWT compiler with no OS-dependent Google tooling.

The main new feature in GWT 1.5 is Java 5 support. There's also a GWT Incubator that's used to host GWT features that may go into GWT's core. You can view the demos online.

GWT's mission statement:

GWT's mission is to radically improve the web experience for users by enabling developers to use existing Java tools to build no-compromise AJAX for any modern browser.

GWT's competition according to Bob - Visual Studio.

Action Packed Week in California

Matt Raible — Tue, 15 Jan 2008 08:33:13 -0700

Morning light on the Golden Gate Bridge
Originally uploaded by TravelnFotog

This week is shaping up to be quite the action-packed week in California. First of all, the MacWorld 2008 Keynote is happening today. This is the event where new Apple products are announced and everyone yawns. I hope they announce some interesting stuff, but I'm not getting my hopes up. The only thing I'd likely buy is a screaming laptop with 8 GB of RAM.

Tonight, there's a presentation on GWT at Google. Since this is only a few blocks from LinkedIn, I'm going to try to make the meeting and checkout the "Googleplex" for the first time.

Finally, tomorrow night a bunch of us are getting together for a Silicon Valley Tech Meetup at The Old Pro in Palo Alto. Start time is 6:30.

Comparing JVM Web Frameworks Presentation

Matt Raible — Wed, 14 Nov 2007 15:14:53 -0700

Early this morning, I assembled a Comparing JVM Web Frameworks presentation in preparation for my talk tomorrow at ApacheCon. As mentioned on Monday, this presentation compares Flex, Grails, GWT, Seam, Struts 2 and Wicket. While I think this presentation would be fun to deliver, I don't believe it has as much meat as the original talk I was planning to give. My original talk compares JSF, Spring MVC, Stripes, Struts 2, Tapestry and Wicket. Since I've used all these frameworks, I'm able to compare them more on their technical features. Since I haven't used Flex, GWT or Seam, there was no way for me to 1) try them all before tomorrow and 2) do a thorough analysis of how well they each handle my desired features.

Since the abstract on ApacheCon's website mentions my original presentation, I don't want to yank out the carpet and present the second without asking. So my plan is to ask the audience which one they'd rather hear and continue from there. I've updated both presentations with the latest statistics and uploaded them for your review. For those of you who've used these frameworks, I'd be interested to hear how accurate you think my Pros and Cons section is. If you know of better pros or cons, please let me know and I'll adjust as needed.

Comparing JSF, Spring MVC, Stripes, Struts 2, Tapestry and Wicket

Comparing Flex, Grails, GWT, Seam, Struts 2 and Wicket

While creating the 2nd presentation, I found a couple things that surprised me. The first is how popular Flex is - not only in job listings, but also in skilled developers and mailing list traffic. Below is a graph that shows how there aren't many jobs for most of the frameworks, but there's lots for Flex.

The following graph illustrates while I chose to use Flex instead of OpenLaszlo as the Flash framework. OpenLaszlo has a much smaller community than Flex.

The second thing that was surprising is Seam doesn't have a logo! How does it ever expect to become a popular open source project without a logo?! It's amazing they've made it this far without having this essential feature. To motivate the creation of a Seam logo, I'm using the following butt-ugly logo in my presentation (found here). Hopefully something better comes along before I deliver my talk tomorrow.

Update: Monday's post started an interesting thread on Stripes' mailing list. Also, I really like Spring MVC's new annotation support. It'd be nice to see it go a step further and use defaults (like ControllerClassNameHandlerMapping + subpackage support) and only require annotations to override the defaults. IMO, Stripes, Spring MVC and Struts 2 are all excellent choices if a request-based framework provides the best architecture for your application.

Update 2: Comparing Flex, Grails, GWT, Seam, Struts 2 and Wicket seems to gave gained a lot of interest (and support) in the blogosphere. Because of this, I'm considering submitting it as a JavaOne talk. If I were to do this, how would you like to see this presentation changed and improved?

Update 3: I received the following Seam logo via e-mail. Thanks Christian!

Update 4: I've updated the Dice.com graph to include "Java" with every search term. To understand the comments on this entry, you might want to view the previous graph.

Update 5: This presentation was posted to the Wicket User mailing list. I followed up asking users to post the pros and cons of Wicket. Now there's a lengthy thread on Wicket's Pros and Cons. Good stuff.

Comparing Web Frameworks: Time for a Change?

Matt Raible — Mon, 12 Nov 2007 16:46:56 -0700

I first came up with the idea to do a "Comparing Web Frameworks" talk in 2004. I submitted a talk to ApacheCon and it got accepted. From there, I outlined, created sample apps and practiced this talk before ApacheCon. Believe it or not, that was my first time speaking in front of a large audience.

Historical note: October 2004 was a pretty cool month - I discovered Rails and Roller had a 1.0 release candidate.

When I created the presentation, it was in large part due to all the WebWork and Tapestry folks harassing me on this very blog. I started using Struts in June 2001 (the same month 1.0 was released) and had used it successfully on many projects. Part of the reason this blog became so popular was I posted lots of tips and tricks that I learned about Struts (and its related project) while using it. After a while, the noise became too heavy to ignore it - especially after I'd tried Spring MVC. So in an effort to learn more about the the other frameworks, I submitted a talk and forced myself to learn them. It seems to have worked out pretty well.

With that being said, I think it's time for a change. The reason I originally wrote this was to educate developers on how the top Java web frameworks differed and encourage developers to try more than one. A while later, I realized there's different tools for different jobs and it's not a one-size-fits-all web framework world. It's not a component vs. request-based framework world either. There's lots of options now. When I've delivered this talk earlier this year, I've always felt like I've left quite a few frameworks out. The solution could be to add more and more frameworks. However, I don't think that's a good idea. The talk is already difficult to squeeze into 90 minutes and it's unlikely that adding more frameworks is going to help.

The change I'd like to do is to reduce the number of frameworks down to (what I consider) the top web frameworks for deploying to the JVM. What are those frameworks? IMHO, they are as follows, in no particular order:

GWT-Ext
Wicket
Grails
Flex/OpenLaszlo
Seam
Struts 2

The RIFE, Tapestry and ZK folks can start bitching now. Sorry - less frameworks make for a more interesting talk. Maybe I'll add you in the future and I can ask the audience which ones they want compared then we can choose four and go from there. Why don't I mention Spring MVC? Because I think Struts 2 is easier to learn and be productive with and I also like it's more open and active community. I've written applications with both and I like Struts 2 better. As for Flex vs. OpenLaszlo, I'm somewhat torn. It seems like learning Flex is going to be better for your career, but it's likely useless without the Flex Builder - which is not open source. However, at $250, it's likely worth its price. I know the Picnik folks used Flex for their UI - I wonder how much they used Flex Builder in the process?

What do you think? Are these the top web frameworks for JVM deployment today? The next time I give this talk is this Thursday at ApacheCon. I may try to re-write my talk and then give the audience a choice of old vs. new. The downside of doing the new talk is I won't have time to write apps with GWT, Flex or Seam. Anyone care to post their top three pros and cons for any of these frameworks?