Search for [jsf] in weblog rd

AppFuse, Reduced

2014-12-16T06:03:31-07:00

In November, I had some time off between clients. To occupy my time, I exercised my body and brain a bit. I spent a couple hours a day exercising and a few hours a day working on AppFuse. AppFuse isn't used to start projects nearly as much as it once was. This makes sense since there's been a ton of innovation on the JVM and there's lots of get-started-quickly frameworks now. Among my favorites are Spring Boot, JHipster, Grails and Play.

You can see that AppFuse's community activity has decreased quite a bit over the years by looking at its mailing list traffic.

Even though there's not a lot of users talking on the mailing list, it still seems to get quite a few downloads from Maven Central.

I think the biggest value that AppFuse provides now is a learning tool for those who work on it. Also, it's a good place to show other developers how they can evolve with open source frameworks (e.g. Spring, Hibernate, JSF, Tapestry, Struts) over several years. Showing how we migrated to Spring MVC Test, for example, might be useful. The upcoming move to Spring Data instead of our Generic DAO solution might be interesting as well.

Regardless of whether AppFuse is used a lot or not, it should be easy to maintain. Over the several weeks, I made some opinionated changes and achieved some pretty good progress on simplifying things and making the project easier to maintain. The previous structure has a lot of duplicate versions, properties and plugin configurations between different projects. I was able to leverage Maven's inheritance model to make a number of improvements:

Changed AppFuse's parent to be based on the Spring IO Platform. This project is a dependency manager that defines version numbers for open source projects that work well with Spring.
Defined plugins, their versions and configurations in <pluginManagement>.
Defined dependencies, their versions and exclusions in <dependencyManagement>.
Simplified archetypes so new projects have minimal dependencies. For example, here's a basic project's pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>com.company</groupId>
    <artifactId>springmvc-project</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>AppFuse Spring MVC Application</name>

    <parent>
        <groupId>org.appfuse</groupId>
        <artifactId>appfuse-web</artifactId>
        <version>3.5.0-SNAPSHOT</version>
    </parent>

    <build>
        <plugins>
            <plugin>
                <groupId>de.juplo</groupId>
                <artifactId>hibernate4-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>dbunit-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.appfuse</groupId>
            <artifactId>appfuse-${web.framework}</artifactId>
            <version>${appfuse.version}</version>
            <type>pom</type>
        </dependency>
    </dependencies>

    <properties>
        <amp.genericCore>true</amp.genericCore>
        <amp.fullSource>false</amp.fullSource>
        <dao.framework>hibernate</dao.framework>
        <db.name>mydatabase</db.name>
        <web.framework>spring</web.framework>

        <!-- Framework/Plugin versions -->
        <appfuse.version>3.5.0-SNAPSHOT</appfuse.version>
        <java.version>1.7</java.version>
    </properties>

    <profiles>
        <profile>
            <id>itest</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.cargo</groupId>
                        <artifactId>cargo-maven2-plugin</artifactId>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>webtest-maven-plugin</artifactId>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <reporting>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>webtest-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </reporting>
</project>

The pull request for these changes says it all:

AppFuse +4,822 -14,369
AppFuse Light +776 -4,687

That's right, I was able to eliminate a good chunk of code without affecting any of AppFuse's functionality¹. I think we can all agree that less code == easier maintenance. This theme will continue as we work on future releases.

Other improvements include migrating all tests to use JUnit4, integrating Spring MVC Test, and configuring the surefire plugin to run tests in parallel. I also The build-helper-maven-plugin is now used to find open ports for Cargo to run and a lot of testing was done to ensure you can build/test multiple AppFuse-derived projects at the same time. Finally, I migrated to the hibernate4-maven-plugin and upgraded to Tapestry 5.4.

In the next version of AppFuse, I plan to remove as much XML as possible - moving all of the configuration to Spring's JavaConfig. We'll also be moving to Java 8 as a minimum. I'm even considering getting rid of all the pom.xml files in favor of another build language that requires less code.

In other words, the upcoming 3.5 release will be the last release that supports Java 7 and uses Spring's XML for configuration. AppFuse 4.0 will strive for #NoXML. The project's roadmap has more details on additional hopes and dreams.

We'd love to hear your feedback on these change. Do you like the simplification theme? Are you OK with having AppFuse as a parent in your projects?

1. For project and code stats, see AppFuse on Open Hub.

Comparing JVM Web Frameworks at vJUG

2014-02-06T10:54:17-07:00

A couple months ago, I was invited to speak at Virtual JUG - an online-only Java User Group organized by the ZeroTurnaround folks. They chose my Comparing JVM Web Frameworks presentation and we agreed I'd speak yesterday morning. They used a combination of Google Hangouts, live streaming on YouTube and IRC to facilitate the meeting. It all went pretty smoothly and produced a comfortable speaking environment. To practice for vJUG, I delivered the same talk on Tuesday night at the Denver Open Source Users Group.

The last time I delivered this talk was at Devoxx France in March 2013. I didn't change any of the format this time, keeping with referencing the Paradox of Choice and encouraging people to define constraints to help them make their decision. I did add a few new slides regarding RebelLabs' Curious Coder’s Java Web Frameworks Comparison: Spring MVC, Grails, Vaadin, GWT, Wicket, Play, Struts and JSF and The 2014 Decision Maker’s Guide to Java Web Frameworks.

I also updated all the pretty graphs (which may or may not have any significance) with the latest stats from Dice.com, LinkedIn, StackOverflow and respective mailing lists. Significant changes I found compared to one year ago:

Job Listings on Dice.com
- Play Framework job listings increased almost 4x
- Tapestry jobs are 1/3 of what they were a year ago
- Wicket jobs are 1/2 of what they were a year ago
- JavaScript framework jobs are up quite a bit: Ember.js up ~300%, AngularJS up 900%, Backbone up 160%
LinkedIn Skills
- Rails down ~30%
- Grails up 25%
- Play Framework up 200%
- Spring Roo up 40%
- Ember.js up 300%
- AngularJS up 840%
- Backbone up 200%

As you can tell from these findings, AngularJS has gained quite a bit of mindshare in the last year. There's a lot of companies looking for JavaScript skills and quite a few folks have added JavaScript frameworks to their LinkedIn profiles.

You can watch the recording on YouTube or click play in the embedded video below.

You can also quickly browse the slide deck below, download the PDF or view it on SlideShare.

Thanks to all the folks who attended these talks. And thanks to @dosug and @virtualjug for giving me the opportunity to speak.

AppFuse 3.0 Released!

2013-12-23T14:31:15-07:00

The AppFuse Team is pleased to announce the release of AppFuse 3.0. This release is AppFuse's first release as a 10-year old and includes a whole slew of improvements.

Java 7 and Maven 3 are now minimal requirements
Replaced MyFaces and Tomahawk with PrimeFaces for JSF
- Removed SiteMesh in favor of JSF's built-in layout support
Added Wicket support
Migrated from jMock to Mockito for tests
Integrated wro4j and WebJars
Migrated to Bootstrap 3 and defaulted to Bootswatch's Spacelab theme

In addition, this release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Spring 4, Spring Security 3.2 and Bootstrap 3. For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is a full-stack framework for building web applications on the JVM. It was originally developed to eliminate the ramp-up time when building new web applications. Over the years, it has matured into a very testable and secure system for creating Java-based webapps.

Demos for this release can be viewed at http://demo.appfuse.org. Please see the QuickStart Guide to get started with this release.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the users mailing list. You can also post them to Stack Overflow with the "appfuse" tag.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix, and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

Happy 10 Year AppFuse!

2013-04-05T08:56:45-06:00

10 years ago yesterday, I released the first version of AppFuse. It started with XDoclet generating ActionForms from POJOs and became very popular for Struts developers that wanted to use Hibernate. The project's popularity peaked in 2006, as you can see from the mailing list traffic below.

It's possible the decrease in traffic is because we re-wrote everything to be based on Maven. It's also possible it was because of more attractive full-stack frameworks like Grails and Rails. However, the real reason is likely that I stopped working on it all the time due to getting a divorce becoming an awesome dad.

Below is a timeline of how the project evolved over its first 4 years.

AppFuse has been a great project for me to work on and it's been a large source of my knowledge about Java, Web Frameworks, Spring, Hibernate - as well as build systems like Ant and Maven. We started with CVS, moved to SVN and now we're on GitHub. We've experienced migrating from Tapestry 4 to Tapestry 5 (thanks Serge Eby!), upgrading to JSF 2 and enjoyed the backwards compatibility of Spring and Struts 2 throughout the years. We've also added REST support, a Web Services archetype and kept up with the latest Spring and Hibernate releases.

Last year, we added Bootstrap and jQuery as foundational front-end frameworks. For our next release, we're switching to PrimeFaces, adding Wicket and changing from jMock to Mockito. Most of these changes are already in source control, we just need to polish them up a bit and add AMP support. I hope to release 3.0 before the bus is done.

Thanks to all the enthusiastic users of and contributors to AppFuse over the years. It's been a great ride!

Devoxx France: A Great Conference in a Magnificent City

2013-03-29T13:14:30-06:00

This week, my lovely fiancé and I traveled to the City of Light. Our journey was designed around some speaking engagements at Devoxx France. Devoxx is one of my favorite conference franchises and Devoxx France has been special to me ever since the Devoxx (Belgium) I spoke at in 2011.

2011 was the year I spoke about my experience with Play, Scala, CoffeeScript and Jade. I wrote the presentation on my flight over, composed the demo video the night before and made it all happen in the nick of time. Of course, this was after 120 hours of research and preparation, so the presentation composition process had all the data I needed. You can imagine my sense of relief after pulling off that talk and getting an enthusiastic applause from the audience for my efforts.

One of the first audience questions I received was from Nicolas Martignole, asking if I'd speak at Devoxx France the following year. I whole-heartedly agreed to do it and was excited for the opportunity. It was with great disappointment that I later found out I couldn't attend Devoxx France in 2012. My client didn't like me taking so much time off and I agreed to scale my two week vacation back to 1 week. This year, I was determined to go, so I submitted some of my favorite talks: Comparing JVM Web Frameworks and The Play vs. Grails Smackdown with James Ward. I was extremely pleased when they both got accepted.

Side Story: I met Martin Odersky shortly when he sat down next to me for the Java Posse presentation in Belgium in 2011. After shaking his hand and introducing myself, I had to politely ask him to leave because it was Trish's seat. Talk about awkward; but Martin was very gracious and promptly found a new seat close by.

Comparing JVM Web Frameworks
Both talks required a bit of updating. For Comparing JVM Web Frameworks, I started reading The Paradox of Choice and found many parallels to the agony that developers experience with choosing a web framework. I described how I didn't think good framework decisions were based on the many, many features that frameworks have, but often on pre-defined constraints. There's those lucky developers that get to choose a Full Stack Framework because they're doing greenfield development. Then there's those that want a better Pure Web Framework that replaces something (e.g. Struts) that's not satisfying their needs. And lastly, there's those that've found it possible to leverage a SOFEA and use a JavaScript MVC framework with an API Framework on the backend. I don't think it makes sense to compare all web frameworks and I tried to use these pre-defined constraints (language, platform and application type) argument to separate into categories and help make choosing easier.

I took out the parts of the presentation that've pissed people off in the past - particular the JSF bashing by James Gosling, the Rails gushing from Craig McClanahan and the Pros and Cons sections of each framework. I added the history of web frameworks and research from InfoQ and devrates.com.

The best part of the JVM Web Frameworks talk was the audience's reaction and enthusiasm. Devoxx always seems to attract passionate developers and Devoxx France was no different. Developers packing the room, clapping after your intro, laughing at your jokes, signifying that they agree with you about JSF. As a speaker, it's an unbelievable experience.

You can view my Comparing JVM Web Frameworks presentation below or on Slideshare.net.

Play Frameworks vs. Grails Smackdown
To prepare for James Ward and my Play vs. Grails Smackdown, we had a number of goals. First of all, we wanted to update our apps to use the latest versions of each framework. I documented what it took for Grails, James just checked in his code to GitHub. It was interesting to see that Grails 2.0.3 -> 2.2.1 caused a number of issues with testing, while Play 2.0.3 -> Play 2.1.0 required API changes, but nothing for tests. Secondly, we updated all the stats for our pretty graphs and ran load tests again.

This is where the fun started. On Tuesday evening, I decided to challenge the notion that Play was twice as fast as Grails. James had proven this with Apache Bench tests. With Play 2.0 and Grails 2.0 (last summer), we clocked Play at 251/requests per second and 198 for Grails. After upgrading each app to the latest releases, we found the numbers to be 233/second for Play and 118 for Grails.

However, Apache Bench only tests until the first byte is received. Since I've done a lot of browser optimizations recently, I fired up whichloadsfaster.com, captured a screenshot and added it to our presentation. The next day, James added a CDN and a bunch of caching to his app and re-ran his AB tests.

Now he was smoking Grails, so I added a CDN and caching as well. However, the best I could do was just over 1000/requests per second, while he was around 2200/second. When he ran live tests during our talk, Play was around 2800/sec and Grails was around 900.

It was great to see how much better performance we could get with caching and a CDN. The best part is this should be available to most applications, not just these frameworks. By adding a CDN (we used Amazon CloudFront) and caching, we were both able to 10x the performance of our apps. You can find our presentation here or view it below.

Summary
This was a very enjoyable conference to attend as a speaker. First of all, it was in one of the most beautiful cities in the world, but it's also a very special place for Trish and I. We got engaged just outside of Paris in Versailles after the last Devoxx conference I spoke at. Trish has some amazing photos from that trip. Secondly, the Devoxx conference attracts a special kind of developer - one that is passionate about and eager for knowledge. Lastly, speaking with my good friend James, in an exotic city about something we love - that was special. Asking for beers and having them brought to us at the start of our Smackdown. That was magical (thanks Nicolas!).

To all the Devoxx organizers and crew - well done on a great show!

Integrating GWT into AppFuse

2013-03-07T18:49:28-07:00

I've been interested in integrating GWT into AppFuse ever since I blogged about it 4 years ago. A few months after that post, I wrote about Enhancing Evite.com with GWT and Grails. After Evite, I had a gig near Boston where I developed with GXT for the remainder of the year. When all was said and done, I ended up spending a year with GWT and really enjoyed my experience. I haven't used it much since.

GWT is scheduled to be integrated into AppFuse in version 4.0. That's quite a ways off. The good news is you might not have to wait that long, thanks to Iván García Sainz-Aja. Iván let us know about his work a couple weeks ago in an email to the appfuse-dev mailing list.

It's still work in progress but it has already most of AppFuse functionality..

If you want to give it a try

https://github.com/ivangsa/appfuse.git

the quickest way to have a go would be

web/gwt> mvn -P gwtDebug -Dgwt.inplace=true gwt:compile jetty:run

at the moment it still requires this fork of gwt-bootstrap to be compiled first

https://github.com/ivangsa/gwt-bootstrap.git

It needs a lot of testing yet but it's getting quite there

As you can imagine, I was very excited to hear about Iván's work. So I cloned his repo, built gwt-bootstrap locally and checked it out. Functionality wise, it was great! However, when I dug into the source code, I found a whole lotta code.

To see how the GWT flavor compared to the other implementations in AppFuse, I created a cloc report on the various web frameworks in AppFuse. I'm sure these reports could be adjusted to be more accurate, but I believe they give a good general overview. I posted some graphs that displays my findings in visual form.

When I sent this to the mailing list, Ivan responded that it was a lot of code and estimated 12 new files would be needed to CRUD an entity. This sure seems like a lot to me, but he defended this yesterday and noted that his implementation follows many of GWT's latest best practices: MVP pattern, Activities and Places, EventBus, Gin and Guice. He also shared a wiki page with explanations and diagrams of how things work.

The reason I'm writing this post is to get more feedback on this implementation. First of all, does GWT really require this much code? Secondly, are there other GWT implementations that reduce a lot of the boilerplate? SmartGWT, Vaadin* and Errai come to mind.

If you were starting a new GWT project and using AppFuse, how would you want it implemented?

* Vaadin 7 claims it can be used as a drop-in replacement for GWT. I tried replacing the gwt-servlet and gwt-user dependencies with Vaadin's, but it didn't work.

Switching AppFuse from MyFaces to PrimeFaces

2013-02-06T12:19:34-07:00

When describing my bias against JSF back in November, I wrote:

... there's a lot of folks praising JSF 2 (and PrimeFaces moreso). That's why I'll be integrating it (or merging your pull request) into the 2.3 release of AppFuse. Since PrimeFaces contains a Bootstrap theme, I hope this is a pleasant experience and my overall opinion of JSF improves.

Shortly after the AppFuse 2.2.1 release in December, Gilberto Andrade contributed a sample project that used Mojarra (the JSF RI) and PrimeFaces instead of MyFaces and its Tomahawk components. Last week, I spent a few hours integrating Gilberto's changes into AppFuse's master branch. You can see all the changes I made (which include a Jetty plugin upgrade and some cleanup) in this Crucible review. Feel free to leave comments on ask questions in the review itself.

The first thing I noticed when integrating PrimeFaces is you have to add a custom repository in order to get its artifacts via Maven.

<repositories>
    <repository>
        <id>prime-repo</id>
        <name>Prime Repo</name>
        <url>http://repository.primefaces.org</url>
    </repository>
</repositories>

This is unfortunate since all of AppFuse's other dependencies can be found in Maven Central. It means that if you're using a JSF archetype, the PrimeFaces repo will be checked for artifacts first, causing an unnecessary slowdown in artifact resolution. I hope the PrimeFaces developers fix this soon.

While integrating these two frameworks, I ran into a number of issues.

An IllegalStateException on startup when using "mvn jetty:run"
The first issue I encountered was that I was unable to run the app in Jetty. It worked fine in Tomcat but I got the following error in Jetty:

2013-01-31 22:28:07.683:WARN:/:unavailable
java.lang.IllegalStateException: Application was not properly initialized at startup, could not find Factory: javax.faces.context.FacesContextFactory
at javax.faces.FactoryFinder$FactoryManager.getFactory(FactoryFinder.java:951)
at javax.faces.FactoryFinder.getFactory(FactoryFinder.java:316)
at javax.faces.webapp.FacesServlet.init(FacesServlet.java:302)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:492)
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:312)

I found the fix for this on Stack Overflow and added the following listener to my web.xml to solve it.

<listener>
    <listener-class>com.sun.faces.config.ConfigureListener</listener-class>
</listener>

Conditionally rendering a button disables its click-ability
The next thing I noticed was the Delete button didn't work when editing a user. It was hidden correctly when adding a user, but clicking on it to delete a user simply refreshes the page. Below is the code I used successfully with MyFaces. For some reason, this doesn't work with PrimeFaces.

<c:if test="${not empty userForm.user.id}">
<h:commandButton value="#{text['button.delete']}" action="#{userForm.delete}"
    styleClass="btn" onclick="return confirmMessage(msgDelConfirm)"/>
</c:if>

I also tried the following, but no dice. This is currently an open issue.

<h:commandButton rendered="${not empty userForm.user.id}" value="#{text['button.delete']}" 
    action="#{userForm.delete}" styleClass="btn" onclick="return confirmMessage(msgDelConfirm)"/>

The PrimeFaces Bootstrap theme 404s on some images
After integrating PrimeFaces' Bootstrap theme, the following error shows up in server logs.

[INFO] [talledLocalContainer] Feb 02, 2013 10:40:25 PM com.sun.faces.application.resource.ResourceHandlerImpl logMissingResource
[WARNING] [talledLocalContainer] WARNING: JSF1064: Unable to find or serve resource, images/ui-bg_highlight-hard_70_000000_1x100.png, from library, primefaces-bootstrap.

This seems to have happened before in previous releases and is currently an open issue.

Canoo WebTest doesn't work with fileUpload nor to set checkbox values
We use Canoo WebTest to run integration tests on the UI in AppFuse. For some reason, performing file uploads and setting checkbox values works fine with MyFaces/Tomahawk, but not with Mojarra/PrimeFaces. I'm not sure if this is caused by the JSF core or the component library, but it remains an open issue. For now, I've just commented out the parts of tests that used to do this.

On a related note, getting the real path of a resource from the ServletContext worked fine before the switch, but results in a null value now.

String uploadDir = getServletContext().getRealPath("/resources") + "/" + request.getRemoteUser() + "/";

PrimeFaces resources served up at /javax.faces.resource/* not found
While I didn't have problems with this in AppFuse, I did encounter it in AppFuse Light. I don't know why there was a difference between the two, but it turned out to be caused by the UrlRewriteFilter and my desire for extensionless URLs. The outbound-rule to strip .xhtml from URLs was the culprit. Adding a condition to it solved the problem. Yeah, the condition seems backwards, but it works.

<outbound-rule match-type="regex">
    <condition type="query-string" operator="equal">ln=primefaces</condition>
    <from>^(.*)\.xhtml(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

Summary
The initial switch to Mojarra/PrimeFaces was pretty easy thanks to Gilberto's sample project. However, the small issues encountered after that turned out to be quite frustrating and you can see that several are still not fixed. I guess it just goes to show that not all web frameworks are perfect. Hopefully we'll get these minor issues fixed before the next release. In the meantime, you can checkout the updated demos for AppFuse JSF and AppFuse Light JSF.

AppFuse Light 2.2.1 Released!

2013-01-24T19:43:20-07:00

In December, the AppFuse Team released 2.2.1. Right before that release, I decided to wait on enhancing its "light" modules, a.k.a. AppFuse Light. I'm glad I did, because it took some effort to get jQuery and Bootstrap integrated, as well as to make it more secure.

The good news is AppFuse Light 2.2.1 is released and it's sitting out on the Central Repository. This release is a refactoring of all archetypes to be up-to-date with the AppFuse 2.2.1 release. This means Java 7 compatibility, Servlet 3, Bootstrap/jQuery integration, Tapestry 5.3.6 upgrade and security improvements. I integrated Bootstrap and jQuery using WebJars Servlet 3 support since it was simple and straightforward.

You can create projects using AppFuse's light archetypes using a command such as the following:

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes 
  -DarchetypeArtifactId=appfuse-light-spring-freemarker-archetype -DarchetypeVersion=2.2.1 
  -DgroupId=com.mycompany -DartifactId=myproject

The list of archetypes is as follows:

appfuse-light-jsf-archetype
appfuse-light-spring-archetype
appfuse-light-spring-freemarker-archetype
appfuse-light-spring-security-archetype
appfuse-light-stripes-archetype
appfuse-light-struts-archetype
appfuse-light-tapestry-archetype
appfuse-light-wicket-archetype

The QuickStart Guide will help you get setup and demos are available at the following links:

If you have questions about AppFuse, we invite you to ask them on the users mailing list or tweet using #appfuse.

For those enjoying Bootstrap in your apps, I encourage you to check out {wrap}bootstrap and Bootswatch.

What's the best way to compare JVM Web Frameworks?

2013-01-09T08:29:17-07:00

I've been comparing web frameworks ever since 2004. It was the first time I'd ever proposed a talk for a conference. ApacheCon was in Vegas that year and my buddy Bruce suggested I speak at it. I submitted the talk, got accepted and went to work learning the frameworks I was talking about. At the time, I had a lot of Struts experience and I'd made a good living learning it, consulting on it and blogging about it. However, there was a new kid on the block (Spring MVC) that was garnishing attention and some other frameworks (WebWork and Tapestry) that had a lot of high praise from developers. I was inspired to learn why so many people hated Struts.

Fast forward 8 years and I'm still comparing web frameworks. Why? Because there still seems to be a large audience that's interested in the topic. Witness InfoQ's Top 20 JVM Web Frameworks, which was one of their most-read articles for two months in a row. One of the beauties of the Java Community is that it's very diverse. There's tons of folks that are part of this community and, like it or not, several folks that are former Java Developers. However, these developers still seem to maintain an interest in the community and it's still one of the largest pools of talent out there. Java is still quite viable and only seems to be getting better with age.

So the topic of web frameworks on the JVM is still hot, and I still like to write about it. For those of you still enthusiastic about the topic, you're in luck. The two best websites for the Java Community, InfoQ and DZone (formerly Javalobby) are still very interested in the topic too!

Both sites emailed me in November to get my advice for their research on the subject. InfoQ (specifically Dio Synodinos) was mostly interested in 1) having me analyze their recently-gathered data, or 2) helping them create a new version. DZone (specifically Mitch Pronschinske) emailed about doing a similar survey to InfoQ's, but with more relevant data points (include GWT, specifying Struts 2 vs. Struts 1, etc.).

My response to Mitch at DZone:

Interestingly enough, the folks at InfoQ contacted me as well as they're thinking of doing a new survey. One of the things I mentioned to them is it'd be interesting to see what folks are using AND which frameworks they admire. Often, devs don't get to choose their web framework at work. I wonder if it'd be possible to collaborate with InfoQ to gather data from developers so it's not being done on two different sites?

I submitted a talk for ApacheCon NA (in February) called Comparing Apache Web Frameworks. When choosing Web Frameworks, I've often found it helps to eliminate frameworks and narrow the scope. Obviously, this makes sense for an Apache Conference, but not for all developers. However, I do plan on analyzing each framework based on a limited set of criteria. Here's what I have so far:

Community, HTML5, REST, Mobile, Performance, Web Performance Optimization

Obviously, community is important for Apache projects, but might not be for the wider audience. It might be good to limit to these 5 criteria, or expand it to 10, but not more. I think it'd be interesting to get the community to rank the various frameworks on these criteria, and also try to find developer's biases while doing it. For example, I wonder if people would be willing to admit they're biased for/against certain frameworks and then take that into account as part of gathering the data?

Unfortunately, ApacheCon denied my submission. This make sense since their schedule seems to be concentrating on HTTPD, Cloud and Big Data.

For InfoQ, Dio asked for a list of web frameworks to include. Below is a list we started with, followed by my response.

Spring MVC
Play
Grails
JSF
Struts 2
Wicket
Lift
Tapestry 5
Seam
JRuby on Rails
Wicket
GWT
Vaadin

I would add Stripes, vert.x and maybe something like Apache Click. VRaptor is probably a good one to add too. There's always a few less-used frameworks that get a lot of complainers if you don't include them.

Comparing to your previous list, I don't think Seam should be in here since they've split the project into separate bundles and are no longer developing Seam as a whole. JRuby on Rails is a tough one because if you say Ruby on Rails, you'll get a ton of responses, but probably not from the Java community. The Ruby community might chime it quite a bit if you can get in touch with them though.

I believe you should include Clojure web frameworks, but I've only heard of one of them: Compojure.

SiteMesh, Netty, etc. - remove them.

I also offered my advice on instructions:

I actually like the two coordinates you used before, importance and adoptability. However, I don't know that everyone read the instructions this way. Most people didn't rank all frameworks and I believe that's part of the point. I only ranked the ones I'd used, but I think it'd be better if people ranked all of them. I also think having these two criteria opens it up to more than just developers. Project/Product Managers and stakeholders that've been successful with certain frameworks should be able to vote too.

For DZone, they wanted to include a set of criteria for ranking:

Project maturity, community support, and documentation quality (one criteria)
UI Features Capability (maybe some are more graphics driven or form driven?)
Code readability
Flexibility (maybe this could be broken down into what types of projects the framework can handle) or several criteria that ask if it is "Good for 'x' type of project"
Performance/Speed
Cross platform support
Extensibility, Plugins, Community Libraries
Architecture (this may just be information for later, not an opinion question)
Web standards support
REST support (is this something you would just rate a yes or no? In that case it wouldn't need to be an opinion question)
Mobile support

My response:

For web frameworks, I believe the 5 I mentioned (Community, HTML5, REST, Mobile, Performance, Web Performance Optimization) are most important, with Security become more and more of a concern. For web standards support, I'd almost change it to "HTML5" and to see how the various frameworks stack up. I think REST is very important, and I think it's cool that Struts 2, Spring MVC and Grails all have great support for it. It'd be interesting to see how the component-based frameworks think of having REST support in the framework (vs. external like Jersey, CXF, etc.).

Of the list you provided, I don't know about Code readability or Flexibility. Code readability is kinda like Learnability. One of the nice things about Spring MVC and Grails is that you can learn how they work very quickly. Then you can use that knowledge and don't have to look things up much. Tapestry and Wicket might be similar for those writing large apps, but I haven't found that to be as true. The more traditional MVC Frameworks just make more sense to me.

UI Feature Capability is a good one because frameworks with widgets are often popular with developers. Flex, GWT, jQuery UI, Sencha all do this very well.

Summary
The reason for this post is to add some transparency to the process of Comparing JVM Web Frameworks. I like to think that I've tried to do this in the past (especially with my reasons for rankings). Now, we'd like to hear from you, the community that uses these web frameworks.

What's the best way to compare JVM Web Frameworks?

While it's nice to hear from the generous folks that create and maintain JVM Web Frameworks, we're mostly interested in hearing from the developers that are using these things on a day-to-day basis. The Blue Collar Developers, if you will. If you could design a JVM Web Framework comparison that answered your questions, how would it look? What questions would it ask? What conclusions would make you happy? Should commercial frameworks like ZK be included?

Your responses are very greatly appreciated.

AppFuse News: Liquibase and wro4j Tutorials from J. García

2012-11-30T14:32:47-07:00

New committer J. García has been doing a lot of work to improve i18n in AppFuse 2.2, as well as our Struts 2 support. In addition, he's written a couple articles that show you how to integrate Liquibase and wro4j in your AppFuse applications. Thanks for the great documentation J!

The 2.2 release is coming along, and we're down to 16 open issues. I've updated the Hibernate, JPA, Services and Web Services tutorials and hope to finish the web tutorials in the next week. You can try the latest code using the QuickStart Guide or check it out on the demo site:

Please see this thread on the mailing list if you have any questions or suggestions.

In related news, Roger Hughes has a good article titled Ten Things You Can Do With Spring Security. Since AppFuse uses Spring Security extensively, hopefully you can use some of Roger's tips to improve the security of your app.

Why the bias against JSF?

2012-11-08T09:24:27-07:00

In my last post about InfoQ's Top 20 Web Frameworks for the JVM, I received a thought-provoking comment from henk53:

There is one little thing that does bother me in those presentations, and that's your fairly obvious bias against JSF.
...
If you are presenting yourself as, more or less, an authority on comparing web frameworks, then having a fairly obvious biased against one of them is just peculiar. I, all of my team, and various clients distrust your ranking of JSF. We do look at your data if the choice is between other frameworks, but as soon as JSF comes into the picture we just look elsewhere.

I'm not really sure where this bias comes from. Yes, JSF 1.0 sucked and 1.2 was only marginally better, but 2.0 is really cool and productive and there are SUPERB component and utility libraries now like PrimeFaces and OmniFaces. As a researcher of this topic I think you should keep up the date and not stick to some old grudge.

This is true, I am biased against JSF. It all started with my first JSF experience back in August 2004. If you remember correctly, 2004 was a big year: JSF 1.0, Spring 1.0 and Flex 1.0 were all released. The "AJAX" term was coined in early 2005.

By 2007 and 2008, JSF still hadn't gotten any better. In late 2009, JSF 2.0 was released and I upgraded in March 2011. As you can see from the aforementioned post, I ran into quite a few issues upgrading. JSF was also the hardest one to get working with extension-less URLs.

Most of my issues with JSF come from having maintained an application built with it since 2004. If I were to start a new application without any legacy migration issues, I imagine it wouldn't be as difficult. However, if you compare it to Struts 2 and Spring MVC, I've had little-to-no issues upgrading those applications over the years.

Also, I'm not just biased against JSF, but most component-based web frameworks. Just ask the Tapestry and Wicket folks. They've felt my criticisms over the years. My reason for preferring request-based frameworks like Struts 2/Spring MVC and Grails/Play has been because I've never seen the appeal in component-based frameworks. Often I've found that their components are just widgets that you can get from any decent JavaScript framework. And chances are that JavaScript framework can work with any web framework. Also, I've worked on a lot of high-traffic web applications that require statelessness for scalability.

I see the value in component-based frameworks, I just don't think components should be authored on the server-side. Most of the Java-based component frameworks require 2+ files for components (one for the component, one for the view, possibly one for the config). I love GWT's component concept in that you can just extract a class and re-use it. With JS frameworks, you can often just include a script. These days, when I think of good component-based frameworks, I think of jQuery UI and Twitter Bootstrap.

All that being said, there's a lot of folks praising JSF 2 (and PrimeFaces moreso). That's why I'll be integrating it (or merging your pull request) into the 2.3 release of AppFuse. Since PrimeFaces contains a Bootstrap theme, I hope this is a pleasant experience and my overall opinion of JSF improves.

In other component-based frameworks in AppFuse news, Tapestry 5 has gotten really fast in the last year. I imagine this is because we have a Tapestry expert, Serge Eby, working on it. And we're planning on adding Wicket in the 2.3 release.

So even though I prefer request-based frameworks with REST support and Bootstrap, that doesn't mean everyone does. I'll do my best to be less-biased in the future. However, please remember that my view on web frameworks is as a developer, not an analyst. And aren't developers supposed to be opinionated?

InfoQ's Top 20 Web Frameworks for the JVM

2012-11-06T12:04:28-07:00

Back in early October, InfoQ.com published a community research article titled Top 20 Web Frameworks for the JVM. Their goal seemed to be fairly simple:

Using the new community research tool, we at InfoQ want to get YOUR opinions on the relative importance and maturity of a variety of web frameworks that are targeted for the JVM. Please vote by dragging each practice across two dimensions – how important is the framework relative to the other frameworks, and how much is it actually used in real teams and projects.

When I first saw this article, I noticed some strange web frameworks listed. Namely, Netty, SiteMesh and Spark. I haven't heard of many folks using Netty for a web framework, but I'm sure it's possible. SiteMesh is certainly not a web framework and I've never even heard of Spark. And where is GWT and Vaadin? Regardless of the choices, I went ahead and voted.

Last week, InfoQ posted their top content for October on Facebook.

Top 20 Web Frameworks for the JVM 25,992 PV
Ruby on Rails vs. Node.js at LinkedIn 11,904 PV
Mobile Webdev: The Horror 11,150 PV
A Brief Introduction to REST 8,872 PV
Kanban and Scrum - making the most of both 7,665 PV

First of all, it's interesting to see that JVM Web Frameworks is still a hot topic for developers. Whenever I do my Comparing JVM Web Frameworks talk at conferences, I always see a few jabs about "he's still doing that talk!?" Yes, it seems strange that a talk I first did in 2004 is still in high demand.

Secondly, I think InfoQ does good in showing how the frameworks ranked and showing their heatmaps. Below are their rankings from 1109 participants.

According to this research, the top 5 web frameworks for the JVM are Spring MVC, Play, Grails, JSF and Struts (I hope those surveyed meant Struts 2, not Struts 1).

In my research from last February (slide 21), I ranked them (with no particular weightings) as Grails, GWT, JRuby on Rails, Spring MVC and Vaadin. So I guess you could say I got 2 out of 5 right (Grails and Spring MVC). Not bad considering InfoQ didn't even consider GWT and Vaadin.

Another intriguing data point in this study is each frameworks' heatmap. For example, below are heatmaps for the top 4 frameworks.

Notice how Grails and Spring MVC are both hotter in the bottom right corner? It seems the community's overall opinions of these two frameworks are more aligned than JSF and Play, which a fair amount of folks rank as hyped and unimportant.

What I really like about this research is it's the community's opinions, visualized. It also confirms that some of my favorite frameworks are still on top. I don't know if JSF belongs as a top framework, however it seems a lot of folks do. I recently thought about removing it from AppFuse, but decided to keep it (at least for the next release). I hope InfoQ does more research projects like this, especially if they get their list of web frameworks right.

Comparing Web Frameworks and HTML5 with Play Scala at Jfokus 2012

2012-02-16T00:01:05-07:00

Stockholm seems a lot like Denver this time of year. Cold, snowy and beautiful. Trish and I arrived in Stockholm (Sweden) on Monday for the Jfokus conference and we're traveling to Madrid today for the Spring I/O conference. I was invited to Jfokus within minutes of delivering my HTML5 with Play Scala talk at Devoxx.

Both the Jfokus and Spring I/O Organizers were interested in my Comparing JVM Web Frameworks talk, so I updated it to reflect my latest thoughts. First of all, I mentioned that there's a lot of great frameworks out there and I think the reason people are so apprehensive to choose one is because they've chosen badly at one point. This might've been Struts back in the day (even thought it was one of the best frameworks at the time) or it might be because a vendor talked them into it. However, if you look at the modern JVM frameworks today, you should be able to see that they're all pretty awesome.

I mentioned how I think Web developers should know JavaScript and CSS. If you're a Java developer and you call yourself a web developer, you're letting your framework do too much of the work for you. I mentioned Rich Manalang's Modern Principles in Web Development, where he talks about his core web development principles.

Designing for mobile first (even if you’re not building a mobile app)
Build only single page apps
Create and use your own REST API
“Sex sells” applies to web apps

I've found these principles to be true in my own experience and suggested that if you want to be a web developer, the frameworks you might want to learn are not traditional JVM web frameworks, but rather client-side MVC frameworks. For those Java developers that don't want to be web developers, I suggest they strengthen their services development knowledge by reading Hot to GET a Cup of Coffee.

You can see my updated presentation below, on Slideshare or as a downloadable PDF. You can also watch the video.

I delivered my 2nd presentation on HTML5 with Play Scala, CoffeeScript and Jade on Wednesday morning. This talk is one of my favorites and I prepared for it over the last several weeks by adding JSON CRUD Services and SecureSocial to my HTML5 Fitness Tracking application. Right before we left for Jfokus, I was able to get everything to work, but didn't spend as much time as I'd like working on the mobile client. If this talk gets accepted for Devoxx France, I plan on spending most of my time enhancing the mobile client. After my latest experience developing, I can see how Rich's first principle (above) makes a lot of sense.

Below is my presentation for this talk. Of course, it's on Slideshare and downloadable as a PDF.

I also updated the Developing Play More demo video to show my latest efforts.

Delivering these talks at Jfokus was a lot of fun. Yes, it was a lot of work and stress to prepare them. However, I also learned a lot creating them and I hope the audience benefitted from that.

The conference itself was incredible. I got to meet Peter Hilton and Helena Hjertén as I was registering. The speaker's dinner at F12 was off-the-hook good and I had the pleasure of finally meeting Rickard Öberg.

I also attended some fantastic presentations, including Peter Hilton's Play Framework 2.0, Bodil Stokke's CoffeeScript: JavaScript without the Fail, Pamela Fox's Client-side Storage and Heiko Seeberger's Scala in Action. I don't know if Heiko has published any slides, but I'm guessing not since most of his presentation was live coding.

I have lots of good memories from Jfokus. Many thanks to Mattias for inviting me!

Refreshing AppFuse's UI with Twitter Bootstrap

2012-01-31T17:12:17-07:00

The last time AppFuse had an update done to its look and feel was in way back in 2006. I've done a lot of consulting since then, which has included a fair bit of page speed optimization, HTML5 development and integrating smarter CSS. It was way back in '05 when we first started looking at adding a CSS Framework to AppFuse. It was Mike Stenhouse's CSS Framework that provided the inspiration and my CSS Framework Design Contest that provided its current themes (puzzlewithstyle, andreas01 and simplicity).

Since then, a lot of CSS Frameworks have been invented, including Blueprint in 2007 and Compass in 2008. However, neither has taken the world by storm like Twitter Bootstrap. From Building Twitter Bootstrap:

A year-and-a-half ago, a small group of Twitter employees set out to improve our team’s internal analytical and administrative tools. After some early meetings around this one product, we set out with a higher ambition to create a toolkit for anyone to use within Twitter, and beyond. Thus, we set out to build a system that would help folks like us build new projects on top of it, and Bootstrap was conceived.
...
Today, it has grown to include dozens of components and has become the most popular project on GitHub with more than 13,000 watchers and 2,000 forks.

The fact that Bootstrap has become the most popular project on GitHub says a lot. For AppFuse.next, I'd like to integrate a lot of my learnings over the past few years, as well as support HTML5 and modern browsers as best we can. This means page speed optimizations, getting rid of Prototype and Scriptaculous in favor of jQuery, adding wro4j for resource optimization and integrating HTML5 Boilerplate. I've used Twitter Bootstrap for my Play More! app, as well as some recent client projects. Its excellent documentation has made it easy to use and I love the way you can simply add classes to elements to make them transform into something beautiful.

Last week, I spent a couple late nights integrating Twitter Bootstrap 2.0 into the Struts 2 and Spring MVC versions of AppFuse. The layout was pretty straightforward thanks to Scaffolding. Creating the Struts Menu Velocity template to produce dropdowns wasn't too difficult. I added class="table table-condensed" to the list screen tables, class="well form-horizontal" to forms and class="btn primary" to buttons.

I also added validation errors with the "help-inline" class. This is also where things got tricky with Struts and Spring MVC. For the form elements in Bootstrap, they recommend you use a "control-group" element that contains your label and a "controls" element. The control contains the input/select/textarea and also the error message if there is one. Here's a sample element waiting for data:

<div class="control-group">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name">
    </div>
</div>

Below is what that element should look like to display a validation error:

<div class="control-group error">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name" value="">
        <span class="help-inline">Please enter your name.</span>
    </div>
</div>

You can see this markup is pretty easy, you just need to add an "error" class to control-group and span to show the error message. With Struts 2, this was pretty easy thanks to its customizable templates for its tags. All I had to do was create a "template/css_xhtml" directory in src/main/webapp and modify checkbox.ftl, controlfooter.ftl, controlheader-core.ftl and controlheader.ftl to match Bootstrap's conventions.

Spring MVC was a bit trickier. Since its tags don't have the concept of writing an entire control (label and field), I had to do a bit of finagling to get things to work. In the current implementation, Struts 2 forms have a single line for a control-group and its control-label and controls.

<s:textfield key="user.firstName" required="true"/>

With Spring MVC, it's a bit more complex:

<spring:bind path="user.firstName">
<fieldset class="control-group${(not empty status.errorMessage) ? ' error' : ''}">
</spring:bind>
    <appfuse:label styleClass="control-label" key="user.firstName"/>
    <div class="controls">
        <form:input path="firstName" id="firstName" maxlength="50"/>
        <form:errors path="firstName" cssClass="help-inline"/>
    </div>
</fieldset>

You could probably overcome this verbosity with Tag Files.

Figuring out if a control-group needed an error class before the input tag was rendered was probably the hardest part of this exercise. This was mostly due to Bootstrap's great documentation and useful examples (viewed by inspecting the markup). Below are some screenshots of the old screens and new ones.

Check out the full set on Flickr if you'd like a closer look.

Even though I like the look of the old UI, I can't help but think a lot of the themes are designed for blogs and content sites, not webapps. The old Wufoo forms were a lot better looking though. And if you're going to develop kick-ass webapps, you need to make them look good. Bootstrap goes a long way in doing this, but it certainly doesn't replace a good UX Designer. Bootstap simply helps you get into HTML5-land, start using CSS3 and it takes the pain out of making things work cross-browser. Its fluid layouts and responsive web design seems to work great for business applications, which I'm guessing AppFuse is used for the most.

I can't thank the Bootstrap developers enough for helping me make this all look good. With Bootstrap 2 dropping this week, I can see myself using this more and more on projects. In the near future, I'll be helping integrate Bootstrap into AppFuse's Tapestry 5 and JSF versions.

What do you think of this CSS change? Do you change your CSS and layout a fair bit when starting with AppFuse archetypes? What can we do to make AppFuse apps look better out-of-the-box?

Update: I updated AppFuse to the final Bootstrap 2.0 release. Also, Johannes Geppert wrote a Struts 2 Bootstrap Plugin. I hope to integrate this into AppFuse in the near future.

My HTML5 with Play Scala, CoffeeScript and Jade Presentation from Devoxx 2011

2011-11-18T11:18:38-07:00

This week, I had the pleasure of traveling to one of my favorite places in the world: Antwerp, Belgium. Like last year, I traveled with the lovely Trish McGinity and spoke at Devoxx 2011. This year, my talk was on developing a web/mobile app with HTML5, Play, Scala, CoffeeScript and Jade. I was inspired to learn Scala at the beginning of this year and added CoffeeScript and Jade to my learning list after talking to James Strachan at TSSJS 2011. You can read more about how my journey began in my first post about learning these technologies.

I started developing with these technologies in August and wrote about my learnings throughout the process. Last week, while writing my presentation, I decided it'd be fun to make my presentation into more of a story-telling-session than a learn-about-new-technologies session. To do this, I focused on talking a bit about the technologies, but more about my experience learning them. I also came up with a challenging idea: create a video that showed the development process, how hard it was to test the app and (hopefully) my success in getting it to work.

It was all a very close call, but I'm happy to say I pulled it off! I got the app to work on an iPhone (thanks to PhoneGap) last Saturday, finished the first draft of my presentation on Sunday night (after pulling an all-nighter) and finished editing the demo video on Wednesday night. My talk was on Thursday afternoon and I had a blast talking about my experience to such a large, enthusiastic audience. You can see the presentation below, on Slideshare or download the PDF.

You can find the "demo" for this talk on YouTube or watch it below.

One of the reasons I really enjoyed this talk is it only represents one milestone in my learning process. I plan on continuing to develop this application and learning more about HTML5, Scala, Play and CoffeeScript and Scalate/Jade. Now that Play 2.0 Beta has been released, I plan on upgrading to it and leveraging its native CoffeeScript and LESS support. I hope to continue using Scalate and its Jade format. And it's very likely PhoneGap will continue to be the bridge that allows everything to run in the background.

I've been talking with the Jfokus folks about doing this talk in Sweden in Feburary and Devoxx France about presenting there in April.

Learning all these technologies has been a challenging, but fun experience so far. As the last slide in my presentation says, I encourage you to do something similar. Pick something new to learn, have fun doing it, but more importantly - get out there and Play!

Update Dec. 20th: A video of this presentation is now available on Parleys.com.

Hyperproductive JSF 2.0 with Ed Burns at Jazoon

2011-06-23T04:53:10-06:00

This morning, I attended Ed Burn's Talk on Hyperproductive JSF 2.0 at Jazoon. As you might know, I've been a critic of JSF for many years. However, it is one of the most used Java web frameworks, so I was hoping to learn how it's improved in the latest version. Below are my notes from Ed's presentation.

Ed's Plan for our Time Investment:

Define a productive development environment
JSF for greenfield and brownfield projects
List the top 9 productivity killers with JSF projects and solutions

"I am always doing that which I cannot do, in order that I may learn how to do it." -- Pablo Picasso

Software is an executable representation of knowledge, a byproduct of learning how to solve a problem. Knowledge is something that changes as more information comes in. A productive environment makes it as easy as possible to learn how to solve a problem. Learning is an iterative process. Early iterations don't need to be optimal. Later iterations do ... and they need to be maintainable. First is the hardest. Fast iterations are they key. Spring Roo and Play are examples of frameworks that make the first iteration very fast.

You should use a tool to jumpstart JSF development: copy from an old project, use a Maven archetype or use your IDE. With greenfield development, you don't have to bother learning the byproduct of other people's learning. It's a great opportunity to pad your resume with the latest hot technologies. With brownfield development, it's vitally important to understand the existing solution and hidden assumptions. You're also much more constrained in your technology choices. If you want to change, you'll need to come up with a migration strategy, which can be difficult. JSF works well for both because it's not just a runtime framework, it's also a conceptual framework. You need to understand how your framework handles data conversion, validation, page flow, persistence integration, I18N, L10N, A11Y, Web standards and user friendliness.

Top 9 JSF Productivity Killers:

Time wasting deployment step
The perils of "there's more than one way to do it"
Lengthy and painful developer on-boarding process
Misused logging and misunderstood error messages
Phantoms
Under-utilized developer tools
Premature optimization
Difficulty in doing TDD
Lack of an app framework

Time wasting deployment step
ZeroTurnaround solves this problem with JRebel, but there's other ways to do it. Some of the killers of flow state: 1) one large war file, 2) underutilizing dynamic language features, 3) complex server environment and 4) build process is redoing work unnecessarily. To stop the time wasting deployment step, the most important things you can do are as follows:

Configure your IDE correctly. In Eclipse, compile directly into WEB-INF/classes and use continuous compilation. With NetBeans, use GlassFish.
Don't do control-flow programming in XML.

Ed then showed a simple demo that showed how you can use Groovy to create a JSF UI Component. He also mentioned that Groovy can be used to author any JSF artifact. The benefit of this is you can simply edit and save a .groovy file without having to recompile or redeploy. Unfortunately, using Groovy didn't eliminate the XML syntax for pages or the XML for defining UI components.

The perils of "there's more than one way to do it"
JSF is very flexible, but flexibility is, more often than not, abused. There's a lack of convention for common things (e.g. master-detail, JSF concepts like converter, validator, etc.). The best way to fix this is to establish the norms for a project and stick with them. For example, Neil Griffin has a good blog entry for the different kind of managed beans you can create. Develop recommendations like Neil's and use them on all your projects.

Lengthy developer on-boarding process
Stick with standards when possible (at least have a common project description and build system across projects). Be committed to periodic cleanup cycles, including documenting for re-use. Pick one JSF component library and stick with it. Support for mixing and switching component libraries has improved with JSF 2, but it's still recommended you use only one.

Misused logging and misunderstood error messages
JSF is notorious for cryptic error messages and very long stack traces. It's still a problem, but the JSF Team is still working on improving them. Good tip: use the <ui:debug> tag. Its recordStateSize="true" attribute can be especially useful. If you're using PrimeFaces, add trace=true to request URLs.

Phantoms
Phantoms is when running code is not the same as the code you are modifying or wrong version of library gets picked up. You should have the capability to hit breakpoints anywhere in your entire software stack, including core Java sources. This is one of the most useful things about open source software. Solutions to phantoms: 1) put a timestamp on every redeploy and have the timestamp appear in the system log 2) write the running library stack to the system log (each library and version being used) and make it easy to compare one developer's runtime stack with another's 3) consider doing all work in tightly controlled VMs (checkout the VM at the beginning of the day, do your work, commit your changes and throw your VM away at the end of the day).

Under-utilizing developer tools
Make sure everyone has the fastest machines available and as much screen real estate as desired. Hardware is much cheaper than developer time. Another tip is to use Hudson as your butler. It's not just the team CI server. In other words, take advantage of automation wherever you can.

Premature Optimization
Keep in mind the trade-offs between readability and performance. When using frameworks such as JSF, don't try to outsmart the implementation. Rather, use the framework as intended and use open-source contributions to treat performance problems. Example, EL expressions got a lot faster between EE5 and EE6. If you spent time trying to optimize EL expressions, you might've been wasting your time.

Difficulty in doing TDD
Try to figure out why TDD is difficult in your company. For JSF, strongly consider JBoss's JSFUnit. Write your testcases to extend from Cactus ServletTestCase and leverage HtmlUnit (JSFUnit does this for you).

Lack of an app framework
Create common components: login panel, CRUD components, etc. If you don't have an app framework, build one over time.

Conclusion
This was an interesting talk by Ed. The dynamics of the room where a bit interesting. Jazoon is held in a movie theater, much like Devoxx. However, it appears there's a spotlight on the speaker that makes it very difficult to see the audience. I don't remember having this problem at Devoxx. Ed asked the audience quite a few questions, but it seemed he had a lot of difficulty in seeing if folks raised their hands. This made for some periods of awkward silence.

Personally, I was hoping to learn some new whizbang tips about JSF that I was not aware of. Unfortunately, I didn't learn anything new and wasn't that impressed with the Groovy demo.

I think Ed's tips about things outside of JSF were good, especially buying developers good hardware. I've seen many companies, including my current client, skimp on developer hardware and cause developer frustration because of it. I think it's great when companies provide developers top-of-the-line hardware and eliminate frustration over CPU and memory resources. LinkedIn and Time Warner Cable both provide their developers with Mac Pros and MacBook Pros as well as huge monitors. IMO, this is one of the best benefits you can provide your engineers.

AppFuse 2.1 Released!

2011-04-04T09:38:05-06:00

The AppFuse Team is pleased to announce the release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are JPA 2, JSF 2, Tapestry 5 and Spring 3. In addition, we've migrated from XFire to CXF and enabled REST for web services. There's even a new appfuse-ws archetype that leverages Enunciate to generate web service endpoints, documentation and downloadable clients. This release fixes many issues with archetypes, improving startup time and allowing jetty:run to be used for quick turnaround while developing. For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is an open source project and application that uses open source frameworks to help you develop Web applications with Java quickly and efficiently. It was originally developed to eliminate the ramp-up time when building new web applications. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project. If you use JRebel with IntelliJ, you can achieve zero-turnaround in your project and develop features without restarting the server.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose with persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run "appfuse:full-source".

AppFuse comes in a number of different flavors. It offers "light", "basic" and "modular" and archetypes. Light archetypes use an embedded H2 database and contain a simple CRUD example. Light archetypes allow code generation and full-source features, but do not currently support Stripes or Wicket. Basic archetypes have web services using CXF, authentication from Spring Security and features including signup, login, file upload and CSS theming. Modular archetypes are similar to basic archetypes, except they have multiple modules which allows you to separate your services from your web project.

AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket. You can see demos of these archetypes at http://demo.appfuse.org.

For information on creating a new project, please see the QuickStart Guide.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the mailing list or create an issue in JIRA.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

Adding Search to AppFuse with Compass

2011-03-15T17:11:12-06:00

Over 5 years ago, I recognized that AppFuse needed to have a search feature and entered an issue in JIRA. Almost 4 years later, a Compass Tutorial was created and shortly after Shay Banon (Compass Founder), sent in a patch. From the message he sent me:

A quick breakdown of enabling search:

Added Searchable annotations to the User and Address.
Defined Compass bean, automatically scanning the model package for mapped searchable classes. It also automatically integrates with Spring transaction manager, and stores the index on the file system ([work dir]/target/test-index).
Defined CompassTemplate (similar in concept to HibernateTemplate).
Defined CompassSearchHelper. Really helps to perform search since it does pagination and so on.
Defined CompassGps, basically it allows for index operation allowing to completely reindex the data from the database. JPA and Hiberante also automatically mirror changes done through their API to the index. iBatis uses AOP.

Fast forward 2 years and I finally found the time/desire to put a UI on the backend Compass implementation that Shay provided. Yes, I realize that Compass is being replaced by ElasticSearch. I may change to use ElasticSearch in the future; now that the search feature exists, I hope to see it evolve and improve.

Since Shay's patch integrated the necessary Spring beans for indexing and searching, the only thing I had to do was to implement the UI. Rather than having an "all objects" results page, I elected to implement it so you could search on an entity's list screen. I started with Spring MVC and added a search() method to the UserController:

@RequestMapping(method = RequestMethod.GET)
public ModelAndView handleRequest(@RequestParam(required = false, value = "q") String query) throws Exception {
    if (query != null && !"".equals(query.trim())) {
        return new ModelAndView("admin/userList", Constants.USER_LIST, search(query));
    } else {
        return new ModelAndView("admin/userList", Constants.USER_LIST, mgr.getUsers());
    }
}

public List<User> search(String query) {
    List<User> results = new ArrayList<User>();
    CompassDetachedHits hits = compassTemplate.findWithDetach(query);
    log.debug("No. of results for '" + query + "': " + hits.length());
    for (int i = 0; i < hits.length(); i++) {
        results.add((User) hits.data(i));
    }
    return results;
}

At first, I used compassTemplate.find(), but got an error because I wasn't using an OpenSessionInViewFilter. I decided to go with findWithDetach() and added the following search form to the top of the userList.jsp page:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms"/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

NOTE: I tried using HTML5's <input type="search">, but found Canoo WebTest doesn't support it.

Next, I wrote a unit test to verify everything worked as expected. I found I had to call compassGps.index() as part of my test to make sure my index was created and up-to-date.

public class UserControllerTest extends BaseControllerTestCase {
    @Autowired
    private CompassGps compassGps;
    @Autowired
    private UserController controller;

    public void testSearch() throws Exception {
        compassGps.index();
        ModelAndView mav = controller.handleRequest("admin");
        Map m = mav.getModel();
        List results = (List) m.get(Constants.USER_LIST);
        assertNotNull(results);
        assertTrue(results.size() >= 1);
        assertEquals("admin/userList", mav.getViewName());
    }
}

After getting this working, I started integrating similar code into AppFuse's other web framework modules (Struts, JSF and Tapestry). When I was finished, they all looked pretty similar from a UI perspective.

Struts:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms..."/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

JSF:

<div id="search">
<h:form id="searchForm">
    <h:inputText id="q" name="q" size="20" value="#{userList.query}"/>
    <h:commandButton value="#{text['button.search']}" action="#{userList.search}"/>
</h:form>
</div>

Tapestry:

<div id="search">
<t:form method="get" t:id="searchForm">
    <t:textfield size="20" name="q" t:id="q"/>
    <input t:type="submit" value="${message:button.search}"/>
</t:form>
</div>

One frustrating thing I found was that Tapestry doesn't support method="get" and AFAICT, neither does JSF 2. With JSF, I had to make my UserList bean session-scoped or the query parameter would be null when it listed the results. Tapestry took me the longest to implement, mainly because I had issues figuring out how it's easy-to-understand-once-you-know onSubmit() handlers worked and I had the proper @Property and @Persist annotations on my "q" property. This tutorial was the greatest help for me. Of course, now that it's all finished, the code looks pretty intuitive.

Feeling proud of myself for getting this working, I started integrating this feature into AppFuse's code generation and found I had to add quite a bit of code to the generated list pages/controllers.

So I went on a bike ride...

While riding, I thought of a much better solution and added the following search method to AppFuse's GenericManagerImpl.java. In the code I added to pages/controllers previously, I'd already refactored to use CompassSearchHelper and I continued to do so in the service layer implementation.

@Autowired
private CompassSearchHelper compass;

public List<T> search(String q, Class clazz) {
    if (q == null || "".equals(q.trim())) {
        return getAll();
    }

    List<T> results = new ArrayList<T>();

    CompassSearchCommand command = new CompassSearchCommand(q);
    CompassSearchResults compassResults = compass.search(command);
    CompassHit[] hits = compassResults.getHits();

    if (log.isDebugEnabled() && clazz != null) {
        log.debug("Filtering by type: " + clazz.getName());
    }

    for (CompassHit hit : hits) {
        if (clazz != null) {
            if (hit.data().getClass().equals(clazz)) {
                results.add((T) hit.data());
            }
        } else {
            results.add((T) hit.data());
        }
    }

    if (log.isDebugEnabled()) {
        log.debug("Number of results for '" + q + "': " + results.size());
    }

    return results;
}

This greatly simplified my page/controller logic because now all I had to do was call manager.search(query, User.class) instead of doing the Compass login in the controller. Of course, it'd be great if I didn't have to pass in the Class to filter by object, but that's the nature of generics and type erasure.

Other things I learned along the way:

To index on startup, I added compassGps.index() to the StartupListener..
In unit tests that leveraged transactions around methods, I had to call compassGps.index() before any transactions started.
To scan multiple packages for searchable classes, I had to add a LocalCompassBeanPostProcessor.

But more than anything, I was reminded it always helps to take a bike ride when you don't like the design of your code.

This feature and many more will be in AppFuse 2.1, which I hope to finish by the end of the month. In the meantime, please feel free to try out the latest snapshot.

JSR 303 and JVM Web Framework Support

2011-03-08T11:33:24-07:00

Emmanuel Bernard recently sent an email to the JSR 303 Experts Group about the next revision of the Bean Validation JSR (303). Rather than sending the proposed changes privately, he blogged about them. I left a comment with what I'd like to see:

+1 for Client-side validation. I'd love to see an API that web frameworks can hook into to add "required" to their tags for HTML5. Or some service that can be registered so the client can make Ajax requests to an API to see if an object is valid.

Emmanuel replied that most of the necessary API already exists for this, but frameworks have been slow to adopt it.

Hi Matt,

The sad thing is that the API is present on the Bean Validation side but presentation frameworks are slow to adopt it and use it :(

RichFaces 4 now has support for it but I wished more presentation frameworks had worked on the integration. If you can convince a few people or have access to a few people, feel free to send them by me :)

The integration API is described here. Let me know if you think some parts are missing or should be improved. We should definitely do some more buzz around it.

In the interest of generating more buzz around it, I decided to do some research and see what JVM Frameworks support JSR 303. Here's what I've come up with so far (in no particular order):

Struts 2 has an open issue, but doesn't seem to support JSR 303. Since I did a quick-n-dirty google search for most of these, I'm not sure if they support client-side JavaScript or HTML5's required. If you know of other JVM-based web frameworks that support JSR 303, please let me know in the comments.

Upgrading to JSF 2

2011-03-07T13:24:53-07:00

Last week, I spent a few hours upgrading AppFuse from JSF 1.2 to JSF 2.0. In reality, I upgraded from MyFaces 1.2.7 to 2.0.4, but all JSF implementations should be the same, right? All in all, it was a pretty easy upgrade with a few minor AppFuse-specific things. My goal in upgrading was to do the bare minimum to get things working and to leave integration of JSF 2 features for a later date.

In addition to upgrading MyFaces, I had to upgrade Tomahawk by changing the dependency's artifactId to tomahawk20. I was also able to remove the following listener from my web.xml:

<listener>
    <listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
<listener>

After that, I discovered that MyFaces uses a new URI (/javax.faces.resource/) for serving up some of its resource files. I kindly asked Spring Security to ignore these requests by adding the following to my security.xml file.

<intercept-url pattern="/javax.faces.resource/**" filters="none"/>

Since JSF 2 includes Facelets by default, I tried removing Facelets as a dependency. After doing this, I received the following error:

ERROR [308855416@qtp-120902214-7] ViewHandlerWrapper.fillChain(158) | Error instantiation parent Faces ViewHandler
java.lang.ClassNotFoundException: com.sun.facelets.FaceletViewHandler
        at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:401)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at org.ajax4jsf.framework.ViewHandlerWrapper.fillChain(ViewHandlerWrapper.java:144)
        at org.ajax4jsf.framework.ViewHandlerWrapper.calculateRenderKitId(ViewHandlerWrapper.java:68)
        at org.apache.myfaces.lifecycle.DefaultRestoreViewSupport.isPostback(DefaultRestoreViewSupport.java:179)
        at org.apache.myfaces.lifecycle.RestoreViewExecutor.execute(RestoreViewExecutor.java:113)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:171)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Figuring this was caused by the following element in my web.xml ...

<context-param>
    <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
    <param-value>com.sun.facelets.FaceletViewHandler</param-value>
</context-param>

... I removed it and tried again. This time I received a NoClassDefFoundError:

java.lang.NoClassDefFoundError: com/sun/facelets/tag/TagHandler
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:392)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.myfaces.shared_impl.util.ClassUtils.classForName(ClassUtils.java:184)
        at org.apache.myfaces.view.facelets.util.ReflectionUtil.forName(ReflectionUtil.java:67)

Since everything seemed to work with Facelets in the classpath, I decided to save this headache for a later date. I entered two issues in AppFuse's JIRA, one for removing Facelets and one for replacing Ajax4JSF with RichFaces.

The next issue I encountered was redirecting from AppFuse's password hint page. The navigation-rule for this page is as follows:

<navigation-rule>
    <from-view-id>/passwordHint.xhtml</from-view-id>
    <navigation-case>
        <from-outcome>success</from-outcome>
        <to-view-id>/login</to-view-id>
        <redirect/>
    </navigation-case>
</navigation-rule>

With JSF 2.0, the rule changes the URL to /login.xhtml when redirecting (where it was left as /login with 1.2) and it was caught by the security setting in my web.xml that prevents users from viewing raw templates.

<security-constraint>
    <web-resource-collection>
        <web-resource-name>Protect XHTML Templates</web-resource-name>
        <url-pattern>*.xhtml</url-pattern>
    </web-resource-collection>
    <auth-constraint/>
</security-constraint>

To solve this issue, I had to make a couple of changes:

Comment out the security-constraint in web.xml and move it to Spring Security's security.xml file.
```
<intercept-url pattern="/**/*.xhtml" access="ROLE_NOBODY"/>
```
Add a rule to urlrewrite.xml that redirects back to login (since login.xhtml doesn't exist and I'm using extensionless URLs).
```
<rule match-type="regex">
    <from>^/login.xhtml$</from>
    <to type="redirect">%{context-path}/login</to>
</rule>
```

After getting the Password Hint feature passing in the browser, I tried running the integration tests (powered by Canoo WebTest). The Password Hint test kept failing with the following error:

[ERROR] /Users/mraible/dev/appfuse/web/jsf/src/test/resources/web-tests.xml:51: JavaScript error loading
page http://localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/passwordHint?username=admin: syntax error (http://
localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/javax.faces.resource/oamSubmit.js.jsf?ln=org.apache.myfaces#122)

Figuring this was caused by my hack to submit the form when the page was loaded, I turned to Pretty Faces, which allows you to call a method directly from a URL. After adding the Pretty Faces dependencies to my pom.xml, I created a src/main/webapp/WEB-INF/pretty-config.xml file with the following XML:

<url-mapping>
    <pattern value="/editProfile"/>
    <view-id value="/userForm.jsf"/>
    <action>#{userForm.edit}</action>
</url-mapping>

<url-mapping>
    <pattern value="/passwordHint/#{username}"/>
    <view-id value="/passwordHint.jsf"/>
    <action>#{passwordHint.execute}</action>
</url-mapping>

This allowed me to remove both editProfile.xhtml and passwordHint.xhtml, both of which simply auto-submitted forms.

At this point, I figured I'd be good to go and ran my integration tests again. The first thing I discovered was that ".jsf" was being tacked onto my pretty URL, most likely by the UrlRewriteFilter. Adding the following to my PasswordHint.java class solved this.

if (username.endsWith(".jsf")) {
    username = username.substring(0, username.indexOf(".jsf"));
}

The next thing was a cryptic error that took me a while to figure out.

DEBUG [1152467051@qtp-144702232-0] PasswordHint.execute(38) | Processing Password Hint...
2011-03-05 05:48:52.471:WARN::/passwordHint/admin
com.ocpsoft.pretty.PrettyException: Exception occurred while processing <:#{passwordHint.execute}> null
        at com.ocpsoft.pretty.faces.beans.ActionExecutor.executeActions(ActionExecutor.java:71)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.processEvent(PrettyPhaseListener.java:214)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.afterPhase(PrettyPhaseListener.java:108)
        at org.apache.myfaces.lifecycle.PhaseListenerManager.informPhaseListenersAfter(PhaseListenerManager.java:111)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:185)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Digging into the bowels of MyFaces, I discovered a class was looking for a viewId with an extension and no view-id was being set. Adding the following to the top of my execute() method solved this.

getFacesContext().getViewRoot().setViewId("/passwordHint.xhtml");

After making this change, all AppFuse's integration tests are passing and the upgrade seems complete. The only other issues I encountered were logging-related. The first is an error about Tomahawk that doesn't seem to affect anything.

Mar 5, 2011 6:44:01 AM com.sun.facelets.compiler.TagLibraryConfig loadImplicit
SEVERE: Error Loading Library: jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml
java.io.IOException: Error parsing [jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml]: 
        at com.sun.facelets.compiler.TagLibraryConfig.create(TagLibraryConfig.java:410)
        at com.sun.facelets.compiler.TagLibraryConfig.loadImplicit(TagLibraryConfig.java:431)
        at com.sun.facelets.compiler.Compiler.initialize(Compiler.java:87)
        at com.sun.facelets.compiler.Compiler.compile(Compiler.java:104)

The second is excessive logging from MyFaces. As far as I can tell, this is because MyFaces switched to java.util.logging instead of commons logging. With all the frameworks that AppFuse leverages, I think it has all the logging frameworks in its classpath now. I was hoping to fix this by posting a message to the mailing list, but haven't received a reply yet.

[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider newInstance
[WARNING] [talledLocalContainer] INFO: Creating instance of org.appfuse.webapp.action.BasePage
[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider destroyInstance
[WARNING] [talledLocalContainer] INFO: Destroy instance of org.appfuse.webapp.action.BasePage

After successfully upgrading AppFuse, I turned to AppFuse Light, where things were much easier.

Now that AppFuse uses JSF 2, I hope to start leveraging some of its new features. If you're yearning to get started with them today, I invite you to grab the source and start integrating them yourself.

Upcoming Conferences: TSSJS in Las Vegas and 33rd Degree in Kraków, Poland

2011-02-25T15:14:59-07:00

It's that time of year again - the beginning of Conference Season. I generally like to speak at a few conferences a year and 2011 is no different. For March Madness, I'll be heading to Las Vegas to speak at TheServerSide Java Symposium. I'll be giving updated talks similar to the ones I gave at last year's Rich Web Experience in Fort Lauderdale:

You might remember my Comparing JVM Web Frameworks talk from Devoxx 2010 and some of the interesting debate it caused. I've done some minor updates to my video presentation and some updates to my JVM Web Frameworks presentation as well. Most notably, I'll be including some findings from Peter Thomas's perfbench project. I also hope to update AppFuse to JSF 2 and integrate extensionless URLs in AppFuse Light. Marcin Zajaczkowski was nice enough to provide an upgrade to Wicket 1.4.15, so it'll be interesting to see how well Wicket supports extensionless URLs.

In April, I'll be presenting Comparing JVM Web Frameworks at the 33rd Degree Conference in Kraków, Poland. While I studied in Russia a couple summers in college, I've never been to Poland, so I'm really looking forward to this trip. With any luck, I'll have AppFuse 2.1 released by then and my knowledge of all its web frameworks' latest versions will be update-to-date. As you know, it's unlikely I'll recommend a best web framework (because there isn't one), but I hope to provide some techniques you can use to decide the best framework for your particular needs.

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and Über Conf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

If you're planning on attending TSSJS or 33rd Degree, hopefully I'll see you there.

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

2011-02-10T16:53:27-07:00

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Filter Name	Dispatchers
securityFilter	request
rewriteFilter	request, forward
struts-prepare	request
sitemesh	request, forward, include
staticFilter	request, forward
struts	request

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

Filter Name	Dispatchers
rewriteFilter	request
securityFilter	request, forward, include
struts-prepare	request, forward
sitemesh	request, forward, include
struts	forward

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.

Making Code Generation Smarter with Maven

2011-01-21T15:26:43-07:00

As you might've read in my last entry, I recently started a new gig with Overstock.com. On my first day, I was quickly immersed into the development process by joining the Conversion Team. The Conversion Team is responsible for developing the checkout UI and handling payments from customers. I quickly discovered Overstock was mostly a Linux + Eclipse Shop and did my best to get my favorite Mac + IntelliJ + JRebel installed and configured. Thanks to my new Team Lead, I was able to get everything up and running the first day, as well as checkin my first contribution: making mvn jetty:run work so I didn't have to use my IDE to deploy to Tomcat.

In setting up my environment, I couldn't help but notice running jetty:run took quite a while to run each time. Specifically, the build process took 45 seconds to start executing the Jetty plugin, then another 23 seconds to startup after that. The first suspicious thing I noticed was that the UI templates were being re-generated and compiled on each execution. The UI Templating Framework at Overstock is Jamon, and is described as follows:

Jamon is a text template engine for Java, useful for generating dynamic HTML, XML, or any text-based content. In a typical Model-View-Controller architecture, Jamon clearly is aimed at the View (or presentation) layer.

Because it is compiled to non-reflective Java code, and statically type-checked, Jamon is ideally suited to support refactoring of template-based UI applications. Using mock objects -like functionality, Jamon also facilitates unit testing of the controller and view.

To generate .java files from .jamon templates, we use the Jamon Plugin for Maven. Remembering that the Maven Compiler Plugin has an incremental-compile feature, I turned to its source code to find out how to implement this in the Jamon plugin. I was pleasantly surprised to find the StaleSourceScanner. This class allows you to easily compare two files to see if the source needs to re-examined for generation or compilation.

I noticed the Jamon Plugin had the following code to figure out which files it should generate into .java files:

private List<File> accumulateSources(File p_templateSourceDir)
{
  final List<File> result = new ArrayList<File>();
  if (p_templateSourceDir == null)
  {
    return result;
  }
  for (File f : p_templateSourceDir.listFiles())
  {
    if (f.isDirectory())
    {
      result.addAll(accumulateSources(f));
    }
    else if (f.getName().toLowerCase(Locale.US).endsWith(".jamon"))
    {
      String filePath = f.getPath();
       // FIXME !?
      String basePath = templateSourceDir().getAbsoluteFile().toString();
      result.add(new File(filePath.substring(basePath.length() + 1)));
    }
  }
  return result;
}

I changed it to be smarter and only generate changed templates with the following code:

private List<File> accumulateSources(File p_templateSourceDir) throws MojoExecutionException
{
  final List<File> result = new ArrayList<File>();
  if (p_templateSourceDir == null)
  {
    return result;
  }
  SourceInclusionScanner scanner = getSourceInclusionScanner( staleMillis );
  SourceMapping mapping = new SuffixMapping( ".jamon", ".java");

  scanner.addSourceMapping( mapping );

  final Set<File> staleFiles = new LinkedHashSet<File>();

  for (File f : p_templateSourceDir.listFiles())
  {
    if (!f.isDirectory())
    {
      continue;
    }

    try
    {
      staleFiles.addAll( scanner.getIncludedSources(f.getParentFile(), templateOutputDir()));
    }
    catch ( InclusionScanException e )
    {
      throw new MojoExecutionException(
        "Error scanning source root: \'" + p_templateSourceDir.getPath()
          + "\' " + "for stale files to recompile.", e );
    }
  }

  // Trim root path from file paths
  for (File file : staleFiles) {
    String filePath = file.getPath();
    String basePath = templateSourceDir().getAbsoluteFile().toString();
    result.add(new File(filePath.substring(basePath.length() + 1)));
  }
}

This method references a getSourceInclusionScanner() method, which is implemented as follows:

protected SourceInclusionScanner getSourceInclusionScanner( int staleMillis )
{
  SourceInclusionScanner scanner;

  if ( includes.isEmpty() && excludes.isEmpty() )
  {
      scanner = new StaleSourceScanner( staleMillis );
  }
  else
  {
      if ( includes.isEmpty() )
      {
          includes.add( "**/*.jamon" );
      }
      scanner = new StaleSourceScanner( staleMillis, includes, excludes );
  }

  return scanner;
}

If you're using Jamon and its Maven Plugin, you can view my patch at SourceForge. If you're looking to include this functionality in your project, I invite you to look at the code I learned from in the Maven Compiler's AbstractCompilerMojo class.

After making this change, I was able to reduce the build execution time by over 50%. Now it takes 20 seconds to hit the Jetty plugin and 42 seconds to finishing starting. Of course, in an ideal world, I'd like to get this down to 20 seconds or less. Strangely enough, the easiest way to do this seems to be simple: use Linux.

On the Linux desktop they provided me, it takes 12 seconds to hit the Jetty plugin and 23 seconds to finish starting. I'd like to think this is a hardware thing, but it only get 20% faster on OS X when using an 8GB RAM + SSD machine (vs. a 4GB + 5400 drive). Since Overstock has provided me with a 4GB MacBook Pro, I'm considering installing Ubuntu on it, just to see what the difference is.

In related news, Overstock.com is looking to hire a whole bunch of Java Developers this year. The pictures of the new Provo office look pretty sweet. Of course, you can also work at HQ, which is a mere 25 minutes from some of the best skiing in the world. Personally, I think Colorado's powder is better, but I can't argue with the convenience of no traffic. In addition to full-time gigs, they've started hiring more remote contractors like myself, so they pretty much have something for everyone. So if you love Java, like to get some turns in before work, and aren't an asshole - you should and I'll try to hook you up.

Update: After writing this post, I received an email from Neil Hartner questioning these numbers. Basically, he was able to get his MacBook Pro to run just as fast as Linux. Turns out, the reason my Mac was so much slower was because JRebel was configured in my MAVEN_OPTS. JRebel's FAQ does state the following:

Does JRebel make the server start up slower?
JRebel needs to do more work on startup (search more places for classes and resources, instrument classes, etc), so some slowdown can be expected. If it's larger than 50% please contact support@zeroturnaround.com.

Since it's right around 50% slower, I guess there's no reason to call them. My guess is the best thing to do is remove JRebel from MAVEN_OPTS, but have an alias that can enable it, or simply run it from your IDE.

How I Calculated Ratings for My JVM Web Frameworks Comparison

2010-12-06T11:55:18-07:00

When I re-wrote my Comparing JVM Web Frameworks presentation from scratch, I decided to add a matrix that allows you to rate a framework based on 20 different criteria. The reason I did this was because I'd used this method when choosing an Ajax framework for Evite last year. The matrix seemed to work well for selecting the top 5 frameworks, but it also inspired a lot of discussion in the community that my ratings were wrong.

I expected this, as I certainly don't know every framework as well as I'd like. The mistake I made was asking for the community to provide feedback on my ratings without describing how I arrived at them. From Peter Thomas's blog:

What you are doing is adjusting ratings based on who in the community shouts the loudest. I can't help saying that this approach comes across as highly arrogant and condescending, you seem to expect framework developers and proponents to rush over and fawn over you to get better ratings, like waiters in a restaurant trying to impress a food-critic for Michelin stars.

I apologize for giving this impression. It certainly wasn't my intent. By having simple numbers (1.0 == framework does well, 0.5 == framework is OK and 0 == framework not good at criteria) with no rationalization, I can see how the matrix can be interpreted as useless (or to put it bluntly, as something you should wipe your ass with). I don't blame folks for getting angry.

For my Rich Web Experience presentation, I documented why I gave each framework the rating I did. Hopefully this will allow folks to critique my ratings more constructively and I can make the numbers more accurate. You can view this document below or on Google Docs.

In the end, what I was hoping to do with this matrix was to simply highlight a technique for choosing a web framework. Furthermore, I think adding a "weight" to each criteria is important because things like books often aren't as important as REST support. To show how this might be done, I added a second sheet to the matrix and made up some weighting numbers. I'd expect anyone that wants to use this to downloaded the matrix, verify the ratings are accurate for your beliefs and weight the criteria accordingly.

Of course, as I and many others have said, the best way to choose a web framework is to try them yourself. I emphasized this at the end of my presentation with the following two slides.

My Comparing JVM Web Frameworks Presentation from Devoxx 2010

2010-11-18T05:23:10-07:00

This week, I've been having a great time in Antwerp, Belgium at the Devoxx Conference. This morning, I had the pleasure of delivering my Comparing JVM Web Frameworks talk. I thoroughly enjoyed giving this presentation, especially to such a large audience. You can view the presentation below (if you have Flash installed) or download it here.

Unlike previous years, I chose to come up with a spreadsheet matrix that shows why I chose the 5 I did. This spreadsheet and rankings given to each framework are likely to be debated, as I don't know all the frameworks as well as I'd like to. Also, the missing column on this spreadsheet is a "weighting" column where you can prioritize certain criteria like I've done in the past when Comparing Ajax Frameworks. If you believe there are incorrect numbers, please let me know and I'll try to get those fixed before I do this talk again at The Rich Web Experience.

One thing that doesn't come across in this presentation is that I believe anyone can use this matrix, and weightings, to make any of these frameworks come out on top. I also believe web frameworks are like spaghetti sauce in The Ketchup Conundrum. That is, the only way to make more happy spaghetti sauce lovers was to make more types of spaghetti sauce. You can read more about this in my There is no "best" web framework article.

Update: If you disagree with the various ratings I gave to web frameworks in this presentation, please provide your opinions by filling out this survey. Thanks to Sebastien Arbogast for setting this up.

Update: Sebastien has posted his survey results at JVM Web Framework Survey, First Results.

Update 12/6: A video of this presentation is now available on Parleys.com.

P.S. My current gig is ending in mid-December. If you're looking for a UI Architect with a passion for open source frameworks, please let me know.

AppFuse 2.1 Milestone 2 Released

2010-11-15T15:28:57-07:00

I'm pleased to announce the 2nd milestone release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Spring 3 and Struts 2.1. This release fixes many issues with archetypes and contains many improvements to support Maven 3. For more details on specific changes see the 2.1.0 M2 release notes.

What is AppFuse?
AppFuse is an open source project and application that uses open source frameworks to help you develop Web applications quickly and efficiently. It was originally developed to eliminate the ramp-up time when building new web applications. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project. If you use JRebel with AppFuse, you can achieve zero-turnaround in your project and develop features without restarting the server.

AppFuse comes in a number of different flavors. It offers "light", "basic" and "modular" and archetypes. Light archetypes use an embedded H2 database and contain a simple CRUD example. In the final 2.1.0 release, the light archetypes will allow code generation like the basic and modular archetypes. Basic archetypes have web services using CXF, authentication from Spring Security and features including signup, login, file upload and CSS theming. Modular archetypes are similar to basic archetypes, except they have multiple modules which allows you to separate your services from your web project.

AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket.

Please note that this release does not contain updates to the documentation. Code generation will work, but it's likely that some content in the tutorials won't match. For example, you can use annotations (vs. XML) for Spring MVC and Tapestry is a whole new framework. I'll be working on documentation over the next several weeks in preparation for the 2.1 final release.

For information on creating a new project, please see the QuickStart Guide.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find bugs, please create an issue in JIRA.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

RE: Moving from Spring to Java EE 6: The Age of Frameworks is Over

2010-10-16T15:19:07-06:00

Last Tuesday, Cameron McKenzie wrote an interesting article on TheServerSide titled Moving from Spring to Java EE 6: The Age of Frameworks is Over. In this article, Cameron says the following:

J2EE represents the past, and Java EE 6 represents the future. Java EE 6 promises us the ability to go beyond frameworks. Frameworks like Spring are really just a bridge between the mistakes of the J2EE past and the success of the Java EE 6 future. Frameworks are out, and extensions to the Java EE 6 platform are in. Now is the time to start looking past Spring, and looking forward to Seam and Weld and CDI technologies.

He then links to an article titled Spring to Java EE - A Migration Experience, an article written by JBoss's Lincoln Baxter. In this article, Lincoln talks about many of the technologies in Java EE 6, namely JPA, EJB, JSF, CDI and JAX-RS. He highlights all the various XML files you'll need to know about and the wide variety of Java EE 6 application servers: JBoss AS 6 and GlassFish v3.

I don't have a problem with Lincoln's article, in fact I think it's very informative and some of the best documentation I've seen for Java EE 6.

I do have some issues with Cameron's statements that frameworks are mistakes of the J2EE past and that Java EE 6 represents the future. Open source frameworks made J2EE successful. Struts and Hibernate came out in the early days of J2EE and still exist today. Spring came out shortly after and has turned into the do-everything J2EE implementation it was trying to fix. Java EE 6 might be a better foundation to build upon, but it's certainly not going to replace frameworks.

To prove my point, let's start by looking at the persistence layer. We used to have Hibernate based on JDBC, now we have JPA implementations built on top of the JPA API. Is JPA a replacement for all persistence frameworks? I've worked with it and think it's a good API, but the 2.0 version isn't available in a Maven repo and Alfresco recently moved away from Hibernate (which == JPA IMO) to iBATIS for greater data access layer control and scalability. Looks like the age of frameworks isn't over for persistence frameworks.

The other areas that Java EE 6 covers that I believe frameworks will continue to excel in: EJB, CDI, JSF and JAX-RS. Personally, I don't have a problem with EJB 3 and think it's a vast improvement on EJB 2.x. I don't have an issue with CDI either, and as long as it resembles Guice for dependency injection, it works for me. However, when you get into the space I've been living in for the last couple years (high-traffic public internet sites), EJB and things like the "conversation-scope" feature of CDI don't buy you much. The way to make web application scale is to eliminate state and cache as much as possible, both of which Java EE doesn't provide much help for. In fact, to disable sessions in a servlet-container, you have to write a Filter like the following:

public class DisabledSessionFilter extends OncePerRequestFilter {

    /**
     * Filters requests to disable URL-based session identifiers.
     */
    @Override
    protected void doFilterInternal(final HttpServletRequest request,
                                    final HttpServletResponse response,
                                    final FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequestWrapper wrappedRequest = new HttpServletRequestWrapper(request) {

            @Override
            public HttpSession getSession(final boolean create) {
                if (create) {
                    throw new UnsupportedOperationException("Session support disabled");
                }
                return null;
            }

            @Override
            public HttpSession getSession() {
                throw new UnsupportedOperationException("Session support disabled");
            }
        };

        // process next request in chain
        chain.doFilter(wrappedRequest, response);
    }
}

What about JAX-RS? Does it replace the need for frameworks? I like the idea of having a REST API in Java. However, its reference implementation is Jersey, which seems more like a framework than just Java EE. If you choose to use JAX-RS in your application, you still have to choose between CXF, Jersey, RESTEasy and Restlet. I compared these frameworks last year and found the Java EE implementation lacking in the features I needed.

Finally, let's talk about my-least-framework-web-framework: JSF. The main reason I don't like JSF is because of its 1.x version. JSF 1.0 was released a year before the Ajax term was coined (see timeline below). Not only did it take forever to develop as a spec, but it tried to be a client-component framework that was very stateful by default.

Now that JSF 2.0 is out, it has Ajax integrated and allows you to use GET instead of POST-for-everything. However, the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn't be developing your UI). Also, the best component development platform for the web is JavaScript. I recommend using an Ajax framework for your components if you really want a rich UI.

Sure you can use the likes of Tapestry and Wicket if you like POJO-based web development, but if you're looking to develop a webapp that's easy to maintain and understand, chances are that you'll do much better with traditional MVC frameworks like Spring MVC and Struts 2. The simplicity and popularity of Rails and Grails further emphasize that developers prefer these types of web frameworks.

Another reason I don't like JSF: there's very few developers in the wild happy with it. The major promoters of JSF are book authors, trainers, Java EE Vendors and MyFaces developers. Whenever I speak at conferences, I ask folks to raise their hands for the various web frameworks they're using. I always ask the JSF users to keep their hands up if they like it. Rarely do they stay up.

So it looks like we still need web frameworks.

Eberhard Wolff has an interesting post where he defends Spring and talks about the productivity comparisons between Spring and Java EE. He recommends using Grails or Spring Roo if you want the level of productivity that Ruby on Rails provides. That's a valid recommendation if you're building CRUD-based webapps, but I haven't developed those in quite some time. Nowadays, the apps I develop are true SOFEA apps, where the backend serves up XML or JSON and the frontend client is HTML/JavaScript/CSS, Android, iPad or Sony Blu-Ray players. On my current project, our services don't even talk to a database, they talk to a CMS via RESTful APIs. We use Spring's RestTemplate for this and HttpClient when it doesn't have the features we need. Not much in Java EE 6 for this type of communication. Sure, Jersey has a client, but it's certainly not part of the Java EE spec.

As far as getting Ruby on Rails' zero-turnaround productivity, I don't need Grails or Spring Roo, I simply use IDEA and JRebel.

Conclusion
I don't see how new features in Java EE 6 can mean the age of frameworks is over. Java SE and J2EE have always been foundations for frameworks. The Java EE 6 features are often frameworks in themselves that can be used outside of a Java EE container. Furthermore, Java EE 6 doesn't provide all the features you need to build a high-scale web app today. There's no caching, no stateless web framework that can serve up JSON and HTML and no hot-reload productivity enhancements like JRebel. Furthermore, there's real excitement in Javaland for languages like Scala, Groovy and JRuby. All of these languages have web frameworks that've made many developers happy.

Here's to the Age of Frameworks - may it live as long as the JVM!

P.S. If you'd like to hear me talk about web frameworks on the JVM, I'll be speaking at The Colorado Springs Open Source Meetup and Devoxx 2010 in the near future.

My TSSJS 2010 Presentations and Summary

2010-03-19T17:29:08-06:00

This afternoon, I delivered my last talk at TSSJS 2010 on The Future of Web Frameworks. It's true that I made some bold statements, but please remember that this is my personal opinion, based on my experience. For the most part, I've been involved in super high-traffic websites for the last few years and this has influenced my opinion on web frameworks. Just because I don't recommend your favorite framework doesn't mean it won't work for you. In fact, many of the best web applications today were built without an open source (or commercial) web framework. In the end, it's not as much about the web framework you're using as it is about hiring smart people. Below is my slide deck from this talk.

Yesterday, I did a GWT vs. Flex Smackdown with James Ward. While there wasn't as much trash talking as I'd hoped, I enjoyed delivering it and disputing the greatness of Flex. Below is the presentation that James and I delivered.

The show itself was great this year. It had more attendees than I've seen in a long time. There were a lot of really interesting sessions and and an often humorous Twitter back-channel. I attended quite a few talks and jotted down my notes from several of them. Please see the links below if you're interested in the sessions I attended. You can view all of the presentations from TSSJS 2010 on SlideShare.

Thanks to everyone who came to Vegas and to TheServerSide for an excellent conference.

My Future of Web Frameworks Presentation

2010-02-26T08:55:39-07:00

Earlier this week, I tweeted about a history of web frameworks timeline I created for my upcoming Future of Web Frameworks talk at TSSJS Vegas 2010. I immediately received a lot of feedback and requests for adding new frameworks and releases. The image below is the result of that Twitter conversation. Thanks to everyone who contributed.

Back in November, I wrote about my proposals for TSSJS. I've been thinking a lot about web frameworks lately and I can't help but think we live in a very exciting time. As a Java developer, I've been exposed to one of the most vibrant language ecosystems on the planet. As Tim Bray talks about, the Java Platform has 3 legs: the language, the virtual machine and a huge, immense library of APIs (both in the JDK and in open source libraries). The diagram below is something I created based on Tim's podcast.

Tim says, "One of those legs is replaceable and that's the language." And he's right, there's many Java.next languages that run efficiently on the JVM. This is one of the most exciting parts of being a Java web developer today. There's many proven web frameworks and languages that you can pick to build your next web application.

The best part is many of the best web frameworks run on the JVM. Not only that, but the best code editors are the IDEs that you're familiar with and have grown to love. Furthermore, much of the literature for Java.next languages is written for Java developers. As someone who knows Java, you have wealth of web frameworks and languages just waiting for you to learn them.

To create my presentation on the future of web frameworks, I followed the outline I posted previously. I plan on explaining the evolution and history of web frameworks and how we got to where we are today. From there, I'll be speculating on what web applications we'll be developing in the future. Finally, I'll touch on the necessary features of web frameworks that will allow us to develop these applications.

Of course, I haven't actually presented this talk yet, so it's likely to change in the coming weeks before the conference. The good news is this gives you the opportunity to provide constructive criticism on this presentation and help make it better. I realize that a presentation rarely represents the conversation that takes place during a conference. However, I believe it can portray the jist of my thinking and lead to a meaningful conversation in the comments of this post. Below is the presentation I created - thanks in advance for any feedback.

For those who will be joining me at TSSJS ... it's gonna be a great show. St. Patrick's Day in Vegas, what more could you ask for?

Update: This article has been re-posted on Javalobby and contains additional community feedback in the comments.

AppFuse 2.1 Milestone 1 Released

2009-11-19T07:16:36-07:00

The AppFuse Team is pleased to announce the first milestone release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Hibernate, Spring and Tapestry 5.

What is AppFuse?
AppFuse is an open source project and application that uses open source tools built on the Java platform to help you develop Web applications quickly and efficiently. It was originally developed to eliminate the ramp-up time found when building new web applications for customers. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose which persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run appfuse:full-source.

In addition, AppFuse Light has been converted to Maven and has archetypes available. AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket.

Other notable improvements:

Added Compass support thanks to a patch from Shay Banon.
Upgraded from XFire to CXF for Web Services.
Moved Maven repository to Sonatype's OSS Repository Hosting for snapshots and releasing to Maven Central. There are no longer any AppFuse-specific artifacts, all are available in central. Thanks to Sonatype for this great service and its excellent repository manager.
Upgraded to Canoo WebTest 3.0. Now if we could just get its Maven Plugin moved to Codehaus.
Added Ajaxified Body to AppFuse Light archetypes.
Infrastructure upgrades, including JIRA 4, Confluence 3, FishEye 2, Bamboo 2 and Crowd 1.6. Many thanks to Atlassian and Contegix for their excellent products and services.
For more details on specific changes see the release notes.

Please note that this release does not contain updates to the documentation. Code generation will work, but it's likely that some content in the tutorials won't match. For example, you can use annotations (vs. XML) for dependency injection and Tapestry is a whole new framework. I'll be working on documentation over the next several weeks in preparation for Milestone 2.

AppFuse is available as several Maven archetypes. For information on creating a new project, please see the QuickStart Guide.

To learn more about AppFuse, please read Ryan Withers' Igniting your applications with AppFuse.

The 2.x series of AppFuse has a minimum requirement of the following specification versions:

Java Servlet 2.4 and JSP 2.0 (2.1 for JSF)
Java 5+

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find bugs, please create an issue in JIRA.

Thanks to everyone for their help contributing code, writing documentation, posting to the mailing lists, and logging issues.

Building SOFEA Applications with GWT and Grails

2009-11-12T09:30:09-07:00

Last night, I spoke at the Denver Java User Group meeting. The consulting panel with Matthew, Tim and Jim a lot of fun and I enjoyed delivering my Building SOFEA Applications with GWT and Grails presentation for the first time. The talk was mostly a story about how we enhanced Evite.com with GWT and Grails and what we did to make both frameworks scale. I don't believe the presentation reflects the story format that well, but it's not about the presentation, it's about the delivery of it.

If you'd like to hear the story about this successful SOFEA implementation at a high-volume site, I'd recommend attending the Rich Web Experience next month. If you attended last night's meeting and have any feedback on how this talk can be improved, I'd love to hear it.

The Future of Web Frameworks at TSSJS

2009-11-10T13:24:39-07:00

For TSSJS Vegas 2010, I submitted two proposals for talks: GWT vs. Flex Smackdown and The Future of Web Frameworks. As of today, the 2nd is the only one that shows up on the conference agenda, but hopefully the former will get accepted too. Here's a description of this talk:

With rich Ajax applications and HTML5 on the horizon, are web frameworks still relevant? Java web frameworks like Struts and Spring MVC were all the rage 5 years ago. Component-based frameworks like Tapestry, JSF and Wicket made it easier to create re-usable applications. But what about the Mobile Web and offline applications?

Are Titanium, Adobe Air and Gears the future? If you're embracing the RESTfulness of the web, do you even need a web framework, or can you use use JAX-RS with an Ajax toolkit?

These questions and many more are examined, answered and debated in this lively session. Bring your opinions and experiences to this session to learn about what's dead, what's rising and what's here to stay. If you're a web framework fan, this session is sure to please.

I believe this talk will be a lot of fun to create and deliver. To create it, I'd like to make it a collaborative effort with the web framework community (users and developers). To kick things off, below is an initial rough outline/agenda:

Title
Introduction
Problem/Purpose
Agenda

How did we get here?
Where are we going?
How do we get there?
Q and A

History of Web Frameworks

Deep History (CGI, etc.)
Java's Rise
PHP
Rails -> Grails
Ajax Frameworks
RESTify!
SOFEA, APIs, etc.

The Future

HTML5
GWT, Cappucino and Spoutcore (compare to Java and compilers)
The Binary Players (Flex, JavaFX and Silverlight)
Getting Rich
Speed (is it a problem? YES!)
IE 6 will die.
Chrome OS
The Mobile Web
Desktop Webapps (Titanium, AIR, etc.)
Or is this the present? Future is bleeding edge.

Getting There: It's all about the APIs

Allows for any client
Web Framework skills transfer to desktop - and phone!
Speed will continue to be *very* important
Innovation, something we haven't thought of

Fallout

Interest in server-side frameworks will continue, but frameworks will become unmaintained
Ajax Frameworks will continue to innovate
HTML5 Frameworks?
IE 6 (hopefully!)
Desktop and Mobile with Web Technologies
Watch out for the next big thing! (or What do you think is the next big thing?)

Conclusion
Q and A

Is there anything I'm missing that's important for the future of web frameworks? Are there items that should be removed? Any advice is most welcome.

Reminder: I'll be speaking at tomorrow's DJUG if you'd like to discuss your thoughts in person.

A Letter to the AppFuse Community

2009-11-04T00:17:17-07:00

The last AppFuse release was way back in May 2008. Many folks have asked when the next release would be ever since. Often, I've said "sometimes this quarter", but obviously, that's never happened. For that, I apologize.

There are many reasons I haven't worked on AppFuse for the past 18 months, but it mostly comes down to the fact that I didn't make time for it. The good news is I'm working on it again and will have a release out sometime this month. Unfortunately, it probably won't be a 2.1 final release, but there's so many things that've changed, I feel like a milestone release is a good idea. Here's a brief summary of changes so far:

Changed archetypes to include all source and tests for the "webapp" portion of the application. No more warpath plugin, merging wars and IDE issues. Using "mvn jetty:run" should work as expected.
Moved from Spring XML to Annotations.
AppFuse Light converted to Maven modules and now depends on AppFuse's backend.
Published easier to use archetype selection form in the QuickStart Guide.
Published archetype selection form for AppFuse Light. I do plan on combining these forms as soon as I figure out the best UI and instructions for users to choose AppFuse or AppFuse Light.
Upgraded all libraries to latest released versions (Spring 3 hasn't had a final release yet).
Upgraded to Tapestry 5 thanks to Serge Eby. I still need to complete tests and code generation for tests.
Added Compass support thanks to a patch from Shay Banon.
Upgraded from XFire to CXF for Web Services.
Moved Maven repository to Sonatype's OSS Repository Hosting for snapshots and releasing to Maven Central. There are no longer any AppFuse-specific artifacts, all are available in central.

I realize there's many full-stack frameworks that do the same thing as AppFuse with less code. Examples include Ruby on Rails, Grails, Seam, Spring Roo and the Play framework. However, there seems to be quite a few folks that continue to use AppFuse and it stills serves the community as a nice example of how to integrate frameworks. Furthermore, it helps me keep up with the latest framework releases, their quirks and issues that happen when you try to integrate them. In short, working on it helps me stay up to speed with Java open source frameworks.

For those folks that like the 1.x, Ant-based version of AppFuse, there will not be a 1.9.5 release. I know I promised it for years, but it's simply something I will not use, so I'd rather not invest my time in it. I'm sorry for lying to those that expected it.

So what's the future of AppFuse? Will it continue to integrate web frameworks with Spring and popular persistence frameworks? Possibly, but it seems more logical to align it with the types of Ajax + REST applications I'm creating these days. I'm currently thinking AppFuse 3.0 would be nice as a RESTful backend with GWT and Flex UIs. I might create the backend with CXF, but it's possible I'd use one of the frameworks mentioned above and simply leverage it to create the default features AppFuse users have come to expect.

More than anything, I'm writing this letter to let you know that the AppFuse project is not dead and you can expect a release in the near future.

Thanks for your support,

Matt

The 2009 Rich Web Experience

2009-08-11T22:16:24-06:00

Late last year, I decided to take a year off from speaking at conferences. My reason was simple: I wanted to hunker down and do some learning. Ever since November 2007, I'd been interested in getting into developing SOFEA applications. This year, I was able to accomplish that and I've learned a lot about GWT and RESTful architectures in the process. Now I'm happy to announce I'll be sharing my experiences at this year's Rich Web Experience.

I look forward to speaking on the following topics:

The first one might look familiar since I used the same title at OSCON 2008. The difference is this time I plan to dive much deeper into the frameworks and help the audience learn more. I also plan to re-visit developing with Flex and Rails in the near future. If there's anything in particular you'd like to see covered in the above talks, please leave a comment.

This is sure to be a fun conference. First of all, it's pretty close to Disney World. A few years back, TSS hosted the "Java In Action" conference in Disney World and while the conference didn't do too well, it was awesome being there with my family. I plan on taking my kids and having some fun when I'm not speaking. Secondly, this conference is the first to offer all-inclusive pricing for attendees - registration, flight (continental U.S.) and 3 nights lodging. This should allow you to get approval without having to jump through the traditional travel hoops. Lastly, the JSF Summit will be held concurrently and you get access to it at no additional cost.

So come on down to Florida in December, listen to all the great speakers, have some fun with your family and look me up for a beer or two. You know it'll be a good time!

Integrating GWT with Spring Security

2009-08-06T08:50:15-06:00

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

How to do cross-domain GWT RPC with a ProxyServlet

2009-08-05T16:06:12-06:00

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Map;
import java.util.zip.GZIPInputStream;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileUploadException;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.StringRequestEntity;
import org.apache.commons.httpclient.methods.multipart.ByteArrayPartSource;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.StringPart;

/**
 * ProxyServlet from http://edwardstx.net/wiki/attach/HttpProxyServlet/ProxyServlet.java
 * (This seems to be a derivative of Noodle -- http://noodle.tigris.org/)
 * 
 * Patched to skip "Transfer-Encoding: chunked" headers, avoid double slashes
 * in proxied URLs, handle GZip and allow GWT RPC.
 */
public class ProxyServlet extends HttpServlet {

    private static final int FOUR_KB = 4196;

    /**
     * Serialization UID.
     */
    private static final long serialVersionUID = 1L;
    /**
     * Key for redirect location header.
     */
    private static final String STRING_LOCATION_HEADER = "Location";
    /**
     * Key for content type header.
     */
    private static final String STRING_CONTENT_ENGINE_HEADER_NAME = "Content-Type";
    /**
     * Key for content length header.
     */
    private static final String STRING_CONTENT_LENGTH_HEADER_NAME = "Content-Length";
    /**
     * Key for host header
     */
    private static final String STRING_HOST_HEADER_NAME = "Host";
    /**
     * The directory to use to temporarily store uploaded files
     */
    private static final File FILE_UPLOAD_TEMP_DIRECTORY = new File(System.getProperty("java.io.tmpdir"));

    // Proxy host params
    /**
     * The host to which we are proxying requests. Default value is "localhost".
     */
    private String stringProxyHost = "localhost";
    /**
     * The port on the proxy host to wihch we are proxying requests. Default value is 80.
     */
    private int intProxyPort = 80;
    /**
     * The (optional) path on the proxy host to wihch we are proxying requests. Default value is "".
     */
    private String stringProxyPath = "";
    /**
     * Setting that allows removing the initial path from client. Allows specifying /twitter/* as synonym for twitter.com.
     */
    private boolean removePrefix;
    /**
     * The maximum size for uploaded files in bytes. Default value is 5MB.
     */
    private int intMaxFileUploadSize = 5 * 1024 * 1024;
    private boolean isSecure;
    private boolean followRedirects;

    /**
     * Initialize the <code>ProxyServlet</code>
     * @param servletConfig The Servlet configuration passed in by the servlet container
     */
    public void init(ServletConfig servletConfig) {
        // Get the proxy host
        String stringProxyHostNew = servletConfig.getInitParameter("proxyHost");
        if (stringProxyHostNew == null || stringProxyHostNew.length() == 0) {
            throw new IllegalArgumentException("Proxy host not set, please set init-param 'proxyHost' in web.xml");
        }
        this.setProxyHost(stringProxyHostNew);
        // Get the proxy port if specified
        String stringProxyPortNew = servletConfig.getInitParameter("proxyPort");
        if (stringProxyPortNew != null && stringProxyPortNew.length() > 0) {
            this.setProxyPort(Integer.parseInt(stringProxyPortNew));
        }
        // Get the proxy path if specified
        String stringProxyPathNew = servletConfig.getInitParameter("proxyPath");
        if (stringProxyPathNew != null && stringProxyPathNew.length() > 0) {
            this.setProxyPath(stringProxyPathNew);
        }
        // Get the maximum file upload size if specified
        String stringMaxFileUploadSize = servletConfig.getInitParameter("maxFileUploadSize");
        if (stringMaxFileUploadSize != null && stringMaxFileUploadSize.length() > 0) {
            this.setMaxFileUploadSize(Integer.parseInt(stringMaxFileUploadSize));
        }
    }

    /**
     * Performs an HTTP GET request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a GET request
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("GET Request URL: " + httpServletRequest.getRequestURL(),
              "Destination URL: " + destinationUrl);
        GetMethod getMethodProxyRequest = new GetMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, getMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
        // Execute the proxy request
        this.executeProxyRequest(getMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Performs an HTTP POST request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doPost(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a standard POST request
        String contentType = httpServletRequest.getContentType();
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("POST Request URL: " + httpServletRequest.getRequestURL(),
              "    Content Type: " + contentType,
              " Destination URL: " + destinationUrl);
        PostMethod postMethodProxyRequest = new PostMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, postMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, postMethodProxyRequest);
        // Check if this is a mulitpart (file upload) POST
        if (ServletFileUpload.isMultipartContent(httpServletRequest)) {
            this.handleMultipartPost(postMethodProxyRequest, httpServletRequest);
        } else {
            if (contentType == null || PostMethod.FORM_URL_ENCODED_CONTENT_ENGINE.equals(contentType)) {
                this.handleStandardPost(postMethodProxyRequest, httpServletRequest);
            } else {
                this.handleContentPost(postMethodProxyRequest, httpServletRequest);
            }
        }
        // Execute the proxy request
        this.executeProxyRequest(postMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Sets up the given {@link PostMethod} to send the same multipart POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a multipart POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the mutlipart POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleMultipartPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest)
            throws ServletException {
        // Create a factory for disk-based file items
        DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
        // Set factory constraints
        diskFileItemFactory.setSizeThreshold(this.getMaxFileUploadSize());
        diskFileItemFactory.setRepository(FILE_UPLOAD_TEMP_DIRECTORY);
        // Create a new file upload handler
        ServletFileUpload servletFileUpload = new ServletFileUpload(diskFileItemFactory);
        // Parse the request
        try {
            // Get the multipart items as a list
            List<FileItem> listFileItems = (List<FileItem>) servletFileUpload.parseRequest(httpServletRequest);
            // Create a list to hold all of the parts
            List<Part> listParts = new ArrayList<Part>();
            // Iterate the multipart items list
            for (FileItem fileItemCurrent : listFileItems) {
                // If the current item is a form field, then create a string part
                if (fileItemCurrent.isFormField()) {
                    StringPart stringPart = new StringPart(
                            fileItemCurrent.getFieldName(), // The field name
                            fileItemCurrent.getString() // The field value
                            );
                    // Add the part to the list
                    listParts.add(stringPart);
                } else {
                    // The item is a file upload, so we create a FilePart
                    FilePart filePart = new FilePart(
                            fileItemCurrent.getFieldName(), // The field name
                            new ByteArrayPartSource(
                            fileItemCurrent.getName(), // The uploaded file name
                            fileItemCurrent.get() // The uploaded file contents
                            ));
                    // Add the part to the list
                    listParts.add(filePart);
                }
            }
            MultipartRequestEntity multipartRequestEntity = new MultipartRequestEntity(
                    listParts.toArray(new Part[]{}),
                    postMethodProxyRequest.getParams());
            postMethodProxyRequest.setRequestEntity(multipartRequestEntity);
            // The current content-type header (received from the client) IS of
            // type "multipart/form-data", but the content-type header also
            // contains the chunk boundary string of the chunks. Currently, this
            // header is using the boundary of the client request, since we
            // blindly copied all headers from the client request to the proxy
            // request. However, we are creating a new request with a new chunk
            // boundary string, so it is necessary that we re-set the
            // content-type string to reflect the new chunk boundary string
            postMethodProxyRequest.setRequestHeader(STRING_CONTENT_ENGINE_HEADER_NAME, multipartRequestEntity.getContentType());
        } catch (FileUploadException fileUploadException) {
            throw new ServletException(fileUploadException);
        }
    }

    /**
     * Sets up the given {@link PostMethod} to send the same standard POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleStandardPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) {
        // Get the client POST data as a Map
        Map<String, String[]> mapPostParameters = (Map<String, String[]>) httpServletRequest.getParameterMap();
        // Create a List to hold the NameValuePairs to be passed to the PostMethod
        List<NameValuePair> listNameValuePairs = new ArrayList<NameValuePair>();
        // Iterate the parameter names
        for (String stringParameterName : mapPostParameters.keySet()) {
            // Iterate the values for each parameter name
            String[] stringArrayParameterValues = mapPostParameters.get(stringParameterName);
            for (String stringParamterValue : stringArrayParameterValues) {
                // Create a NameValuePair and store in list
                NameValuePair nameValuePair = new NameValuePair(stringParameterName, stringParamterValue);
                listNameValuePairs.add(nameValuePair);
            }
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestBody(listNameValuePairs.toArray(new NameValuePair[]{}));
    }

    /**
     * Sets up the given {@link PostMethod} to send the same content POST
     * data (JSON, XML, etc.) as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    private void handleContentPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) throws IOException, ServletException {
        StringBuilder content = new StringBuilder();
        BufferedReader reader = httpServletRequest.getReader();
        for (;;) {
            String line = reader.readLine();
            if (line == null) break;
            content.append(line);
        }

        String contentType = httpServletRequest.getContentType();
        String postContent = content.toString();

        if (contentType.startsWith("text/x-gwt-rpc")) {
            String clientHost = httpServletRequest.getLocalName();
            if (clientHost.equals("127.0.0.1")) {
                clientHost = "localhost";
            }

            int clientPort = httpServletRequest.getLocalPort();
            String clientUrl = clientHost + ((clientPort != 80) ? ":" + clientPort : "");
            String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + intProxyPort : "") + httpServletRequest.getServletPath();
            //debug("Replacing client (" + clientUrl + ") with server (" + serverUrl + ")");
            postContent = postContent.replace(clientUrl , serverUrl);
        }

        String encoding = httpServletRequest.getCharacterEncoding();
        debug("POST Content Type: " + contentType + " Encoding: " + encoding,
              "Content: " + postContent);
        StringRequestEntity entity;
        try {
            entity = new StringRequestEntity(postContent, contentType, encoding);
        } catch (UnsupportedEncodingException e) {
            throw new ServletException(e);
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestEntity(entity);
    }

    /**
     * Executes the {@link HttpMethod} passed in and sends the proxy response
     * back to the client via the given {@link HttpServletResponse}
     * @param httpMethodProxyRequest An object representing the proxy request to be made
     * @param httpServletResponse An object by which we can send the proxied
     *                             response back to the client
     * @throws IOException Can be thrown by the {@link HttpClient}.executeMethod
     * @throws ServletException Can be thrown to indicate that another error has occurred
     */
    private void executeProxyRequest(
            HttpMethod httpMethodProxyRequest,
            HttpServletRequest httpServletRequest,
            HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a default HttpClient
        HttpClient httpClient = new HttpClient();
        httpMethodProxyRequest.setFollowRedirects(false);
        // Execute the request
        int intProxyResponseCode = httpClient.executeMethod(httpMethodProxyRequest);
        String response = httpMethodProxyRequest.getResponseBodyAsString();

        // Check if the proxy response is a redirect
        // The following code is adapted from org.tigris.noodle.filters.CheckForRedirect
        // Hooray for open source software
        if (intProxyResponseCode >= HttpServletResponse.SC_MULTIPLE_CHOICES /* 300 */ && intProxyResponseCode < HttpServletResponse.SC_NOT_MODIFIED /* 304 */) {
            String stringStatusCode = Integer.toString(intProxyResponseCode);
            String stringLocation = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
            if (stringLocation == null) {
                throw new ServletException("Received status code: " + stringStatusCode + " but no " + STRING_LOCATION_HEADER + " header was found in the response");
            }
            // Modify the redirect to go to this proxy servlet rather that the proxied host
            String stringMyHostName = httpServletRequest.getServerName();
            if (httpServletRequest.getServerPort() != 80) {
                stringMyHostName += ":" + httpServletRequest.getServerPort();
            }
            stringMyHostName += httpServletRequest.getContextPath();
            if (followRedirects) {
                if (stringLocation.contains("jsessionid")) {
                    Cookie cookie = new Cookie("JSESSIONID", stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
                    cookie.setPath("/");
                    httpServletResponse.addCookie(cookie);
                    //debug("redirecting: set jessionid (" + cookie.getValue() + ") cookie from URL");
                } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
	                Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
                    String[] cookieDetails = header.getValue().split(";");
					String[] nameValue = cookieDetails[0].split("=");

					Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
					cookie.setPath("/");
					//debug("redirecting: setting cookie: " + cookie.getName() + ":" + cookie.getValue() + " on " + cookie.getPath());
					httpServletResponse.addCookie(cookie);
                }
                httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() + this.getProxyPath(), stringMyHostName));
                return;
            }
        } else if (intProxyResponseCode == HttpServletResponse.SC_NOT_MODIFIED) {
            // 304 needs special handling.  See:
            // http://www.ics.uci.edu/pub/ietf/http/rfc1945.html#Code304
            // We get a 304 whenever passed an 'If-Modified-Since'
            // header and the data on disk has not changed; server
            // responds w/ a 304 saying I'm not going to send the
            // body because the file has not changed.
            httpServletResponse.setIntHeader(STRING_CONTENT_LENGTH_HEADER_NAME, 0);
            httpServletResponse.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
            return;
        }

        // Pass the response code back to the client
        httpServletResponse.setStatus(intProxyResponseCode);

        // Pass response headers back to the client
        Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
        for (Header header : headerArrayResponse) {
            if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
                    header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") || // don't copy gzip header
                    header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header so browser doesn't prompt on failed basic auth
                // proxy servlet does not support chunked encoding
            } else {
                httpServletResponse.setHeader(header.getName(), header.getValue());
            }
        }

        List<Header> responseHeaders = Arrays.asList(headerArrayResponse);

        if (isBodyParameterGzipped(responseHeaders)) {
            debug("GZipped: true");
            if (!followRedirects && intProxyResponseCode == HttpServletResponse.SC_MOVED_TEMPORARILY) {
                response = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
                httpServletResponse.setStatus(HttpServletResponse.SC_OK);
                intProxyResponseCode = HttpServletResponse.SC_OK;
                httpServletResponse.setHeader(STRING_LOCATION_HEADER, response);
            } else {
                response = new String(ungzip(httpMethodProxyRequest.getResponseBody()));
            }
            httpServletResponse.setContentLength(response.length());
        }

        // Send the content to the client
        debug("Received status code: " + intProxyResponseCode,
              "Response: " + response);

        httpServletResponse.getWriter().write(response);
    }


    /**
     * The response body will be assumed to be gzipped if the GZIP header has been set.
     *
     * @param responseHeaders of response headers
     * @return true if the body is gzipped
     */
    private boolean isBodyParameterGzipped(List<Header> responseHeaders) {
        for (Header header : responseHeaders) {
            if (header.getValue().equals("gzip")) {
                return true;
            }
        }
        return false;
    }

    /**
     * A highly performant ungzip implementation. Do not refactor this without taking new timings.
     * See ElementTest in ehcache for timings
     *
     * @param gzipped the gzipped content
     * @return an ungzipped byte[]
     * @throws java.io.IOException when something bad happens
     */
    private byte[] ungzip(final byte[] gzipped) throws IOException {
        final GZIPInputStream inputStream = new GZIPInputStream(new ByteArrayInputStream(gzipped));
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(gzipped.length);
        final byte[] buffer = new byte[FOUR_KB];
        int bytesRead = 0;
        while (bytesRead != -1) {
            bytesRead = inputStream.read(buffer, 0, FOUR_KB);
            if (bytesRead != -1) {
                byteArrayOutputStream.write(buffer, 0, bytesRead);
            }
        }
        byte[] ungzipped = byteArrayOutputStream.toByteArray();
        inputStream.close();
        byteArrayOutputStream.close();
        return ungzipped;
    }

    public String getServletInfo() {
        return "GWT Proxy Servlet";
    }

    /**
     * Retrieves all of the headers from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestHeaders(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an Enumeration of all of the header names sent by the client
        Enumeration enumerationOfHeaderNames = httpServletRequest.getHeaderNames();
        while (enumerationOfHeaderNames.hasMoreElements()) {
            String stringHeaderName = (String) enumerationOfHeaderNames.nextElement();
            if (stringHeaderName.equalsIgnoreCase(STRING_CONTENT_LENGTH_HEADER_NAME)) {
                continue;
            }
            // As per the Java Servlet API 2.5 documentation:
            //		Some headers, such as Accept-Language can be sent by clients
            //		as several headers each with a different value rather than
            //		sending the header as a comma separated list.
            // Thus, we get an Enumeration of the header values sent by the client
            Enumeration enumerationOfHeaderValues = httpServletRequest.getHeaders(stringHeaderName);
            while (enumerationOfHeaderValues.hasMoreElements()) {
                String stringHeaderValue = (String) enumerationOfHeaderValues.nextElement();
                // In case the proxy host is running multiple virtual servers,
                // rewrite the Host header to ensure that we get content from
                // the correct virtual server
                if (stringHeaderName.equalsIgnoreCase(STRING_HOST_HEADER_NAME)) {
                    stringHeaderValue = getProxyHostAndPort();
                }
                Header header = new Header(stringHeaderName, stringHeaderValue);
                // Set the same header on the proxy request
                httpMethodProxyRequest.setRequestHeader(header);
            }
        }
    }

    /**
     * Retrieves all of the cookies from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestCookies(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an array of all of all the cookies sent by the client
        Cookie[] cookies = httpServletRequest.getCookies();
        if (cookies == null) {
            return;
        }

        for (Cookie cookie : cookies) {
            cookie.setDomain(stringProxyHost);
            cookie.setPath(httpServletRequest.getServletPath());
            httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() + "=" + cookie.getValue() + "; Path=" + cookie.getPath());
        }
    }

    // Accessors
    private String getProxyURL(HttpServletRequest httpServletRequest) {
        // Set the protocol to HTTP
        String protocol = (isSecure) ? "https://" : "http://";
        String stringProxyURL = protocol + this.getProxyHostAndPort();

        // simply use whatever servlet path that was part of the request as opposed to getting a preset/configurable proxy path
        if (!removePrefix) {
            stringProxyURL += httpServletRequest.getServletPath();
        }
        stringProxyURL += "/";
        
        // Handle the path given to the servlet
        String pathInfo = httpServletRequest.getPathInfo();
        if (pathInfo != null && pathInfo.startsWith("/")) {
            if (stringProxyURL != null && stringProxyURL.endsWith("/")) {
                // avoid double '/'
                stringProxyURL += pathInfo.substring(1);
            }
        } else {
            stringProxyURL += httpServletRequest.getPathInfo();
        }
        // Handle the query string
        if (httpServletRequest.getQueryString() != null) {
            stringProxyURL += "?" + httpServletRequest.getQueryString();
        }
        
        return stringProxyURL;
    }

    private String getProxyHostAndPort() {
        if (this.getProxyPort() == 80) {
            return this.getProxyHost();
        } else {
            return this.getProxyHost() + ":" + this.getProxyPort();
        }
    }

    protected String getProxyHost() {
        return this.stringProxyHost;
    }

    protected void setProxyHost(String stringProxyHostNew) {
        this.stringProxyHost = stringProxyHostNew;
    }

    protected int getProxyPort() {
        return this.intProxyPort;
    }

    protected void setSecure(boolean secure) {
        this.isSecure = secure;
    }
    
    protected void setFollowRedirects(boolean followRedirects) {
        this.followRedirects = followRedirects;
    }

    protected void setProxyPort(int intProxyPortNew) {
        this.intProxyPort = intProxyPortNew;
    }

    protected String getProxyPath() {
        return this.stringProxyPath;
    }

    protected void setProxyPath(String stringProxyPathNew) {
        this.stringProxyPath = stringProxyPathNew;
    }

    protected void setRemovePrefix(boolean removePrefix) {
        this.removePrefix = removePrefix;
    }

    protected int getMaxFileUploadSize() {
        return this.intMaxFileUploadSize;
    }

    protected void setMaxFileUploadSize(int intMaxFileUploadSizeNew) {
        this.intMaxFileUploadSize = intMaxFileUploadSizeNew;
    }

    private void debug(String ... msg) {
        for (String m : msg) {
            System.out.println("[DEBUG] " + m);
        }
    }
}

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

Ajax Framework Analysis Results

2009-04-23T20:34:44-06:00

Way back in January, I wrote about how my colleagues and I were evaluating Ajax frameworks to build a SOFEA-style architecture. To make our choice, we used the following process:

Choose a short list of frameworks to prototype with.
Create an application prototype with each framework.
Document findings and create a matrix with important criteria.
Create presentation to summarize document.
Deliver document, presentation and recommendation.

When I wrote that entry, we had just finished step 2 and were starting step 3. I first wrote this blog post a week later, when we delivered step 5. Here is the comparison and conclusion sections of the analysis document we composed.

Framework Comparison
In order to evaluate the different frameworks against important criteria, we created a matrix with weights and ranks for each framework. This matrix shows how our weighting and rankings lead us to the winner for our project. You can view this matrix online or see below for a summary.

Note: Criteria whose values were identical across all candidates were weighted at zero. Charting capability was weighted at zero b/c we decided to use Flash for this.

This matrix indicates that GWT is the best candidate for our team to develop SOFEA-style applications with. In addition to the matrix, below are graphs that illustrate interesting (and possibly meaningless) statistics about each project.

Conclusion
After working with the various frameworks, we believe that all the frameworks were very good and could be used to write applications with. If all weights are equal, these frameworks were almost even when compared against our evaluation criteria. The graph below illustrates this.

Even after applying the weighted criteria, the evenness doesn't change a whole lot.

Without considering the even or weighted criteria, we believe the decision all comes down to what the developers on the project feel they will be most comfortable with. If you're developing with Dojo or YUI, chances are you're dressing up existing HTML and possibly using progressive enhancement to add more rich functionality. On the other hand, Ext JS and GWT are similar to Swing programming where you build the UI with code (JavaScript for Ext JS, Java for GWT).

The tools available for JavaScript development have gotten increasingly better in recent years. IntelliJ IDEA has a JavaScript Editor that provides many of the same features as its Java editor. Aptana Studio also has excellent support for authoring and debugging JavaScript. However, we believe the Java debugging and authoring support in IDEs is much better. Furthermore, we are more familiar with organizing code in Java projects and feel more comfortable in this development environment.

Based on this evaluation, we believe that GWT is the best framework for our team to develop SOFEA-style applications with.

Flash Forward to Today...
The core GWT library from Google doesn't have a whole lot of widgets, nor do they look good out-of-the-box. So early on, we experimented with two alternative implementations that continue to leverage GWT concepts and tools:

GXT: a GWT version of Ext JS
SmartGWT: a GWT version of SmartClient

Unfortunately, over the past few months, we've found that both of these implementations are too heavy for our requirements, mostly because of the file size of the generated JavaScript code. For example, a feature I wrote generated a 275K *.cache.html file using GXT. After determining that was too slow to give users the initial "pop", I re-wrote it without GXT. After a day, we had an application with *.cache.html files of 133K. Yes, that's over a 50% reduction in size!*

Because of these findings, we are proceeding with the core GWT library from Google and adding in new components as needed. It is cool to know you can make a UI "pop" with GWT, as long as you stick to the core - close-to-the-metal - components. For those applications that can afford an initial "loading..." state, I'd definitely recommend looking at GXT and SmartGWT.

* To make refactoring easier, I copied GXT MVC into our source tree and modified all imports.

My Drunk on Software Interview

2009-04-05T22:23:57-06:00

Back in February, I met up with James Ward and Jon Rose for a Drunk on Software interview. We enjoyed some good beer and had a great conversation about SOFEA, open source and RIA. See larger video here.

GWT and AppFuse

2009-03-04T22:50:26-07:00

Someone recently sent me the following e-mail asking about GWT integration in AppFuse.

I see from your blog that you're spending some time with GWT at the moment. What's your plan, are you going to integrate GWT as another UI Option for AppFuse?

The reason I'm asking is that I actually checked out all of the AppFuse code from the svn repository yesterday, with the intention of starting off adding some GWT stuff in there. My intention was to start by getting a basic Maven archetype together for GWT as an AppFuse UI.

However, if you're planning on doing this yourself in the near future, then there's no point in me starting doing it, I'd have to learn how to write archetype's for a start (not that it looks too difficult) but you'd obviously do it much quicker.

Being a good open-source developer, I moved the discussion to the developer mailing list and replied there:

It's likely I'll create a version of AppFuse Light with GWT, but I doubt I'll do it in the near future. I hope to release AppFuse 2.1 first (which will include "light" archetypes). I wouldn't get your hopes up in waiting for me to do the work. However, I'd be happy to assist you in doing it. AppFuse Light is now modular and uses the AppFuse backend.

http://raibledesigns.com/rd/entry/appfuse_light_converted_to_maven

Here's how I believe GWT should be integrated:

Create an appfuse-ws archetype that serves up RESTful services (http://issues.appfuse.org/browse/APF-897).
Create an appfuse-gwt archetype that consumes those services. This archetype would contain a proxy servlet that allows #1 to be on a separate host/port.

In addition to #1, I hope to convert the Struts 2 and Spring MVC archetypes to use those frameworks' REST support.

For #2, we could use SmartGWT or GXT. SmartGWT might be better since Sanjiv is a committer on this project.

I know I've been slacking on AppFuse development, but it is ski season and running to work seems to drain my late-night coding ambitions. With that being said, I'm committed to getting AppFuse 2.1 released by JavaOne (hopefully sooner). I figure it's a good week's worth of work and I'll probably have to do it late at night to find the time. That's OK though, I usually really start to enjoy it once I get into it.

Comparing Web Frameworks Book

2009-02-23T09:49:15-07:00

A publisher recently sent me an e-mail asking some advice. They received a proposal for a book that compares CakePHP, Symfony, Zend, TurboGears, Django, Struts, RoR. Here's a quote from the proposal:

We would like to compare a couple of frameworks and present their advantages and disadvantages in various applications.

Obviously, that kind of manual would be very useful for readers who are starting their 'adventures' with web applications, as it would facilitate their choosing the best framework for their particular application. The manuscript would offer a comparison of the most popular solutions (CakePHP, Symfony, Zend Framework, TurboGears, Django, Struts, Ruby on Rails) and demonstrate the main differences between each.

Therefore, the target audience would mainly be project managers, responsible for deciding on the technologies to be used for in-house projects, as well as less experienced, web application beginners.

Another purpose of the book would be to present 'good practices' in various frameworks, such as code re-factoring, design patterns and application security. From this point of view, it could become a valuable asset for experienced and learner programmers alike.

Since I got a lot of feedback from my tweet on this subject, I figured I'd ask it here.

What do you think of such a book?

Here's my response:

How do PHP books do these days? Of the list of frameworks (CakePHP, Symfony, Zend Framework, TurboGears, Django, Struts, Ruby on Rails), I think there's interest in Django and Rails, but not so much the others. And Struts sucks, so having that as a comparison is obviously going to make it look bad. I wouldn't buy it, but I'm a Java guy that's mostly interested in web frameworks that make developing SOFEA-based applications easier. In my mind, these are Flex and GWT.

The book I'd like to see would cover developing RESTful backends and SOFEA front-ends. RoR, Grails or Django could be used to develop the backend and Flex, GWT and X could be for the front-end. In reality, this is probably a tough book to write b/c things move so fast. If you decide to do it, I'd keep it short and sweet so you can get it to market and update it quickly.

What's the best Java Hosting Solution?

2009-02-07T10:21:28-07:00

A friend recently asked me who I'd recommend for a Java hosting provider. Since I get asked this question every-so-often, it seemed appropriate to post my answer here.

KGB Internet - I use KGB for this site. I have my own JVM and have full control over what I want to install. I can control Tomcat versions and upgrade as needed. I don't know if I'd recommend him for a business site as he can take up to 12 hours to respond to requests.
Kattare - These guys will give you your own Tomcat instance and seem to have reasonable prices. They do seem to take quite some time to respond to requests (24-48 hours). I have a free instance that I use for a non-profit, so that could be the reason.
Contegix - These guys are far-and-away the best company for Java-based hosting. They're not cheap though. However, they have the best customer service in the business - often responding to e-mails in less than a minute.

Do you agree with these recommendations? If not, who do you recommend for Java hosting and why?

Choosing an Ajax Framework

2009-01-08T21:36:22-07:00

This past week, my colleagues and I have been researching Ajax Frameworks. We're working on a project that's following SOFEA-style architecture principles and we want the best framework for our needs. I'm writing this post to see 1) if you, the community, agree with our selection process and 2) to learn about your experiences with the frameworks we're evaluating. Below is the process we're following to make our choice.

Choose a short list of frameworks to prototype with.
Create an application prototype with each framework.
Document findings and create a matrix with important criteria.
Create presentation to summarize document.
Deliver document, presentation (with demos) and recommendation.

For #1, we chose Ext JS, Dojo, YUI and GWT because we feel these Ajax libraries offer the most UI widgets. We also considered Prototype/Scriptaculous, jQuery and MooTools, but decided against them because of their lack of UI widgets.

For #2, we time-boxed ourselves to 3 days of development. In addition to basic functionality, we added several features (i.e. edit in place, drag and drop, calendar widgets, transitions, charts, grid) that might be used in the production application. We all were able to complete most of the functionality of the application. Of course, there's still some code cleanup as well as styling to make each app look good for the demo. The nice thing about doing this is we're able to look at each others code and see how the same thing is done in each framework. None of us are experts in any of the frameworks, so it's possible we could do things better. However, I think it's good we all started somewhat green because it shows what's possible for someone relatively new to the frameworks.

For #3, we're creating a document with the following outline:

Introduction

Ajax Framework Candidates
(intro and explanation)

  Project Information
  (history)
  (license / cost)
  (number of committers)
  (support options)
  (mailing list traffic (nov/dec 2008))

Matrix and Notes

Conclusion

For the Matrix referenced in the outline above, we're using a table with weights and ranks:

Weight	Criteria	Dojo	YUI	GWT	Ext JS	Notes
#	Important Criteria for Customer	0..1	0..1	0..1	0..1	Notes about rankings

Our strategy for filling in this matrix:

Customer adjusts the weight for each criteria (removing/adding as needed) so all weights add up to 1.
We rank each framework with 0, .5 or 1 where 0 = doesn't satisfy criteria, .5 = partially satisfies, 1 = satisfies.

The list of criteria provided to us by our client is as follows (in no particular order).

Quality of Documentation/Tutorials/Self Help
Browser support (most important browsers/versions based on web stats)
Testability (esp. Selenium compatibility)
Licensing
Project health/adoption
Performance
Scalability
Flexibility/extensibility
Productivity (app dev, web dev)
Richness of widget/component library
Charting capability
Ability to create new widgets
Match to existing Java team skill-set
Ease of deployment (on Ops, QA, Users)
Degree of risk generally
Ability to integrate with existing site (which includes Prototype)
Easy to style with CSS
Validation (esp. marking form elements invalid)
Component Theme-ing/Decoration
CDN Availability (i.e. Google's Ajax Libraries API or Ext CDN)

What do you think? How could this process be improved? Of course, if you have framework answers (0, .5 or 1) for our matrix, we'd love to hear your opinions.

Dojo/Comet support in Java Web Frameworks

2008-12-18T15:58:37-07:00

This week I'm doing a research project for a client. The main purpose of the project is to find out which Java-based web framework works best with Dojo and Comet. Here's the key requirement from the client:

It's all about Comet, we want Comet everywhere we can put it, but we want to isolate the icky bits of fiddling with pages with JavaScript. We're kind of wed to the Dojo implementation of the client-side bit, so we may as well use more of the Dojo widgets for a richer UI. For us, "works best with" needs to pay a certain amount of consideration to "fits naturally with", if you understand what I mean. I know that any framework that lets you spit out raw HTML will let you hand code in your Dojo / Comet, but that's certain to become very tiresome very quickly.

The candidate frameworks they asked me to look at are Wicket and Tapestry 5. They're willing to upgrade to Struts 2 since they're already using Struts 1. However, they don't feel that action-based frameworks naturally lead to rich UIs, so they'd prefer a component-based framework. They're currently using Seam for an administration-type application and feel it's too heavy for their customer-facing application.

Here's what I've found so far in my research. Please let me know if anything is incorrect.

Tapestry 5 doesn't have Dojo or Comet support (Prototype and Scriptaculous are the baked-in Ajax frameworks).
Struts 2 has old (version 0.4.3) and somewhat deprecated Dojo support. The developers seem to be in favor of removing it and promoting people hand-code Dojo instead. Struts 2 doesn't have support for Comet.
Wicket has support for Dojo 1.1 that includes Comet support. This was written by Stefan Fußenegger and posted to the mailing list last month. I e-mailed Stefan and asked him about documentation. His response: "I lost my ambition to document it properly since I didn't receive any feedback on the mailing list. :)"

At this point, it seems that if the client really wants to use Dojo, they should use Wicket, and possibly pay Stefan to document it properly. However, they're willing to consider other options, as long as they have Comet support.

One option I thought of is to use DWR and its Reverse Ajax/Comet support. Another option would be to add better Dojo support to Tapestry 5. However, I don't think this is possible since the Prototype/Scriptaculous code is generated by the framework and would likely require a changes to switch it to Dojo.

Are there any other Java-based web frameworks that support easily creating Dojo widgets and working with Comet? Keith Donald tweeted that Spring MVC has Dojo support. However, I believe it's only for widgets and it still requires you to write JavaScript. If your framework doesn't have Dojo/Comet support, how hard would it be to add it?

Update: I also posted this question on LinkedIn. Make sure and check my question for additional thoughts from folks.

RESTful Web Applications with Subbu Allamaraju

2008-10-24T09:52:02-06:00

Subbu works at Yahoo! developing standards, patterns and practices for HTTP web languages. In the past, he was a web service and Java developer. He was also a standards contributor at BEA and an author of books on Java EE. His current passion is HTTP and REST. Subbu confesses that he's not a web developer, has no interest in the internals of programming models used for web frameworks and he's only interested in the visible aspects of the architecture.

"The Web is Mostly Restful"

Being RESTful in an abstract sense means:

Resources are named by URIs
Resources have representations (Atom, HTML, JSON, XML)
Resources contain contextual links to allow navigation of state
There's a Uniform Interface

In the web today, some resources and URIs are personalized, but most are not. Some depend on sessions, but most do not. The consequence of a personalized UI with a non-unique URI is you cannot rely on browser caching.

The web is full of different representations (HTML, XML, JS, PDF, CSS, Flash). The problem with HTML is you can't tell links that you want a particular representation based of a link. The links are hard-coded to be a particular content-type. However, some media types on responses are ignored. This is often a problem with browsers and whether the user has plugins installed.

The Uniform Interface for the web is HTML and primarily links and forms (GET and POST). There's still some misconceptions (e.g. POST is secure). However, it's not about security, it's about idempotency and safety. You need to make sure you only use POST when you're changing data. POSTs are not repeatable. GET URIs are not always refreshable, which is quite unfortunate. Users shouldn't have to fight the back button.

Caching is a fundamental aspect of the web. Even in a personalized site, most of the content can be cached. The web is read-only for the most part. However, many enterprise web applications don't take advantage of caching. This is fine when there's not that many users, but it's bad when you want to scale to thousands of users. There's several frameworks that use cache-busting and prefer backend caching over HTTP caching. These frameworks are not using the web like they should.

Backend caching (e.g. Memcached) uses a non-uniform interface and you need to explicitly program to it. Frontend/HTTP caching has a uniform interface that's pluggable. Backend caching is generally more expensive to develop and deploy. There are cases where data should be cached on the backend, but you shouldn't focus all on backend caching w/o doing some frontend caching.

With Ajax, you get more opportunities to be RESTful. XMLHttpRequest is another HTTP client that can be programmed to. It has full support for the uniform interface, which allows content negotiation, optimistic concurrency and caching. Cross-domain hacks can be done with <script> and <iframe> to tunnel requests over GET. The W3C has been working for the last two years on how to do cross-domain Ajax w/o using hacks. The problem with current cross-domain implementations is they often use GET for everything, which isn't very RESTful. Subbu recommends using a proxy on the same domain if you do need to talk to other domains. This will allow your Ajax code to remain RESTful.

Web Frameworks
Web development is hard because of all the moving pieces that exist. Because of this, many web frameworks have been created to solve the various problems. In 1997, there were servlets. They provided basic plumbing and closely reflected HTTP/1.1. Servlets provided a poor programming model, but it allowed a lot of frameworks to be built on top of it. We don't use servlets to write applications, only to write application frameworks. The second era came about in 2001 when Action-oriented frameworks became popular. In 2004, JSF and friends came to play. JSF is a component-based framework with known limitations (complex, slow, uses POST for almost everything, Ajax is difficult). These limitations have resulted in a number of third-party patches being developed to solve these issues.

JSF was designed to use the request to create a component tree that maintains state. Unfortunately, the state is not something the developer has control over. It's not the state of the application, it's the state of the components. The client's knowledge of the state is mentioned with a cookie and the server keeps the state in the session. The problem with JSF is you don't have a choice of state in your application - you can't write stateless applications like you can with servlets.

JSF uses overloaded URIs for its resources. When you have one URI with multiple representations, there's no way to tell how a representation was chosen. JSF's compromise is to allow client-side state saving. However, they do this by putting hidden field in the form and requiring POST for navigation.

JSF vs. REST
Basically, these two are at opposite extremes. JSF is focused heavily on a UI component model. The people that developed it misinterpreted how the web works and made some fundamental questionable choices. You can patch it, but you can not fix it.

Web 2.0 Frameworks
GWT is a cross-compilation based framework. You write Java to generate JavaScript (b/c everyone hates writing JavaScript). It mashes client and server code into a single source. These layers communicate using GWT-RPC. Typical RPC concerns do not apply since code generation handles coupling and the client is downloaded from the same application. GWT-PRC does POSTs to the server and uses HTTP like a transport layer. To be fair, GWT does allow you to use a RequestBuilder to use the web like it should be used. This class allows more control over HTTP requests, it supports GET and POST and it allows so-called RESTful layers (GWT-REST and GET-Restlet). GWT is focused heavily on ease-of-use, which is good. It's modeled after RPC and breaks the uniform interface and focuses on backend caching. Unlike JSF, GWT is fixable, but the community tends to use RPC instead of RequestBuilder.

SOFEA has a central promise of SOA. Business logic is a reusable service that changes less often. The presentation application calls those services and changes more often. The nice thing about this type of architecture is it allows a separation of concerns and loose coupling. However, it doesn't embrace REST like it should. Appcelerator is an implementation of SOFEA that has a Ruby on Rails-like usability. However, it uses a SOAP/HTTP style with messaging and POSTs to a single URI. Appcelerator is interesting, but it introduces a different style of coupling. It breaks URI opacity and client deals with POX instead of links.

Conclusion
Don't fight the architecture of the web. Innovate and enhance instead of breaking. If nothing else, break judiciously. As developers, we should demand more from our frameworks and make sure they use the web and HTTP like it should be used.

Xebia RIA Framework Contest

2008-10-07T08:06:48-06:00

Last year, I blogged about the Xebia Web Framework Contest where Struts 2, GWT, Wicket and MyFaces (JSF) were all used to develop the same applications. It seems they've done it again this year, comparing RIA frameworks (English translation) this time.

Five teams were formed, five frameworks have been selected:

Flex 3 Flex 3
Silverlight 2.0 beta 2 Silverlight 2.0 beta 2
Google GWT 1.5 Google GWT 1.5
Echo3 Echo3
JavaFX Preview SDK JavaFX SDK Preview

The number of teams is limited, the list of frameworks is also selected, and we have therefore necessarily set aside some frameworks that would have certainly had their place in the contest. Examples Ext JS, Yahoo! UI, Curl, XUL, ZK or OpenLaszlo. If you have experience on one of these frameworks, feel free to share in the comments on this article! [Read More »]

The winner? Flex - which doesn't surprise me a whole lot.

After this day, the Flex team, has clearly demarcated from its competitors. After two sprints of development and ownership, it could devote the third and final sprint to get rich quick and easy application, focusing only on the features and user experience. It is thanks to the wealth of high-level components, the wealth of documentation available and a maturity framework that the team has made the Flex application's most successful.

Good stuff - thanks Xebia!

Ajaxified Body

2008-10-03T14:33:09-06:00

I've often wondered if it was possible to use Ajax to reload the main content of a web application without reloading the header, menu and footer. SiteMesh allows you to move these common elements to a decorator that gets wrapped around each page. Below is a diagram of how SiteMesh works.

You can read the Introduction to SiteMesh article if you're interested in learning more about SiteMesh's basic features. By default, SiteMesh decorates text/html responses and ignores any other content type (e.g. image/gif). It also contains an <excludes> configuration element that allows you to turn off decoration when a URL matches a certain pattern. For example, the following allows you to disable decoration when "ajax=true" is passed in the URL.

<excludes>
    <pattern>**ajax=true</pattern>
</excludes>

To optimize the loading of an application so the common elements aren't loaded for each page, it should be possible to create an Ajaxified Body where the primary content area (PCA) of the site is loaded via Ajax rather than refreshing the whole page. The header, footer and navigation bar often stays the same from page-to-page, so it doesn't seem to make a whole lot of sense to load them each time the page loads. The image below shows the PCA (of an AppFuse Light application) as a grey square with a red border.

Implementing an Ajaxified Body consists of the following steps:

Adding SiteMesh and moving common elements to a decorator.
Remove common elements from each individual page (if you're using includes).
Configure SiteMesh so decoration is disabled when the requested URL contains "ajax=true".
Write JavaScript that modifies all <a href=""> links (and buttons with onclick='location.href') in the PCA to have an onclick handler.
The onclick handler should call a JavaScript function that loads the link's URL + ajax=true using XMLHttpRequest (XHR).
Add XHR success handling to replace the PCA with the loaded content.
Add XHR error handling to go to the URL normally when response.status != 200.
Inspect the response HTML for <title> element and replace document.title if exists.
Inspect the response HTML for <head> element and append to current if exists.
Inspect the response HTML for <script> and <link> elements (JavaScript and CSS) and evaluate them if they exist.

As a proof of concept, I created a prototype using AppFuse Light (Prototype/Scriptaculous for Ajax). You can see a demo at the following URL. You can also download a patch or the source for this project.

http://demo.raibledesigns.com/ajaxifiedbody

Below are a number of things I discovered while writing this prototype:

The hardest part of implementing this seems to be coding the exceptions. It's possible you'll have some links with existing onclick handlers and you may have to disable "ajaxifying links" for those links.
A progress indicator is important or the page might load so fast that the user doesn't visually detect it changed. This can lead to a worse user experience because they don't see the flash of the blank page they're used to when a page load occurs.
While forms can be submitted via Ajax, there's no harm in leaving existing form behavior in place where the full site is reloaded after submitting a form.
If a particular page needs to change the common elements (header, menu, footer), it should be possible to do that with JavaScript after the PCA content loads.
If the success/error indicator is outside the PCA, it may need to be populated and displayed/hidden with JavaScript after the PCA loads.

I'm sure my implementation can be improved, but I'm also curious to see what you think of this idea. I know it's not revolutionary, but it's something I'm considering adding by default to AppFuse and AppFuse Light. Do any Ajax frameworks do something like this out-of-the-box?

Update: Thanks to everyone for the great feedback - keep it coming. I agree that adding history support is a must. I'll try to do that in the next day or two. This post has also been featured on Javalobby and Ajaxian.

Update 2: Added history support.

What's wrong with JSF

2008-08-25T18:53:31-06:00

The developers of Seam have come up with a list of major issues with JSF. I'm assuming many of these issues are fixed by Seam, but it's interesting to note how they've somewhat admitted that JSF has many flaws. Note that there's a lot of references to Struts and WebWork.

Hopefully many of these will be fixed in JSF 2. If REST support is an important feature for web frameworks, it'll be be interesting to see how the component frameworks handle it. It'd be great if they provided native support. Oh wait, then they'd be action-based frameworks.

GWT and REST

2008-07-21T10:31:58-06:00

I've posted two message to the GWT Google Group in the last couple of days. However, new member messages are moderated and neither has shown up yet. I'm reposting my questions here in hopes of getting some answers.

Is there a way to easily use a REST backend with GWT? I tried GWT-REST. It works, but it seems to be centered towards Rails (I'm using Grails) and it suffers from the SOP issue.

JSONRequest looks promising for cross-domain support, but I can't get it to work either. The provided examples work, but not my simple hello world that returns:

{"response":"Hello World!"}

Also, the example implementation only has GET support, not PUT, DELETE or POST. I can post my REST backend on the public internet if anyone is interested in seeing the issues I'm having.

Thanks in advance for any advice.

RE: Which is the Hottest Java Web Framework?

2008-06-10T22:39:08-06:00

The "Break it Down" Blog has a lengthy post on Which is the Hottest Java Web Framework? Or Maybe Not Java? Comparing Java Web Frameworks is hard because so many people are passionate about the framework they know best. Add a couple more like Flex and Ruby on Rails and its downright difficult. Nevertheless, this post is good in that it contains a lot of pretty trend graphs and it looks like the author has done some good research. It's likely the folks that will scream foul are the ones that did poor in the comparison (Tapestry and Stripes, I'm talking about you).

Surprising among the top Java Web Frameworks is the rise of Struts 2:

To quote:

Which is much more interesting I think is how Wicket adoption has stayed almost flat while Struts 2 adoption has spiked. Spring MVC/WebFlow seems to be going no where fast and racing JBoss Seam there.

The popularity of Struts 2 really caught me off guard with it being quite a bit different from Struts 1, I figured it got thrown into the "just another web framework" category, but I guess there is something in a name and it's doing quite well.

Regardless of what you think of the post and trends, you have to appreciate the amount of time the author put into it.

Should we change AppFuse to be Struts 2-specific?

2008-05-29T08:29:44-06:00

Dusty recently posted an interesting idea to the AppFuse developers mailing list:

After thinking/coding/reading for a while I think the more interesting task is: Retool AppFuse to be one or more Struts2 plugins based on various higher level app patterns. (AppFuse Facebook, AppFuse Employee DB, AppFuse Blog, AppFuse Basic LDAP, AppFuse Basic Crowd).

This all comes from the fact, that I have been wanting to refactor the AppFuse web layer for Struts. One of the interesting aspects of AppFuse is that it works pretty much the same across all its web frameworks. It does so with some lowest common denominator abstractions that can be ported and look and work the same across frameworks. I have picked my tool(s): Struts 2 and Ruby On Rails when I want to pretend I am young again. I know Spring MVC, JSF, etc. but I have no desire to build significant apps on those platforms. It's not because they suck and Struts2 rules, it is because I know Struts 2 the best, I am most efficient there and it provides everything I need to build great webapps (Let's not devolve to a framework debate). So, I would rather have a more Struts 2-specific web stack, that really leverages conventions born and raised there. The nice thing about the Struts 2 web stack is that it is complemented nicely by AppFuse's data/service layer, since unlike Grails or Rails, Struts 2 has no data or service layer. [Read More »]

Seems like a good idea to me. What do you think?

Someday I'd like to come up with a "compatibility test" that allows others to improve upon the ideas in AppFuse and develop their stacks independently. A suite of Selenium tests that require extensionless URLs might be a good start.