Matt RaibleMatt Raible is a writer with a passion for software. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.
You searched this site for "matt". 1,142 entries found.

You can also try this same search on Google.

Consulting, SOFEA, Grails and GWT at next week's Denver JUG

Next Wednesday, I'll be at Denver's JUG meeting to talk about Independent Consulting and Building SOFEA Applications with Grails and GWT. The first talk will be a a panel discussion among local independent consultants, including James Goodwill, Matthew McCullough, Tim Berglund and myself.

This session explores the trials and tribulations of an independent consultant. How do you find contracts? Should you setup an LLC, an S-Corp or just be a sole proprietorship? What about health insurance and benefits? Are recruiters helpful or hurtful? Learn lots of tips and tricks to get your dream job and your ideal lifestyle.

The Grails and GWT talk is a preview of a talk I'll be doing at the Rich Web Experience in December. Below is a rewrite of the abstract in first-person.

Earlier this year, I participated in a major enhancement of a high-traffic well-known internet site. The company wanted us to quickly re-architect their site and use a modern Ajax framework to do it with. An Ajax Framework evaluation was done to help the team choose the best framework for their skillset. The application was built with a SOFEA architecture using GWT on the frontend and Grails/REST on the backend.

This talk will cover how Bryan Noll, Scott Nicholls, James Goodwill and I came to choose GWT and Grails, as well as stumbling blocks we encountered along the way. In addition, we'll explore many topics such as raw GWT vs. GXT/SmartGWT, the Maven GWT Plugin, modularizing your code, multiple EntryPoints, MVP, integration testing and JSON parsing with Overlay Types.

If you're in Denver next Wednesday night (November 11th), you should stop by the Denver JUG meeting. It'll be a fun night and there's sure to be a few beers afterward. ;-)

Posted in Java at Nov 05 2009, 10:52:37 PM MST 5 Comments

A Letter to the AppFuse Community

The last AppFuse release was way back in May 2008. Many folks have asked when the next release would be ever since. Often, I've said "sometimes this quarter", but obviously, that's never happened. For that, I apologize.

There are many reasons I haven't worked on AppFuse for the past 18 months, but it mostly comes down to the fact that I didn't make time for it. The good news is I'm working on it again and will have a release out sometime this month. Unfortunately, it probably won't be a 2.1 final release, but there's so many things that've changed, I feel like a milestone release is a good idea. Here's a brief summary of changes so far:

  • Changed archetypes to include all source and tests for the "webapp" portion of the application. No more warpath plugin, merging wars and IDE issues. Using "mvn jetty:run" should work as expected.
  • Moved from Spring XML to Annotations.
  • AppFuse Light converted to Maven modules and now depends on AppFuse's backend.
  • Published easier to use archetype selection form in the QuickStart Guide.
  • Published archetype selection form for AppFuse Light. I do plan on combining these forms as soon as I figure out the best UI and instructions for users to choose AppFuse or AppFuse Light.
  • Upgraded all libraries to latest released versions (Spring 3 hasn't had a final release yet).
  • Upgraded to Tapestry 5 thanks to Serge Eby. I still need to complete tests and code generation for tests.
  • Added Compass support thanks to a patch from Shay Banon.
  • Upgraded from XFire to CXF for Web Services.
  • Moved Maven repository to Sonatype's OSS Repository Hosting for snapshots and releasing to Maven Central. There are no longer any AppFuse-specific artifacts, all are available in central.

I realize there's many full-stack frameworks that do the same thing as AppFuse with less code. Examples include Ruby on Rails, Grails, Seam, Spring Roo and the Play framework. However, there seems to be quite a few folks that continue to use AppFuse and it stills serves the community as a nice example of how to integrate frameworks. Furthermore, it helps me keep up with the latest framework releases, their quirks and issues that happen when you try to integrate them. In short, working on it helps me stay up to speed with Java open source frameworks.

For those folks that like the 1.x, Ant-based version of AppFuse, there will not be a 1.9.5 release. I know I promised it for years, but it's simply something I will not use, so I'd rather not invest my time in it. I'm sorry for lying to those that expected it.

So what's the future of AppFuse? Will it continue to integrate web frameworks with Spring and popular persistence frameworks? Possibly, but it seems more logical to align it with the types of Ajax + REST applications I'm creating these days. I'm currently thinking AppFuse 3.0 would be nice as a RESTful backend with GWT and Flex UIs. I might create the backend with CXF, but it's possible I'd use one of the frameworks mentioned above and simply leverage it to create the default features AppFuse users have come to expect.

More than anything, I'm writing this letter to let you know that the AppFuse project is not dead and you can expect a release in the near future.

Thanks for your support,

Matt

Posted in Java at Nov 04 2009, 12:17:17 AM MST 44 Comments

What would you like to see at TSSJS 2010?

The Venetian A couple months ago, I was asked by TheServerSide to speak at next year's TheServerSide Java Symposium in Las Vegas. In addition, they asked me to help them evaluate presentation proposals and suggest topics/speakers.

First of all, I think the biggest thing that TSSJS could do to improve is to host more networking events. With the JavaOne Party being over, I think there's a tremendous opportunity to fill a gap in the networking needs of the Java Community. When I first attended TSSJS in 2006, there were a fair amount of parties and everyone got to interact quite a bit. In 2008, there were no networking events. I believe having a strong networking story would attract a lot more attendees, companies and sponsors.

Secondly, I think it's possible that TSSJS has too many server-side related sessions. IMO, the server-side (and middleware in general) isn't that exciting. TechTarget appears to own TheClientSide, so why not add some more client-side stuff to the mix? For example, I'd love to see a Struts 1 app-makeover using different technologies (for example, Flex, GWT and jQuery). I think HTML5 and Google Wave's Architecture sessions would be interesting too. If adding client-side sessions is too far away from TheServerSide, maybe it should be renamed to TheServerSide JVM Symposium and there can be all kinds of sessions on JVM languages (e.g. Scala, JRuby, Groovy) and all the great things those languages can accomplish.

Lastly, I've been asked to send a couple session proposals. Currently, I'm thinking about a doing GWT vs. Flex Smackdown with James Ward, but I'm open to other ideas. It's been quite awhile since I did a "Comparing Web Frameworks" talk. Maybe "Hot Web Frameworks for 2010" is more appropriate? I also think it'd be interesting to do a somewhat philosophical talk on "The State of Web Frameworks" and where we're headed in the next year.

What would make you want to attend TSSJS next year? Let me know your thoughts and I'll do my best to make them a reality.

Update October 22, 2009: Whoo hoo! It looks like TheClientSide will be a part of TSSJS Vegas next year. Should be a great show.

Posted in Java at Oct 12 2009, 11:28:21 AM MDT 3 Comments

Packaging a SOFEA Application for Distribution

The project I'm working on is a bit different from those I'm used to. I'm used to working on web applications that are hosted on servers and customers access with their browser. SaaS if you will. My current client is different. They're a product company that sells applications and distributes them to customers via download and CD. Their customers install these applications on internal servers (supported servers include WebSphere, WebLogic and Tomcat).

The product I'm currently working on is structured as a SOFEA application and therefore consists of two separate modules - a backend and a frontend. Since it's installed in a servlet container, both modules are WARs and can be installed separately.

Building the backend and frontend as separate projects makes a lot of sense for two reasons:

  • In development, different teams can work on the frontend and backend projects.
  • Having them as separate projects allows them to be versioned separately.

However, having them as two separate projects does make it a bit more difficult for distribution. I'm writing this post to show you how I recently added support for distributing our application as 2 WARs or 1 WAR using the power of Maven, war overlays and the UrlRewriteFilter.

Project Setup
First of all, we have several different Maven modules, but the most important ones are as follows:

  • product-services
  • product-client
  • product-integration-tests

Of course, our modules aren't really named "product", but you get the point. The services project is really just a WAR project with Spring Security configured. It depends on other JAR modules that the services exist in. The client project is a GWT WAR that has a proxy servlet defined in its web.xml that makes it easier to develop. It also contains some UrlRewrite configuration that allows GWT Log's Remote Logging feature to work. The proxy servlet is something we don't want to ship with our product, so we have a separate web.xml for production vs. development. We do the substitution using the maven-war-plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.0.2</version>
    <configuration>
        <!-- Production web.xml -->
        <webXml>src/main/resources/web.xml</webXml>
        <warSourceDirectory>war</warSourceDirectory>
        <!-- Exclude everything but urlrewrite JAR -->
        <warSourceExcludes>
            WEB-INF/lib/aop**,WEB-INF/lib/commons-**,WEB-INF/lib/gin-**,
            WEB-INF/lib/guice-**,WEB-INF/lib/gwt-**,WEB-INF/lib/gxt-**,
            WEB-INF/lib/junit-**
        </warSourceExcludes>
    </configuration>
</plugin>

I could exclude WEB-INF/lib/** and WEB-INF/classes/**, but in my particular project, we still want UrlRewrite in standalone mode, and we have some i18n properties files in WEB-INF/classes that are served up for Selenium tests.

With this configuration, we have a services WAR and a client WAR that can be installed and used by clients. To collapse them into one and make it possible to ship a single war, I turned to our product-integration-tests module. This module contains Selenium tests that test both types of distributions.

Merging 2 WARs into 1
The most important thing in the product-integration-tests module is that it creates a single WAR. First of all, it uses <packaging>war</packaging> to make this possible. The rest is done using the following 3 steps.

1. Its dependencies include the client and servlet WARs (and Selenium RC for testing).

<dependencies>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-services</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-client</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium.client-drivers</groupId>
        <artifactId>selenium-java-client-driver</artifactId>
        <version>1.0.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

2. The WAR created excludes the "integration-tests" part of the name:

<build>
    <finalName>product-${project.version}</finalName>
    ...
</build>

3. WAR overlays are configured so the everything in the client's WEB-INF directory is excluded from the merged WAR.

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <!-- http://maven.apache.org/plugins/maven-war-plugin/overlays.html -->
        <overlays>
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-services</artifactId>
                <excludes>
                    <!-- TODO: Rename to api.html (this is the Enunciate-generated documentation) -->
                    <exclude>index.html</exclude>
                </excludes>
            </overlay>
            <!-- No server needed in product-client -->
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-client</artifactId>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
            <!-- Only include META-INF/context.xml to set the ROOT path -->
            <overlay>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
        </overlays>
    </configuration>
</plugin>

That's it! Using this configuration, it's possible to distribute a Maven-based SOFEA project as single or multiple WARs. However, there are some nuances.

One thing you might notice is the reference to META-INF/context.xml in the overlays configuration. This subtly highlights one issue I experienced when merging the WARs. In our GWT client, we're using URLs that point to our services at /product-services/*. This works in development (via a proxy servlet) and when the WARs are installed separately - as long as the services WAR is installed at /product-services. However, when they're merged, a little URL rewriting needs to happen. To do this, I added the UrlRewriteFilter to the product-services module and configured a simple rule.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCENGINE urlrewrite PUBLIC "-//tuckey.org//DTD UrlRewrite 3.0//EN"
        "http://tuckey.org/res/dtds/urlrewrite3.0.dtd">

<urlrewrite use-query-string="true">
    <!-- Used when services are merged into WAR with GWT client -->
    <rule>
        <from>^/product-services/(.*)$</from>
        <to type="forward">/$1</to>
    </rule>
</urlrewrite>

Because the services URLs point to the root (/product-services), the merged WAR has to be installed as the ROOT application. When you're using Cargo with Tomcat and want to deploy to ROOT, you have to have a META-INF/context.xml with a path="" reference (ref: CARGO-516).

<Context path=""/>

It is possible to change the URLs in the client to be relative, but this gets seems to get messy when you're using separate WARs. When using relative URLs, I found I had to do solution using cross-context forwarding to get the results I wanted. Using a redirect instead of a forward worked, but resulted in the client talking to the server twice (once to get redirected, a second time for the actual call). Cross-context forwarding is supported by the UrlRewriteFilter and Tomcat, but I'm not sure WebSphere or WebLogic support it. The best solution is probably to change the URLs dynamically at runtime, possibly using some sort of deferred binding technique.

Testing with Cargo and Selenium
Once I had everything merged, I wanted to configure Cargo and Selenium to allow testing both distribution types. If I installed all 3 wars at the same time, the "product-services" WAR would be used by both the product-client.war and the product.war, so I had to use profiles to allow installing the single merged WAR or both WARs. Below is the profile I used for starting Cargo, deploying the merged WAR, starting Selenium RC and running Selenium tests.

<properties>
    <cargo.container>tomcat6x</cargo.container>
    <cargo.container.url>
        http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.zip
    </cargo.container.url>
    <cargo.host>localhost</cargo.host>
    <cargo.port>23433</cargo.port>
    <cargo.wait>false</cargo.wait>
    <cargo.version>1.0</cargo.version>

    <!-- *safari and *iexplore are additional options -->
    <selenium.browser>*firefox</selenium.browser>
</properties>
...
<profile>
    <id>itest-bamboo</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <wait>${cargo.wait}</wait>
                    <container>
                        <containerId>${cargo.container}</containerId>
                        <log>${project.build.directory}/${cargo.container}/cargo.log</log>
                        <zipUrlInstaller>
                            <url>${cargo.container.url}</url>
                            <installDir>${installDir}</installDir>
                        </zipUrlInstaller>
                    </container>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <!-- Deploy as ROOT since XHR requests are made to /product-services -->
                        <deployables>
                            <deployable>
                                <properties>
                                    <context>ROOT</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
                <executions>
                    <execution>
                        <id>start-container</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-container</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>selenium-maven-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start-server</goal>
                        </goals>
                        <configuration>
                            <background>true</background>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <excludes>
                                <exclude>none</exclude>
                            </excludes>
                            <includes>
                                <include>**/*SeleniumTest.java</include>
                            </includes>
                            <systemProperties>
                                <property>
                                    <name>selenium.browser</name>
                                    <value>${selenium.browser}</value>
                                </property>
                                <property>
                                    <name>cargo.port</name>
                                    <value>${cargo.port}</value>
                                </property>
                            </systemProperties>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

This profile is run by our Bamboo nightly tests with mvn install -Pitest-bamboo. The 2nd profile I added doesn't install the project's WAR, but instead installs the two separate WARs. Running mvn install -Pitest-bamboo,multiple-wars executes the Selenium tests against the multi-WAR distribution.

<profile>
    <id>multiple-wars</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <deployables>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-client</artifactId>
                                <pingURL>http://${cargo.host}:${cargo.port}/product-client/index.html</pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-client</context>
                                </properties>
                            </deployable>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-services</artifactId>
                                <pingURL>
                                    http://${cargo.host}:${cargo.port}/project-services/index.jspx
                                </pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-services</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

I won't be including any information on authoring Selenium tests because there's already many good references. I encourage you to checkout the following if you're looking for Selenium testing techniques.

Summary
This article has shown you how I used Maven, war overlays and the UrlRewriteFilter to allow create different distributions of a SOFEA application. I'm still not sure which packaging (1 WAR vs. 2) mechanism is best, but it's nice to know there's options. If you package and distribute SOFEA applications, I'd love to hear about your experience in this area.

Posted in Java at Oct 06 2009, 01:17:38 AM MDT 2 Comments

Lean Teams: Doing more with less

This evening I attended the Denver Rails User Group (a.k.a. DeRailed) to hear a presentation by Marty Haught. It was titled "Lean Teams: Doing more with less" and the following are my notes from the event.

Today's talk is about "Rocking with Ramen" - a.k.a. working with less funds to make great things. Lean comes from the manufacturing world in that you should Add Nothing but Value. The most important thing you should do is add business value.

The Seven Wasteful Sins for manufacturing are:

  • Overproduction
  • Inventory
  • Extra Processing Steps
  • Motion
  • Defects
  • Waiting
  • Transportation

The key to fighting overproduction in software is to trim features to those that achieve the greatest value. You should do "the simplest thing that could possibly work" and delay commitment as long as you can because YAGNI.

A minimum viable product is a starting place for validated learning with the least amount of effort. It should be embarrassing. Early adopters see the potential. Rails Rumble and Startup Weekend are good examples of promoting this type of development.

Unused and useless features are best solved by feedback-driven development. This is a process for validating value and creating software that people use. The end result is that you create software that people use and you're able to pivot your plan as you learn. The benefit of this is you stay humble and you don't drink the Kool-Aid (e.g. VC's tell you you're going to be the next Twitter).

The first part of feedback is "Pirate Metrics" by Dave McClure. The main things to track are acquisition, activation, retention, referral and revenue (AARRRR!). The main things you should gather from metrics is they're actionable and should help you make decisions. Vanity metrics like hits-per-month and such should be ignored.

Other feedback options include net promoter score (popup question to ask if users would recommend to a friend), feedback form (make it easy for users to tell you what you think about your product), A/B testing, and usability testing.

The final point is that it's OK to remove features.

To reduce extra processing and waiting, you should implement "Kanban". It's a pull-based system for a continuous flow of work and can be used in software projects to manage/schedule work for cross functional teams. It's an expression of just-in-time and has an emphasis on flow. It's all about getting across the board as fast as possible. In agile development, this is often expressed as a card-based system on a wall in the same room as your development team. Things can only move from the left-to-right as there is space for them. Marty is showing a screenshot of a "Zen" tool he uses on his projects. It has 3 columns (Definition, Work and Verification) from left-to-right that allows you to easily move stuff.

The most important thing about Kanban is it helps to eliminate constraints. The Zen tool only allows a certain amount of items in the "Work" column and it visually communicates blocked items by moving them to the top and highlighting them with a red border. The Zen tool that Marty is showing looks similar to Rally, but is much more visually appealing.

The benefits of Kanban include:

  • simple, less process
  • less inventory of requirements/stories
  • limit work in progress, maximize throughput
  • less time in meetings
  • more naturally represents story lifecycle
  • more easily spot bottlenecks
  • estimate only if it adds value

Kanban promotes tracking how long it takes for a story get across the board and into production vs. tracking velocity of a team.

On my current project, we use Rally, a small team and have two week iterations. Because the things that Marty is talking about seem to be things we're already doing, I asked him how Kanban differs from Scrum with small teams. He explained that this biggest difference is Kanban is most useful when you're pushing things to production with each iteration.

The most controversial practice that Marty promotes is Continuous Deployment. This is the automated deployment of code to production. It includes automated testing and continuous integration, simple deployment/rollback scripts, a successful CI build triggers deployment, and there's real-time alerts in production. When shit goes wrong, you should use the "five whys" to perform root cause analysis. Marty admits that this is only a good idea when there's a high-level of trust in your development team and lots of tests to prove nothing is broken.

The benefits of continuous deployment is there's a lower story cycle time, you eliminate waste in deploying code, you deliver features/bugs fixes faster and you find integration issues quicker and in isolation. It's also a great way to promote not checking in shitty code.

The skeptics think this is a bad idea because 1) it's scary, 2) they believe it causes lower quality and 3) it causes more issues in production. The good news is you can still control production deployments with your source control system (e.g. branches and such). More than anything, it forces you to have a high quality continuous integration system that acts as the gatekeeper for what goes to production.

You can learn more about topics Marty covered in this talk at the following sites:

If you're lucky enough to be attending Aloha on Rails, Marty will be presenting there. I recommend you attend his talk if you're trying to get stuff done quickly and get it into production even quicker. His techniques seem to be invaluable for developers that are trying to maximize their efficiency and reduce the time it takes to get their code into production.

Posted in Open Source at Sep 23 2009, 09:20:28 PM MDT 4 Comments

Building GWT Applications with MVP and Issues with Overlay Types

MVP has recently become a popular strategy for structuring GWT applications. This is largely due to its testability and Ray Ryan's Best Practices For Architecting Your GWT App from this year's Google I/O. GWT, by itself, is simply a widget toolkit and doesn't ship with any sort of MVC (or MVP) framework.

On my current project, we're using GXT, a GWT implementation based on ExtJS. It has its own MVC framework, but it has very little documentation and can be confusing when using it with GWT's History management. At one point, I attempted to make it more understandable by writing a blog entry on GXT's MVC Framework.

One of my initial assignments was to decide if we should use MVP or MVC. Regardless of which one was chosen, I was also tasked with deciding if we should use an existing framework or write our own. After watching Ray Ryan's session on YouTube and recalling my frustration with GXT MVC on my last project, I quickly became convinced MVP was the answer.

To test my "MVP is best for our project" theory, I did a spike to implement it. I used the GWT MVP Example tutorial as a starting point and added the following libraries to my project.

Implementing the MVP pattern itself was relatively straightforward, but I did encounter a few issues. I'm writing this post to see if anyone has solved these issues.

MVP Implementation Issues
The first issue I ran across was GXT's widgets don't implement standard GWT interfaces. To try and figure out a solution, I posted the following on Twitter:

"Wondering if it's possible to do MVP with GXT since it's buttons don't implement standard GWT interfaces."

The best response I received was from Simon Stewart (founder of the WebDriver project, works for Google):

"Put the GXT buttons in the View. Let that turn DOM events into semantic events."

He also pointed me to his tdd-gwt-gae project which shows many techniques for unit testing (with jMock) and integration testing (with GWTTestCase). Using Simon's examples, I was able to determine an initial strategy for implementing MVP with GXT.

My strategy is instead of getting widgets from the view and adding handlers, you add handlers to to the view and it takes care of adding them to the widgets that should listen for them. This seems to work, but my View interface has a lot of void methods, which is a bit different than standard MVP patterns I've seen.

The 2nd issue I encountered is with unit testing. Unit testing can be be performed on MVP applications by mocking out any dependencies that use JSNI. Classes that use JSNI are not testable with plain ol' JUnit and typically requires you to use GWTTestCase, which can be slow and cumbersome. This isn't to say that GWTTestCase isn't useful, just that it has its place.

When unit testing MVP applications, the recommended practice seems to be you should test presenters and not views. Services fall into a similar "don't unit test" category because they'll need to connect to the server-side, which won't be running in a unit testing environment.

This is where I ran into a major issue that I don't have a solution for.

I'm using Overlay Types (as described in JSON Parsing with JavaScript Overlay Types) to do JSON parsing in callbacks. Since Overlay Types use JSNI, it's not possible to do any JSON parsing in unit tests. The problem with not being able to do any JSON parsing is the callbacks will often call eventBus.fireEvent(GwtEvent) after the JSON parsing has happened. This means I can't fully test the flow of a presenter if event firing happens in a callback.

In attempt to try different mocking techniques for callbacks, I created a test that uses two recommended EasyMock-ing strategies. The first is a "CallbackSuccessMatcher" and is described in more detail in Testing GWT without GwtTestCase. The second technique uses a "CallbackMockSupport" class to allow EasyMock expectations such as expectLastCallAsync() and expectLastCallAsyncSuccess(T). You can read more about this technique in Test driven development for GWT UI code with asynchronous RPC.

Both of these examples use RPC, which typically has callbacks that have an onSuccess(T type) method. It's easy to use these callbacks in unit tests since T is a POJO and the onSuccess() method contains no JSNI code.

Currently, I see a few possible solutions to this problem:

  • Figure out a way to detect when unit tests are running and add if/else logic to callbacks.
  • Modify presenters and services so a callback can be set that is unit test-friendly.
  • Make JSOModel an interface that can be replaced/mocked in tests.

The last solution seems like best one, but I'm also curious to know what others are doing. My hunch is that most GWT apps use RPC and haven't run into this issue.

Posted in Java at Sep 22 2009, 01:41:36 PM MDT 17 Comments

Labor Day Weekend at The Farm

The Castle (very solar efficient) This past weekend, the kids and I journeyed into the Colorado mountains for a weekend of camping, hiking and fishing. It was my good friend Matt Good's 50th Birthday Celebration at his place affectionately known as The Farm. We had beautiful weather, wide open spaces and lots of laughs with good friends. Jack even managed to land a girlfriend (Amanda), who somehow became his "ex" in the same hot tubbing session. They got back together the next morning. ;-)

On the way home, we took the long way and traveled from Ute Pass Road to Frasier, stopping at Lake Evelyn Trail for an hour hike and some fishing. I even let the kids drive a few times on the "bumpy road" and we managed to skip most of the traffic on the freeway.

More than anything, I was impressed with Abbie and Jack's ambition on this trip. They really seem to love hiking and fishing and don't drag their feet like they used to. They're also enthusiastic about camping and sleeping in a tent. I'm awful proud of my little munchkins. Below are some pictures from our weekend.

Picking out a campsite Jack getting a cooking lesson Jack and his new girlfriend

Side View of The Castle Crazy Kids Fishy

Thanks to Matt and Pam for hosting us and congrats to Matt on officially becoming an "old man". For more pictures, please see my Labor Day 2009 set on Flickr.

Posted in General at Sep 09 2009, 08:22:40 AM MDT 1 Comment

My Experience with Java REST Frameworks (specifically Jersey and CXF)

Recently I was tasked with developing a server-side REST strategy for a client. When I started working with them, they were using GWT RPC for exposing services. They wanted to move to RESTful services to allow for a more open platform, that multiple types of clients could talk to. There were interested in supporting SOAP and GWT RPC as well, but it wasn't required. They are using Spring, a custom namespace (for easily creating remote services) and an HTML documentation generator to expose their API in human readable form.

When I first starting developing, I chose to try Enunciate. From Enunciate's homepage:

Enunciate is an engine for creating, maintaining, and deploying your rich Web service API on the Java platform. If that sounds complicated, it's not. All you have to do is define your service interfaces in Java source code. ... Then invoke Enunciate.

Sounds pretty sweet, eh? At first glance, the things I liked about Enunciate were:

  1. The ability to generate multiple endpoints (and clients).
  2. Generates nice-looking documentation.
  3. Allows selecting different frameworks (for example, CXF instead of JAX-WS RI).

Initially, the hardest part of using Enunciate was integrating it into my project. This was somewhat related to having a multi-module project, but moreso related to getting the settings right in my enunciate.xml file. After getting everything working, I encountered a few Spring wiring issues with the GWT Endpoints and with Jersey's Spring support.

The good news is I believe most of these issues were related to my project and how it proxies Spring beans that Jersey couldn't find. Jersey's Spring support only supports annotations at this time, so I was unable to tell it the proxied bean names via XML (or even its own annotations). I'm sure this problem is solvable, but after struggling with it for a day or two, I decided to give up on Enunciate. I had to get something working, and fast.

At this point, I was back to the drawing board. I knew there were plenty of good Java REST frameworks available, but Spring and CXF (for SOAP) were already available dependencies in my project, so I chose that route. I was able to get something up and running fairly quickly with this tutorial and CXF's JAX-RS documentation. I ran into an Invalid JSON Namespace issue, but was able to solve it by adding a custom namespace.

It's not all rosy though, there are still a couple CXF issues I haven't solved:

While CXF does take a bit of XML for each service, it's kinda slick in that it only requires you to annotate your service interfaces. It also generates documentation for your services (WSDL and WADL), but it's not as pretty as Enunciate's. To solve this, I added Enunciate for documentation only. To make Enunciate work, I did have to add some annotations to my service implementation classes and parse the generated HTML to fix some links to CXF's services, but ultimately it works pretty well.

In the end, I recommended my client not use Enunciate for generating endpoints. This was primarily related to their unique Spring configuration, but also because I was able to easily use the same classes for REST, SOAP and GWT Endpoints. However, I will continue to keep my eye on Enunciate. It could be very useful for the upcoming appfuse-ws archetype. I'm also eager to see better GWT support and the ability to generate Overlay types in future releases.

Posted in Java at Aug 27 2009, 02:20:51 PM MDT 15 Comments

Integrating GWT with Spring Security

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
 

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

Posted in Java at Aug 06 2009, 08:50:15 AM MDT 10 Comments

How to do cross-domain GWT RPC with a ProxyServlet

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

Posted in Java at Aug 05 2009, 04:06:12 PM MDT 17 Comments