Matt RaibleMatt Raible is a Web Architecture Consultant specializing in open source frameworks.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

What's New in Maven 3.0 with Matthew McCullough

Last night, I attended the Denver JUG meeting to hear some excellent talks by Matthew McCullough and Tim Berglund. I took notes during Matthew's talk, but my battery ran out before Tim's talk started. Below are my notes.

Matthew started out by described the differences between Maven 2 and Maven 3. As he began, he emphasized it wasn't a beginner talk, but mostly for existing Maven users that understand how to read a pom.xml and such.

The Roadmap
Commits to Maven 3 have been happening for the last 3 years. Matthew is not an employee of Sonatype, but he mentioned their name quite a bit in his talk. Sonatype has hired several committers (7 that Matthew knows of by name) that now work on Maven 3 full-time. For compatibility with Maven 2, the project has 450 integration tests and they test it against 100s of Maven 2 projects. Maven 3 has plugin classloader partitioning and a legacy simulation layer for old plugins.

The main improvement in Maven 3 is speed. It's been performance tuned to be 50% to 400% faster. Benchmarks (guaranteed by integration tests) include better: Disk I/O, Network I/O, CPU and Memory. Another new feature is extensibility so Maven is a better library rather than just a command-line tool. Now there's a library and APIs that you can use to do the things that Maven does. Plexus has been replaced with Guice and it's now much easier to embed Maven (Polyglot Maven and Maven Shell are examples of this).

Below are a number of other changes between Maven 2 and Maven 3.

  • Syntax: pom.xml still uses <modelVersion>4.0.0</modelVersion> so it can be a drop-in replacement for Maven 2 projects.
  • Validations: poms are heavily validated against common mistakes, warns when plugin versions are not specified (use mvn validate to see issues), blocks duplicate dependencies (examined in same POM only, conflict resolution used otherwise).
  • Help URLs: wiki page URLs now shown for all error messages. One of the first Apache projects to do this.
  • Removals: profiles.xml external file support removed, Maven 1.0 repository support removed <layout>legacy</layout> (it's been 5 years since any commits to Maven 1).
  • Behavior: SNAPSHOTs always deployed with date-stamps, artifact resolution caching has been improved to do less checking (override with mvn <phase> -U).
  • Plugins: version auto-selection favors RELEASEs over SNAPSHOTs (opposite for Maven 2), versions cannot be specified as RELEASE or LATEST, plugins only resolved from <pluginRepository> locations.
  • See the Plugin Compatibility Matrix to see if your favorite plugins are compatible.

Maven 3 hopes to be a drop-in replacement for Maven 2, but non backwards-compatible changes will be happening in Maven 3.1. It's anticipated release is Q1 of 2011 and will likely contain the following features.

  • "Mixins" for direct dependencies
  • Site plugin takes over <reporting>
  • Backwards compatibility by <modelVersion
  • There's a good chance 3.1 breaks compatibility with legacy POMs

Another new thing in Maven 3 is Toolchain. Toolchain a common way to configure your JDK for multiple plugins. There are only a handful of plugins that are toolchain-enabled. User tool chain definitions are defined in ~/.m2/toolchains.xml. To use different toolchains (JDKs), you specify a vendor and version as part of your plugin configuration.

Maven Shell is a high performance console that's a Maven 3 add-on. It's hosted at GitHub to make community contributions easier. It goes on your command line and it offers syntax highlighting and context-sensitive help (by typing ? at the command prompt).

Another major improvement in Maven 3 is Polyglot Maven. Tools like Gant and Buildr have made Maven look ancient, but they've also given it a good challenge. Maven 3 is likely to leapfrog these tools because of its ability to use different languages for your build configuration. Currently, 6 languages are supported. Polyglot Maven is a super-set distribution of Maven 3. It's not shipped with Maven 3 core because it contains all the other language implementations and is quite large. Polyglot Maven also contains a translate tool that allows you to convert any-to-any language. It has a DSL framework with Macros and Lifecycle Hooks. Macros allows for more concise syntax.

After talking about Polyglot Maven a bit, Matthew shows us a demo translating pom.xml to pom.yaml and then running the build. After that, he showed us examples of what a pom looks like when defined in Clojure, Scala and Groovy. Someone asked about file parsing performance and Matthew said different languages would cause a single-digit performance difference as part of your build process. Personally, I can't help but think any non-XML parser would be faster than the XML parser.

In regards to m2eclipse, a new drop (0.10) occurred a few weeks ago and it's one of the highest quality releases to date. It has major refactoring and many performance improvements.

For sample Maven projects see Matthew's Maven Samples.

I very much enjoyed Matthew's talk, both because of his presentation techniques and because he had a lot of good information. While I've tried Maven 3 and Shell in the past, I've been newly inspired to start using them again on a daily basis.

Tim's talk on Decision Making was also excellent. The biggest things I learned were that conflict is good (idea-wise, not personal) and things to look out for between teams (fault lines). Hopefully both Tim and Matthew post their slides so I can link to them here.

Posted in Java at May 13 2010, 03:54:21 PM MDT 1 Comment

Software Quality: The Quest for the Holy Grail?

This afternoon, I attended a session on software quality by Jesper Pedersen at TSSJS. Jesper is a Core Developer at JBoss by Red Hat. He's the project lead of JBoss JCA, Tattletale, Papaki and JBoss Profiler 2. He's also currently the chairman of the Boston JBoss User Group. In this session, Jesper hopes to define basic requirements for a development environment and offer ideas on how to clean up a messy project.

Is software quality a friend or a foe? By implementing software quality processes, are you introducing unnecessary overhead? Development platforms are different today. We write a lot more business-specific code. We depend on standard frameworks for the core functionality. We depend on vendors to implement the standards correctly. We also depend on vendors to provide the necessary integration layer.

Since the platform is now a large slice of the pie, we must make sure we know where the issue is located. We must have proper integration tests; we must manage dependencies. Today, we must treat dependencies as if they are part of the application.

Defining the platform for your project helps narrow down the dependencies for your project. The platform is composed of corporate guidelines, standards, vendors and backend systems that you have to integrate with. Documentation is key for a successful project. Key documents types: User Guide, Developer Guide, API Guide, Architect Design, Implementation and Test.

It helps to define a project-wide configuration management system. Define a code-formatting guide will add consistency in your source tree. Also make sure you have separate build, test and execution environments. Use a Maven repository for your dependencies; both to support your project's artifacts as well as vendor artifacts.

"Maven today is an industry standard." -- Jesper Pederson

Define your tool chain as you would for your application. Back your Maven repository with SCM tools like Subversion or Git. For testing, use JUnit (unit testing), Corbertura (test coverage) and Hudson (continuous integration). Furthermore, you can add Checkstyle and Findbugs to verify coding conventions and find basic issues with code.

For the build environment, you need to make sure your dependency metadata is correct. Also, make sure you use the best practices for your given build tool. For example, with Maven and Ivy, it's a good idea to extract the version numbers into a common area of your pom.xml/build.xml so you can easily view all the versions in use. After you've done this, you can externalize the version information from the environment. Watch out for transitive dependencies as they can be tricky. Make sure you know what gets pulled in. Use enforcers to approve/ban dependencies or turn it off (Ivy). You can also vote for MNG-2315. Finally, snapshot dependencies are evil: define your release process so that releases are easy.

What can you do if your project is already a mess? Signs that your project is a mess: you look at your platform as a big black box, you use different dependencies than your deployment platform or you don't have integration tests for sub-systems or dependencies. To fix this, you can use a tool to get an overview of the entire project. Enter Tattletale.

Tattletale can give you an overview of your dependencies (Ant and Maven integration). It's a static analysis tool that doesn't depend on metadata, scanning your class files instead. Using Tattletale, you can produce a number of reports about your dependencies, what they're dependent on and what's dependent on you.

To maintain the lead in your project, make sure to define a checklist for each stage of your development cycle. Do reviews on documentation, architecture, component design and code. Enforce your rules of your project with your build system.

Jesper's final thoughts:

  • Maintaining dependencies for a software project can be a tricky task.
  • Using an Open Source platform as the foundation will ease the investigation of issues and increase trust.
  • Defining a project-wide tool chain is key.
  • Enforce all the rules on the project (better up-front than "fixing it" afterwards)

As Dusty mentioned, this session has a lot of good (basic) information, but there wasn't much new stuff. My team is using many of the technologies and practices that Jesper has mentioned. I guess that's validation that we're doing it right. I've heard of Tattletale, but never had a need for it since I haven't been on any "messy" projects recently.

Posted in Java at Mar 17 2010, 03:00:46 PM MDT 2 Comments

AppFuse 2.1 Milestone 1 Released

The AppFuse Team is pleased to announce the first milestone release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Hibernate, Spring and Tapestry 5.

What is AppFuse?
AppFuse is an open source project and application that uses open source tools built on the Java platform to help you develop Web applications quickly and efficiently. It was originally developed to eliminate the ramp-up time found when building new web applications for customers. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose which persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run appfuse:full-source.

In addition, AppFuse Light has been converted to Maven and has archetypes available. AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket.

Other notable improvements:

Please note that this release does not contain updates to the documentation. Code generation will work, but it's likely that some content in the tutorials won't match. For example, you can use annotations (vs. XML) for dependency injection and Tapestry is a whole new framework. I'll be working on documentation over the next several weeks in preparation for Milestone 2.

AppFuse is available as several Maven archetypes. For information on creating a new project, please see the QuickStart Guide.

To learn more about AppFuse, please read Ryan Withers' Igniting your applications with AppFuse.

The 2.x series of AppFuse has a minimum requirement of the following specification versions:

  • Java Servlet 2.4 and JSP 2.0 (2.1 for JSF)
  • Java 5+

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find bugs, please create an issue in JIRA.

Thanks to everyone for their help contributing code, writing documentation, posting to the mailing lists, and logging issues.

Posted in Java at Nov 19 2009, 07:16:36 AM MST 8 Comments

Testing GWT Libraries with Selenium and Maven

On Tuesday, I wrote about Running Hosted Mode in GWT Libraries. Today I added an additional module to our project to run Selenium tests against our GWT library. In the process, I discovered some things I needed to modify in my GWT library's pom.xml. I'm writing this post so others can use this setup to write GWT libraries and package them for testing with Selenium.

First of all, I noticed that when you're using the GWT Maven Plugin with a JAR project, it doesn't automatically run gwt:compile or gwt:test in the compile and test phases. I had to explicitly configure the compile goal to run in the compile phase. I also had to add <webappDirectory> to the configuration to compile the JavaScript files into the war directory.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>gwt-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <module>org.appfuse.gwt.core.CoreUI</module>
        <runTarget>index.html</runTarget>
        <webappDirectory>war</webappDirectory>
    </configuration>
    <executions>
        <execution>
            <phase>compile</phase>
            <goals>
                <goal>compile</goal>
            </goals>
        </execution>
    </executions>
</plugin>

To package the generated JavaScript and index.html in the JAR, I added the following <resources> section to the maven-resources-plugin configuration I mentioned in my previous post.

<resource>
    <directory>war</directory>
    <includes>
        <include>core.ui/**</include>
        <include>index.html</include>
    </includes>
</resource>

In addition, I discovered some javax.servlet.* classes in my JAR after running "mvn package". I believe this is caused by the GWT plugin sucking these in when it compiles my ProxyServlet. I excluded them by adding the maven-jar-plugin.

<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
        <excludes>
            <exclude>javax/servlet/**</exclude>
        </excludes>
    </configuration>
</plugin>

After doing this, I was able to publish my JAR with all the contents I needed to run Selenium tests against it.

Testing the GWT Library with Selenium
The module that contains the Selenium tests is a WAR project that uses war overlays, Cargo and Selenium RC. You can read about the Maven setup I use for running Selenium tests in Packaging a SOFEA Application for Distribution.

The major difference when testing a JAR (vs. a WAR), is I had to use the maven-dependency-plugin to unpack the JAR so its contents would get included in the WAR for testing. Below is the configuration I used to accomplish this:

<plugin>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <id>unpack</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>unpack</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>org.appfuse</groupId>
                        <artifactId>gwt-core</artifactId>
                        <version>1.0-SNAPSHOT</version>
                        <type>jar</type>
                        <overWrite>false</overWrite>
                        <excludes>META-INF/**,org/**,javax/**</excludes>
                    </artifactItem>
                </artifactItems>
                <outputDirectory>
                    ${project.build.directory}/${project.build.finalName}
                </outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

Hopefully this will help you develop GWT libraries and run Selenium tests against them. If you have any suggestions for simplifying this configuration, please let me know.

NOTE: I did considering a couple of other options for running Selenium tests against our GWT library:

  1. Add something to the existing project that 1) creates a WAR and 2) fires up Cargo/Selenium in a profile to test it.
  2. Create the tests in a GWT (war) project that includes widgets from the library.

I decided on the solution documented above because it seemed like the best option.

Posted in Java at Nov 04 2009, 10:09:27 PM MST 2 Comments

Running Hosted Mode in GWT Libraries (when using Maven)

Earlier this year, I wrote about Modularizing GWT Applications with GWT-Maven. Fast forward 8 months and I'm still working with GWT and using this same technique. However, this time I'm working with the Maven GWT Plugin from Codehaus. In my last post, I wrote:

The results of modularizing your application are beneficial (shared code) and detrimental (you have to mvn install gwt-core whenever you make changes in shared classes). If you know of a way to configure the gwt-maven plugin to read sources from both gwt-core and gwt-webapp in hosted mode, I'd love to hear about it.

The good news is I found a solution for this, using the Builder Helper Maven Plugin. The GWT Maven Plugin's Productivity tip for multi-project setup has more information on how to configure this (note: we use IntelliJ and Eclipse on my project and did not need to configure this in a profile).

All was fine and dandy with this configuration until I wanted to be able to run hosted mode to develop/test everything in my library before including it in my main project. Luckily, you can still run mvn gwt:run on a JAR project. However, when you configure your pom.xml so sources are included in your JAR, you run into an issue: your *.java files will be copied to war/WEB-INF/classes and hosted mode will use these files as source rather than the ones you're editing in src/main/java.

To solve this, I changed my pom.xml to do two things:

  • Only copy resources right before packaging (in the test phase).
  • When packaging is complete, delete the *.java files from war/WEB-INF/classes (using Ant).

Below is the XML I used to make this possible. Please let me know if you have a way to simplify this configuration.

<plugin>
    <artifactId>maven-resources-plugin</artifactId>
    <version>2.4.1</version>
    <executions>
        <execution>
            <phase>test</phase>
            <goals>
                <goal>copy-resources</goal>
            </goals>
            <configuration>
                <outputDirectory>${project.build.outputDirectory}</outputDirectory>
                <resources>
                    <resource>
                        <directory>src/main/java</directory>
                    </resource>
                    <resource>
                        <directory>src/main/resources</directory>
                    </resource>
                </resources>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.3</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>run</goal>
            </goals>
            <configuration>
                <tasks>
                    <delete>
                        <fileset dir="${project.build.outputDirectory}" includes="**/*.java"/>
                    </delete>
                </tasks>
            </configuration>
        </execution>
    </executions>
</plugin>

This solution seems to work pretty well. As far as developing your library in hosted mode, you'll need to configure two *.gwt.xml files, one that doesn't have an <entry-point> defined and one that does. Configure the one with the entry point as the <module> in your gwt-maven-plugin configuration.

As a side note, I found a few issues with the 1.1 version of the Maven GWT Archetype. Below are the steps I used to fix these issues and upgrade to GWT 1.7.0 (I realize 1.7.1 is out, but gwt-dev-1.7.1-mac.jar doesn't exist in Maven central).

First, create a new project by running the following from the command line:

mvn archetype:generate \
  -DarchetypeGroupId=org.codehaus.mojo \
  -DarchetypeArtifactId=gwt-maven-plugin \
  -DarchetypeVersion=1.1 \
  -DgroupId=com.yourcompany \
  -DartifactId=gwt-project -Dversion=1.0-SNAPSHOT -B

After creating the project, you'll need to modify the pom.xml as follows:

  1. Change the gwt-maven-plugin's version to 1.1.
  2. Change the ${gwtVersion} property to 1.7.0.
  3. Add <runTarget>Application.html</runTarget> to the <configuration> element of the plugin.
  4. Move Application.html and web.xml so they're under the "war" directory.
  5. Update Application.html to prepend the GWT module name in the <script> tag.

I hope these instructions help you create modular GWT projects with Maven. This setup is working great on my current project.

Posted in Java at Nov 03 2009, 09:37:07 AM MST 3 Comments

Packaging a SOFEA Application for Distribution

The project I'm working on is a bit different from those I'm used to. I'm used to working on web applications that are hosted on servers and customers access with their browser. SaaS if you will. My current client is different. They're a product company that sells applications and distributes them to customers via download and CD. Their customers install these applications on internal servers (supported servers include WebSphere, WebLogic and Tomcat).

The product I'm currently working on is structured as a SOFEA application and therefore consists of two separate modules - a backend and a frontend. Since it's installed in a servlet container, both modules are WARs and can be installed separately.

Building the backend and frontend as separate projects makes a lot of sense for two reasons:

  • In development, different teams can work on the frontend and backend projects.
  • Having them as separate projects allows them to be versioned separately.

However, having them as two separate projects does make it a bit more difficult for distribution. I'm writing this post to show you how I recently added support for distributing our application as 2 WARs or 1 WAR using the power of Maven, war overlays and the UrlRewriteFilter.

Project Setup
First of all, we have several different Maven modules, but the most important ones are as follows:

  • product-services
  • product-client
  • product-integration-tests

Of course, our modules aren't really named "product", but you get the point. The services project is really just a WAR project with Spring Security configured. It depends on other JAR modules that the services exist in. The client project is a GWT WAR that has a proxy servlet defined in its web.xml that makes it easier to develop. It also contains some UrlRewrite configuration that allows GWT Log's Remote Logging feature to work. The proxy servlet is something we don't want to ship with our product, so we have a separate web.xml for production vs. development. We do the substitution using the maven-war-plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.0.2</version>
    <configuration>
        <!-- Production web.xml -->
        <webXml>src/main/resources/web.xml</webXml>
        <warSourceDirectory>war</warSourceDirectory>
        <!-- Exclude everything but urlrewrite JAR -->
        <warSourceExcludes>
            WEB-INF/lib/aop**,WEB-INF/lib/commons-**,WEB-INF/lib/gin-**,
            WEB-INF/lib/guice-**,WEB-INF/lib/gwt-**,WEB-INF/lib/gxt-**,
            WEB-INF/lib/junit-**
        </warSourceExcludes>
    </configuration>
</plugin>

I could exclude WEB-INF/lib/** and WEB-INF/classes/**, but in my particular project, we still want UrlRewrite in standalone mode, and we have some i18n properties files in WEB-INF/classes that are served up for Selenium tests.

With this configuration, we have a services WAR and a client WAR that can be installed and used by clients. To collapse them into one and make it possible to ship a single war, I turned to our product-integration-tests module. This module contains Selenium tests that test both types of distributions.

Merging 2 WARs into 1
The most important thing in the product-integration-tests module is that it creates a single WAR. First of all, it uses <packaging>war</packaging> to make this possible. The rest is done using the following 3 steps.

1. Its dependencies include the client and servlet WARs (and Selenium RC for testing).

<dependencies>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-services</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-client</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium.client-drivers</groupId>
        <artifactId>selenium-java-client-driver</artifactId>
        <version>1.0.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

2. The WAR created excludes the "integration-tests" part of the name:

<build>
    <finalName>product-${project.version}</finalName>
    ...
</build>

3. WAR overlays are configured so the everything in the client's WEB-INF directory is excluded from the merged WAR.

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <!-- http://maven.apache.org/plugins/maven-war-plugin/overlays.html -->
        <overlays>
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-services</artifactId>
                <excludes>
                    <!-- TODO: Rename to api.html (this is the Enunciate-generated documentation) -->
                    <exclude>index.html</exclude>
                </excludes>
            </overlay>
            <!-- No server needed in product-client -->
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-client</artifactId>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
            <!-- Only include META-INF/context.xml to set the ROOT path -->
            <overlay>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
        </overlays>
    </configuration>
</plugin>

That's it! Using this configuration, it's possible to distribute a Maven-based SOFEA project as single or multiple WARs. However, there are some nuances.

One thing you might notice is the reference to META-INF/context.xml in the overlays configuration. This subtly highlights one issue I experienced when merging the WARs. In our GWT client, we're using URLs that point to our services at /product-services/*. This works in development (via a proxy servlet) and when the WARs are installed separately - as long as the services WAR is installed at /product-services. However, when they're merged, a little URL rewriting needs to happen. To do this, I added the UrlRewriteFilter to the product-services module and configured a simple rule.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCENGINE urlrewrite PUBLIC "-//tuckey.org//DTD UrlRewrite 3.0//EN"
        "http://tuckey.org/res/dtds/urlrewrite3.0.dtd">

<urlrewrite use-query-string="true">
    <!-- Used when services are merged into WAR with GWT client -->
    <rule>
        <from>^/product-services/(.*)$</from>
        <to type="forward">/$1</to>
    </rule>
</urlrewrite>

Because the services URLs point to the root (/product-services), the merged WAR has to be installed as the ROOT application. When you're using Cargo with Tomcat and want to deploy to ROOT, you have to have a META-INF/context.xml with a path="" reference (ref: CARGO-516).

<Context path=""/>

It is possible to change the URLs in the client to be relative, but this gets seems to get messy when you're using separate WARs. When using relative URLs, I found I had to do solution using cross-context forwarding to get the results I wanted. Using a redirect instead of a forward worked, but resulted in the client talking to the server twice (once to get redirected, a second time for the actual call). Cross-context forwarding is supported by the UrlRewriteFilter and Tomcat, but I'm not sure WebSphere or WebLogic support it. The best solution is probably to change the URLs dynamically at runtime, possibly using some sort of deferred binding technique.

Testing with Cargo and Selenium
Once I had everything merged, I wanted to configure Cargo and Selenium to allow testing both distribution types. If I installed all 3 wars at the same time, the "product-services" WAR would be used by both the product-client.war and the product.war, so I had to use profiles to allow installing the single merged WAR or both WARs. Below is the profile I used for starting Cargo, deploying the merged WAR, starting Selenium RC and running Selenium tests.

<properties>
    <cargo.container>tomcat6x</cargo.container>
    <cargo.container.url>
        http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.zip
    </cargo.container.url>
    <cargo.host>localhost</cargo.host>
    <cargo.port>23433</cargo.port>
    <cargo.wait>false</cargo.wait>
    <cargo.version>1.0</cargo.version>

    <!-- *safari and *iexplore are additional options -->
    <selenium.browser>*firefox</selenium.browser>
</properties>
...
<profile>
    <id>itest-bamboo</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <wait>${cargo.wait}</wait>
                    <container>
                        <containerId>${cargo.container}</containerId>
                        <log>${project.build.directory}/${cargo.container}/cargo.log</log>
                        <zipUrlInstaller>
                            <url>${cargo.container.url}</url>
                            <installDir>${installDir}</installDir>
                        </zipUrlInstaller>
                    </container>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <!-- Deploy as ROOT since XHR requests are made to /product-services -->
                        <deployables>
                            <deployable>
                                <properties>
                                    <context>ROOT</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
                <executions>
                    <execution>
                        <id>start-container</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-container</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>selenium-maven-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start-server</goal>
                        </goals>
                        <configuration>
                            <background>true</background>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <excludes>
                                <exclude>none</exclude>
                            </excludes>
                            <includes>
                                <include>**/*SeleniumTest.java</include>
                            </includes>
                            <systemProperties>
                                <property>
                                    <name>selenium.browser</name>
                                    <value>${selenium.browser}</value>
                                </property>
                                <property>
                                    <name>cargo.port</name>
                                    <value>${cargo.port}</value>
                                </property>
                            </systemProperties>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

This profile is run by our Bamboo nightly tests with mvn install -Pitest-bamboo. The 2nd profile I added doesn't install the project's WAR, but instead installs the two separate WARs. Running mvn install -Pitest-bamboo,multiple-wars executes the Selenium tests against the multi-WAR distribution.

<profile>
    <id>multiple-wars</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <deployables>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-client</artifactId>
                                <pingURL>http://${cargo.host}:${cargo.port}/product-client/index.html</pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-client</context>
                                </properties>
                            </deployable>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-services</artifactId>
                                <pingURL>
                                    http://${cargo.host}:${cargo.port}/project-services/index.jspx
                                </pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-services</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

I won't be including any information on authoring Selenium tests because there's already many good references. I encourage you to checkout the following if you're looking for Selenium testing techniques.

Summary
This article has shown you how I used Maven, war overlays and the UrlRewriteFilter to allow create different distributions of a SOFEA application. I'm still not sure which packaging (1 WAR vs. 2) mechanism is best, but it's nice to know there's options. If you package and distribute SOFEA applications, I'd love to hear about your experience in this area.

Posted in Java at Oct 06 2009, 01:17:38 AM MDT 2 Comments

How to use GWT 2.0 with Maven and Generate SOYC Reports

One of the most interesting features coming in GWT 2.0 is code splitting and the ability to use GWT.runAsync() to reduce the size of your application's initial download. This week, I learned how to use GWT 2.0 with my GWT 1.6/Maven project. Below are instructions on how to build and use the latest GWT with Maven.

  • Checkout GWT and setup GWT_TOOLS.
  • Set a GWT_VERSION environment variable to 2.0.0-SNAPSHOT (export GWT_VERSION=2.0.0-SNAPSHOT).
  • Build GWT with the ant command.
  • After building completes, install the GWT artifacts into your local Maven repository using the following commands:
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-user -Dversion=2.0.0-SNAPSHOT \
    -Dpackaging=jar -Dfile=build/lib/gwt-user.jar
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-servlet -Dversion=2.0.0-SNAPSHOT \
    -Dpackaging=jar -Dfile=build/lib/gwt-servlet.jar
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
    -Dclassifier=mac -Dpackaging=jar -Dfile=build/lib/gwt-dev-mac.jar
    
    mkdir temp
    tar -zxf build/dist/gwt-mac-2.0.0-SNAPSHOT.tar.gz -C temp
    cd temp/gwt-mac-2.0.0-SNAPSHOT
    zip -0 gwt-mac-2.0.0-SNAPSHOT.zip lib*.jnilib
    cd ../..
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
    -Dclassifier=mac-libs -Dpackaging=zip \
    -Dfile=temp/gwt-mac-2.0.0-SNAPSHOT/gwt-mac-2.0.0-SNAPSHOT.zip
    
    Thanks to Jason for his help with this script.
  • Modify the pom.xml of your GWT project to use the the gwt-maven-plugin from Codehaus. Of course, you'll need to modify the <runTarget> to fit your project.
    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>gwt-maven-plugin</artifactId>
        <version>1.1</version>
        <configuration>
            <runTarget>org.appfuse.gwt.mvc.${entry.point}/${entry.point}.html</runTarget>
        </configuration>
        <executions>
            <execution>
                <goals>
                    <goal>compile</goal>
                    <goal>test</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
    
  • Modify your dependencies to match the ones below. With the Codehaus plugin, dependencies are much more concise.
    <dependency>
        <groupId>com.google.gwt</groupId>
        <artifactId>gwt-servlet</artifactId>
        <version>${gwt.version}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>com.google.gwt</groupId>
        <artifactId>gwt-user</artifactId>
        <version>${gwt.version}</version>
        <scope>provided</scope>
    </dependency>
    
  • Add <gwt.version>2.0.0-SNAPSHOT</gwt.version> to the properties section of your pom.xml.
  • At this point, you should be able to compile your project with mvn gwt:compile and run it in hosted mode using mvn gwt:run.

Generate SOYC Reports
In Google's code splitting documentation, it mentions The Story of Your Compile (SOYC). From the documentation:

To obtain a SOYC report for your application, there are two steps necessary. First, add -soyc to the compilation options that are passed to the GWT compiler. This will cause the compiler to emit raw information about the compile to XML files in an -aux directory beside the rest of the compiled output. In that directory, you will see an XML file for each permutation and a manifest.xml file that describes the contents of all the others.

The second step is to convert that raw information into viewable HTML. This is done with the SoycDashboard tool.

The first step is not currently possible with the gwt-maven-plugin, so I created a patch for it.

If you patch the gwt-maven-plugin and install it locally, make sure and change the version in your pom.xml to 1.2-SNAPSHOT.

To use the SoycDashboard tool, you'll need to install the gwt-soyc-vis.jar.

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-soyc-vis -Dversion=2.0.0-SNAPSHOT \
-Dpackaging=jar -Dfile=build/lib/gwt-soyc-vis.jar

Now you can generate SOYC reports with mvn gwt:compile -Dgwt.compiler.soyc=true. You can also add <soyc>true</soyc> to the <configuration> section of the gwt-maven-plugin.

The second step (converting the raw information into viewable HTML) is possible using java from the command-line, or by using the exec-maven-plugin. Here's the (lengthy) command-line version:

java -Xmx1024m -cp /Users/mraible/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:/Users/mraible/.m2/repository/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-mac.jar com.google.gwt.soyc.SoycDashboard -resources ~/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar -out target/soyc-report target/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz

In this example, I'm using the files from stories0.xml.gz, dependencies0.xml.gz, splitPoints0.xml.gz. In the soycReport output directory, there's 5 of each these files and I'm not sure what the difference between reports is. Hopefully someone on the GWT team can elaborate. The exec-maven-plugin version is as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <executable>java</executable>
        <arguments>
            <argument>-cp</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:${settings.localRepository}/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-${platform}.jar
            </argument>
            <argument>com.google.gwt.soyc.SoycDashboard</argument>
            <argument>-out</argument>
            <argument>target/soyc-report</argument>
            <argument>-resources</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar
            </argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz</argument>
        </arguments>
    </configuration>
</plugin>

After configuring this plugin in your project, you should be able to run mvn gwt:compile exec:exec and open the generated report (at target/soyc-report/SoycDashboard-index.html). Currently, there doesn't seem to be much documentation on SOYC. Fred Sauer's recent presentation talks a bit about SOYC and GWT.runAsync(), but that's about it.

To figure out how to use GWT 2.0 with Maven, I used my GWT MVC Example project. The first SOYC report I generated said the initial download was 108,967 KB. To integrate GWT.runAsync(), I modified all the project's controllers so their handleEvent() methods changed from this:

public void handleEvent(AppEvent event) {
    onViewHome(event);
}

To this:

public void handleEvent(final AppEvent event) {
    GWT.runAsync(new RunAsyncCallback() {
        public void onFailure(Throwable throwable) {
            Window.alert(throwable.getMessage());
        }

        public void onSuccess() {
            onViewHome(event);
        }
    });
}

When I generated a new SOYC report, the initial download size was reduced to 56,718 KB. Furthermore, I was able to see that my "Leftovers code" consisted of 63,175 KB. I'm sure there's better ways to split my project using GWT.runAsync(), but I'm happy to see I was able to reduce the initial download by 50%.

If you'd like to try GWT 2.0, you can can download my gwt-mvc example project. To build/run this project, you'll need to 1) build and install GWT, 2) patch gwt-maven-plugin and 3) run mvn gwt:compile exec:exec to generate the SOYC report. In an ideal world, the gwt-maven-plugin can be enhanced to generate the SOYC report (rather than using the exec-maven-plugin). In the meantime, I think it's pretty cool that you can try out GWT 2.0 features while they're still being developed.

Posted in Java at Jun 25 2009, 11:45:04 PM MDT 11 Comments

Optimizing a GWT Application with Multiple EntryPoints

Building a GWT application is an easy way for Java Developers to write Ajax applications. However, it can be difficult to release a GWT application to production before it's finished. One of the most important things I've learned in Software Development is to get a new application into production as soon as possible. Not only does getting it from dev → qa → prod verify your process works, it also can do a lot to test the viability of the new application.

One of the biggest issues with GWT applications is size. The project I'm working on compiles Java to JavaScript and creates ~570K *.cache.html files (one for each modern browser). These files end up being around 180K gzipped. I believe this is an OK size for an entire application. However, if you're going to release early, release often with GWT, chances are you'll just want to release one feature at a time.

When the first feature was completed on my project, the *.cache.html files were around 300K. Rather than using branches to release to QA and UAT, bug fixes and new features were developed on trunk. Unfortunately, the QA and UAT process took several weeks longer than expected so by the time the feature was ready to release, the *.cache.html files had grown to around ~570K. The reason the file had grown so much was because it included all of the other features.

Earlier this week, while running to a dentist appointment, I thought of a solution to this problem. The basic idea was to optimize the compilation process so only the to-be-released feature was included. Even better, the solution didn't require more modularization. The results:

Before: *.cache.html -> 569K, gzipped 175K
After: *.cache.html -> 314K, gzipped 100K

According to my calculations, that's a 56% reduction in size. How did I do it?

  1. Created a new FeatureName.java EntryPoint with only the to-be-released features imported.
  2. Created a new FeatureName.gwt.xml that references the new EntryPoint.
  3. Copied old (kitchen-sink) EntryPoint.html to FeatureName.html and changed the reference to the nocache.js file.
  4. Created a Maven profile that allows using -PFeatureName to build a FeatureName-only module.

One downside to doing things this way is it's possible to create a WAR that has the same name and different features. Surely the Maven Overlords would frown upon this. Since this is just a temporary solution to release features incrementally, I'm not too worried about it. A possible workaround is to create different WAR names when a feature's profile is activated. I believe the true "Maven way" would be to make the "kitchen sink" application into a JAR and have several WAR modules with the different EntryPoints. Seems a bit complicated to me.

Other than this Maven publishing issue, the only other issue I can foresee is keeping the two EntryPoints and HTML files in synch. Then again, the separate files allow a feature to be customized for the release and can be deleted when its no longer needed.

What do you think? Do you know of a better way to compile a GWT application so it only contains certain features?

Posted in Java at Mar 25 2009, 04:00:37 PM MDT 12 Comments

Modularizing GWT Applications with GWT-Maven

Last week, I spent some time modularizing the GWT application I'm working on. By modularizing, I mean splitting the code from one GWT module into a "core" and "webapp" module. The reason for doing this was so the "core" module could be used by another GWT application. Creating GWT Modules is fairly straightforward, but it wasn't as intuitive as expected when using the gwt-maven-plugin.

The hardest part of moving the code was figuring out how to run tests in the new "core" module. After getting it all working, it seems easy enough. Hopefully this post will make it easy for others. Here's the steps I'd recommend:

  1. Convert your GWT project into a multi-module project where you have a top-level pom.xml and two sub-modules (e.g. gwt-core and gwt-webapp).
  2. Do the normal single-to-multi-project Maven stuff like declaring the <parent> element in the modules and moving plugins/dependencies to the top-level pom.xml.
  3. Refactor your gwt-webapp project to push down all shared classes (and their tests) to gwt-core.
  4. In the gwt-core project, include *.xml and *.java in your JAR so GWT can extract/compile the source code when building gwt-webapp.
    <resources>
        <resource>
            <directory>src/main/java</directory>
            <includes>
                <include>**/*.java</include>
                <include>**/*.xml</include>
            </includes>
        </resource>
    </resources>
    
  5. In gwt-core/src/main/java, create a Core.gwt.xml that references the modules you'd like to use in all your applications. For example:
    <module>
        <inherits name="com.google.gwt.user.User"/>
        <inherits name="com.google.gwt.i18n.I18N"/>
        <inherits name="com.extjs.gxt.ui.GXT"/>
        <inherits name="pl.rmalinowski.gwt2swf.GWT2SWF"/>
    </module>
    
  6. Now the tricky part begins, mostly because of how the gwt-maven plugin currently works. In src/test/java, create a NoOpEntryPoint.gwt.xml that inherits your Core module and defines an EntryPoint.
    <module>
        <inherits name="com.company.app.Core"/>
        <entry-point class="com.company.app.NoOpEntryPoint"/>
    </module>
    
  7. Create a NoOpEntryPoint.java class in the same directory as NoOpEntryPoint.gwt.xml.
    public class NoOpEntryPoint implements EntryPoint {
        
        public void onModuleLoad() {
            // do nothing
        }
    }
    
  8. In any class that extends GWTTestCase (I usually create a parent class for all tests), reference the NoOpEntryPoint in the getModuleName() method.
        @Override
        public String getModuleName() {
            return "com.company.app.NoOpEntryPoint";
        }
    
  9. Lastly, in the gwt-maven plugin's configuration (in gwt-core/pom.xml), reference the NoOpEntryPoint in <compileTargets>, a non-existent file in <runTarget> and only the "test" goal in the executions.
    <plugin>
        <groupId>com.totsp.gwt</groupId>
        <artifactId>maven-googlewebtoolkit2-plugin</artifactId>
        <version>2.0-beta26</version>
        <configuration>
            <compileTargets>
                <value>com.company.app.NoOpEntryPoint</value>
            </compileTargets>
            <runTarget>com.company.app.NoOpEntryPoint/doesntexist.html</runTarget>
            <logLevel>INFO</logLevel>
            <style>OBF</style>
            <noServer>false</noServer>
            <extraJvmArgs>-Xmx512m</extraJvmArgs>
            <gwtVersion>${gwtVersion}</gwtVersion>
            <testFilter>*GwtTestSuite.java</testFilter>
            <testSkip>${skipTests}</testSkip>
        </configuration>
        <executions>
            <execution>
                <goals>
                    <goal>test</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
    

The results of modularizing your application are beneficial (shared code) and detrimental (you have to mvn install gwt-core whenever you make changes in shared classes). If you know of a way to configure the gwt-maven plugin to read sources from both gwt-core and gwt-webapp in hosted mode, I'd love to hear about it.

Posted in Java at Mar 23 2009, 10:36:08 AM MDT 11 Comments

Nexus is a kick-ass Repository Manager

I started my current gig at the end of last year. I've been enjoying the work and especially the project infrastructure we've been using. We're using the usual suspects: JIRA, Confluence, Hudson and Subversion. We're also using a couple new ones, namely sventon and Nexus. For building, we're using Maven and Ivy (as a Grails plugin).

Nexus I'm writing this post to talk about Nexus and how much I've enjoyed using it. I like Nexus for two reasons: it's aesthetically pleasing and it's well-documented. Another reason I really dig it is because I haven't had to touch it since I first configured it. Software that just keeps on humming is always fun to work with.

Initially, I remember having some issues setting up repositories. I also remember solving them after learning how groups work.

In addition to on-the-job, I've started to use Nexus more and more in my open source life. With the help of Jason van Zyl, I recently moved AppFuse's repository to Sonatype's oss.sonatype.org. I also noticed there's a Nexus instance for Apache projects. If that's not enough, you can get Nexus Pro free if you're an open source project.

Personally, the open source version of Nexus seems good enough for me. While the Staging Suite looks nice, I think it's possible to do a lot of similar things with good communication. After all, it's not going to free you from having to wrestle with the maven-release-plugin.

Next week, I'm helping to polish and document our entire release process (from dev → qa → production). If you have any advice on how to best perform releases with Maven, Grails and/or Nexus, I'd love to hear about it. My goal is extreme efficiency so releases can be done very quickly and with minimal effort.

Posted in Java at Mar 05 2009, 11:59:02 PM MST 13 Comments