Matt RaibleMatt Raible is a writer with a passion for software. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.
You searched this site for "maven". 270 entries found.

You can also try this same search on Google.

RE: Moving from Spring to Java EE 6: The Age of Frameworks is Over

Last Tuesday, Cameron McKenzie wrote an interesting article on TheServerSide titled Moving from Spring to Java EE 6: The Age of Frameworks is Over. In this article, Cameron says the following:

J2EE represents the past, and Java EE 6 represents the future. Java EE 6 promises us the ability to go beyond frameworks. Frameworks like Spring are really just a bridge between the mistakes of the J2EE past and the success of the Java EE 6 future. Frameworks are out, and extensions to the Java EE 6 platform are in. Now is the time to start looking past Spring, and looking forward to Seam and Weld and CDI technologies.

He then links to an article titled Spring to Java EE - A Migration Experience, an article written by JBoss's Lincoln Baxter. In this article, Lincoln talks about many of the technologies in Java EE 6, namely JPA, EJB, JSF, CDI and JAX-RS. He highlights all the various XML files you'll need to know about and the wide variety of Java EE 6 application servers: JBoss AS 6 and GlassFish v3.

I don't have a problem with Lincoln's article, in fact I think it's very informative and some of the best documentation I've seen for Java EE 6.

I do have some issues with Cameron's statements that frameworks are mistakes of the J2EE past and that Java EE 6 represents the future. Open source frameworks made J2EE successful. Struts and Hibernate came out in the early days of J2EE and still exist today. Spring came out shortly after and has turned into the do-everything J2EE implementation it was trying to fix. Java EE 6 might be a better foundation to build upon, but it's certainly not going to replace frameworks.

To prove my point, let's start by looking at the persistence layer. We used to have Hibernate based on JDBC, now we have JPA implementations built on top of the JPA API. Is JPA a replacement for all persistence frameworks? I've worked with it and think it's a good API, but the 2.0 version isn't available in a Maven repo and Alfresco recently moved away from Hibernate (which == JPA IMO) to iBATIS for greater data access layer control and scalability. Looks like the age of frameworks isn't over for persistence frameworks.

The other areas that Java EE 6 covers that I believe frameworks will continue to excel in: EJB, CDI, JSF and JAX-RS. Personally, I don't have a problem with EJB 3 and think it's a vast improvement on EJB 2.x. I don't have an issue with CDI either, and as long as it resembles Guice for dependency injection, it works for me. However, when you get into the space I've been living in for the last couple years (high-traffic public internet sites), EJB and things like the "conversation-scope" feature of CDI don't buy you much. The way to make web application scale is to eliminate state and cache as much as possible, both of which Java EE doesn't provide much help for. In fact, to disable sessions in a servlet-container, you have to write a Filter like the following:

public class DisabledSessionFilter extends OncePerRequestFilter {

    /**
     * Filters requests to disable URL-based session identifiers.
     */
    @Override
    protected void doFilterInternal(final HttpServletRequest request,
                                    final HttpServletResponse response,
                                    final FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequestWrapper wrappedRequest = new HttpServletRequestWrapper(request) {

            @Override
            public HttpSession getSession(final boolean create) {
                if (create) {
                    throw new UnsupportedOperationException("Session support disabled");
                }
                return null;
            }

            @Override
            public HttpSession getSession() {
                throw new UnsupportedOperationException("Session support disabled");
            }
        };

        // process next request in chain
        chain.doFilter(wrappedRequest, response);
    }
}

What about JAX-RS? Does it replace the need for frameworks? I like the idea of having a REST API in Java. However, its reference implementation is Jersey, which seems more like a framework than just Java EE. If you choose to use JAX-RS in your application, you still have to choose between CXF, Jersey, RESTEasy and Restlet. I compared these frameworks last year and found the Java EE implementation lacking in the features I needed.

Finally, let's talk about my-least-framework-web-framework: JSF. The main reason I don't like JSF is because of its 1.x version. JSF 1.0 was released a year before the Ajax term was coined (see timeline below). Not only did it take forever to develop as a spec, but it tried to be a client-component framework that was very stateful by default.

History of Web Frameworks

Now that JSF 2.0 is out, it has Ajax integrated and allows you to use GET instead of POST-for-everything. However, the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn't be developing your UI). Also, the best component development platform for the web is JavaScript. I recommend using an Ajax framework for your components if you really want a rich UI.

Sure you can use the likes of Tapestry and Wicket if you like POJO-based web development, but if you're looking to develop a webapp that's easy to maintain and understand, chances are that you'll do much better with traditional MVC frameworks like Spring MVC and Struts 2. The simplicity and popularity of Rails and Grails further emphasize that developers prefer these types of web frameworks.

Another reason I don't like JSF: there's very few developers in the wild happy with it. The major promoters of JSF are book authors, trainers, Java EE Vendors and MyFaces developers. Whenever I speak at conferences, I ask folks to raise their hands for the various web frameworks they're using. I always ask the JSF users to keep their hands up if they like it. Rarely do they stay up.

So it looks like we still need web frameworks.

Eberhard Wolff has an interesting post where he defends Spring and talks about the productivity comparisons between Spring and Java EE. He recommends using Grails or Spring Roo if you want the level of productivity that Ruby on Rails provides. That's a valid recommendation if you're building CRUD-based webapps, but I haven't developed those in quite some time. Nowadays, the apps I develop are true SOFEA apps, where the backend serves up XML or JSON and the frontend client is HTML/JavaScript/CSS, Android, iPad or Sony Blu-Ray players. On my current project, our services don't even talk to a database, they talk to a CMS via RESTful APIs. We use Spring's RestTemplate for this and HttpClient when it doesn't have the features we need. Not much in Java EE 6 for this type of communication. Sure, Jersey has a client, but it's certainly not part of the Java EE spec.

As far as getting Ruby on Rails' zero-turnaround productivity, I don't need Grails or Spring Roo, I simply use IDEA and JRebel.

Conclusion
I don't see how new features in Java EE 6 can mean the age of frameworks is over. Java SE and J2EE have always been foundations for frameworks. The Java EE 6 features are often frameworks in themselves that can be used outside of a Java EE container. Furthermore, Java EE 6 doesn't provide all the features you need to build a high-scale web app today. There's no caching, no stateless web framework that can serve up JSON and HTML and no hot-reload productivity enhancements like JRebel. Furthermore, there's real excitement in Javaland for languages like Scala, Groovy and JRuby. All of these languages have web frameworks that've made many developers happy.

Here's to the Age of Frameworks - may it live as long as the JVM!

P.S. If you'd like to hear me talk about web frameworks on the JVM, I'll be speaking at The Colorado Springs Open Source Meetup and Devoxx 2010 in the near future.

Posted in Java at Oct 16 2010, 03:19:07 PM MDT 37 Comments

Running Selenium Tests on Sauce Labs

Recently I embarked on a mission to configure my team's Selenium testing process to support multiple browsers. We use Hudson for our continuous integration server. Since our Hudson instance runs on Solaris, testing with Firefox on Solaris didn't seem like a good representation of our clients. Our browser support matrix currently looks as follows:

Platform Browser
Supported
Windows IE7.x and 8.x, Firefox 2.x and 3.x
Mac Safari 3.x, 4.x
Best Effort
Windows and Mac Chrome 4.x

At first, I attempted to use Windows VMs to run Selenium tests on IE. This was a solution that didn't work too well. The major reasons it didn't work:

  1. I had issues getting the Selenium Plugin for Hudson working. Upgrading the plugin to use Selenium RC 1.0.5 may solve this issue.
  2. We had some unit tests that failed on Windows. I tried using the Cygpath Plugin for Hudson (which allows you to emulate a Unix environment on Windows), but failed to get it to work.
  3. We quickly realized it might become a maintenance nightmare to keep all the different VMs up-to-date.

Frustrated by these issues, I turned to Sauce Labs. They have a cloud-based model that runs Selenium tests on VMs that point back to your application. They also support many different browser/OS combinations. We asked them about support for OS X and various Windows versions and they indicated that their experience shows browsers are the same across OSes.

I'm writing this article to show you how we've configured our build process to support 1) testing locally and 2) testing on Sauce Labs. In a future post, I hope to write about how to run Selenium tests concurrently for faster execution.

Running Selenium Tests Locally
We use Maven to build our project and run our Selenium tests. Our configuration is very similar to the poms referenced in Integrating Selenium with Maven 2. Basically, we have an "itest" profile that gets invoked when we pass in -Pitest. It downloads/starts Tomcat (using Cargo), deploys our WAR, starts Selenium RC (using the selenium-maven-plugin) and executes JUnit-based tests using the maven-surefire-plugin. All of this configuration is pretty standard and something I've used on many projects over the past several years.

Beyond that, we have a custom BlockJUnit4ClassRunner class that takes screenshots and captures the HTML source for tests that fail.

public class SeleniumJUnitRunner extends BlockJUnit4ClassRunner {
    public SeleniumJUnitRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    protected Statement methodInvoker(FrameworkMethod method, Object test) {
        if (!(test instanceof AbstractSeleniumTestCase)) {
            throw new RuntimeException("Only works with AbstractSeleniumTestCase");
        }

        final AbstractSeleniumTestCase stc = ((AbstractSeleniumTestCase) test);
        stc.setDescription(describeChild(method));

        return new InvokeMethod(method, test) {
            @Override
            public void evaluate() throws Throwable {
                try {
                    super.evaluate();
                } catch (Throwable throwable) {
                    stc.takeScreenshot("FAILURE");
                    stc.captureHtmlSource("FAILURE");
                    throw throwable;
                }
            }
        };
    }
}

To use the functionality SeleniumJUnitRunner provides, we have a parent class for all our tests. This class uses the @RunWith annotation as follows:

@RunWith(SeleniumJUnitRunner.class)
public abstract class AbstractSeleniumTestCase {
    // convenience methods
}

This class looks up the Selenium RC Server, the app location and what browser to use based on system properties. If system properties are not set, it has defaults for running locally.

public static String SERVER = System.getProperty("selenium.server");
public static String APP = System.getProperty("selenium.application");
public static String BROWSER = System.getProperty("selenium.browser");

protected Selenium selenium;

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    selenium = new DefaultSelenium(SERVER, 4444, BROWSER, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

The system properties are specified as part of the surefire-plugin's configuration. The reason we default them in the above code is so tests can be run from IDEA as well.

<artifactId>maven-surefire-plugin</artifactId>
<version>2.5</version>
<configuration>
    <systemPropertyVariables>
        <selenium.application>${selenium.application}</selenium.application>
        <selenium.browser>${selenium.browser}</selenium.browser>
        <selenium.server>${selenium.server}</selenium.server>
    </systemPropertyVariables>
</configuration>

Running Selenium Tests in the Cloud
To run tests in the cloud, you have to do a bit of setup first. If you're behind a firewall, you'll need to setup SSH tunneling so Sauce Labs can see your machine. You'll also need to setup SSH Tunneling on your Hudson server, but installing/configuring/running locally is usually a good first step. Below are the steps I used to configure Sauce Labs' SSH Tunneling on OS X.

1. Install the Python version in /opt/tools/saucelabs. If you get an error (No local packages or download links found for install) download the egg and run it with:

sudo sh setuptools-0.6c11-py2.6.egg

NOTE: If you get an error (unable to execute gcc-4.2: No such file or directory) when installing pycrypto on OS X, you'll need to install the OS X Developer Tools.

2. Create a /opt/tools/saucelabs/local.sh script with the following in it. You should change the last parameter to use your username (instead of mraible) since Sauce Labs uses unique tunnel names.

python tunnel.py {sauce.username} {sauce.key} localhost 9000:80 mraible.local

3. Start the tunnel by executing local.sh. You should see output similar to the following.

$ sh local.sh 
/System/../Python.framework/../2.6/../twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead
 import itertools, md5
/System/../Python.framework/../2.6/../twisted/conch/ssh/keys.py:13: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
 import sha, md5
Launching tunnel ... 
Status: new
Status: booting
Status: running
Tunnel host: ec2-75-101-216-8.compute-1.amazonaws.com
Tunnel ID: 70f15fb59d2e7ebde55a6274ddfa54dd
<sshtunnel.TunnelTransport instance at 0x10217ad88> created
requesting remote forwarding for tunnel 70f15fb59d2e7ebde55a6274ddfa54dd 80=>localhost:9000
accepted remote forwarding for tunnel 70f15fb59d2e7ebde55a6274ddfa54dd 80=>localhost:9000

After setting up the SSH Tunnel, I modified AbstractSeleniumTestCase's setUp() method to allow running tests on Sauce Labs.

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    } else if (BROWSER.split(":").length == 3) {
        String[] platform = BROWSER.split(":");

        String os = platform[0];
        String browser = platform[1];

        // if Google Chrome, don't use a version #
        String version = (platform[1].equals("googlechrome") ? "" : platform[2]);
        String printableVersion = ((version.length() > 0) ? " " + platform[2].charAt(0) : "");

        String jobName = description.getMethodName() + " [" + browser + printableVersion + "]";

        BROWSER = "{\"username\":\"{your-username}\",\"access-key\":\"{your-access-key}\"," +
                "\"os\":\"" + platform[0] + "\",\"browser\": \"" + platform[1] + "\"," +
                "\"browser-version\":\"" + version + "\"," +
                "\"job-name\":\"" + jobName + "\"}";

        log.debug("Testing with " + browser + printableVersion + " on " + os);
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    selenium = new DefaultSelenium(SERVER, 4444, BROWSER, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

After making this change, I was able to run Selenium tests from IDEA using the following steps:

  1. Start Jetty on port 9000 (since that's what the tunnel points to). In IDEA's Maven panel, create a run/debug configuration for jetty:run, click the "Runner" tab and enter "-Djetty.port=9000" in the VM Parameters box.
  2. Right-click on the test to run and create a run/debug configuration. Enter the following in the VM Parameters box. The last two parameters allow skipping the xvfb and Selenium RC startup process.
    -Dselenium.browser="Windows 2003:iexplore:8." -Dselenium.application=mraible.local -Dselenium.server=saucelabs.com -Dxvfb.skip=true -Dselenium.server.skip=true

These same parameters can be used if you want to run all tests from the command line:

mvn install -Pitest -Dselenium.browser="Windows 2003:iexplore:8." -Dselenium.application=mraible.local -Dselenium.server=saucelabs.com -Dxvfb.skip=true -Dselenium.server.skip=true -Dcargo.port=9000

To simplify things, we create profiles for the various browsers. For example, below are profiles for IE8 and Firefox 3.6.

<profile>
    <id>firefox-win</id>
    <properties>
        <cargo.port>9000</cargo.port>
        <selenium.application>http://${user.name}.local</selenium.application>
        <selenium.browser>Windows 2003:firefox:3.6.</selenium.browser>
        <selenium.server>saucelabs.com</selenium.server>
        <selenium.server.skip>true</selenium.server.skip>
        <xvfb.skip>true</xvfb.skip>
    </properties>
</profile>
<profile>
    <id>ie-win</id>
    <properties>
        <cargo.port>9000</cargo.port>
        <selenium.application>http://${user.name}.local</selenium.application>
        <selenium.browser>Windows 2003:iexplore:8.</selenium.browser>
        <selenium.server>saucelabs.com</selenium.server>
        <selenium.server.skip>true</selenium.server.skip>
        <xvfb.skip>true</xvfb.skip>
    </properties>
</profile>

Issues
Since we've started using Sauce Labs, we've run into a number of issues. Some of these are Selenium-related and some are simply things we learned since we started testing on multiple browsers.

  • SSH Tunnels Keep Restarting This happens on our Hudson server that runs the tunnels as a service. This seems to happen daily and screws up our Hudson results because builds fail.
  • XPath vs. CSS Selectors One of the first things we noticed was that our IE tests were 2-3 times slower than the same tests on Firefox. We discovered this is because Internet Explorer has a very slow XPath engine. To fix this issue, it's recommended that ids or CSS Selectors be used whenever trying to locate elements. For more information on CSS Selectors and Selenium, see CSS Selectors in Selenium Demystified. To test CSS Selectors, I found Firefinder to be a very useful Firefox plugin. Note that many pseudo elements won't work in IE.
  • IE7 fails to initialize on Sauce Labs There's no errors in our JUnit reports, so we're not sure what's causing this. It could very well be bugs in our code/configuration, but IE8 works fine.
  • The Job Names on Sauce Labs don't get set correctly and often results in duplicate job names. This could certainly be related to my code. Finding videos that show failed tests is difficult when the job names aren't set correctly.
  • It would be slick if you could download the video of a failed test, similar to what we do by taking screenshots.
  • Google Chrome works on Sauce Labs, but I'm unable to get it working locally (on Windows or OS X). This seems to be a Selenium issue.
  • Safari 4 works, but when it fails, the screenshot shows a Safari can't find the file error. Since there's no real error to debug, it's difficult to figure out why the test fails. Since Safari 4 is not listed on platforms supported by Selenium, I'm unsure how to fix this.

Overall, Sauce Labs seems to work pretty well. However, in the process of messing with Hudson, build agents and Selenium infrastructure, it's become readily apparent that we need a team member to devote their full-attention to it. Having a developer or two work on it every now-and-then is inefficient, especially when we're still in the process of ironing everything out and making it all stable.

If you have any tips on how you've solved issues with Sauce Labs (ssh tunnels, IE7) or Selenium (Safari 4, Google Chrome), I'd love to hear them. I'm also interested to hear from anyone with experience running Selenium tests concurrently (locally or in the cloud).

Update: I discovered a bug in my AbstractSeleniumTest's setUp() method where job names weren't being set correctly. I've since changed the code in this class to the following:

private static String browser, printableVersion;

@BeforeClass
public static void parseBrowser() {

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    } else if (BROWSER.split(":").length == 3) {
        String[] platform = BROWSER.split(":");

        String os = platform[0];
        browser = platform[1];

        // if Google Chrome, don't use a version #
        String version = (platform[1].equals("googlechrome") ? "" : platform[2]);
        printableVersion = ((version.length() > 0) ? " " + platform[2].charAt(0) : "");

        BROWSER = "{\"username\":\"{your-username}\",\"access-key\":\"{your-access-key}\"," +
                "\"os\":\"" + os + "\",\"browser\": \"" + browser + "\"," +
                "\"browser-version\":\"" + version + "\", " +
                "\"job-name\": \"jobName\"}";
    }
}

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    String seleniumBrowser = BROWSER;
    if (BROWSER.startsWith("{")) { // sauce labs
        String jobName = description.getMethodName() + " [" + browser + printableVersion + "]";
        log.debug("=> Running job: " + jobName);

        seleniumBrowser = BROWSER.replace("jobName", jobName);
    }

    selenium = new DefaultSelenium(SERVER, 4444, seleniumBrowser, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

Posted in Java at Jun 06 2010, 07:50:20 PM MDT 4 Comments

Versioning Static Assets with UrlRewriteFilter

A few weeks ago, a co-worker sent me interesting email after talking with the Zoompf CEO at JSConf.

One interesting tip mentioned was how we querystring the version on our scripts and css. Apparently this doesn't always cache the way we expected it would (some proxies will never cache an asset if it has a querystring). The recommendation is to rev the filename itself.

This article explains how we implemented a "cache busting" system in our application with Maven and the UrlRewriteFilter. We originally used querystring in our implementation, but switched to filenames after reading Souders' recommendation. That part was figured out by my esteemed colleague Noah Paci.

Our Requirements

  • Make the URL include a version number for each static asset URL (JS, CSS and SWF) that serves to expire a client's cache of the asset.
  • Insert the version number into the application so the version number can be included in the URL.
  • Use a random version number when in development mode (based on running without a packaged war) so that developers will not need to clear their browser cache when making changes to static resources. The random version number should match the production version number formats which is currently: x.y-SNAPSHOT-revisionNumber
  • When running in production, the version number/cachebust is computed once (when a Filter is initialized). In development, a new cachebust is computed on each request.

In our app, we're using Maven, Spring and JSP, but the latter two don't really matter for the purposes of this discussion.

Implementation Steps
1. First we added the buildnumber-maven-plugin to our project's pom.xml so the build number is calculated from SVN.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>buildnumber-maven-plugin</artifactId>
    <version>1.0-beta-4</version>
    <executions>
        <execution>
            <phase>validate</phase>
            <goals>
                <goal>create</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <doCheck>false</doCheck>
        <doUpdate>false</doUpdate>
        <providerImplementations>
            <svn>javasvn</svn>
        </providerImplementations>
    </configuration>
</plugin>

2. Next we used the maven-war-plugin to add these values to our WAR's MANIFEST.MF file.

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.0.2</version>
    <configuration>
        <archive>
            <manifest>
                <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
            </manifest>
            <manifestEntries>
                <Implementation-Version>${project.version}</Implementation-Version>
                <Implementation-Build>${buildNumber}</Implementation-Build>
                <Implementation-Timestamp>${timestamp}</Implementation-Timestamp>
            </manifestEntries>
        </archive>
    </configuration>
</plugin>

3. Then we configured a Filter to read the values from this file on startup. If this file doesn't exist, a default version number of "1.0-SNAPSHOT-{random}" is used. Otherwise, the version is calculated as ${project.version}-${buildNumber}.

private String buildNumber = null;

...
@Override
public void initFilterBean() throws ServletException {
    try {
        InputStream is = 
            servletContext.getResourceAsStream("/META-INF/MANIFEST.MF");
        if (is == null) {
            log.warn("META-INF/MANIFEST.MF not found.");
        } else {
            Manifest mf = new Manifest();
            mf.read(is);
            Attributes atts = mf.getMainAttributes();
            buildNumber = atts.getValue("Implementation-Version") + "-" + atts.getValue("Implementation-Build");
            log.info("Application version set to: " + buildNumber);
        }
     } catch (IOException e) {
        log.error("I/O Exception reading manifest: " + e.getMessage());
     }
}

...

    // If there was a build number defined in the war, then use it for
    // the cache buster. Otherwise, assume we are in development mode 
    // and use a random cache buster so developers don't have to clear 
    // their browswer cache.
    requestVars.put("cachebust", buildNumber != null ? buildNumber : "1.0-SNAPSHOT-" + new Random().nextInt(100000));

4. We then used the "cachebust" variable and appended it to static asset URLs as indicated below.

<c:set var="version" scope="request" 
    value="${requestScope.requestConfig.cachebust}"/>
<c:set var="base" scope="request"
    value="${pageContext.request.contextPath}"/>

<link rel="stylesheet" type="text/css" 
    href="${base}/v/${version}/assets/css/style.css" media="all"/>

<script type="text/javascript" 
    src="${base}/v/${version}/compressed/jq.js"></script>

The injection of /v/[CACHEBUSTINGSTRING]/(assets|compressed) eventually has to map back to the actual asset (that does not include the two first elements of the URI). The application must remove these two elements to map back to the actual asset. To do this, we use the UrlRewriteFilter. The UrlRewriteFilter is used (instead of Apache's mod_rewrite) so when developers run locally (using mvn jetty:run) they don't have to configure Apache.

5. In our application, "/compressed/" is mapped to wro4j's WroFilter. In order to get UrlRewriteFilter and WroFilter to work with this setup, the WroFilter has to accept FORWARD and REQUEST dispatchers.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
    <filter-name>WebResourceOptimizer</filter-name>
    <url-pattern>/compressed/*</url-pattern>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>REQUEST</dispatcher>
</filter-mapping>

Once this was configured, we added the following rules to our urlrewrite.xml to allow rewriting of any assets or compressed resource request back to its "correct" URL.

<rule match-type="regex">
    <from>^/v/[0-9A-Za-z_.\-]+/assets/(.*)$</from>
    <to>/assets/$1</to>
</rule>
<rule match-type="regex">
    <from>^/v/[0-9A-Za-z_.\-]+/compressed/(.*)$</from>
    <to>/compressed/$1</to>
</rule>
<rule>
    <from>/compressed/**</from>
    <to>/compressed/$1</to>
</rule>

Of course, you can also do this in Apache. This is what it might look like in your vhost.d file:

RewriteEngine    on
RewriteLogLevel  0!
RewriteLog       /srv/log/apache22/app_rewrite_log
RewriteRule      ^/v/[.A-Za-z0-9_-]+/assets/(.*) /assets/$1 [PT]
RewriteRule      ^/v/[.A-Za-z0-9_-]+/compressed/(.*) /compressed/$1 [PT]

Whether it's a good idea to implement this in Apache or using the UrlRewriteFilter is up for debate. If we're able to do this with the UrlRewriteFilter, the benefit of doing this at all in Apache is questionable, especially since it creates a duplicate of code.

Posted in Java at Jun 04 2010, 09:27:42 AM MDT 4 Comments

What's New in Maven 3.0 with Matthew McCullough

Last night, I attended the Denver JUG meeting to hear some excellent talks by Matthew McCullough and Tim Berglund. I took notes during Matthew's talk, but my battery ran out before Tim's talk started. Below are my notes.

Matthew started out by described the differences between Maven 2 and Maven 3. As he began, he emphasized it wasn't a beginner talk, but mostly for existing Maven users that understand how to read a pom.xml and such.

The Roadmap
Commits to Maven 3 have been happening for the last 3 years. Matthew is not an employee of Sonatype, but he mentioned their name quite a bit in his talk. Sonatype has hired several committers (7 that Matthew knows of by name) that now work on Maven 3 full-time. For compatibility with Maven 2, the project has 450 integration tests and they test it against 100s of Maven 2 projects. Maven 3 has plugin classloader partitioning and a legacy simulation layer for old plugins.

The main improvement in Maven 3 is speed. It's been performance tuned to be 50% to 400% faster. Benchmarks (guaranteed by integration tests) include better: Disk I/O, Network I/O, CPU and Memory. Another new feature is extensibility so Maven is a better library rather than just a command-line tool. Now there's a library and APIs that you can use to do the things that Maven does. Plexus has been replaced with Guice and it's now much easier to embed Maven (Polyglot Maven and Maven Shell are examples of this).

Below are a number of other changes between Maven 2 and Maven 3.

  • Syntax: pom.xml still uses <modelVersion>4.0.0</modelVersion> so it can be a drop-in replacement for Maven 2 projects.
  • Validations: poms are heavily validated against common mistakes, warns when plugin versions are not specified (use mvn validate to see issues), blocks duplicate dependencies (examined in same POM only, conflict resolution used otherwise).
  • Help URLs: wiki page URLs now shown for all error messages. One of the first Apache projects to do this.
  • Removals: profiles.xml external file support removed, Maven 1.0 repository support removed <layout>legacy</layout> (it's been 5 years since any commits to Maven 1).
  • Behavior: SNAPSHOTs always deployed with date-stamps, artifact resolution caching has been improved to do less checking (override with mvn <phase> -U).
  • Plugins: version auto-selection favors RELEASEs over SNAPSHOTs (opposite for Maven 2), versions cannot be specified as RELEASE or LATEST, plugins only resolved from <pluginRepository> locations.
  • See the Plugin Compatibility Matrix to see if your favorite plugins are compatible.

Maven 3 hopes to be a drop-in replacement for Maven 2, but non backwards-compatible changes will be happening in Maven 3.1. It's anticipated release is Q1 of 2011 and will likely contain the following features.

  • "Mixins" for direct dependencies
  • Site plugin takes over <reporting>
  • Backwards compatibility by <modelVersion
  • There's a good chance 3.1 breaks compatibility with legacy POMs

Another new thing in Maven 3 is Toolchain. Toolchain a common way to configure your JDK for multiple plugins. There are only a handful of plugins that are toolchain-enabled. User tool chain definitions are defined in ~/.m2/toolchains.xml. To use different toolchains (JDKs), you specify a vendor and version as part of your plugin configuration.

Maven Shell is a high performance console that's a Maven 3 add-on. It's hosted at GitHub to make community contributions easier. It goes on your command line and it offers syntax highlighting and context-sensitive help (by typing ? at the command prompt).

Another major improvement in Maven 3 is Polyglot Maven. Tools like Gant and Buildr have made Maven look ancient, but they've also given it a good challenge. Maven 3 is likely to leapfrog these tools because of its ability to use different languages for your build configuration. Currently, 6 languages are supported. Polyglot Maven is a super-set distribution of Maven 3. It's not shipped with Maven 3 core because it contains all the other language implementations and is quite large. Polyglot Maven also contains a translate tool that allows you to convert any-to-any language. It has a DSL framework with Macros and Lifecycle Hooks. Macros allows for more concise syntax.

After talking about Polyglot Maven a bit, Matthew shows us a demo translating pom.xml to pom.yaml and then running the build. After that, he showed us examples of what a pom looks like when defined in Clojure, Scala and Groovy. Someone asked about file parsing performance and Matthew said different languages would cause a single-digit performance difference as part of your build process. Personally, I can't help but think any non-XML parser would be faster than the XML parser.

In regards to m2eclipse, a new drop (0.10) occurred a few weeks ago and it's one of the highest quality releases to date. It has major refactoring and many performance improvements.

For sample Maven projects see Matthew's Maven Samples.

I very much enjoyed Matthew's talk, both because of his presentation techniques and because he had a lot of good information. While I've tried Maven 3 and Shell in the past, I've been newly inspired to start using them again on a daily basis.

Tim's talk on Decision Making was also excellent. The biggest things I learned were that conflict is good (idea-wise, not personal) and things to look out for between teams (fault lines). Hopefully both Tim and Matthew post their slides so I can link to them here.

Posted in Java at May 13 2010, 03:54:21 PM MDT 1 Comment

Software Quality: The Quest for the Holy Grail?

This afternoon, I attended a session on software quality by Jesper Pedersen at TSSJS. Jesper is a Core Developer at JBoss by Red Hat. He's the project lead of JBoss JCA, Tattletale, Papaki and JBoss Profiler 2. He's also currently the chairman of the Boston JBoss User Group. In this session, Jesper hopes to define basic requirements for a development environment and offer ideas on how to clean up a messy project.

Is software quality a friend or a foe? By implementing software quality processes, are you introducing unnecessary overhead? Development platforms are different today. We write a lot more business-specific code. We depend on standard frameworks for the core functionality. We depend on vendors to implement the standards correctly. We also depend on vendors to provide the necessary integration layer.

Since the platform is now a large slice of the pie, we must make sure we know where the issue is located. We must have proper integration tests; we must manage dependencies. Today, we must treat dependencies as if they are part of the application.

Defining the platform for your project helps narrow down the dependencies for your project. The platform is composed of corporate guidelines, standards, vendors and backend systems that you have to integrate with. Documentation is key for a successful project. Key documents types: User Guide, Developer Guide, API Guide, Architect Design, Implementation and Test.

It helps to define a project-wide configuration management system. Define a code-formatting guide will add consistency in your source tree. Also make sure you have separate build, test and execution environments. Use a Maven repository for your dependencies; both to support your project's artifacts as well as vendor artifacts.

"Maven today is an industry standard." -- Jesper Pederson

Define your tool chain as you would for your application. Back your Maven repository with SCM tools like Subversion or Git. For testing, use JUnit (unit testing), Corbertura (test coverage) and Hudson (continuous integration). Furthermore, you can add Checkstyle and Findbugs to verify coding conventions and find basic issues with code.

For the build environment, you need to make sure your dependency metadata is correct. Also, make sure you use the best practices for your given build tool. For example, with Maven and Ivy, it's a good idea to extract the version numbers into a common area of your pom.xml/build.xml so you can easily view all the versions in use. After you've done this, you can externalize the version information from the environment. Watch out for transitive dependencies as they can be tricky. Make sure you know what gets pulled in. Use enforcers to approve/ban dependencies or turn it off (Ivy). You can also vote for MNG-2315. Finally, snapshot dependencies are evil: define your release process so that releases are easy.

What can you do if your project is already a mess? Signs that your project is a mess: you look at your platform as a big black box, you use different dependencies than your deployment platform or you don't have integration tests for sub-systems or dependencies. To fix this, you can use a tool to get an overview of the entire project. Enter Tattletale.

Tattletale can give you an overview of your dependencies (Ant and Maven integration). It's a static analysis tool that doesn't depend on metadata, scanning your class files instead. Using Tattletale, you can produce a number of reports about your dependencies, what they're dependent on and what's dependent on you.

To maintain the lead in your project, make sure to define a checklist for each stage of your development cycle. Do reviews on documentation, architecture, component design and code. Enforce your rules of your project with your build system.

Jesper's final thoughts:

  • Maintaining dependencies for a software project can be a tricky task.
  • Using an Open Source platform as the foundation will ease the investigation of issues and increase trust.
  • Defining a project-wide tool chain is key.
  • Enforce all the rules on the project (better up-front than "fixing it" afterwards)

As Dusty mentioned, this session has a lot of good (basic) information, but there wasn't much new stuff. My team is using many of the technologies and practices that Jesper has mentioned. I guess that's validation that we're doing it right. I've heard of Tattletale, but never had a need for it since I haven't been on any "messy" projects recently.

Posted in Java at Mar 17 2010, 03:00:46 PM MDT 2 Comments

My Future of Web Frameworks Presentation

Earlier this week, I tweeted about a history of web frameworks timeline I created for my upcoming Future of Web Frameworks talk at TSSJS Vegas 2010. I immediately received a lot of feedback and requests for adding new frameworks and releases. The image below is the result of that Twitter conversation. Thanks to everyone who contributed.

History of Web Frameworks

Back in November, I wrote about my proposals for TSSJS. I've been thinking a lot about web frameworks lately and I can't help but think we live in a very exciting time. As a Java developer, I've been exposed to one of the most vibrant language ecosystems on the planet. As Tim Bray talks about, the Java Platform has 3 legs: the language, the virtual machine and a huge, immense library of APIs (both in the JDK and in open source libraries). The diagram below is something I created based on Tim's podcast.

Java has 3 Legs

Tim says, "One of those legs is replaceable and that's the language." And he's right, there's many Java.next languages that run efficiently on the JVM. This is one of the most exciting parts of being a Java web developer today. There's many proven web frameworks and languages that you can pick to build your next web application.

The best part is many of the best web frameworks run on the JVM. Not only that, but the best code editors are the IDEs that you're familiar with and have grown to love. Furthermore, much of the literature for Java.next languages is written for Java developers. As someone who knows Java, you have wealth of web frameworks and languages just waiting for you to learn them.

To create my presentation on the future of web frameworks, I followed the outline I posted previously. I plan on explaining the evolution and history of web frameworks and how we got to where we are today. From there, I'll be speculating on what web applications we'll be developing in the future. Finally, I'll touch on the necessary features of web frameworks that will allow us to develop these applications.

Of course, I haven't actually presented this talk yet, so it's likely to change in the coming weeks before the conference. The good news is this gives you the opportunity to provide constructive criticism on this presentation and help make it better. I realize that a presentation rarely represents the conversation that takes place during a conference. However, I believe it can portray the jist of my thinking and lead to a meaningful conversation in the comments of this post. Below is the presentation I created - thanks in advance for any feedback.

For those who will be joining me at TSSJS ... it's gonna be a great show. St. Patrick's Day in Vegas, what more could you ask for? ;-)

Update: This article has been re-posted on Javalobby and contains additional community feedback in the comments.

Posted in Java at Feb 26 2010, 08:55:39 AM MST 5 Comments

Using JRebel with IntelliJ IDEA on OS X

Yesterday afternoon, I figured out how to use JRebel with IntelliJ IDEA. I wrote up some AppFuse documentation for it at Using JRebel with IntelliJ IDEA and figured I'd repost it here for those developers using IDEA 9 and Maven.

  1. Download and install IntelliJ IDEA 9 Ultimate Edition (in /Applications on OS X).
  2. Download and install JRebel.
    1. java -jar jrebel-setup.jar
    2. Install the JRebel Plugin for IDEA. Shortcut: File > Settings > Search for plugins and find JRebel.
  3. On OS X, Modify /etc/launchd.conf and add the following so M2_HOME is available to GUI apps. You'll need to reboot after making this change.
    setenv M2_HOME /opt/tools/maven2
    

    More info on this setting is available on Stack Overflow.

  4. Modify your project's pom.xml to include the JRebel Maven plugin (for generating the configuration of which directories and files to watch for changes).
    <plugin>
        <groupId>org.zeroturnaround</groupId>
        <artifactId>javarebel-maven-plugin</artifactId>
        <version>1.0.5</version>
        <executions>
            <execution>
              <id>generate-rebel-xml</id>
              <phase>process-resources</phase>
              <goals>
                  <goal>generate</goal>
              </goals>
            </execution>
        </executions>
    </plugin>
    
  5. If you're using the Maven Jetty plugin, change your pom.xml so Jetty doesn't reload the app when classes are compiled. Specifically, change scanIntervalSeconds to 0. If you're not using this plugin, you should definitely check it out for Java webapp development.
  6. Use the JRebel icons to start jetty:run in your IDE. JRebel with IntelliJ IDEA
  7. Command Line Integration: Set a JREBEL_HOME environment variable that points to your JRebel install (/Applications/ZeroTurnaround/JRebel on OS X) and set your MAVEN_OPTS to use JRebel's settings. For example:
    export JAVA_OPTS="-Xmx512M -XX:PermSize=256m -XX:MaxPermSize=512m -Djava.awt.headless=true"
    export JREBEL_HOME=/Applications/ZeroTurnaround/JRebel 
    export MAVEN_OPTS="$JAVA_OPTS -noverify -javaagent:$JREBEL_HOME/jrebel.jar"
    

After making these changes, you should able to compile classes in IDEA and refresh your browser. Log messages like the following should show up in your console.

JRebel: Reloading class 'org.appfuse.webapp.action.UserAction'.

To simplify things further, you can map Command+S to compile (instead of Shift+F9). Just look for Keymaps in Settings, rename the default one and search for Compile to remap.

I'm assuming the steps to make things work on Windows and Linux are similar. Please let me know if you have any issues with these instructions.

Posted in Java at Feb 02 2010, 10:34:08 AM MST 3 Comments

What's your preferred development infrastructure stack?

Over the years, I've used many different source control systems, wikis, bug trackers and continuous integration servers. On many projects, I've been responsible for recommending and helping to install these systems. For the most part, they've often been disparate, meaning there wasn't a whole lot of integration between the various applications. Here's a list of all the different systems I've used:

I believe all of these applications are useful in supporting an efficient development process. When clients have asked me to help them build this type of infrastructure, I've often asked if they wanted to pay for it or not. If not, I'd recommend Trac (since it has a wiki, source viewer and bug tracker all-in-one) and Hudson. If they were willing to pay, I'd recommend the Atlassian Suite (Confluence, JIRA and Bamboo).

These stacks all seem to work pretty well and the Atlassian Suite certainly works great for AppFuse and other open source projects. However, I recently had the pleasure of working at Chordiant Software where we used Chordiant Mesh to collaborate and develop software. Their Mesh system is powered by Jive Clearspace and provides a wealth of tools for each project, including a dashboard, discussions, documents, notifications and widgets providing status + links to JIRA and Bamboo.

Even though Clearspace's rich text editor caused me some early frustration, I really enjoyed the fact that a solid development infrastructure existed. It made it much easier to collaborate, document and execute our development process. I realize that it's difficult to build and maintain a custom development infrastructure stack. Chordiant had a whole team that developed, enhanced and supported their environment. But that doesn't mean it's impossible and not worth striving for.

I think there's a number of best-of-breed applications you can use to build a sweet development infrastructure stack.

  • Source Control: Git
  • Source Viewer: FishEye
  • Wiki: Jive SBS
  • Bug Tracker: JIRA
  • Continuous Integration: Hudson

I've only used Git for a few weeks, but I can easily tell it's better than Subversion. I don't think it's easy to convince companies to switch their source control system, so it's probably not worth arguing if you're already using Subversion. I can also envision using Confluence instead of Jive SBS, but then you lose forum support and have to use something like Mailman or Google Groups. JIRA Studio looks close to my dream stack, except it doesn't support Git or a forum + mailing list system.

What is your preferred development infrastructure stack? Why?

Posted in Java at Jan 12 2010, 09:54:46 PM MST 30 Comments

GWT OAuth and LinkedIn APIs

LinkedIn Logo When I worked at LinkedIn last year, I received a lot of inquiries from friends and developers about LinkedIn's APIs. After a while, I started sending the following canned response:

For API access to build LinkedIn features into your application, fill out the following form:

   http://www.linkedin.com/static?key=developers_apis

For requests to build an application, go to:

   http://www.linkedin.com/static?key=developers_opensocial

I talked with the API team and they did say they look at every request that's sent via these forms. They don't respond to all of them b/c they know that many people would be angry if they told them "no", so they'd rather not have that headache.

Yesterday, I was pumped to see that they've finally decided to open up their API to Developers.

Starting today, developers worldwide can integrate LinkedIn into their business applications and Web sites. Developer.linkedin.com is now live and open for business.

First of all, congratulations to the API team on finally making this happen! I know it's no small feat. Secondly, it's great to see them using Jive SBS for their API documentation and developer community. My current client uses this to facilitate development and I love how it integrates a wiki, JIRA, FishEye, Crucible and Bamboo into one central jumping off point.

I've always been a fan of LinkedIn, ever since I joined way back in May 2003. However, I've longed for a way to access my data. LinkedIn Widgets are nice, but there's something to be said for the full power of an API. Last night, I sat down for a couple hours and enhanced my Implementing OAuth with GWT example to support LinkedIn's API.

I'm happy to report my experiment was a success and you can download GWT OAuth 1.2 or view it online. For now, I'm simply authenticating with OAuth and accessing the Profile API.

OAuth with GWT

In the process, I learned a couple things:

// For LinkedIn's OAuth API, convert request parameters to an AuthorizationHeader
if (httpServletRequest.getRequestURL().toString().contains("linkedin-api")) {
    String[] parameters = httpServletRequest.getQueryString().split("&");
    StringBuilder sb = new StringBuilder("OAuth realm=\"http://api.linkedin.com/\",");
    for (int i = 0; i < parameters.length; i++) {
        sb.append(parameters[i]);
        if (i < parameters.length - 1) {
            sb.append(",");
        }
    }

    Header authorization = new Header("Authorization", sb.toString());
    httpMethodProxyRequest.setRequestHeader(authorization);
}

You might recall that my previous example had issues authenticating with Google, but worked well with Twitter. LinkedIn's authentication seems to work flawlessly. This leads me to believe that Twitter and LinkedIn have a much more mature OAuth implementation than Google.

Related OAuth News: Apache Roller 5 will be shipping with OAuth support. See Dave Johnson's What's New in Roller 5 presentation for more information.

Update December 6, 2009: I modified the gwt-oauth project to use GWT 1.7.1 and changed to the Maven GWT Plugin from Codehaus. Download GWT OAuth 1.3 or view it online.

Posted in The Web at Nov 24 2009, 03:46:05 PM MST 7 Comments

AppFuse 2.1 Milestone 1 Released

The AppFuse Team is pleased to announce the first milestone release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Hibernate, Spring and Tapestry 5.

What is AppFuse?
AppFuse is an open source project and application that uses open source tools built on the Java platform to help you develop Web applications quickly and efficiently. It was originally developed to eliminate the ramp-up time found when building new web applications for customers. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose which persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run appfuse:full-source.

In addition, AppFuse Light has been converted to Maven and has archetypes available. AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket.

Other notable improvements:

Please note that this release does not contain updates to the documentation. Code generation will work, but it's likely that some content in the tutorials won't match. For example, you can use annotations (vs. XML) for dependency injection and Tapestry is a whole new framework. I'll be working on documentation over the next several weeks in preparation for Milestone 2.

AppFuse is available as several Maven archetypes. For information on creating a new project, please see the QuickStart Guide.

To learn more about AppFuse, please read Ryan Withers' Igniting your applications with AppFuse.

The 2.x series of AppFuse has a minimum requirement of the following specification versions:

  • Java Servlet 2.4 and JSP 2.0 (2.1 for JSF)
  • Java 5+

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find bugs, please create an issue in JIRA.

Thanks to everyone for their help contributing code, writing documentation, posting to the mailing lists, and logging issues.

Posted in Java at Nov 19 2009, 07:16:36 AM MST 8 Comments