Raible Designs

RE: Moving from Spring to Java EE 6: The Age of Frameworks is Over

Last Tuesday, Cameron McKenzie wrote an interesting article on TheServerSide titled Moving from Spring to Java EE 6: The Age of Frameworks is Over. In this article, Cameron says the following:

J2EE represents the past, and Java EE 6 represents the future. Java EE 6 promises us the ability to go beyond frameworks. Frameworks like Spring are really just a bridge between the mistakes of the J2EE past and the success of the Java EE 6 future. Frameworks are out, and extensions to the Java EE 6 platform are in. Now is the time to start looking past Spring, and looking forward to Seam and Weld and CDI technologies.

He then links to an article titled Spring to Java EE - A Migration Experience, an article written by JBoss's Lincoln Baxter. In this article, Lincoln talks about many of the technologies in Java EE 6, namely JPA, EJB, JSF, CDI and JAX-RS. He highlights all the various XML files you'll need to know about and the wide variety of Java EE 6 application servers: JBoss AS 6 and GlassFish v3.

I don't have a problem with Lincoln's article, in fact I think it's very informative and some of the best documentation I've seen for Java EE 6.

I do have some issues with Cameron's statements that frameworks are mistakes of the J2EE past and that Java EE 6 represents the future. Open source frameworks made J2EE successful. Struts and Hibernate came out in the early days of J2EE and still exist today. Spring came out shortly after and has turned into the do-everything J2EE implementation it was trying to fix. Java EE 6 might be a better foundation to build upon, but it's certainly not going to replace frameworks.

To prove my point, let's start by looking at the persistence layer. We used to have Hibernate based on JDBC, now we have JPA implementations built on top of the JPA API. Is JPA a replacement for all persistence frameworks? I've worked with it and think it's a good API, but the 2.0 version isn't available in a Maven repo and Alfresco recently moved away from Hibernate (which == JPA IMO) to iBATIS for greater data access layer control and scalability. Looks like the age of frameworks isn't over for persistence frameworks.

The other areas that Java EE 6 covers that I believe frameworks will continue to excel in: EJB, CDI, JSF and JAX-RS. Personally, I don't have a problem with EJB 3 and think it's a vast improvement on EJB 2.x. I don't have an issue with CDI either, and as long as it resembles Guice for dependency injection, it works for me. However, when you get into the space I've been living in for the last couple years (high-traffic public internet sites), EJB and things like the "conversation-scope" feature of CDI don't buy you much. The way to make web application scale is to eliminate state and cache as much as possible, both of which Java EE doesn't provide much help for. In fact, to disable sessions in a servlet-container, you have to write a Filter like the following:

public class DisabledSessionFilter extends OncePerRequestFilter {

    /**
     * Filters requests to disable URL-based session identifiers.
     */
    @Override
    protected void doFilterInternal(final HttpServletRequest request,
                                    final HttpServletResponse response,
                                    final FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequestWrapper wrappedRequest = new HttpServletRequestWrapper(request) {

            @Override
            public HttpSession getSession(final boolean create) {
                if (create) {
                    throw new UnsupportedOperationException("Session support disabled");
                }
                return null;
            }

            @Override
            public HttpSession getSession() {
                throw new UnsupportedOperationException("Session support disabled");
            }
        };

        // process next request in chain
        chain.doFilter(wrappedRequest, response);
    }
}

What about JAX-RS? Does it replace the need for frameworks? I like the idea of having a REST API in Java. However, its reference implementation is Jersey, which seems more like a framework than just Java EE. If you choose to use JAX-RS in your application, you still have to choose between CXF, Jersey, RESTEasy and Restlet. I compared these frameworks last year and found the Java EE implementation lacking in the features I needed.

Finally, let's talk about my-least-framework-web-framework: JSF. The main reason I don't like JSF is because of its 1.x version. JSF 1.0 was released a year before the Ajax term was coined (see timeline below). Not only did it take forever to develop as a spec, but it tried to be a client-component framework that was very stateful by default.

Now that JSF 2.0 is out, it has Ajax integrated and allows you to use GET instead of POST-for-everything. However, the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn't be developing your UI). Also, the best component development platform for the web is JavaScript. I recommend using an Ajax framework for your components if you really want a rich UI.

Sure you can use the likes of Tapestry and Wicket if you like POJO-based web development, but if you're looking to develop a webapp that's easy to maintain and understand, chances are that you'll do much better with traditional MVC frameworks like Spring MVC and Struts 2. The simplicity and popularity of Rails and Grails further emphasize that developers prefer these types of web frameworks.

Another reason I don't like JSF: there's very few developers in the wild happy with it. The major promoters of JSF are book authors, trainers, Java EE Vendors and MyFaces developers. Whenever I speak at conferences, I ask folks to raise their hands for the various web frameworks they're using. I always ask the JSF users to keep their hands up if they like it. Rarely do they stay up.

So it looks like we still need web frameworks.

Eberhard Wolff has an interesting post where he defends Spring and talks about the productivity comparisons between Spring and Java EE. He recommends using Grails or Spring Roo if you want the level of productivity that Ruby on Rails provides. That's a valid recommendation if you're building CRUD-based webapps, but I haven't developed those in quite some time. Nowadays, the apps I develop are true SOFEA apps, where the backend serves up XML or JSON and the frontend client is HTML/JavaScript/CSS, Android, iPad or Sony Blu-Ray players. On my current project, our services don't even talk to a database, they talk to a CMS via RESTful APIs. We use Spring's RestTemplate for this and HttpClient when it doesn't have the features we need. Not much in Java EE 6 for this type of communication. Sure, Jersey has a client, but it's certainly not part of the Java EE spec.

As far as getting Ruby on Rails' zero-turnaround productivity, I don't need Grails or Spring Roo, I simply use IDEA and JRebel.

Conclusion
I don't see how new features in Java EE 6 can mean the age of frameworks is over. Java SE and J2EE have always been foundations for frameworks. The Java EE 6 features are often frameworks in themselves that can be used outside of a Java EE container. Furthermore, Java EE 6 doesn't provide all the features you need to build a high-scale web app today. There's no caching, no stateless web framework that can serve up JSON and HTML and no hot-reload productivity enhancements like JRebel. Furthermore, there's real excitement in Javaland for languages like Scala, Groovy and JRuby. All of these languages have web frameworks that've made many developers happy.

Here's to the Age of Frameworks - may it live as long as the JVM!

P.S. If you'd like to hear me talk about web frameworks on the JVM, I'll be speaking at The Colorado Springs Open Source Meetup and Devoxx 2010 in the near future.

Happy 8th Birthday to this blog!

Eight years ago today, this blog was born in the wee hours of the morning. I was inspired to start it after reading Dave Johnson's article on Roller. I have to say, it's been a great ride and I remember the early days like they were yesterday. Many of the Java bloggers wrote daily and shared short tips, tricks and snippets on their blogs; much in the same way we do on Twitter today.

A lot has happened in my life since this blog was started: Abbie was born, Jack was born, I started AppFuse, wrote Spring Live, had some really cool gigs and gained a whole new perspective on my life.

For those long time readers, you might've noticed the vacation posts have picked up recently and the technology posts have subsided somewhat. The good news is this indicates I'm having a lot of fun; the bad news is I'm not learning as much as I'd like. Hopefully that'll change soon and I'll be writing about developing apps for the online video space in the near future. There's a good chance the posts about my life and how much fun I'm having will continue, especially as Abbie and Jack continue to grow into world-class skiers.

As usual, I have many ambitions for this fall, including helping Apache Roller, finishing AppFuse 2.1 and learning how to play the guitar. Along the way, I'll be helping build/release some kick-ass software for a major cable provider, building a sauna in my basement and enjoying the hell out of Devoxx 2010. You can sure I'll be blogging about these along the way, as well as many years into the future.

Thanks for reading all these years, it's been a fantastic experience.

Scaling Flash Movies to match Browser Zoom Levels

Recently I was tasked with figuring out how to scale the Flash assets in the web application I'm working on. In the app, there's two different Flash assets: a Spotlight (cycles through images) and a Video Player. Before I started working on the issue, our assets would stay the same fixed size no matter what the browser zoom level. You can see this issue in action by going to Hulu or Fancast and zooming in/out (Command +/- on Mac, Control +/- on Windows). The Flash assets don't scale with the browser's text.

I found a lot of references for how to trap and handle resizing in JavaScript, so that's the initial path I took. I ended up having issues trapping the resize event in IE, as well as persisting the appropriate zoom level on page reload. Because of this, I ended up using a pure ActionScript solution that works much better. This article shows how I implemented both solutions.

Regardless of implementation, the first change I had to make was to move the height and width from the Flash asset (object/embed/JS) to its surrounding tag (<section> in our app). Then I changed the height/width to 100% on the Flash asset.

JavaScript Implementation
To allow zooming in ActionScript, I modified our main class to expose a "zoom" method to JavaScript:

ExternalInterface.addCallback("zoom", _zoom);

...

private function _zoom(scale:Number):void {
    _view.scaleX = _view.scaleX * scale;
    _view.scaleY = _view.scaleY * scale;
}

var windowHeight;
var documentHeight;

$(document).ready(function() { 
    ...
    windowHeight = $(window).height();
    documentHeight = $(document).height();

    $(window).resize(resizeWindow);
}

// Resize Flash assets when page is zoomed
function resizeWindow() {
    var newWindowHeight = $(window).height();
    var newDocumentHeight = $(document).height();
    // if document height hasn't changed, it's a browser window resize
    // event instead of a text zoom - don't change anything
    if (newDocumentHeight === documentHeight) {
        return;
    } else {
        documentHeight = newDocumentHeight;
    }
    var scale = (windowHeight / newWindowHeight); 

    var player = getFlashMovie('playerId');
    if (player && player.zoom) {
        player.zoom(scale);
    }
    var spotlight = getFlashMovie('spotlightId');
    if (spotlight && spotlight.zoom) {
        spotlight.zoom(scale);
    }

    windowHeight = newWindowHeight;
}

This seemed to work well in Firefox, Safari and Opera, but not in IE. I found this explanation about why it might not work, but I was unsuccessful in getting IE to recognize a resize/zoom event.

To fix scaling in our Spotlight asset, I used a similar solution. However, since the Spotlight didn't have all its elements in a container (they were being added directly to the stage), I had to refactor the code to add a SpotlightView (extends Sprite) that contains the bulk of the code.

Browsers persist the zoom level you've set for a site. The problem with the solution used here is it only scales up and down properly if you start from scale = 1 and revert to scale = 1 before leaving the site. If you zoom in and close your browser, when you come back the flash movies will be scale = 1 while the rest of the site is zoomed in. To solve this problem, I attempted to save the scale value in a cookie. This worked, and I was able to read the cookie in the *.as files to scale the movie correctly. However, I experienced some issues with this approach and didn't like having to delete cookies when I wanted the Flash assets to scale correctly.

ActionScript Implementation
After discovering issues with the JavaScript implementation, I did some research to see if it was possible to listen for the browser resize event in ActionScript. The Flash Fluid Layouts and Stage Resize in AS3 tutorial clued me in that the stage could listen for a resize event.

stage.addEventListener(Event.RESIZE, resizeListener); 

After adding the above line in the initialization, I added a resizeListener function that scales based on the default dimensions. It also ensures no scaling happens in full screen mode.

private function resizeListener(e:Event):void {
    // don't scale if entering full screen mode
    if (stage.displayState == StageDisplayState.FULL_SCREEN)  {
        _view.scaleX = 1;
        _view.scaleY = 1;
    } else {
        _view.scaleX = stage.stageWidth / 964;
        _view.scaleY = stage.stageHeight / 586;
    }
}

For the Spotlight asset, there are a number of different layouts (home, featured and news). The main class has a resizeListener function that scales accordingly to which layout type is being used.

private function resizeListener(e:Event):void {
    var type:String = _view.getLayoutType();

    if (type == "featured") { 
        _view.scaleX = stage.stageWidth / 958;
       _view.scaleY = stage.stageHeight / 180;
   } else if (type == "home") { 
        _view.scaleX = stage.stageWidth / 964;
        _view.scaleY = stage.stageHeight / 428;
    } else if (type == "news") {
        _view.scaleX = stage.stageWidth / 964;
        _view.scaleY = stage.stageHeight / 189;
    }
}

Because the layout type isn't set until the XML is loaded, I listen for that event in my URLLoader.

xmlLoader.addEventListener(Event.COMPLETE, resizeListener);

With the pure ActionScript implementation, the zoom level is automatically persisted. The Event.RESIZE event is fired by the Flash plugin when the page first loads if the dimensions are not the default.

That's it! Special thanks to James Ward for clueing me into scaleX and scaleY. Hopefully Hulu and Comcast can use this tutorial to scale their video players too.

My Incredible Trip to Ireland

If you ever get a chance to travel to Ireland, take it! I don't know when I heard these words, or how they came into my head, but I remembered them clearly when I was first introduced to Barry Alistair by Jeff Genender. Soon after, I was able to negotiate my way into being a speaker at The 2010 Irish Software Show.

The show was last week and I had a blast traveling to Dublin to speak and explore. My sister came me on this trip, but missed a connection in Seattle and had to join me a day late. I left Denver at noon on Monday and arrived at Dublin Airport at 7 am. I was on the same flight as Josh Long and thoroughly enjoyed my iPad as a travel companion. When I got off the plane, my battery life was at 60% and I'd been watching movies and listening to music for 6 hours.

I took a cab through the misty, cool morning to my hotel. I grabbed a coffee, cleaned up, and walked a few blocks to Trinity College for the conference. I made it in time for the opening keynote by Chris Horn. It was an interesting talk, focusing on what needed to happen to make Ireland the Innovation Hub of Europe. After that, I attended Tim Berglund's session on Complexity Theory and Software Development. After lunch and a few more talks, I teamed up with Andres Almiray and Josh Long for a pint at the hotel bar.

That evening, we attended Jeff Genender's talk on Getting into Open Source. The free drinks loosened everyone up and Jeff did a great job with a humorous presentation on how to get Committer Status. After Jeff's talk, about 10 of us headed to a Moroccan restaurant for a late dinner. I was in bed around midnight.

Wednesday morning, my sister arrived in my hotel room at 8 and promptly fell into bed. I set my alarm to sleep an hour and closed the Vegas-style, no-light-allowed curtains. We awoke much later (12:30) than we'd planned (9:00). We quickly got up and headed for some sight-seeing in Dublin. First off, we hit Dublinia and Christ Church Cathedral. Both sites were spectacular and we both learned a lot about the history of Dublin. From there, we skipped across the bridge to The Old Jameson Distillery for a tour and a bit of whiskey.

The picture below was taken on the Ha'penny Bridge as we were heading back from Jameson. The expression of the girl on the left is priceless.

A couple hours later and I was delivering my talk on The Future of Web Frameworks. The crowd was lively; the Guinness I drank while talking was lovely. My session was followed by a Web Framework Experts Panel with Peter Ledbrook (Grails), Jamie van Dyke (Rails), Shay Friedman (ASP.NET MVC), Julian Fitzell (Seaside) and myself (Java Frameworks). The debate was good and there was much discussion about the right apps for each framework and how important statelessness is for scalable applications. After 3 hours of talking, my sister and I headed back to the hotel. I was particularly happy about the evening since it was the first time a family member of mine had seen me speak.

Correction from my Dad: This wasn't the first time a family member saw me speak. He attended my talk at ApacheCon EU 2007.

A block from the hotel, we spotted a nice looking pub (Doyles) and stopped in for a pint. As we bellied up to the end of the bar, we recognized Jamie (from the panel) and got introduced to his friend Rob. We quickly got lost in conversation, stories and laughter and were surprised when we discovered it was 2:30am. Since I had a talk first thing in the morning, we ducked out shortly after.

Thursday started with my talk Comparing Kick-Ass Web Frameworks. Then my sister and I did some more site-seeing, starting at the Guinness Storehouse. We met Josh and John Willis as they were leaving and they advised we go straight to The Gravity Bar at the top. We took there advise and were getting great views of Dublin and savoring sweet pints of Guinness moments later. The tour facility was freakin' awesome and I loved how it was shaped like a pint glass.

We grabbed some gear from the gift shopped and landed (by accident) at The Brazen Head (Ireland's Oldest Pub, Est. 1198) for a pint of cider and Guinness. Since my sister used to be in the cider business, she was particularly happy there was so much on tap in Ireland.

From the pub, we headed to John Willis's session on The Cambrian Cloud Explosion. Following John's session, we headed to the Speaker's Dinner for a very fun evening with the hosts and speakers of the conference.

On Friday, we woke up in the early afternoon and quickly decided the Book of Kells was our best chance of getting some site seeing in. After visiting the Book of Kells, my favorite quote of the conference happened in the courtyard.

Josh looked at Jamie (with his bad hangover) and exclaimed, "My God Man. Your skin is so white it's hurting my eyes!". You probably had to be there (or know Josh) to enjoy the humor, but I wanted to capture the memory in this post so I could laugh whenever I read this in the future. After that, Jamie, Josh, Kalin and I enjoyed a Starbuck's patio talking about living in the South of France for a couple hours. Then we walked 2 blocks to the Porterhouse Brewing Co. to watch the World Cup and enjoy more interesting conversations.

Jamie left the conference that evening and we joined a whole slew of other speakers for dinner at an excellent Lebanese restaurant near Temple Bar. Good times where had afterwards at a nearby Silent Disco.

Saturday, we woke up early to catch a tour bus out to Glendalough with Josh and John. The bus ride was not pleasant, but the destination was spectacular. We hung out there for several hours, exploring the buildings, walking to the lake and humoring each other.

Our last night in Dublin was an early, relaxing one. As you can tell, I really enjoyed this trip, particularly hanging out with my sister and all the cool people we met. I can easily say that this trip registers as one of my favorite conference experiences to date.

To see all the pictures I took on this trip, check out my Irish Software Show 2010 set on Flickr.

My Presentations from The Irish Software Show 2010

This week I've been enjoying Dublin, Ireland thanks to the 2nd Annual Irish Software Show. On Wednesday night, I spoke about The Future of Web Frameworks and participated in a panel with Grails, Rails, ASP.NET MVC and Seaside developers. It was a fun night with lots of lively discussion. Below is my presentation from this event.

This morning, I delivered my Comparing Kick-Ass Web Frameworks talk. This presentation contains updated statistics for various metrics comparing Rails vs. Grails and Flex vs. GWT.

Thanks to all who attended my talks this week!

P.S. I believe audio was recorded on Wednesday night, but I'm unsure how it turned out. I'm pretty sure no recordings were done on this morning's session.

Running Selenium Tests on Sauce Labs

Recently I embarked on a mission to configure my team's Selenium testing process to support multiple browsers. We use Hudson for our continuous integration server. Since our Hudson instance runs on Solaris, testing with Firefox on Solaris didn't seem like a good representation of our clients. Our browser support matrix currently looks as follows:

At first, I attempted to use Windows VMs to run Selenium tests on IE. This was a solution that didn't work too well. The major reasons it didn't work:

1. I had issues getting the Selenium Plugin for Hudson working. Upgrading the plugin to use Selenium RC 1.0.5 may solve this issue.
2. We had some unit tests that failed on Windows. I tried using the Cygpath Plugin for Hudson (which allows you to emulate a Unix environment on Windows), but failed to get it to work.
3. We quickly realized it might become a maintenance nightmare to keep all the different VMs up-to-date.

Frustrated by these issues, I turned to Sauce Labs. They have a cloud-based model that runs Selenium tests on VMs that point back to your application. They also support many different browser/OS combinations. We asked them about support for OS X and various Windows versions and they indicated that their experience shows browsers are the same across OSes.

I'm writing this article to show you how we've configured our build process to support 1) testing locally and 2) testing on Sauce Labs. In a future post, I hope to write about how to run Selenium tests concurrently for faster execution.

Running Selenium Tests Locally
We use Maven to build our project and run our Selenium tests. Our configuration is very similar to the poms referenced in Integrating Selenium with Maven 2. Basically, we have an "itest" profile that gets invoked when we pass in -Pitest. It downloads/starts Tomcat (using Cargo), deploys our WAR, starts Selenium RC (using the selenium-maven-plugin) and executes JUnit-based tests using the maven-surefire-plugin. All of this configuration is pretty standard and something I've used on many projects over the past several years.

Beyond that, we have a custom BlockJUnit4ClassRunner class that takes screenshots and captures the HTML source for tests that fail.

public class SeleniumJUnitRunner extends BlockJUnit4ClassRunner {
    public SeleniumJUnitRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    protected Statement methodInvoker(FrameworkMethod method, Object test) {
        if (!(test instanceof AbstractSeleniumTestCase)) {
            throw new RuntimeException("Only works with AbstractSeleniumTestCase");
        }

        final AbstractSeleniumTestCase stc = ((AbstractSeleniumTestCase) test);
        stc.setDescription(describeChild(method));

        return new InvokeMethod(method, test) {
            @Override
            public void evaluate() throws Throwable {
                try {
                    super.evaluate();
                } catch (Throwable throwable) {
                    stc.takeScreenshot("FAILURE");
                    stc.captureHtmlSource("FAILURE");
                    throw throwable;
                }
            }
        };
    }
}

To use the functionality SeleniumJUnitRunner provides, we have a parent class for all our tests. This class uses the @RunWith annotation as follows:

@RunWith(SeleniumJUnitRunner.class)
public abstract class AbstractSeleniumTestCase {
    // convenience methods
}

This class looks up the Selenium RC Server, the app location and what browser to use based on system properties. If system properties are not set, it has defaults for running locally.

public static String SERVER = System.getProperty("selenium.server");
public static String APP = System.getProperty("selenium.application");
public static String BROWSER = System.getProperty("selenium.browser");

protected Selenium selenium;

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    selenium = new DefaultSelenium(SERVER, 4444, BROWSER, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

The system properties are specified as part of the surefire-plugin's configuration. The reason we default them in the above code is so tests can be run from IDEA as well.

<artifactId>maven-surefire-plugin</artifactId>
<version>2.5</version>
<configuration>
    <systemPropertyVariables>
        <selenium.application>${selenium.application}</selenium.application>
        <selenium.browser>${selenium.browser}</selenium.browser>
        <selenium.server>${selenium.server}</selenium.server>
    </systemPropertyVariables>
</configuration>

Running Selenium Tests in the Cloud
To run tests in the cloud, you have to do a bit of setup first. If you're behind a firewall, you'll need to setup SSH tunneling so Sauce Labs can see your machine. You'll also need to setup SSH Tunneling on your Hudson server, but installing/configuring/running locally is usually a good first step. Below are the steps I used to configure Sauce Labs' SSH Tunneling on OS X.

1. Install the Python version in /opt/tools/saucelabs. If you get an error (No local packages or download links found for install) download the egg and run it with:

sudo sh setuptools-0.6c11-py2.6.egg

NOTE: If you get an error (unable to execute gcc-4.2: No such file or directory) when installing pycrypto on OS X, you'll need to install the OS X Developer Tools.

2. Create a /opt/tools/saucelabs/local.sh script with the following in it. You should change the last parameter to use your username (instead of mraible) since Sauce Labs uses unique tunnel names.

python tunnel.py {sauce.username} {sauce.key} localhost 9000:80 mraible.local

3. Start the tunnel by executing local.sh. You should see output similar to the following.

$ sh local.sh 
/System/../Python.framework/../2.6/../twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead
 import itertools, md5
/System/../Python.framework/../2.6/../twisted/conch/ssh/keys.py:13: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
 import sha, md5
Launching tunnel ... 
Status: new
Status: booting
Status: running
Tunnel host: ec2-75-101-216-8.compute-1.amazonaws.com
Tunnel ID: 70f15fb59d2e7ebde55a6274ddfa54dd
<sshtunnel.TunnelTransport instance at 0x10217ad88> created
requesting remote forwarding for tunnel 70f15fb59d2e7ebde55a6274ddfa54dd 80=>localhost:9000
accepted remote forwarding for tunnel 70f15fb59d2e7ebde55a6274ddfa54dd 80=>localhost:9000

After setting up the SSH Tunnel, I modified AbstractSeleniumTestCase's setUp() method to allow running tests on Sauce Labs.

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    } else if (BROWSER.split(":").length == 3) {
        String[] platform = BROWSER.split(":");

        String os = platform[0];
        String browser = platform[1];

        // if Google Chrome, don't use a version #
        String version = (platform[1].equals("googlechrome") ? "" : platform[2]);
        String printableVersion = ((version.length() > 0) ? " " + platform[2].charAt(0) : "");

        String jobName = description.getMethodName() + " [" + browser + printableVersion + "]";

        BROWSER = "{\"username\":\"{your-username}\",\"access-key\":\"{your-access-key}\"," +
                "\"os\":\"" + platform[0] + "\",\"browser\": \"" + platform[1] + "\"," +
                "\"browser-version\":\"" + version + "\"," +
                "\"job-name\":\"" + jobName + "\"}";

        log.debug("Testing with " + browser + printableVersion + " on " + os);
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    selenium = new DefaultSelenium(SERVER, 4444, BROWSER, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

After making this change, I was able to run Selenium tests from IDEA using the following steps:

1. Start Jetty on port 9000 (since that's what the tunnel points to). In IDEA's Maven panel, create a run/debug configuration for jetty:run, click the "Runner" tab and enter "-Djetty.port=9000" in the VM Parameters box.
2. Right-click on the test to run and create a run/debug configuration. Enter the following in the VM Parameters box. The last two parameters allow skipping the xvfb and Selenium RC startup process.
   -Dselenium.browser="Windows 2003:iexplore:8." -Dselenium.application=mraible.local -Dselenium.server=saucelabs.com -Dxvfb.skip=true -Dselenium.server.skip=true

These same parameters can be used if you want to run all tests from the command line:

mvn install -Pitest -Dselenium.browser="Windows 2003:iexplore:8." -Dselenium.application=mraible.local -Dselenium.server=saucelabs.com -Dxvfb.skip=true -Dselenium.server.skip=true -Dcargo.port=9000

To simplify things, we create profiles for the various browsers. For example, below are profiles for IE8 and Firefox 3.6.

<profile>
    <id>firefox-win</id>
    <properties>
        <cargo.port>9000</cargo.port>
        <selenium.application>http://${user.name}.local</selenium.application>
        <selenium.browser>Windows 2003:firefox:3.6.</selenium.browser>
        <selenium.server>saucelabs.com</selenium.server>
        <selenium.server.skip>true</selenium.server.skip>
        <xvfb.skip>true</xvfb.skip>
    </properties>
</profile>
<profile>
    <id>ie-win</id>
    <properties>
        <cargo.port>9000</cargo.port>
        <selenium.application>http://${user.name}.local</selenium.application>
        <selenium.browser>Windows 2003:iexplore:8.</selenium.browser>
        <selenium.server>saucelabs.com</selenium.server>
        <selenium.server.skip>true</selenium.server.skip>
        <xvfb.skip>true</xvfb.skip>
    </properties>
</profile>

Issues
Since we've started using Sauce Labs, we've run into a number of issues. Some of these are Selenium-related and some are simply things we learned since we started testing on multiple browsers.

• SSH Tunnels Keep Restarting This happens on our Hudson server that runs the tunnels as a service. This seems to happen daily and screws up our Hudson results because builds fail.
• XPath vs. CSS Selectors One of the first things we noticed was that our IE tests were 2-3 times slower than the same tests on Firefox. We discovered this is because Internet Explorer has a very slow XPath engine. To fix this issue, it's recommended that ids or CSS Selectors be used whenever trying to locate elements. For more information on CSS Selectors and Selenium, see CSS Selectors in Selenium Demystified. To test CSS Selectors, I found Firefinder to be a very useful Firefox plugin. Note that many pseudo elements won't work in IE.
• IE7 fails to initialize on Sauce Labs There's no errors in our JUnit reports, so we're not sure what's causing this. It could very well be bugs in our code/configuration, but IE8 works fine.
• The Job Names on Sauce Labs don't get set correctly and often results in duplicate job names. This could certainly be related to my code. Finding videos that show failed tests is difficult when the job names aren't set correctly.
• It would be slick if you could download the video of a failed test, similar to what we do by taking screenshots.
• Google Chrome works on Sauce Labs, but I'm unable to get it working locally (on Windows or OS X). This seems to be a Selenium issue.
• Safari 4 works, but when it fails, the screenshot shows a Safari can't find the file error. Since there's no real error to debug, it's difficult to figure out why the test fails. Since Safari 4 is not listed on platforms supported by Selenium, I'm unsure how to fix this.

Overall, Sauce Labs seems to work pretty well. However, in the process of messing with Hudson, build agents and Selenium infrastructure, it's become readily apparent that we need a team member to devote their full-attention to it. Having a developer or two work on it every now-and-then is inefficient, especially when we're still in the process of ironing everything out and making it all stable.

If you have any tips on how you've solved issues with Sauce Labs (ssh tunnels, IE7) or Selenium (Safari 4, Google Chrome), I'd love to hear them. I'm also interested to hear from anyone with experience running Selenium tests concurrently (locally or in the cloud).

Update: I discovered a bug in my AbstractSeleniumTest's setUp() method where job names weren't being set correctly. I've since changed the code in this class to the following:

private static String browser, printableVersion;

@BeforeClass
public static void parseBrowser() {

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    } else if (BROWSER.split(":").length == 3) {
        String[] platform = BROWSER.split(":");

        String os = platform[0];
        browser = platform[1];

        // if Google Chrome, don't use a version #
        String version = (platform[1].equals("googlechrome") ? "" : platform[2]);
        printableVersion = ((version.length() > 0) ? " " + platform[2].charAt(0) : "");

        BROWSER = "{\"username\":\"{your-username}\",\"access-key\":\"{your-access-key}\"," +
                "\"os\":\"" + os + "\",\"browser\": \"" + browser + "\"," +
                "\"browser-version\":\"" + version + "\", " +
                "\"job-name\": \"jobName\"}";
    }
}

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    String seleniumBrowser = BROWSER;
    if (BROWSER.startsWith("{")) { // sauce labs
        String jobName = description.getMethodName() + " [" + browser + printableVersion + "]";
        log.debug("=> Running job: " + jobName);

        seleniumBrowser = BROWSER.replace("jobName", jobName);
    }

    selenium = new DefaultSelenium(SERVER, 4444, seleniumBrowser, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

Versioning Static Assets with UrlRewriteFilter

A few weeks ago, a co-worker sent me interesting email after talking with the Zoompf CEO at JSConf.

One interesting tip mentioned was how we querystring the version on our scripts and css. Apparently this doesn't always cache the way we expected it would (some proxies will never cache an asset if it has a querystring). The recommendation is to rev the filename itself.

This article explains how we implemented a "cache busting" system in our application with Maven and the UrlRewriteFilter. We originally used querystring in our implementation, but switched to filenames after reading Souders' recommendation. That part was figured out by my esteemed colleague Noah Paci.

Our Requirements

• Make the URL include a version number for each static asset URL (JS, CSS and SWF) that serves to expire a client's cache of the asset.
• Insert the version number into the application so the version number can be included in the URL.
• Use a random version number when in development mode (based on running without a packaged war) so that developers will not need to clear their browser cache when making changes to static resources. The random version number should match the production version number formats which is currently: x.y-SNAPSHOT-revisionNumber
• When running in production, the version number/cachebust is computed once (when a Filter is initialized). In development, a new cachebust is computed on each request.

In our app, we're using Maven, Spring and JSP, but the latter two don't really matter for the purposes of this discussion.

Implementation Steps
1. First we added the buildnumber-maven-plugin to our project's pom.xml so the build number is calculated from SVN.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>buildnumber-maven-plugin</artifactId>
    <version>1.0-beta-4</version>
    <executions>
        <execution>
            <phase>validate</phase>
            <goals>
                <goal>create</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <doCheck>false</doCheck>
        <doUpdate>false</doUpdate>
        <providerImplementations>
            <svn>javasvn</svn>
        </providerImplementations>
    </configuration>
</plugin>

2. Next we used the maven-war-plugin to add these values to our WAR's MANIFEST.MF file.

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.0.2</version>
    <configuration>
        <archive>
            <manifest>
                <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
            </manifest>
            <manifestEntries>
                <Implementation-Version>${project.version}</Implementation-Version>
                <Implementation-Build>${buildNumber}</Implementation-Build>
                <Implementation-Timestamp>${timestamp}</Implementation-Timestamp>
            </manifestEntries>
        </archive>
    </configuration>
</plugin>

3. Then we configured a Filter to read the values from this file on startup. If this file doesn't exist, a default version number of "1.0-SNAPSHOT-{random}" is used. Otherwise, the version is calculated as ${project.version}-${buildNumber}.

private String buildNumber = null;

...
@Override
public void initFilterBean() throws ServletException {
    try {
        InputStream is = 
            servletContext.getResourceAsStream("/META-INF/MANIFEST.MF");
        if (is == null) {
            log.warn("META-INF/MANIFEST.MF not found.");
        } else {
            Manifest mf = new Manifest();
            mf.read(is);
            Attributes atts = mf.getMainAttributes();
            buildNumber = atts.getValue("Implementation-Version") + "-" + atts.getValue("Implementation-Build");
            log.info("Application version set to: " + buildNumber);
        }
     } catch (IOException e) {
        log.error("I/O Exception reading manifest: " + e.getMessage());
     }
}

...

    // If there was a build number defined in the war, then use it for
    // the cache buster. Otherwise, assume we are in development mode 
    // and use a random cache buster so developers don't have to clear 
    // their browswer cache.
    requestVars.put("cachebust", buildNumber != null ? buildNumber : "1.0-SNAPSHOT-" + new Random().nextInt(100000));

4. We then used the "cachebust" variable and appended it to static asset URLs as indicated below.

<c:set var="version" scope="request" 
    value="${requestScope.requestConfig.cachebust}"/>
<c:set var="base" scope="request"
    value="${pageContext.request.contextPath}"/>

<link rel="stylesheet" type="text/css" 
    href="${base}/v/${version}/assets/css/style.css" media="all"/>

<script type="text/javascript" 
    src="${base}/v/${version}/compressed/jq.js"></script>

The injection of /v/[CACHEBUSTINGSTRING]/(assets|compressed) eventually has to map back to the actual asset (that does not include the two first elements of the URI). The application must remove these two elements to map back to the actual asset. To do this, we use the UrlRewriteFilter. The UrlRewriteFilter is used (instead of Apache's mod_rewrite) so when developers run locally (using mvn jetty:run) they don't have to configure Apache.

5. In our application, "/compressed/" is mapped to wro4j's WroFilter. In order to get UrlRewriteFilter and WroFilter to work with this setup, the WroFilter has to accept FORWARD and REQUEST dispatchers.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
    <filter-name>WebResourceOptimizer</filter-name>
    <url-pattern>/compressed/*</url-pattern>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>REQUEST</dispatcher>
</filter-mapping>

Once this was configured, we added the following rules to our urlrewrite.xml to allow rewriting of any assets or compressed resource request back to its "correct" URL.

<rule match-type="regex">
    <from>^/v/[0-9A-Za-z_.\-]+/assets/(.*)$</from>
    <to>/assets/$1</to>
</rule>
<rule match-type="regex">
    <from>^/v/[0-9A-Za-z_.\-]+/compressed/(.*)$</from>
    <to>/compressed/$1</to>
</rule>
<rule>
    <from>/compressed/**</from>
    <to>/compressed/$1</to>
</rule>

Of course, you can also do this in Apache. This is what it might look like in your vhost.d file:

RewriteEngine    on
RewriteLogLevel  0!
RewriteLog       /srv/log/apache22/app_rewrite_log
RewriteRule      ^/v/[.A-Za-z0-9_-]+/assets/(.*) /assets/$1 [PT]
RewriteRule      ^/v/[.A-Za-z0-9_-]+/compressed/(.*) /compressed/$1 [PT]

Whether it's a good idea to implement this in Apache or using the UrlRewriteFilter is up for debate. If we're able to do this with the UrlRewriteFilter, the benefit of doing this at all in Apache is questionable, especially since it creates a duplicate of code.

What's New in Maven 3.0 with Matthew McCullough

Last night, I attended the Denver JUG meeting to hear some excellent talks by Matthew McCullough and Tim Berglund. I took notes during Matthew's talk, but my battery ran out before Tim's talk started. Below are my notes.

Matthew started out by described the differences between Maven 2 and Maven 3. As he began, he emphasized it wasn't a beginner talk, but mostly for existing Maven users that understand how to read a pom.xml and such.

The Roadmap
Commits to Maven 3 have been happening for the last 3 years. Matthew is not an employee of Sonatype, but he mentioned their name quite a bit in his talk. Sonatype has hired several committers (7 that Matthew knows of by name) that now work on Maven 3 full-time. For compatibility with Maven 2, the project has 450 integration tests and they test it against 100s of Maven 2 projects. Maven 3 has plugin classloader partitioning and a legacy simulation layer for old plugins.

The main improvement in Maven 3 is speed. It's been performance tuned to be 50% to 400% faster. Benchmarks (guaranteed by integration tests) include better: Disk I/O, Network I/O, CPU and Memory. Another new feature is extensibility so Maven is a better library rather than just a command-line tool. Now there's a library and APIs that you can use to do the things that Maven does. Plexus has been replaced with Guice and it's now much easier to embed Maven (Polyglot Maven and Maven Shell are examples of this).

Below are a number of other changes between Maven 2 and Maven 3.

• Syntax: pom.xml still uses <modelVersion>4.0.0</modelVersion> so it can be a drop-in replacement for Maven 2 projects.
• Validations: poms are heavily validated against common mistakes, warns when plugin versions are not specified (use mvn validate to see issues), blocks duplicate dependencies (examined in same POM only, conflict resolution used otherwise).
• Help URLs: wiki page URLs now shown for all error messages. One of the first Apache projects to do this.
• Removals: profiles.xml external file support removed, Maven 1.0 repository support removed <layout>legacy</layout> (it's been 5 years since any commits to Maven 1).
• Behavior: SNAPSHOTs always deployed with date-stamps, artifact resolution caching has been improved to do less checking (override with mvn <phase> -U).
• Plugins: version auto-selection favors RELEASEs over SNAPSHOTs (opposite for Maven 2), versions cannot be specified as RELEASE or LATEST, plugins only resolved from <pluginRepository> locations.
• See the Plugin Compatibility Matrix to see if your favorite plugins are compatible.

Maven 3 hopes to be a drop-in replacement for Maven 2, but non backwards-compatible changes will be happening in Maven 3.1. It's anticipated release is Q1 of 2011 and will likely contain the following features.

• "Mixins" for direct dependencies
• Site plugin takes over <reporting>
• Backwards compatibility by <modelVersion
• There's a good chance 3.1 breaks compatibility with legacy POMs

Another new thing in Maven 3 is Toolchain. Toolchain a common way to configure your JDK for multiple plugins. There are only a handful of plugins that are toolchain-enabled. User tool chain definitions are defined in ~/.m2/toolchains.xml. To use different toolchains (JDKs), you specify a vendor and version as part of your plugin configuration.

Maven Shell is a high performance console that's a Maven 3 add-on. It's hosted at GitHub to make community contributions easier. It goes on your command line and it offers syntax highlighting and context-sensitive help (by typing ? at the command prompt).

Another major improvement in Maven 3 is Polyglot Maven. Tools like Gant and Buildr have made Maven look ancient, but they've also given it a good challenge. Maven 3 is likely to leapfrog these tools because of its ability to use different languages for your build configuration. Currently, 6 languages are supported. Polyglot Maven is a super-set distribution of Maven 3. It's not shipped with Maven 3 core because it contains all the other language implementations and is quite large. Polyglot Maven also contains a translate tool that allows you to convert any-to-any language. It has a DSL framework with Macros and Lifecycle Hooks. Macros allows for more concise syntax.

After talking about Polyglot Maven a bit, Matthew shows us a demo translating pom.xml to pom.yaml and then running the build. After that, he showed us examples of what a pom looks like when defined in Clojure, Scala and Groovy. Someone asked about file parsing performance and Matthew said different languages would cause a single-digit performance difference as part of your build process. Personally, I can't help but think any non-XML parser would be faster than the XML parser.

In regards to m2eclipse, a new drop (0.10) occurred a few weeks ago and it's one of the highest quality releases to date. It has major refactoring and many performance improvements.

For sample Maven projects see Matthew's Maven Samples.

I very much enjoyed Matthew's talk, both because of his presentation techniques and because he had a lot of good information. While I've tried Maven 3 and Shell in the past, I've been newly inspired to start using them again on a daily basis.

Tim's talk on Decision Making was also excellent. The biggest things I learned were that conflict is good (idea-wise, not personal) and things to look out for between teams (fault lines). Hopefully both Tim and Matthew post their slides so I can link to them here.

My TSSJS 2010 Presentations and Summary

This afternoon, I delivered my last talk at TSSJS 2010 on The Future of Web Frameworks. It's true that I made some bold statements, but please remember that this is my personal opinion, based on my experience. For the most part, I've been involved in super high-traffic websites for the last few years and this has influenced my opinion on web frameworks. Just because I don't recommend your favorite framework doesn't mean it won't work for you. In fact, many of the best web applications today were built without an open source (or commercial) web framework. In the end, it's not as much about the web framework you're using as it is about hiring smart people. Below is my slide deck from this talk.

Yesterday, I did a GWT vs. Flex Smackdown with James Ward. While there wasn't as much trash talking as I'd hoped, I enjoyed delivering it and disputing the greatness of Flex. Below is the presentation that James and I delivered.

The show itself was great this year. It had more attendees than I've seen in a long time. There were a lot of really interesting sessions and and an often humorous Twitter back-channel. I attended quite a few talks and jotted down my notes from several of them. Please see the links below if you're interested in the sessions I attended. You can view all of the presentations from TSSJS 2010 on SlideShare.

• What's Happening in the Java World?
• Software Quality: The Quest for the Holy Grail?
• The Cloud Computing Continuum with Bob McWhirter
• Highly Interactive Software with Java and Flex
• C++, Java and .NET: Lessons Learned from the Internet Age
• Developing Rich Web Service APIs with Java
• What's New in Spring 3.0

Thanks to everyone who came to Vegas and to TheServerSide for an excellent conference.

What's New in Spring 3.0

This morning, I attended Rod Johnson's What's New in Spring 3.0 keynote at TSSJS. Rod ditched his slides for the talk and mentioned that this might be risky. Especially since he was pretty jetlagged (flew in from Paris at 11pm last night). Below are my notes from his talk.

The most important thing for the future of Java is productivity and cloud computing. The focus at SpringSource is heavily on productivity and not just on improving the Spring codebase. If you look at the comparisons out there between Rails and Spring, it's not an apples-to-apples comparison. The philosophy with Spring has always been the developer is always right. However, if you look at something like Rails, you'll see it's far more prescriptive. That layer of opinionated frameworks is important in that it improves your productivity greatly.

SpringSource is putting a lot of emphasis on improving developer productivity with two opinionated frameworks: Grails and Spring Roo. To show how productive developers can be, Rod started to build a web app with Spring Roo. As part of this demo, he mentioned we'd see many of the new features of Spring 3: RestTemplate, @Value and Spring EL.

Rod used STS to write the application and built a Twitter client. After creating a new project using File -> New Roo Project, a Roo Shell tab shows up at the bottom. Typing "hint" tells you what you should do write away. The initial message is "Roo requires the installation of a JPA provider and associated database." The initial command is "persistence setup --provider HIBERNATE --database HYPERSONIC_IN_MEMORY". After running this, a bunch of log messages are shown on the console, most of them indicating that pom.xml has been modified.

The first file that Rod shows is src/main/resources/META-INF/spring/applicationContext.xml. It's the only XML file you'll need in your application and includes a PropertyPlaceHolderConfigurer, a context:component-scan for finding annotations and a transaction manager.

After typing "hint" again, Roo indicates that Rod should create entities. He does this by running "ent --class ~.domain.Term --testAutomatically". A Term class (with a bunch of annotations) is created, as well as a number of *.aj files and an integration test. Most of the files don't have anything in them but annotations. The integration test uses @RooIntegrationTest(entity=Term.class) on its class to fire up a Spring container in the test and do dependency injection (if necessary). From there, Rod demonstrated that he could easily modify the test to verify the database existed.

private SimpleJdbcTemplate jt;

@Autowired
public void init(DataSource ds) {
    this.jt = new SimpleJdbcTemplate(ds);
}

@Test 
public void testDb() {
    jt.queryForInt("SELECT COUNT(0) FROM TERM");
}

Interestingly, after running the test, you could see a whole bunch of tests being run, not just the one that was in the class itself. From there, he modified the Term class to add two new properties: name and searchTerms. He also used JSR 303's @NotNull annotation to make the fields required.

@Entity
@RooJavaBean
@RooToString
@RooEntity
public class Term {

    @NotNull
    private String name;

    @NotNull
    private String searchTerms;
}

Next, Rod added a new test and showed that the setters for these properties were automatically created and he never had to write getters and setters. This is done by aspects that are generated beside your Java files. Roo is smart enough that if you write toString() methods in your Java code, it will delete the aspect that normally generates the toString() method.

To add fields to an entity from the command lie, you can run commands like "field string --fieldName text --notNull" and "field number --type java.lang.Long --fieldName twitterId --notNull". The Roo Shell is also capable of establishing relationships between entities.

After successfully modifying his Entities, Rod started creating code to talk to Twitter's API. He used RestTemplate to do this and spent a good 5 minutes trying to get Eclipse to import the class properly. The best part of this demo was watching him do what most developers do: searching Google for RestTemplate to get the package name to import.

After awkward silence and some fumbling, he opened an existing project (that had the dependencies properly configured) and used Java Config to configure beans for the project. This was done with a @Configuration annotation on the class, @Value annotations on properties (that read from a properties file) and @Bean annotations for the beans to expose. The first time Rod tried to run the test it failed because a twitter.properties file didn't exist. After creating it, he successfully ran the test and successfully searched Twitter's API.

The nice thing about @Configuration is the classes are automatically picked up and you don't need to configure any XML to recognize them. Also, in your Java classes, you don't have to use @Autowired to get @Bean references injected.

After this, Rod attempted to show a web interface of the application. He started the built-in SpringSource tc Server and proceeded to show us Tomcat's 404 page. Unfortunately, Tomcat seemed to startup OK (no errors in the logs), but obviously something didn't work well. For the next few silent moments, we watched him try to delete web.xml from Eclipse. Unfortunately, this didn't work and we weren't able to see the scaffolding the entities that Rod created.

At this point, Rod opened a completed version of the app and was able to show it to us in a browser. You could hear the murmur of the crowd as everyone realized he was about to show the the Twitter search results for #tssjs. Most of the tweets displayed were from folks commenting about how some things didn't work in the demo.

In summary, there's some really cool things in Spring 3: @Configuration, @Value, task scheduling with @Scheduled and one-way methods with @Async.

Final points of SpringSource and VMWare: they're committed to Java and middleware. Their big focus is providing an integrated experience from productivity to cloud. There's other languages that are further along than Java and SpringSource is trying to fix that. One thing they're working on is a private Java cloud that companies can use and leverage as a VMWare appliance.

I think there's a lot of great things in Spring 3 and most users of Roo seem to be happy with it. It's unfortunate that the Demo Gods frowned upon Rod, but it was cool to see him do the "no presentation" approach.