Raible Designs

Getting Started + Testing with Angular CLI and Angular 2 (RC5)

I started creating Angular 2 applications when it was in beta (back in March). To keep up with Angular 2's changes, I wrote a tutorial about developing with RC1 in June. Earlier this month, RC5 was released and many things changed once again. I think Scott Davis sums it up nicely in a tweet.

To keep up with the rapid pace of change in Angular 2, I decided to write another tutorial, this time using Angular CLI. The biggest change I found since writing the last tutorial is testing infrastructure changes. Since Angular's Testing documentation hasn't been updated recently, hopefully this tutorial will help.

Testing AngularJS Applications

This article is the second in a series about learning AngularJS. It describes how to test a simple AngularJS application. In a previous article, Getting Started with AngularJS, I showed how to develop a simple search and edit feature.

What you'll learn

You'll learn to use Jasmine for unit testing controllers and Protractor for integration testing. Angular's documentation has a good developer's guide to unit testing if you'd like more information on testing and why it's important.

The best reason for writing tests is to automate your testing. Without tests, you'll likely be testing manually. This manual testing will take longer and longer as your codebase grows.

What you'll need

• About 15-30 minutes
• A favorite text editor or IDE. We recommend IntelliJ IDEA.
• Git installed.
• Node.js and NPM installed.

Documenting your Spring API with Swagger

Over the last several months, I've been developing a REST API using Spring Boot. My client hired an outside company to develop a native iOS app, and my development team was responsible for developing its API. Our main task involved integrating with Epic, a popular software system used in Health care. We also developed a Crowd-backed authentication system, based loosely on Philip Sorst's Angular REST Security.

To document our API, we used Spring MVC integration for Swagger (a.k.a. swagger-springmvc). I briefly looked into swagger4spring-web, but gave up quickly when it didn't recognize Spring's @RestController. We started with swagger-springmvc 0.6.5 and found it fairly easy to integrate. Unfortunately, it didn't allow us to annotate our model objects and tell clients which fields were required. We were quite pleased when a new version (0.8.2) was released that supports Swagger 1.3 and its @ApiModelProperty.

What is Swagger?
The goal of Swagger is to define a standard, language-agnostic interface to REST APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection.

To demonstrate how Swagger works, I integrated it into Josh Long's x-auth-security project. If you have a Boot-powered project, you should be able to use the same steps.

Integration Testing with HTTP, HTTPS and Maven

Earlier this week, I was tasked with getting automated integration tests working in my project at Overstock.com. By automated, I mean that ability to run "mvn install" and have the following process cycled through:

• Start a container
• Deploy the application
• Run all integration tests
• Stop the container

Since it makes sense for integration tests to run in Maven's integration-test phase, I first configured the maven-surefire-plugin to skip tests in the test phase and execute them in the integration-test phase. I used the <id>default-phase</id> syntax to override the plugins' usual behavior.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <executions>
    <execution>
      <id>default-test</id>
      <configuration>
        <excludes>
          <exclude>**/*Test*.java</exclude>
        </excludes>
      </configuration>
    </execution>
    <execution>
      <id>default-integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/*Test.java</include>
        </includes>
        <excludes>
          <exclude>none</exclude>
          <exclude>**/TestCase.java</exclude>
        </excludes>
      </configuration>
    </execution>
  </executions>
</plugin>

After I had this working, I moved onto getting the container started and stopped properly. In the past, I've done this using Cargo and it's always worked well for me. Apart from the usual setup I use in AppFuse archetypes (example pom.xml), I added a couple additional items:

• Added <timeout>180000</timeout> so the container would wait up to 3 minutes for the WAR to deploy.
• In configuration/properties, specified <context.path>ROOT</context.path> so the app would deploy at the / context path.
• In configuration/properties, specified <cargo.protocol>https</cargo.protocol> since many existing unit tests made requests to secure resources.

I started by using Cargo with Tomcat and had to create certificate keystore in order to get Tomcat to start with SSL enabled. After getting it to start, I found the tests failed with the following errors in the logs:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: 
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1649)

Co-workers told me this was easily solved by adding my 'untrusted' cert to my JVM keystore. Once all this was working, I thought I was good to go, but found that some tests were still failing. The failures turned out to be because they were talking to http and https was the only protocol enabled. After doing some research, I discovered that Cargo doesn't support starting on both http and https ports.

So back to the drawing board I went. I ended up turning to the maven-jetty-plugin and the tomcat-maven-plugin to get the functionality I was looking for. I also automated the certificate keystore generation using the keytool-maven-plugin. Below is the extremely-verbose 95-line profiles section of my pom.xml that allows either container to be used.

Sidenote: I wonder how this same setup would look using Gradle?

<profiles>
  <profile>
    <id>jetty</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <build>
      <plugins>
        <plugin>
          <groupId>org.mortbay.jetty</groupId>
          <artifactId>maven-jetty-plugin</artifactId>
          <version>6.1.26</version>
          <configuration>
            <contextPath>/</contextPath>
            <connectors>
              <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <!-- forwarded == true interprets x-forwarded-* headers -->
                <!-- http://docs.codehaus.org/display/JETTY/Configuring+mod_proxy -->
                <forwarded>true</forwarded>
                <port>8080</port>
                <maxIdleTime>60000</maxIdleTime>
              </connector>
              <connector implementation="org.mortbay.jetty.security.SslSocketConnector">
                <forwarded>true</forwarded>
                <port>8443</port>
                <maxIdleTime>60000</maxIdleTime>
                <keystore>${project.build.directory}/ssl.keystore</keystore>
                <password>overstock</password>
                <keyPassword>overstock</keyPassword>
              </connector>
            </connectors>
            <stopKey>overstock</stopKey>
            <stopPort>9999</stopPort>
          </configuration>
          <executions>
            <execution>
              <id>start-jetty</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
              <configuration>
                <daemon>true</daemon>
              </configuration>
            </execution>
            <execution>
              <id>stop-jetty</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>stop</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
  <profile>
    <id>tomcat</id>
    <build>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>tomcat-maven-plugin</artifactId>
          <version>1.1</version>
          <configuration>
            <addContextWarDependencies>true</addContextWarDependencies>
            <fork>true</fork>
            <path>/</path>
            <port>8080</port>
            <httpsPort>8443</httpsPort>
            <keystoreFile>${project.build.directory}/ssl.keystore</keystoreFile>
            <keystorePass>overstock</keystorePass>
          </configuration>
          <executions>
            <execution>
              <id>start-tomcat</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
            </execution>
            <execution>
              <id>stop-tomcat</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>shutdown</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

With this setup in place, I was able to automate running our integration tests by simply typing "mvn install" (for Jetty) or "mvn install -Ptomcat" (for Tomcat). For running in Hudson, it's possible I'll have to further enhance things to randomize the port and pass that into tests as a system property. The build-helper-maven-plugin and its reserve-network-port goal is a nice way to do this. Note: if you want to run more than one instance of Tomcat at a time, you might have to randomize the ajp and rmi ports to avoid collisions.

The final thing I encountered was our app didn't shutdown gracefully. Luckily, this was fixed in a newer version of our core framework and upgrading fixed the problem. Here's the explanation from an architect on the core framework team.

The hanging problem was caused by the way the framework internally aggregated statistics related to database connection usage and page response times. The aggregation runs on a separate thread but not as a daemon thread. Previously, the aggregation threads weren't being terminated on shutdown so the JVM would hang waiting for them to finish. In the new frameworks, the aggregation threads are terminated on shutdown.

Hopefully this post helps you test your secure and unsecure applications at the same time. At the same time, I'm hoping it motivates the Cargo developers to add simultaneous http and https support.

Update: In the comments, Ron Piterman recommended I use the Maven Failsafe Plugin because its designed to run integration tests while Surefire Plugin is for unit tests. I changed my configuration to the following and everything still passes. Thanks Ron!

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <skipTests>true</skipTests>
  </configuration>
</plugin>
<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <includes>
      <include>**/*Test.java</include>
    </includes>
    <excludes>
      <exclude>**/TestCase.java</exclude>
    </excludes>
  </configuration>
  <executions>
    <execution>
      <id>integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <phase>verify</phase>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Update 2: In addition to application changes to solve hanging issues, I also had to change my Jetty Plugin configuration to use a different SSL connector implementation. This also required adding the jetty-sslengine dependency, which has been renamed to jetty-ssl for Jetty 7.

<connector implementation="org.mortbay.jetty.security.SslSelectChannelConnector">
...
<dependencies>
  <dependency>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-sslengine</artifactId>
    <version>6.1.26</version>
  </dependency>
</dependencies>

Running Selenium Tests on Sauce Labs

Recently I embarked on a mission to configure my team's Selenium testing process to support multiple browsers. We use Hudson for our continuous integration server. Since our Hudson instance runs on Solaris, testing with Firefox on Solaris didn't seem like a good representation of our clients. Our browser support matrix currently looks as follows:

At first, I attempted to use Windows VMs to run Selenium tests on IE. This was a solution that didn't work too well. The major reasons it didn't work:

1. I had issues getting the Selenium Plugin for Hudson working. Upgrading the plugin to use Selenium RC 1.0.5 may solve this issue.
2. We had some unit tests that failed on Windows. I tried using the Cygpath Plugin for Hudson (which allows you to emulate a Unix environment on Windows), but failed to get it to work.
3. We quickly realized it might become a maintenance nightmare to keep all the different VMs up-to-date.

Frustrated by these issues, I turned to Sauce Labs. They have a cloud-based model that runs Selenium tests on VMs that point back to your application. They also support many different browser/OS combinations. We asked them about support for OS X and various Windows versions and they indicated that their experience shows browsers are the same across OSes.

I'm writing this article to show you how we've configured our build process to support 1) testing locally and 2) testing on Sauce Labs. In a future post, I hope to write about how to run Selenium tests concurrently for faster execution.

Running Selenium Tests Locally
We use Maven to build our project and run our Selenium tests. Our configuration is very similar to the poms referenced in Integrating Selenium with Maven 2. Basically, we have an "itest" profile that gets invoked when we pass in -Pitest. It downloads/starts Tomcat (using Cargo), deploys our WAR, starts Selenium RC (using the selenium-maven-plugin) and executes JUnit-based tests using the maven-surefire-plugin. All of this configuration is pretty standard and something I've used on many projects over the past several years.

Beyond that, we have a custom BlockJUnit4ClassRunner class that takes screenshots and captures the HTML source for tests that fail.

public class SeleniumJUnitRunner extends BlockJUnit4ClassRunner {
    public SeleniumJUnitRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    protected Statement methodInvoker(FrameworkMethod method, Object test) {
        if (!(test instanceof AbstractSeleniumTestCase)) {
            throw new RuntimeException("Only works with AbstractSeleniumTestCase");
        }

        final AbstractSeleniumTestCase stc = ((AbstractSeleniumTestCase) test);
        stc.setDescription(describeChild(method));

        return new InvokeMethod(method, test) {
            @Override
            public void evaluate() throws Throwable {
                try {
                    super.evaluate();
                } catch (Throwable throwable) {
                    stc.takeScreenshot("FAILURE");
                    stc.captureHtmlSource("FAILURE");
                    throw throwable;
                }
            }
        };
    }
}

To use the functionality SeleniumJUnitRunner provides, we have a parent class for all our tests. This class uses the @RunWith annotation as follows:

@RunWith(SeleniumJUnitRunner.class)
public abstract class AbstractSeleniumTestCase {
    // convenience methods
}

This class looks up the Selenium RC Server, the app location and what browser to use based on system properties. If system properties are not set, it has defaults for running locally.

public static String SERVER = System.getProperty("selenium.server");
public static String APP = System.getProperty("selenium.application");
public static String BROWSER = System.getProperty("selenium.browser");

protected Selenium selenium;

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    selenium = new DefaultSelenium(SERVER, 4444, BROWSER, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

The system properties are specified as part of the surefire-plugin's configuration. The reason we default them in the above code is so tests can be run from IDEA as well.

<artifactId>maven-surefire-plugin</artifactId>
<version>2.5</version>
<configuration>
    <systemPropertyVariables>
        <selenium.application>${selenium.application}</selenium.application>
        <selenium.browser>${selenium.browser}</selenium.browser>
        <selenium.server>${selenium.server}</selenium.server>
    </systemPropertyVariables>
</configuration>

Running Selenium Tests in the Cloud
To run tests in the cloud, you have to do a bit of setup first. If you're behind a firewall, you'll need to setup SSH tunneling so Sauce Labs can see your machine. You'll also need to setup SSH Tunneling on your Hudson server, but installing/configuring/running locally is usually a good first step. Below are the steps I used to configure Sauce Labs' SSH Tunneling on OS X.

1. Install the Python version in /opt/tools/saucelabs. If you get an error (No local packages or download links found for install) download the egg and run it with:

sudo sh setuptools-0.6c11-py2.6.egg

NOTE: If you get an error (unable to execute gcc-4.2: No such file or directory) when installing pycrypto on OS X, you'll need to install the OS X Developer Tools.

2. Create a /opt/tools/saucelabs/local.sh script with the following in it. You should change the last parameter to use your username (instead of mraible) since Sauce Labs uses unique tunnel names.

python tunnel.py {sauce.username} {sauce.key} localhost 9000:80 mraible.local

3. Start the tunnel by executing local.sh. You should see output similar to the following.

$ sh local.sh 
/System/../Python.framework/../2.6/../twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead
 import itertools, md5
/System/../Python.framework/../2.6/../twisted/conch/ssh/keys.py:13: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
 import sha, md5
Launching tunnel ... 
Status: new
Status: booting
Status: running
Tunnel host: ec2-75-101-216-8.compute-1.amazonaws.com
Tunnel ID: 70f15fb59d2e7ebde55a6274ddfa54dd
<sshtunnel.TunnelTransport instance at 0x10217ad88> created
requesting remote forwarding for tunnel 70f15fb59d2e7ebde55a6274ddfa54dd 80=>localhost:9000
accepted remote forwarding for tunnel 70f15fb59d2e7ebde55a6274ddfa54dd 80=>localhost:9000

After setting up the SSH Tunnel, I modified AbstractSeleniumTestCase's setUp() method to allow running tests on Sauce Labs.

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    } else if (BROWSER.split(":").length == 3) {
        String[] platform = BROWSER.split(":");

        String os = platform[0];
        String browser = platform[1];

        // if Google Chrome, don't use a version #
        String version = (platform[1].equals("googlechrome") ? "" : platform[2]);
        String printableVersion = ((version.length() > 0) ? " " + platform[2].charAt(0) : "");

        String jobName = description.getMethodName() + " [" + browser + printableVersion + "]";

        BROWSER = "{\"username\":\"{your-username}\",\"access-key\":\"{your-access-key}\"," +
                "\"os\":\"" + platform[0] + "\",\"browser\": \"" + platform[1] + "\"," +
                "\"browser-version\":\"" + version + "\"," +
                "\"job-name\":\"" + jobName + "\"}";

        log.debug("Testing with " + browser + printableVersion + " on " + os);
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    selenium = new DefaultSelenium(SERVER, 4444, BROWSER, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

After making this change, I was able to run Selenium tests from IDEA using the following steps:

1. Start Jetty on port 9000 (since that's what the tunnel points to). In IDEA's Maven panel, create a run/debug configuration for jetty:run, click the "Runner" tab and enter "-Djetty.port=9000" in the VM Parameters box.
2. Right-click on the test to run and create a run/debug configuration. Enter the following in the VM Parameters box. The last two parameters allow skipping the xvfb and Selenium RC startup process.
   -Dselenium.browser="Windows 2003:iexplore:8." -Dselenium.application=mraible.local -Dselenium.server=saucelabs.com -Dxvfb.skip=true -Dselenium.server.skip=true

These same parameters can be used if you want to run all tests from the command line:

mvn install -Pitest -Dselenium.browser="Windows 2003:iexplore:8." -Dselenium.application=mraible.local -Dselenium.server=saucelabs.com -Dxvfb.skip=true -Dselenium.server.skip=true -Dcargo.port=9000

To simplify things, we create profiles for the various browsers. For example, below are profiles for IE8 and Firefox 3.6.

<profile>
    <id>firefox-win</id>
    <properties>
        <cargo.port>9000</cargo.port>
        <selenium.application>http://${user.name}.local</selenium.application>
        <selenium.browser>Windows 2003:firefox:3.6.</selenium.browser>
        <selenium.server>saucelabs.com</selenium.server>
        <selenium.server.skip>true</selenium.server.skip>
        <xvfb.skip>true</xvfb.skip>
    </properties>
</profile>
<profile>
    <id>ie-win</id>
    <properties>
        <cargo.port>9000</cargo.port>
        <selenium.application>http://${user.name}.local</selenium.application>
        <selenium.browser>Windows 2003:iexplore:8.</selenium.browser>
        <selenium.server>saucelabs.com</selenium.server>
        <selenium.server.skip>true</selenium.server.skip>
        <xvfb.skip>true</xvfb.skip>
    </properties>
</profile>

Issues
Since we've started using Sauce Labs, we've run into a number of issues. Some of these are Selenium-related and some are simply things we learned since we started testing on multiple browsers.

• SSH Tunnels Keep Restarting This happens on our Hudson server that runs the tunnels as a service. This seems to happen daily and screws up our Hudson results because builds fail.
• XPath vs. CSS Selectors One of the first things we noticed was that our IE tests were 2-3 times slower than the same tests on Firefox. We discovered this is because Internet Explorer has a very slow XPath engine. To fix this issue, it's recommended that ids or CSS Selectors be used whenever trying to locate elements. For more information on CSS Selectors and Selenium, see CSS Selectors in Selenium Demystified. To test CSS Selectors, I found Firefinder to be a very useful Firefox plugin. Note that many pseudo elements won't work in IE.
• IE7 fails to initialize on Sauce Labs There's no errors in our JUnit reports, so we're not sure what's causing this. It could very well be bugs in our code/configuration, but IE8 works fine.
• The Job Names on Sauce Labs don't get set correctly and often results in duplicate job names. This could certainly be related to my code. Finding videos that show failed tests is difficult when the job names aren't set correctly.
• It would be slick if you could download the video of a failed test, similar to what we do by taking screenshots.
• Google Chrome works on Sauce Labs, but I'm unable to get it working locally (on Windows or OS X). This seems to be a Selenium issue.
• Safari 4 works, but when it fails, the screenshot shows a Safari can't find the file error. Since there's no real error to debug, it's difficult to figure out why the test fails. Since Safari 4 is not listed on platforms supported by Selenium, I'm unsure how to fix this.

Overall, Sauce Labs seems to work pretty well. However, in the process of messing with Hudson, build agents and Selenium infrastructure, it's become readily apparent that we need a team member to devote their full-attention to it. Having a developer or two work on it every now-and-then is inefficient, especially when we're still in the process of ironing everything out and making it all stable.

If you have any tips on how you've solved issues with Sauce Labs (ssh tunnels, IE7) or Selenium (Safari 4, Google Chrome), I'd love to hear them. I'm also interested to hear from anyone with experience running Selenium tests concurrently (locally or in the cloud).

Update: I discovered a bug in my AbstractSeleniumTest's setUp() method where job names weren't being set correctly. I've since changed the code in this class to the following:

private static String browser, printableVersion;

@BeforeClass
public static void parseBrowser() {

    if (BROWSER == null) {
        BROWSER = "*firefox3";
    } else if (BROWSER.split(":").length == 3) {
        String[] platform = BROWSER.split(":");

        String os = platform[0];
        browser = platform[1];

        // if Google Chrome, don't use a version #
        String version = (platform[1].equals("googlechrome") ? "" : platform[2]);
        printableVersion = ((version.length() > 0) ? " " + platform[2].charAt(0) : "");

        BROWSER = "{\"username\":\"{your-username}\",\"access-key\":\"{your-access-key}\"," +
                "\"os\":\"" + os + "\",\"browser\": \"" + browser + "\"," +
                "\"browser-version\":\"" + version + "\", " +
                "\"job-name\": \"jobName\"}";
    }
}

@Before
public void setUp() throws Exception {
    if (SERVER == null) {
        SERVER = "localhost";
    }

    if (APP == null) {
        APP = "http://localhost:9000";
    }

    String seleniumBrowser = BROWSER;
    if (BROWSER.startsWith("{")) { // sauce labs
        String jobName = description.getMethodName() + " [" + browser + printableVersion + "]";
        log.debug("=> Running job: " + jobName);

        seleniumBrowser = BROWSER.replace("jobName", jobName);
    }

    selenium = new DefaultSelenium(SERVER, 4444, seleniumBrowser, APP);
    selenium.start("captureNetworkTraffic=true");
    selenium.getEval("window.moveTo(1,1); window.resizeTo(1021,737);");
    selenium.setTimeout("60000");
}

Web Application Testing with Selenium by Jason Huggins

This evening, I attended Agile Denver's monthly meeting to listen to Jason Huggins talk about Selenium. The meeting started off with a panel on UI testing that I participated in. The most interesting part of this panel (for me) was meeting the other panelists and learning about their expertise. Folks from Red Pine Studios in Boulder video taped both the panel and presentation. Hopefully it will be published online in the near future.

Below are my notes from Jason's talk. Please keep in mind that most of these are his words, not mine.

Jason is the Executive Software Chef at Sauce Labs. He often experiments with new recipes and is one of the creators of Selenium. He worked at Google and helped them build and use a Selenium Farm to test Gmail and Google Docs. Selenium was inspired by ThoughtWorks Expense Report system and its "Add Row" button. The button caused so many issues, they needed a way to write a test that could be run in multiple browsers.

The first thing they tried was jWebUnit (a wrapper around HtmlUnit). Since HtmlUnit simulates the browser, it wasn't "real world" enough. The 2nd attempt was DriftWood. It was a Mozilla extension that drove a real browser so it could handle JavaScript UI features. The downside it was it didn't work for IE or Safari. It also used XML Syntax for tests. The 3rd attempt was JsUnit. It worked in all browsers, but its emphasis was on a single page unit test; it had no page-to-page workflow support. Also, you couldn't see what it was doing while it was running. The 4th attempt was FIT (Framework for Integration Testing). It allowed more readable tests, but the API wasn't that intuitive and there was too much magic behind the curtain. So basically, they had to fork FIT.

The first attempt was called "Selenese" and consisted of a 3-column table where each row had an Action, Target and Value. In the beginning, Selenium Core was a TestRunner that ran in any browser. It was written in plain ol' JavaScript and HTML. The next thing that came about was the Selenium IDE for Firefox. It maintains the echo of Selenium Core and FIT.

Selenium Remote Control (RC) was the next product produced by the project. Selenium RC allows you to write your tests in any language. A Selenium server interprets the requests and turns them into browser manipulation events. Finally, Selenium Grid was developed to leverage Selenium's HTTP architecture to allow parallel execution across servers.

Cloud computing is a wonderful use case for functional testing. Selenium Hub is a gateway into the Selenium Grid that routes the test request to particular browsers and platforms. Sauce Labs has a version of Selenium Grid that runs in the cloud.

Selenium Issues
Selenium is slow. Functional tests will always be slower than unit tests. Until the browsers can launch faster, there's always going to be speed issues. Parallel-ization can solve some of these and is something you should think about right away.

The JavaScript sandbox, Flash, Java Applets, Silverlight and Canvas all present problems in Selenium. Silverlight was shipped without any testing APIs. There are several libraries that provide a bridge for testing Flash. The Selenium project has though about including FlexMonkey, but its GPL license prevents it.

Practical Advice
Everyone seems to build a framework on top of Selenium. If you do this, make sure and write your DSL in terms of intent and then map it to Selenium actions.

Look for abstractions so you're not writing your Selenium tests with its API. It's too much like Assembler.

K.I.S.S. - don't write large tests, just do small ones. Often, when functional tests fail, they tell you something failed, but they don't tell you what failed. The shortest possible functional tests help reduce the scope of where a problem can be. Other benefits of short tests are they're easier to read and easier to write.

Selenium 2.0
The big thing in Selenium 2.0 is a merger with WebDriver. The nice thing about WebDriver is it gets rid of Selenium RC and allows you to drive the browser with a low-level API. For example, you use C++ to drive IE. Basically, every language will talk to the C driver. Except for Firefox, the connection and control is done through telnet. Selenium 2 should fix all the problems with Selenium 1, but also allow you to still use Selenium RC if you want to do grid-style testing.

Selenium 2's API is about finding elements and interacting with those elements. Also, it's entirely backwards compatible, so you can use the old API.

At this point, my laptop's battery died and I was unable to take any more notes. However, I was able to see some pretty slick demos, particularly Jason's company's Sauce onDemand cloud testing services. All you need to do to run your tests in the cloud is change how you initialize Selenium. A kick-ass feature this service provides is video playback (a.k.a. Castro). I'm currently using Selenium's screenshot functionality, but it doesn't hold a candle to the ability to watch a video playback of your tests. Jason also showed us a demo of using Castro and Selenium 2 to create a screencast on-the-fly. Very cool stuff.

My only question after seeing this talk is what's the difference between BrowserMob and Sauce Labs? Both companies were founded by Selenium committers and seem to offer competing projects. My gut feel is that BrowserMob is best for performance/load testing and Sauce Labs is best for running your tests in the cloud.

Web Application Testing with Selenium at Agile Denver next Monday

Next Monday evening, Jason Huggins will be in Denver to talk about web application testing with Selenium. Below is an outline of Jason's talk:

• Overview of Selenium, what it does, what it doesn't do
• How to get started with Selenium IDE, RC, and Grid
• Why I started Sauce Labs (Selenium as a Cloud Service)
• Selenium in 2010... Selenium 2
• Future trends: Adobe Flex, mobile (iphone/ipad, android), HTML5 (canvas, video)
• Best practices for succeeding with Selenium

This is an Agile Denver event and I'm really looking forward to it. Prior to Jason's talk (at 6:30), I'll be participating in a panel with several other Denver Testing Experts to discuss Selenium and related topics. This event will be held at the PPA Event Center (directions). We expect a great turnout, so make sure and get there early! For more information, see the meeting announcement.

On a related meetup note, don't forget next Tuesday's Denver Tech Meetup. Last month was a lot of fun and next week's meetup happens to land on Fat Tuesday. I can't think of a good reason not to come.

If you'd like to RSVP for the Denver Tech Meetup, please do so on LinkedIn or Facebook.

Consulting, SOFEA, Grails and GWT at next week's Denver JUG

Next Wednesday, I'll be at Denver's JUG meeting to talk about Independent Consulting and Building SOFEA Applications with Grails and GWT. The first talk will be a a panel discussion among local independent consultants, including James Goodwill, Matthew McCullough, Tim Berglund and myself.

This session explores the trials and tribulations of an independent consultant. How do you find contracts? Should you setup an LLC, an S-Corp or just be a sole proprietorship? What about health insurance and benefits? Are recruiters helpful or hurtful? Learn lots of tips and tricks to get your dream job and your ideal lifestyle.

The Grails and GWT talk is a preview of a talk I'll be doing at the Rich Web Experience in December. Below is a rewrite of the abstract in first-person.

Earlier this year, I participated in a major enhancement of a high-traffic well-known internet site. The company wanted us to quickly re-architect their site and use a modern Ajax framework to do it with. An Ajax Framework evaluation was done to help the team choose the best framework for their skillset. The application was built with a SOFEA architecture using GWT on the frontend and Grails/REST on the backend.

This talk will cover how Bryan Noll, Scott Nicholls, James Goodwill and I came to choose GWT and Grails, as well as stumbling blocks we encountered along the way. In addition, we'll explore many topics such as raw GWT vs. GXT/SmartGWT, the Maven GWT Plugin, modularizing your code, multiple EntryPoints, MVP, integration testing and JSON parsing with Overlay Types.

If you're in Denver next Wednesday night (November 11th), you should stop by the Denver JUG meeting. It'll be a fun night and there's sure to be a few beers afterward.

Testing GWT Libraries with Selenium and Maven

On Tuesday, I wrote about Running Hosted Mode in GWT Libraries. Today I added an additional module to our project to run Selenium tests against our GWT library. In the process, I discovered some things I needed to modify in my GWT library's pom.xml. I'm writing this post so others can use this setup to write GWT libraries and package them for testing with Selenium.

First of all, I noticed that when you're using the GWT Maven Plugin with a JAR project, it doesn't automatically run gwt:compile or gwt:test in the compile and test phases. I had to explicitly configure the compile goal to run in the compile phase. I also had to add <webappDirectory> to the configuration to compile the JavaScript files into the war directory.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>gwt-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <module>org.appfuse.gwt.core.CoreUI</module>
        <runTarget>index.html</runTarget>
        <webappDirectory>war</webappDirectory>
    </configuration>
    <executions>
        <execution>
            <phase>compile</phase>
            <goals>
                <goal>compile</goal>
            </goals>
        </execution>
    </executions>
</plugin>

To package the generated JavaScript and index.html in the JAR, I added the following <resources> section to the maven-resources-plugin configuration I mentioned in my previous post.

<resource>
    <directory>war</directory>
    <includes>
        <include>core.ui/**</include>
        <include>index.html</include>
    </includes>
</resource>

In addition, I discovered some javax.servlet.* classes in my JAR after running "mvn package". I believe this is caused by the GWT plugin sucking these in when it compiles my ProxyServlet. I excluded them by adding the maven-jar-plugin.

<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
        <excludes>
            <exclude>javax/servlet/**</exclude>
        </excludes>
    </configuration>
</plugin>

After doing this, I was able to publish my JAR with all the contents I needed to run Selenium tests against it.

Testing the GWT Library with Selenium
The module that contains the Selenium tests is a WAR project that uses war overlays, Cargo and Selenium RC. You can read about the Maven setup I use for running Selenium tests in Packaging a SOFEA Application for Distribution.

The major difference when testing a JAR (vs. a WAR), is I had to use the maven-dependency-plugin to unpack the JAR so its contents would get included in the WAR for testing. Below is the configuration I used to accomplish this:

<plugin>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <id>unpack</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>unpack</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>org.appfuse</groupId>
                        <artifactId>gwt-core</artifactId>
                        <version>1.0-SNAPSHOT</version>
                        <type>jar</type>
                        <overWrite>false</overWrite>
                        <excludes>META-INF/**,org/**,javax/**</excludes>
                    </artifactItem>
                </artifactItems>
                <outputDirectory>
                    ${project.build.directory}/${project.build.finalName}
                </outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

Hopefully this will help you develop GWT libraries and run Selenium tests against them. If you have any suggestions for simplifying this configuration, please let me know.

Packaging a SOFEA Application for Distribution

The project I'm working on is a bit different from those I'm used to. I'm used to working on web applications that are hosted on servers and customers access with their browser. SaaS if you will. My current client is different. They're a product company that sells applications and distributes them to customers via download and CD. Their customers install these applications on internal servers (supported servers include WebSphere, WebLogic and Tomcat).

The product I'm currently working on is structured as a SOFEA application and therefore consists of two separate modules - a backend and a frontend. Since it's installed in a servlet container, both modules are WARs and can be installed separately.

Building the backend and frontend as separate projects makes a lot of sense for two reasons:

• In development, different teams can work on the frontend and backend projects.
• Having them as separate projects allows them to be versioned separately.

However, having them as two separate projects does make it a bit more difficult for distribution. I'm writing this post to show you how I recently added support for distributing our application as 2 WARs or 1 WAR using the power of Maven, war overlays and the UrlRewriteFilter.

Project Setup
First of all, we have several different Maven modules, but the most important ones are as follows:

• product-services
• product-client
• product-integration-tests

Of course, our modules aren't really named "product", but you get the point. The services project is really just a WAR project with Spring Security configured. It depends on other JAR modules that the services exist in. The client project is a GWT WAR that has a proxy servlet defined in its web.xml that makes it easier to develop. It also contains some UrlRewrite configuration that allows GWT Log's Remote Logging feature to work. The proxy servlet is something we don't want to ship with our product, so we have a separate web.xml for production vs. development. We do the substitution using the maven-war-plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.0.2</version>
    <configuration>
        <!-- Production web.xml -->
        <webXml>src/main/resources/web.xml</webXml>
        <warSourceDirectory>war</warSourceDirectory>
        <!-- Exclude everything but urlrewrite JAR -->
        <warSourceExcludes>
            WEB-INF/lib/aop**,WEB-INF/lib/commons-**,WEB-INF/lib/gin-**,
            WEB-INF/lib/guice-**,WEB-INF/lib/gwt-**,WEB-INF/lib/gxt-**,
            WEB-INF/lib/junit-**
        </warSourceExcludes>
    </configuration>
</plugin>

I could exclude WEB-INF/lib/** and WEB-INF/classes/**, but in my particular project, we still want UrlRewrite in standalone mode, and we have some i18n properties files in WEB-INF/classes that are served up for Selenium tests.

With this configuration, we have a services WAR and a client WAR that can be installed and used by clients. To collapse them into one and make it possible to ship a single war, I turned to our product-integration-tests module. This module contains Selenium tests that test both types of distributions.

Merging 2 WARs into 1
The most important thing in the product-integration-tests module is that it creates a single WAR. First of all, it uses <packaging>war</packaging> to make this possible. The rest is done using the following 3 steps.

1. Its dependencies include the client and servlet WARs (and Selenium RC for testing).

<dependencies>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-services</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>com.company.app</groupId>
        <artifactId>product-client</artifactId>
        <version>1.0-SNAPSHOT</version>
        <type>war</type>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium.client-drivers</groupId>
        <artifactId>selenium-java-client-driver</artifactId>
        <version>1.0.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

2. The WAR created excludes the "integration-tests" part of the name:

<build>
    <finalName>product-${project.version}</finalName>
    ...
</build>

3. WAR overlays are configured so the everything in the client's WEB-INF directory is excluded from the merged WAR.

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <!-- http://maven.apache.org/plugins/maven-war-plugin/overlays.html -->
        <overlays>
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-services</artifactId>
                <excludes>
                    <!-- TODO: Rename to api.html (this is the Enunciate-generated documentation) -->
                    <exclude>index.html</exclude>
                </excludes>
            </overlay>
            <!-- No server needed in product-client -->
            <overlay>
                <groupId>com.company.app</groupId>
                <artifactId>product-client</artifactId>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
            <!-- Only include META-INF/context.xml to set the ROOT path -->
            <overlay>
                <excludes>
                    <exclude>WEB-INF/**</exclude>
                </excludes>
            </overlay>
        </overlays>
    </configuration>
</plugin>

That's it! Using this configuration, it's possible to distribute a Maven-based SOFEA project as single or multiple WARs. However, there are some nuances.

One thing you might notice is the reference to META-INF/context.xml in the overlays configuration. This subtly highlights one issue I experienced when merging the WARs. In our GWT client, we're using URLs that point to our services at /product-services/*. This works in development (via a proxy servlet) and when the WARs are installed separately - as long as the services WAR is installed at /product-services. However, when they're merged, a little URL rewriting needs to happen. To do this, I added the UrlRewriteFilter to the product-services module and configured a simple rule.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCENGINE urlrewrite PUBLIC "-//tuckey.org//DTD UrlRewrite 3.0//EN"
        "http://tuckey.org/res/dtds/urlrewrite3.0.dtd">

<urlrewrite use-query-string="true">
    <!-- Used when services are merged into WAR with GWT client -->
    <rule>
        <from>^/product-services/(.*)$</from>
        <to type="forward">/$1</to>
    </rule>
</urlrewrite>

Because the services URLs point to the root (/product-services), the merged WAR has to be installed as the ROOT application. When you're using Cargo with Tomcat and want to deploy to ROOT, you have to have a META-INF/context.xml with a path="" reference (ref: CARGO-516).

<Context path=""/>

It is possible to change the URLs in the client to be relative, but this gets seems to get messy when you're using separate WARs. When using relative URLs, I found I had to do solution using cross-context forwarding to get the results I wanted. Using a redirect instead of a forward worked, but resulted in the client talking to the server twice (once to get redirected, a second time for the actual call). Cross-context forwarding is supported by the UrlRewriteFilter and Tomcat, but I'm not sure WebSphere or WebLogic support it. The best solution is probably to change the URLs dynamically at runtime, possibly using some sort of deferred binding technique.

Testing with Cargo and Selenium
Once I had everything merged, I wanted to configure Cargo and Selenium to allow testing both distribution types. If I installed all 3 wars at the same time, the "product-services" WAR would be used by both the product-client.war and the product.war, so I had to use profiles to allow installing the single merged WAR or both WARs. Below is the profile I used for starting Cargo, deploying the merged WAR, starting Selenium RC and running Selenium tests.

<properties>
    <cargo.container>tomcat6x</cargo.container>
    <cargo.container.url>
        http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.zip
    </cargo.container.url>
    <cargo.host>localhost</cargo.host>
    <cargo.port>23433</cargo.port>
    <cargo.wait>false</cargo.wait>
    <cargo.version>1.0</cargo.version>

    <!-- *safari and *iexplore are additional options -->
    <selenium.browser>*firefox</selenium.browser>
</properties>
...
<profile>
    <id>itest-bamboo</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <wait>${cargo.wait}</wait>
                    <container>
                        <containerId>${cargo.container}</containerId>
                        <log>${project.build.directory}/${cargo.container}/cargo.log</log>
                        <zipUrlInstaller>
                            <url>${cargo.container.url}</url>
                            <installDir>${installDir}</installDir>
                        </zipUrlInstaller>
                    </container>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <!-- Deploy as ROOT since XHR requests are made to /product-services -->
                        <deployables>
                            <deployable>
                                <properties>
                                    <context>ROOT</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
                <executions>
                    <execution>
                        <id>start-container</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-container</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>selenium-maven-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start-server</goal>
                        </goals>
                        <configuration>
                            <background>true</background>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <excludes>
                                <exclude>none</exclude>
                            </excludes>
                            <includes>
                                <include>**/*SeleniumTest.java</include>
                            </includes>
                            <systemProperties>
                                <property>
                                    <name>selenium.browser</name>
                                    <value>${selenium.browser}</value>
                                </property>
                                <property>
                                    <name>cargo.port</name>
                                    <value>${cargo.port}</value>
                                </property>
                            </systemProperties>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

This profile is run by our Bamboo nightly tests with mvn install -Pitest-bamboo. The 2nd profile I added doesn't install the project's WAR, but instead installs the two separate WARs. Running mvn install -Pitest-bamboo,multiple-wars executes the Selenium tests against the multi-WAR distribution.

<profile>
    <id>multiple-wars</id>
    <activation>
        <activeByDefault>false</activeByDefault>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.cargo</groupId>
                <artifactId>cargo-maven2-plugin</artifactId>
                <version>${cargo.version}</version>
                <configuration>
                    <configuration>
                        <home>${project.build.directory}/${cargo.container}/container</home>
                        <properties>
                            <cargo.hostname>${cargo.host}</cargo.hostname>
                            <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                        </properties>
                        <deployables>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-client</artifactId>
                                <pingURL>http://${cargo.host}:${cargo.port}/product-client/index.html</pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-client</context>
                                </properties>
                            </deployable>
                            <deployable>
                                <groupId>com.company.app</groupId>
                                <artifactId>product-services</artifactId>
                                <pingURL>
                                    http://${cargo.host}:${cargo.port}/project-services/index.jspx
                                </pingURL>
                                <type>war</type>
                                <properties>
                                    <context>/product-services</context>
                                </properties>
                            </deployable>
                        </deployables>
                    </configuration>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

I won't be including any information on authoring Selenium tests because there's already many good references. I encourage you to checkout the following if you're looking for Selenium testing techniques.

• Carlos Sanchez's Selenium Test Environment for AppFuse
• Testing Ext JS & Ext GWT Applications With Selenium
• Testing Nexus with Selenium: A lesson in complex UI testing (Part 1)
• Testing Nexus with Selenium: A lesson in complex UI testing (Part 2)
• Testing Nexus with Selenium: A lesson in complex UI testing (Part 3)

Summary
This article has shown you how I used Maven, war overlays and the UrlRewriteFilter to allow create different distributions of a SOFEA application. I'm still not sure which packaging (1 WAR vs. 2) mechanism is best, but it's nice to know there's options. If you package and distribute SOFEA applications, I'd love to hear about your experience in this area.