I spent some time this past week integrating Selenium with Maven 2. This post is designed to show you how to do this in your Maven 2 projects.
First of all, there were two types of testing scenarios I wanted to make possible. The first was to allow HTML-based tests that web designers could create and run with Selenium IDE. As far as I know, Selenium IDE is capable of recording and exporting Java-based tests (powered by TestNG or JUnit), but I don't believe it's capable of playing them back. So for Java Developers, I wanted to allow them to write their tests in Java.
To get Maven to run HTML-based tests, the easiest way seems to be using the <selenese> Ant task. I tried Mavenium as well, but it 1) doesn't use the latest version of Selenium RC and 2) reports success when tests fail. Below is a Maven profile that I'm using in an AppFuse-based project to run HTML tests.
<profiles>
<profile>
<id>integration-test</id>
<activation>
<property>
<name>!maven.test.skip</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>0.3-SNAPSHOT</version>
<configuration>
<wait>${cargo.wait}</wait>
<container>
<containerId>${cargo.container}</containerId>
<!--home>${cargo.container.home}</home-->
<zipUrlInstaller>
<url>${cargo.container.url}</url>
<installDir>${installDir}</installDir>
</zipUrlInstaller>
</container>
<configuration>
<home>${project.build.directory}/${cargo.container}/container</home>
<properties>
<cargo.hostname>${cargo.host}</cargo.hostname>
<cargo.servlet.port>${cargo.port}</cargo.servlet.port>
</properties>
</configuration>
</configuration>
<executions>
<execution>
<id>start-container</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-container</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>launch-selenium</id>
<phase>integration-test</phase>
<configuration>
<tasks>
<taskdef resource="selenium-ant.properties">
<classpath refid="maven.plugin.classpath"/>
</taskdef>
<selenese suite="src/test/resources/selenium/TestSuite.html"
browser="*firefox" timeoutInSeconds="180" port="5555"
results="${project.build.directory}/selenium-firefox-results.html"
startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>ant</groupId>
<artifactId>ant-nodeps</artifactId>
<version>1.6.5</version>
</dependency>
<dependency>
<groupId>org.openqa.selenium.server</groupId>
<artifactId>selenium-server</artifactId>
<version>0.9.1-SNAPSHOT</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>selenium-ie</id>
<activation>
<os>
<family>windows</family>
</os>
</activation>
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>launch-selenium</id>
<phase>integration-test</phase>
<configuration>
<tasks>
<taskdef resource="selenium-ant.properties">
<classpath refid="maven.plugin.classpath"/>
</taskdef>
<selenese suite="src/test/resources/selenium/TestSuite.html"
browser="*firefox" timeoutInSeconds="180" port="5555"
results="${project.build.directory}/selenium-firefox-results.html"
startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
<selenese suite="src/test/resources/selenium/TestSuite.html"
browser="*iexplore" timeoutInSeconds="180" port="5555"
results="${project.build.directory}/selenium-ie-results.html"
startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
The above setup will allow you to run Selenium tests in Firefox, and in IE as well when you're on Windows. I tried to get Safari to work on the Mac, but it just opens Safari and hangs.
HTML tests are great for non-programmers, but what about developers that prefer Java and want test reports to be included in the surefire-plugin's reports? That's easy enough. First of all, put your tests in a particular package so they can be excluded from the normal testing cycle. I used a webapp.selenium package. I configured the surefire-plugin to exclude these tests:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/selenium/*Test.java</exclude>
</excludes>
</configuration>
</plugin>
Then I added the newly released selenium-maven-plugin to my "integration-test" profile and configured surefire to run the Selenium Java tests.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>selenium-maven-plugin</artifactId>
<version>1.0-beta-1</version>
<executions>
<execution>
<id>start-selenium</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start-server</goal>
</goals>
<configuration>
<background>true</background>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/selenium/*Test.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
Yeah, Maven can be quite verbose when configuring profiles. I contacted the Maven list to see if it's possible to simplify all this XML, but so far haven't found a solution.
If you'd like to see a pom.xml with the Selenium bits and a profile that runs both HTML and Java-based tests, click here (JavaScript needs to be enabled for this to work).
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/selenium/*Test.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>openqa</id>
<url>http://maven.openqa.org</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.openqa.selenium.client-drivers</groupId>
<artifactId>selenium-java-client-driver</artifactId>
<version>0.9.1-SNAPSHOT</version>
<scope>test</scope>
</dependency>
</dependencies>
<profiles>
<profile>
<id>integration-test</id>
<activation>
<property>
<name>!maven.test.skip</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>0.3-SNAPSHOT</version>
<configuration>
<wait>${cargo.wait}</wait>
<container>
<containerId>${cargo.container}</containerId>
<!--home>${cargo.container.home}</home-->
<zipUrlInstaller>
<url>${cargo.container.url}</url>
<installDir>${installDir}</installDir>
</zipUrlInstaller>
</container>
<configuration>
<home>${project.build.directory}/${cargo.container}/container</home>
<properties>
<cargo.hostname>${cargo.host}</cargo.hostname>
<cargo.servlet.port>${cargo.port}</cargo.servlet.port>
</properties>
</configuration>
</configuration>
<executions>
<execution>
<id>start-container</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-container</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- Configuration to run Java-based Selenium tests -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>selenium-maven-plugin</artifactId>
<version>1.0-beta-1</version>
<executions>
<execution>
<id>start-selenium</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start-server</goal>
</goals>
<configuration>
<background>true</background>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/selenium/*Test.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<!-- End of configuration for Java-based tests -->
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>launch-selenium</id>
<phase>integration-test</phase>
<configuration>
<tasks>
<taskdef resource="selenium-ant.properties">
<classpath refid="maven.plugin.classpath"/>
</taskdef>
<selenese suite="src/test/resources/selenium/TestSuite.html"
browser="*firefox" timeoutInSeconds="180" port="5555"
results="${project.build.directory}/selenium-firefox-results.html"
startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>ant</groupId>
<artifactId>ant-nodeps</artifactId>
<version>1.6.5</version>
</dependency>
<dependency>
<groupId>org.openqa.selenium.server</groupId>
<artifactId>selenium-server</artifactId>
<version>0.9.1-SNAPSHOT</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>selenium-ie</id>
<activation>
<os>
<family>windows</family>
</os>
</activation>
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>launch-selenium</id>
<phase>integration-test</phase>
<configuration>
<tasks>
<taskdef resource="selenium-ant.properties">
<classpath refid="maven.plugin.classpath"/>
</taskdef>
<selenese suite="src/test/resources/selenium/TestSuite.html"
browser="*firefox" timeoutInSeconds="180" port="5555"
results="${project.build.directory}/selenium-firefox-results.html"
startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
<selenese suite="src/test/resources/selenium/TestSuite.html"
browser="*iexplore" timeoutInSeconds="180" port="5555"
results="${project.build.directory}/selenium-ie-results.html"
startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
NOTE: I used 0.9.1-SNAPSHOT of Selenium Server because it solves issues with the latest version of Firefox.
This brings up a related question I asked on the AppFuse mailing list a couple of days ago:
Do you use the Canoo WebTests? If not, how do you do UI testing? If so, would you prefer Selenium?
If you've tried AppFuse 2.x and have an opinion, please add a comment. Personally, I like Selenium, but I like how Canoo WebTest can be somewhat friendly to designers and allow i18n testing with Ant's property file support. With Selenium, you have to use parse/replace or Java tests to do i18n testing. Then again, if you need to test a lot of Ajax functionality, it's likely that Selenium will work much better for you.
