« Java CMS Systems | Main | The Playoffs Begin »
20070309 Friday March 09, 2007

Integrating Selenium with Maven 2 I spent some time this past week integrating Selenium with Maven 2. This post is designed to show you how to do this in your Maven 2 projects.

First of all, there were two types of testing scenarios I wanted to make possible. The first was to allow HTML-based tests that web designers could create and run with Selenium IDE. As far as I know, Selenium IDE is capable of recording and exporting Java-based tests (powered by TestNG or JUnit), but I don't believe it's capable of playing them back. So for Java Developers, I wanted to allow them to write their tests in Java.

To get Maven to run HTML-based tests, the easiest way seems to be using the <selenese> Ant task. I tried Mavenium as well, but it 1) doesn't use the latest version of Selenium RC and 2) reports success when tests fail. Below is a Maven profile that I'm using in an AppFuse-based project to run HTML tests. 

<profiles>
    <profile>
        <id>integration-test</id>
        <activation>
            <property>
                <name>!maven.test.skip</name>
            </property>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.codehaus.cargo</groupId>
                    <artifactId>cargo-maven2-plugin</artifactId>
                    <version>0.3-SNAPSHOT</version>
                    <configuration>
                        <wait>${cargo.wait}</wait>
                        <container>
                            <containerId>${cargo.container}</containerId>
                            <!--home>${cargo.container.home}</home-->
                            <zipUrlInstaller>
                                <url>${cargo.container.url}</url>
                                <installDir>${installDir}</installDir>
                            </zipUrlInstaller>
                        </container>
                        <configuration>
                            <home>${project.build.directory}/${cargo.container}/container</home>
                            <properties>
                                <cargo.hostname>${cargo.host}</cargo.hostname>
                                <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                            </properties>
                        </configuration>
                    </configuration>
                    <executions>
                        <execution>
                            <id>start-container</id>
                            <phase>pre-integration-test</phase>
                            <goals>
                                <goal>start</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>stop-container</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>stop</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <artifactId>maven-antrun-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>launch-selenium</id>
                            <phase>integration-test</phase>
                            <configuration>
                                <tasks>
                                    <taskdef resource="selenium-ant.properties">
                                        <classpath refid="maven.plugin.classpath"/>
                                    </taskdef>
                                    <selenese suite="src/test/resources/selenium/TestSuite.html"
                                              browser="*firefox" timeoutInSeconds="180" port="5555"
                                              results="${project.build.directory}/selenium-firefox-results.html"
                                              startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
                                </tasks>
                            </configuration>
                            <goals>
                                <goal>run</goal>
                            </goals>
                        </execution>
                    </executions>
                    <dependencies>
                        <dependency>
                            <groupId>ant</groupId>
                            <artifactId>ant-nodeps</artifactId>
                            <version>1.6.5</version>
                        </dependency>
                        <dependency>
                            <groupId>org.openqa.selenium.server</groupId>
                            <artifactId>selenium-server</artifactId>
                            <version>0.9.1-SNAPSHOT</version>
                        </dependency>
                    </dependencies>
                </plugin>
            </plugins>
        </build>
    </profile>
    <profile>
        <id>selenium-ie</id>
        <activation>
            <os>
                <family>windows</family>
            </os>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-antrun-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>launch-selenium</id>
                            <phase>integration-test</phase>
                            <configuration>
                                <tasks>
                                    <taskdef resource="selenium-ant.properties">
                                        <classpath refid="maven.plugin.classpath"/>
                                    </taskdef>
                                    <selenese suite="src/test/resources/selenium/TestSuite.html"
                                              browser="*firefox" timeoutInSeconds="180" port="5555"
                                              results="${project.build.directory}/selenium-firefox-results.html"
                                              startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
                                    <selenese suite="src/test/resources/selenium/TestSuite.html"
                                              browser="*iexplore" timeoutInSeconds="180" port="5555"
                                              results="${project.build.directory}/selenium-ie-results.html"
                                              startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/"/>
                                </tasks>
                            </configuration>
                            <goals>
                                <goal>run</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

The above setup will allow you to run Selenium tests in Firefox, and in IE as well when you're on Windows. I tried to get Safari to work on the Mac, but it just opens Safari and hangs.

HTML tests are great for non-programmers, but what about developers that prefer Java and want test reports to be included in the surefire-plugin's reports? That's easy enough. First of all, put your tests in a particular package so they can be excluded from the normal testing cycle. I used a webapp.selenium package. I configured the surefire-plugin to exclude these tests: 

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <excludes>
            <exclude>**/selenium/*Test.java</exclude>
        </excludes>
    </configuration>
</plugin>

Then I added the newly released selenium-maven-plugin to my "integration-test" profile and configured surefire to run the Selenium Java tests. 

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>selenium-maven-plugin</artifactId>
    <version>1.0-beta-1</version>  
    <executions>
        <execution>
            <id>start-selenium</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start-server</goal>
            </goals>
            <configuration>
                <background>true</background>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <executions>
        <execution>
            <id>surefire-it</id>
            <phase>integration-test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <excludes>
                    <exclude>none</exclude>
                </excludes>
                <includes>
                    <include>**/selenium/*Test.java</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>

Yeah, Maven can be quite verbose when configuring profiles. I contacted the Maven list to see if it's possible to simplify all this XML, but so far haven't found a solution.

If you'd like to see a pom.xml with the Selenium bits and a profile that runs both HTML and Java-based tests, click here (JavaScript needs to be enabled for this to work).

NOTE: I used 0.9.1-SNAPSHOT of Selenium Server because it solves issues with the latest version of Firefox.

This brings up a related question I asked on the AppFuse mailing list a couple of days ago:

Do you use the Canoo WebTests? If not, how do you do UI testing? If so, would you prefer Selenium?

If you've tried AppFuse 2.x and have an opinion, please add a comment. Personally, I like Selenium, but I like how Canoo WebTest can be somewhat friendly to designers and allow i18n testing with Ant's property file support. With Selenium, you have to use parse/replace or Java tests to do i18n testing. Then again, if you need to test a lot of Ajax functionality, it's likely that Selenium will work much better for you. Posted in Java at Mar 09 2007, 10:35:04 AM MST 16 Comments

Comments:

I did largely the same thing, except I start up a local in-process copy of Jetty to serve up my web application, as well as run Selenium Server (also in-process). I've started rolling this together into a tapestry-test module: http://tapestry.apache.org/tapestry5/tapestry-test/

Posted by Howard Lewis Ship on March 09, 2007 at 02:13 PM MST #

+1 for Selenium. It's much simpler and we can hope for a better i18n support in the future.

Posted by sebnoumea on March 09, 2007 at 04:18 PM MST #

What about the use of Selenium to test JSF UI's? Don't the auto-generated html element id's throw selenium for a loop?

Posted by Charles Crouch on March 09, 2007 at 04:31 PM MST #

Matt, With respect to Selenium IDE, its quite cool to record your tests in it, then export them to java/junit. Some very small cleanup can be required, but it is otherwise a good approach. - Paul

Posted by Paul Hammant on March 11, 2007 at 08:26 AM MDT #

http://docs.codehaus.org/display/MAVENUSER/Maven+and+Selenium

Posted by Carlos Sanchez on March 12, 2007 at 07:44 PM MDT #

How to deploy app as ROOT: http://jira.codehaus.org/browse/CARGO-516

Posted by ros on June 21, 2007 at 04:29 PM MDT #

Ros - the answer is described in the bug above - you need to configure Cargo and a META-INF/context.xml in your webapp.

Posted by Matt Raible on June 21, 2007 at 04:33 PM MDT #

[Trackback] Selenium Selenium ???? Selenium ??? ???? Selenium Reference ???? Selenium? Maven2?? ???? ??

Posted by Confluence: WEB2.0 on July 16, 2007 at 09:14 AM MDT #

Thanks for the detailed example.

In order to get a very similar pom to run with the selenese ant task under windows I had to change the used ant taskdef-classpathref to maven.test.classpath (in your example it's maven.plugin.classpath). maven.plugin.classpath includes under windows only the core Ant and Maven jars. The Selenium jars (with the SeleneseAntTask) are missing -- don't know why, with MacOSX maven.plugin.classpath ran fine.

Posted by Jonas Fleer on August 09, 2007 at 09:36 AM MDT #

I would like to contribute a very important addendum to this tutorial. If you provide custom excludes for the maven-surefire-plugin, as you have done to exclude the selenium tests from the unit test run, it is imperative that you restore the inner class filter, or else the runner will fail attempting to scan inner classes for test methods.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <excludes>
      <exclude>**/selenium/*Test.java</exclude>
      <exclude>**/*$*</exclude>
    </excludes>
  </configuration>
</plugin>

Posted by Dan Allen on March 14, 2008 at 01:34 PM MDT #

At kicktipp we ran into a few problems while running our intergation tests: we don't have a x-Server on our Continous integration (CI) server and wanted to run selenes HTML Tests as some more complex Selenium JUnit Tests using the selenium RC.

If running on linux you won't always have a running X-Server for starting firefox, especially when running on a CI. Therefore you need a Xvfb. Add this execution to your selenium plugin: 

<execution>
  <id>xvfb</id>
  <phase>pre-integration-test</phase>
  <goals>
    <goal>xvfb</goal>
  </goals>
  </execution>

Additionally you don't need an ant task if you want to run your selenese Tests, you can use selenium:selenese goal:

<execution>
  <id>selenese</id>
  <phase>integration-test</phase>
  <goals>
    <goal>selenese</goal>
  </goals>
  <configuration>
    <port>4445</port>
    <browser>*firefox</browser>
    <suite>src/test/resources/tipprunde/tippspiel.html</suite>
    <startURL>http://localhost:8080/</startURL>
  </configuration>
</execution>

You need a diffrent port here as the selenium-start-server goal is still executing on port 4444 and selenese does start its own server.

But there is another problem to fix: the selenese goal does not respect the running xvfb. Other than the server-start goal it does not read the display properties from target/selenium/display.properties. As there is no way to pass the display to this goal, you should set an environment variable 

export DISPLAY=\:20
in your start-up script of your favourite CI-server

It worked for us at least.

Posted by Janning Vygen on October 08, 2008 at 03:53 AM MDT #

[Trackback] Tagged your site as selenium at iLinkShare!

Posted by iLinkShare (Web 2.0 linksharing) on October 31, 2008 at 11:03 AM MDT #

Matt, Great writeup! We got things working relatively similar to the way you describe. However, we've run into a glitch on Linux where all tests pass, but mvn throws an exit status of "1" when xvfb is run from a goal as above. This is causing TeamCity to break. Have you seen this? Any idea how to circumvent it? Cheers!

Posted by Aaron Stewart on December 12, 2008 at 11:40 AM MST #

Hi,

I see that your startUrl looks like this: startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/

isn't the last part stripped off? so that it will look like startURL="http://${cargo.host}:${cargo.port}/ ? It does not produce a error.

In my case i kinda need this but it is stripped off. Anyone have more information about this?

Posted by kukudas on February 19, 2009 at 08:17 AM MST #

If you want to deploy your app at ROOT (which I didn't), you can use the following configuration.

<plugin>
    <groupId>org.codehaus.cargo</groupId>
    <artifactId>cargo-maven2-plugin</artifactId>
    <version>1.0-beta-2</version>
    <configuration>
        <wait>${cargo.wait}</wait>
        <container>
            <containerId>${cargo.container}</containerId>
            <!--home>${cargo.container.home}</home-->
            <zipUrlInstaller>
                <url>${cargo.container.url}</url>
                <installDir>${installDir}</installDir>
            </zipUrlInstaller>
        </container>
        <configuration>
            <home>${project.build.directory}/${cargo.container}/container</home>
            <properties>
                <cargo.hostname>${cargo.host}</cargo.hostname>
                <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
            </properties>
            <deployables>
                <deployable>
                    <properties>
                        <context>ROOT</context>
                    </properties>
                </deployable>
            </deployables>
        </configuration>
    </configuration>
    <executions>
        <execution>
            <id>start-container</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start</goal>
            </goals>
        </execution>
        <execution>
            <id>stop-container</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>stop</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Posted by Matt Raible on February 19, 2009 at 10:40 AM MST #

Hi, thx for your response Matt. I don't want to deploy my app at root. I want to do it like you but if i use startURL="http://${cargo.host}:${cargo.port}/${project.build.finalName}/ the last part gets stripped off. Do i have something missed? thx kukudas

Posted by kukudas on February 20, 2009 at 01:01 AM MST #

Post a Comment:
  • HTML Syntax: Allowed
Click me to subscribe
Matt Raible is a UI Architect specializing in open source web frameworks. Contact me for rates.
« July 2009
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
 
       
Today

Recent Entries

Tag Cloud

abbie ajax appfuse denver grails gwt hibernate jack java jsf linkedin maven myfaces oscon rails skiing softwaresummit spring springmvc struts struts2 tapestry travel webframeworks wicket