Web Application Testing with Selenium by Jason Huggins
This evening, I attended Agile Denver's monthly meeting to listen to Jason Huggins talk about Selenium. The meeting started off with a panel on UI testing that I participated in. The most interesting part of this panel (for me) was meeting the other panelists and learning about their expertise. Folks from Red Pine Studios in Boulder video taped both the panel and presentation. Hopefully it will be published online in the near future.
Below are my notes from Jason's talk. Please keep in mind that most of these are his words, not mine.
Jason is the Executive Software Chef at Sauce Labs. He often experiments with new recipes and is one of the creators of Selenium. He worked at Google and helped them build and use a Selenium Farm to test Gmail and Google Docs. Selenium was inspired by ThoughtWorks Expense Report system and its "Add Row" button. The button caused so many issues, they needed a way to write a test that could be run in multiple browsers.
Selenium Remote Control (RC) was the next product produced by the project. Selenium RC allows you to write your tests in any language. A Selenium server interprets the requests and turns them into browser manipulation events. Finally, Selenium Grid was developed to leverage Selenium's HTTP architecture to allow parallel execution across servers.
Cloud computing is a wonderful use case for functional testing. Selenium Hub is a gateway into the Selenium Grid that routes the test request to particular browsers and platforms. Sauce Labs has a version of Selenium Grid that runs in the cloud.
Selenium is slow. Functional tests will always be slower than unit tests. Until the browsers can launch faster, there's always going to be speed issues. Parallel-ization can solve some of these and is something you should think about right away.
Everyone seems to build a framework on top of Selenium. If you do this, make sure and write your DSL in terms of intent and then map it to Selenium actions.
Look for abstractions so you're not writing your Selenium tests with its API. It's too much like Assembler.
K.I.S.S. - don't write large tests, just do small ones. Often, when functional tests fail, they tell you something failed, but they don't tell you what failed. The shortest possible functional tests help reduce the scope of where a problem can be. Other benefits of short tests are they're easier to read and easier to write.
The big thing in Selenium 2.0 is a merger with WebDriver. The nice thing about WebDriver is it gets rid of Selenium RC and allows you to drive the browser with a low-level API. For example, you use C++ to drive IE. Basically, every language will talk to the C driver. Except for Firefox, the connection and control is done through telnet. Selenium 2 should fix all the problems with Selenium 1, but also allow you to still use Selenium RC if you want to do grid-style testing.
Selenium 2's API is about finding elements and interacting with those elements. Also, it's entirely backwards compatible, so you can use the old API.
At this point, my laptop's battery died and I was unable to take any more notes. However, I was able to see some pretty slick demos, particularly Jason's company's Sauce onDemand cloud testing services. All you need to do to run your tests in the cloud is change how you initialize Selenium. A kick-ass feature this service provides is video playback (a.k.a. Castro). I'm currently using Selenium's screenshot functionality, but it doesn't hold a candle to the ability to watch a video playback of your tests. Jason also showed us a demo of using Castro and Selenium 2 to create a screencast on-the-fly. Very cool stuff.
My only question after seeing this talk is what's the difference between BrowserMob and Sauce Labs? Both companies were founded by Selenium committers and seem to offer competing projects. My gut feel is that BrowserMob is best for performance/load testing and Sauce Labs is best for running your tests in the cloud.
Posted by uberVU - social comments on February 16, 2010 at 09:34 AM MST #
Having worked with both of these products they offer 2 different things from Selenium.
BrowserMob is concerned with monitoring your site and doing load and performance testings.
Sauce labs is all about running your functional tests as fast as you can by parallelizing your scripts.
Posted by David Burns on February 16, 2010 at 09:34 AM MST #
ceo, cofounder, sauce labs inc
Posted by john dunham on February 16, 2010 at 04:19 PM MST #
Great post Matt!
We are starting to use Selenium to test our tool that was developed using GWT/GXT. I remember that in one of your posts you told something about creating fixed ids for all GXT widgets. Do you have any other experiences and best practices regarding texting GXT with Selenium?
Posted by Otavio Silva on February 21, 2010 at 01:59 PM MST #
Posted by Marc Guillemot on February 21, 2010 at 02:00 PM MST #