Matt RaibleMatt Raible is a Java Champion and Developer Advocate at Okta. developer.okta.com

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Canoo WebTest vs. Selenium

From a message I sent earlier to the AppFuse user mailing list:

After getting Dojo's DropdownDatePicker working with both Struts and Spring MVC, I've run into a snag:

You can no longer use Canoo WebTest to enter dates in a form.

It doesn't find the form element because HtmlUnit (the underlying browser implementation for Canoo) doesn't understand/render Dojo's components.

For users, workarounds include:

1. Don't write tests for entering data on forms that have required dates.
2. Don't make dates required.

Either way, it seems logical to move to Selenium for UI testing.

What do you think? Are Dojo-based components the future of Ajax support in Java web frameworks? If so, I wish Dojo allowed more "progressive enhancement" features instead of the invalid HTML feature it seems to rely on now. I'm more than willing to support Dojo if that's what AppFuse users want. However, in my experience with Struts 2's datetimepicker, it seems pretty slow. Of course, if I could render my own input element and then add a datepicker next to it, I'd be much happier. AFAICT, the Struts' datetimepicker doesn't allow you to do this, but I could be wrong.

Of course, if Dojo 0.9 lives up to the hype and is much faster - maybe I should quit caring about Hijax and just start writing those Selenium tests?

For users of those "next-gen" Java frameworks (like Seam, Grails and Wicket), what do you use for UI testing? Does Canoo WebTest work for you when you have a lot of Ajax functionality? If not, do you use Selenium or leave UI testing up to the QA folks?

Update: I ended up using jscalendar for both Spring MVC and Struts 2's datepickers. It's simply the best tool for the job. Dojo is too bloated IMO (240K for a datepicker!) for such a small feature. This allows us to worry about getting AppFuse 2.0 out the door without migrating to Selenium first. We will eventually migrate to Selenium (it's a better tool for the job IMO), but probably not until 2.1. The only question is what's the best way to distribute Selenium tests?

Posted in Java at May 31 2007, 12:25:03 AM MDT 19 Comments
Comments:

What about "Yahoo! User Interface Library": http://developer.yahoo.com/yui/

And http://developer.yahoo.com/yui/examples/calendar/quickstart/1.html

Posted by Gilberto on May 31, 2007 at 07:00 AM MDT #

Good article: http://www.ddj.com/dept/webservices/199203087?pgno=1

Gilberto

Posted by Gilberto on May 31, 2007 at 07:04 AM MDT #

From the AppFuse mailing list:

jQuery, Yahoo's YUI and MooTools are all nice libraries. However, since Tapestry and Struts 2 bake in Dojo by default, it makes it a little difficult to not include it. Maybe we should just make it easier to include a different Ajax framework than Scriptaculous? Right now, we only use it for highlighting and setting form focus - maybe we should move those into global.js (which I've thought of renaming, suggestions welcome) and make them independent of any framework. Or we could continue our abstractions on top extractions and use Ext JS or jMaki - I believe both libraries allow you to use other libraries underneath.

The good news is we don't plan on Ajaxifying pages by default until AppFuse 2.2, so we have some time to make these decisions. For those of you who have experience with any of these libraries, now is the time to speak up. Personally, I still like Scriptaculous and Prototype, but I haven't worked as much with the others.

Posted by Matt Raible on May 31, 2007 at 12:23 PM MDT #

I find Dojo hard to work with, from release to release all the Struts widgets break :(, documentation is not good and now I find plenty of broken links everywhere. It is getting better and better, but honestly, it is somewhat painful (if you look at the code for the Struts widgets you will find many bug fixes and workarounds for Dojo problems). If I had it my way, Struts would be using YUI instead, but I got a little bit late to the party :) By the way the DatePicker is going to be a lot better on Struts 2.1 (it is actually gonna work!)

Posted by musachy on May 31, 2007 at 01:25 PM MDT #

musachy - have their been any discussions on allowing alternate Ajax frameworks to be plugged in? Or allowing progressive enhancement so the Ajaxified output from Struts isn't an all-or-nothing proposition? This question should probably be aimed at Jesse as well. However, he's a committer on Dojo, so he might be a bit biased on Tapestry's default Ajax framework. ;-)

Posted by Matt Raible on May 31, 2007 at 02:00 PM MDT #

Well, some people have mentioned, but there hasn't been any discussion on it. The Dojo tags will be on their own plugin in 2.1, so there is no reason why a YUI/other_framework plugin couldn't be done. I want to start working on such a plugin(YUI), outside of Struts (googlecode), but not until the Dojo one is stable (should be on 2.1). But I've been really busy lately, playing Guild Wars ;)

Posted by Musachy Barroso on May 31, 2007 at 06:46 PM MDT #

For Nuxeo 5, which is based, on the front-end, on Seam, JSF and Ajax4Jsf, I had very limited success with Canoo webtest (I only could test 1 or 2 screens until I got untractable JavaScript errors).

We switched afterwards to Selenium, and it just works! Another benefit is that our tests don’t need to be written by the software engineers (using Selenium IDE), but still can be run using automated tools (TestNG and Hudson).

Stefane Fermigier, CEO, Nuxeo

Posted by Stefane Fermigier on June 01, 2007 at 01:06 AM MDT #

Just a question: have you tried to replace the htmlunit jar provided with WebTest with HtmlUnit's latest build? Many changes have been made since 1.11 what makes it work nicely for instance with prototype (no idea what concerns Dojo).

More generally concerning WebTest vs Selenium: Selenium is a nice tool (and it's weaknesses are not the ones that are often presented) but in test automation you have "automation" which means that it is a software engineering activity. If recorded tests are what you want, then Selenium is probably fine but you have to know that it's the less cost effective way to perform test automation (even if it makes sense for some very special cases). If you want to perform what I call "professional test automation", then using WebTest is more effective (ok, I should write a special post to develop that but you can have a look at WebTest new "Key features" doc).

[for info: as I'm HtmlUnit and WebTest committer, I'm surely a bit partial]

Posted by Marc Guillemot on June 01, 2007 at 04:00 AM MDT #

I think that this discussion is more about the testing tool than the problem with the date picker. It identifies something that is very reveling about the capability of the two tools. Selenium is the "real deal." You are writing web applications for the browser and Selenium exercises that client. In fact, it can exercise different implementations of that client. You cannot beat that, no matter how good your emulation layer may be, just as Wine will never be better than VMware.

That isn't to say that WebTest isn't useful, it just cannot be the final line of defense. The QA team could always use Selenium (or something comparable, including manual testing). Either way, it has to be done or else your client becomes part of your testing strategy. Selenium may not be perfect, but that project is going in the right direction.

Posted by Dan Allen on June 01, 2007 at 06:54 AM MDT #

AFAIK Tapestry 5 will use Selenium

Posted by Renat Zubairov on June 01, 2007 at 01:27 PM MDT #

That's by the way a very valid concern about AJAX/NON AJAX approaches as all or nothing approach. We've discussed it allot on our application and concluded that we would do as much as we can to enable our application to be executed (and tested!) without Javascript. Fortunately Tapestry help us with that because AJAX part even in Tap4.1 is so nicely done (thanks Jesse) that in majority of the cases (90%) application works with or without javascript absolutely consistently. It's quite obvious that in component oriented frameworks AJAX will only redraw required components (when it's there) and when Javascript is disabled (not interpreted) whole page is redrawn.

Posted by Renat Zubairov on June 01, 2007 at 01:34 PM MDT #

Different tools for different tasks. As Dan says above, you can't get better than the real deal. Unless you don't have the real deal available. Or the Selenium APIs or reporting tools get in your way. Or you want to minimize resource consumption. Selenium has its place, as do WebTest and HtmlUnit.

I should mention that HtmlUnit (the engine behind WebTest) has integrated the unit tests from the Prototype and jQuery projects into its own unit tests, though they don't all pass right now.

Posted by Daniel Gredler on June 02, 2007 at 11:26 PM MDT #

I have a client where the Internet Explorer of the QA team members is configured not to display a popup message when a js error occurs (standard company settings). The consequence is that they are really unlikely to discover js errors in the applications they test.
Sounds silly, isn't it? In fact you would have exactly the same with Selenium!

Automated tests are here to be run as often as possible and to give the best feedback to find errors as early as possible (when they are cheap to fix) to improve the application's quality. Under this aspect, a simulated browser can catch more errors and therefore be a greater help.

I don't know what the "direction" of Selenium is. I can imagine that for instance Selenium IDE could catch js errors. I don't know for the hta but this can surely not be done with Selenium Core. Js errors are only one example, other kinds of errors should be detected as soon as possible, even if a "normal" browser hide them.

In fact the discussion here is a bit different and depends not only of the quality of the tools but of the kind of automated testers the AppFuse users are. For instance Selenium is more "visual" and some people better understand what they "see". On the other side, WebTest is for instance easier to setup and tests are cheaper to maintain.

Posted by Marc Guillemot on June 05, 2007 at 02:21 AM MDT #

Hi, I'm a WebTest committer (full disclosure).

Selenium and Canoo WebTest are representatives of two distinct classes of test tools. Test tools consist of a simulation and a verification part. We're currently only discussing the first aspect (the second would also be worth discussing).

Type A tools like WebTest use an internal abstraction of a browser and work against an expectation (if not even specification) of protocols. Type B tools like Selenium do the simulation part by remote-controlling a browser.

Some claim that only B is the "real deal" but be reminded that you depend on all the accidental behaviour that your chosen browser happens to show. You are doomed to fall for the same traps that manual testers do: assuming the webapp is correct because the errors are hidden. That's only as good as "programming by coincidence". Unfortunately, that is the "real deal" in many projects.

I for myself would never feel safe behind such a weak "defense line".

Posted by Dierk on June 05, 2007 at 06:20 AM MDT #

For testing, definitely go with Selenium.

>If not, do you use Selenium or leave UI testing up to the QA folks? QA is time consuming. Keep selenium tests for widgets in the repository also helps other developers can work on the code.

QA drawbacks include having invalid and duplicate bugs that just take time.

Selenium is useful for testing widgets (like a date picker) and pages. You might also want to check out Selenium-RC to write all your tests in Java.

The Dojo framework depends on eval, which can make debugging very hard. The date picker is slow and heavy and has more features than you need. Stick with the DynArch one.

Posted by Garrett on June 22, 2007 at 02:29 PM MDT #

Hi, everyone... (full disclosure: I'm the creator of Selenium Core)...

2 words regarding WebTest vs Selenium: "Use Both!"

1) Selenium (and all browser-drivers) gives you "last mile" coverage of the actual customer/client usage in a real browser. But there's still work to be done. My current project at Google is to work on all those annoying things like unexpected modal dialog windows that currently stop Selenium in its tracks without browser-specific workarounds. The downside to browser-driving is that it's slow... unless you have a grid of servers from every platform (Mac, Linux, Windows) where you can run your tests in parallel on. :-) (Work is being done at that, too!) In the meantime, one could get that massive grid now if they start playing around with Amazon's Elastic Compute Cloud (EC2) (at least for Linux/Firefox testing).

2) WebTest (and all protocol-drivers) gives you speed and simplicity. You can do more testing of different kinds of scenarios faster with a protocol driver than you can with Selenium. Also, a protocol driver will make it easier to test more "failure modes" like simulating what happens if the network connection drops mid-request. Think of it like designing an airplane. Before a manufacture sells a new airplane on the market, they use simulators *AND* real-world flight tests. It's a BOTH/AND decision not EITHER/OR when it comes to product testing.

I'm eagely following the WebDriver project (http://code.google.com/p/webdriver/). It provides 1 common API that lets you toggle between using HtmlUnit for browser emulation and driving actual browsers (via COM for IE, a custom plug-in for Firefox).

While I'm waiting for WebDriver to stabilize... in the meantime... write many unit tests, fewer Canoo WebTests, and fewer still Selenium tests. But use all of them. The more ways you get coverage the better off you'll be.

Cheers,
Hugs

Posted by Jason Huggins on August 02, 2007 at 11:37 AM MDT #

Hello,

Using Canoo Web Tool it Only Test Java Coding 's Or All Domain

Thanks,
-Rajkumar

Posted by Rajkumar on January 31, 2008 at 03:57 AM MST #

Hi, I have developed a sample application using spring MVC, Ajax, JavaScript. So, I wanted to know which test suite I should use, to have test coverage for all the components. I am not sure if canoo has the features to satisfy my requirement. Thanks in advance, Arin

Posted by Arin on July 04, 2009 at 12:05 AM MDT #

@Arin - I believe Selenium or jWebUnit are the best solutions for testing Ajax applications. Canoo Web Test is very good as well. I would try them out and see which one you like best.

Posted by Matt Raible on July 09, 2009 at 09:00 AM MDT #

Post a Comment:
  • HTML Syntax: Allowed