Matt RaibleMatt Raible is a Web Developer and Java Champion. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

RECENT ENTRIES


Tag Cloud


abbie appfuse birthday denver family grails gwt jack java javascript jhipster jsf linkedin maven roller skiing softwaresummit spring springmvc struts2 travel trish vacation webframeworks wicket
« Resin now Open Sourc... | Main | The New Gig »
Tuesday October 05, 2004

Aren't out-of-container tests supposed to be faster?

Earlier this week, I converted my StrutsTestCase tests from using CactusStrutsTestCase to MockStrutsTestCase. At first, this seemed like a great thing - on my Windows XP box. To run all the tests in AppFuse, it takes around 2:20. I know, this is quite a bit of time for unit tests - but it was 3:15 in version 1.5. At least it's faster than it was. 1:20 of this is for starting Tomcat and running Canoo WebTests.

This all seemed great until I ran "ant test-all" on my PowerBook tonight. It averages around 7:00 minutes. WFT?! I know PowerBooks are slow - but they're not that slow. I'm guessing the reason is because Spring's ApplicationContext is loaded by each test - whereas the Cactus versions would always grab it out of the ServletContext. Sounds like I need a TestSuite.

What if I isolate and compare times for just the "test-web" target? This target runs all the Action, Filter and Listener tests. Time to execute on Windows: 33 seconds, PowerBook: 2 minutes 21 seconds. What about just testing the JSPs (with Cargo)? Windows: 1:24, PowerBook: 3:32. My Windows desktop has 1.5 GB RAM and a 2.6 GHz processor. My PowerBook has 1 GB RAM and 1.5 GHz. I agree that it seems like an unfair comparison - but AppFuse tests ran in around 4 minutes on the PowerBook in the last release. I guess it's back to the drawing board.

Update: Nevermind. I just downloaded and checked out AppFuse 1.5 - almost 8 minutes for running all the tests. If I were going to work on my PowerBook for my next project, I'd refactor AppFuse Managers and Actions to use jMock. Luckily I'm using my Windows machine. ;-)

Update 2: I found a solution! Using Ant 1.6.2's forkmode="once", I was able to reduce the time of the test-web target from 2:21 to 24 seconds on my PowerBook!! Windows: :33 -> :18. Running "ant test-all" on the Mac is now 2:15 - while the Windows version is 2:20.

WTF!? My PowerBook is actually faster than my Windows box? I never thought I'd see the day - Yeeee haaawwww!!

Now I just have to figure out how to detect 1.6.2 and warn users if they aren't using it.

Posted in Java at Oct 05 2004, 09:26:56 PM MDT 8 Comments
Comments:

One of the real problems with StrutsTest is that it needs the struts.xml AND it parses it for each test case. This means, as you guesed, the context is loaded each time. I'm fond of pointing out that StrutsTest isn't _really_ a unit test framework, because it requires your whole Struts environment to be in place even though you're only testing part of it. "If it needs the web.xml, it's not a unit test". It is, however, a very effective _integration_ test framework.

Posted by Robert Watkins on October 06, 2004 at 06:42 AM MDT #

I agree with robert - I don't think StrutsTest is really an out of container test, its an 'in a not-the-real container' test. I don't like to use it because I feel it misses the point.

Posted by Perryn Fowler on October 06, 2004 at 08:31 AM MDT #

If components/tiers/systems are interacting it isn't a unit test =) Have a smaller suite of system/integration tests which can take a lot longer and only need to be run before checkins (and are used to catch config errors). This is where anything that needs to touch a config file goes. Configuration isn't code. Usually you can even automate this in a better way by having a test which does a sanity check on all the configs. Struts, is, an admittedly hard one to test -- but it also predates JUnit and the unit testing fad. It comes from an older age when tests were something QA wrote ;-) -Brian

Posted by Brian McCallister on October 06, 2004 at 11:15 AM MDT #

Hmm, for some reason my trackback script doesn't work on your site 8-( Here's a way to detect Ant 1.6.2: http://stefanbodewig.blogger.de/stories/158341/

Posted by Stefan Bodewig on October 07, 2004 at 06:40 AM MDT #

FWIW, loading Spring app contexts from and XML file will slow down unit tests a fair amount also. I found that actually creating the app context by hand in Java (e.g. new FooObject( param1, param2) ... ) is much faster and much easier to manage. I also think that your tests are more of the integration type than the unit type. Unit tests are very pointed and only test the inputs and outputs of a given method. I've used EasyMock for this to abstract away many things, including an entire JDBC persistence layer so that I could achieve a unit test without calls going to a database. There is a fair amount of work involved in this but once you get the hang of it, it's like any other code you write. I also started to reuse EasyMock code which wound up being a big time saver. Just to completed my original thought here, IMO unit tests are run during the build process whereas integration tests are something separate. I like to place integration tests in a directory other than src/tests so that they are isolated and the build isn't slowed down by them.

Posted by Bruce Snyder on October 07, 2004 at 12:00 PM MDT #

I think the reason your PowerBook is faster is that Apple has optimized its JVM so that multiple JVM instances startup quicker and share resources. I think they contributed this code back to Sun who then was going to incorporate that into the Windows JVM for 1.5 or 1.6.

Posted by Kurt Wiersma on October 07, 2004 at 01:53 PM MDT #

Bruce and Brian - I know the difference b/w a <em>unit</em> test and an <em>integration</em> test. Can't we just call them programmer tests and agree that they're a good way to develop software? I don't want to have the debate b/w unit and integration tests again. I know AppFuse uses integration tests for pretty much everything. I do plan on (someday) isolating things a bit more and using jMock for Manager, Spring Controller and WebWork Actions. I don't know about Struts Actions - those might be a bit more difficult to mock dependencies. I have found that Spring Controllers can be a pain to test sans-container. Mine tend to have lots of dependencies that can lead to a large setUp() method.

In reality, I've found that the way I'm doing things in AppFuse makes it easier to write tests than using EasyMock or JMock. Since most tests only take a second or two to run, and they test full integration - the current tests certainly aren't a <em>bad thing</em>. Isn't the most important thing to get our jobs done fast and efficiently instead of playing with all the latest mock stuff anyway. ;-) I do think my perspective is scewed because I tend to write full apps rather than just a layer. If I were working on a team where I was responsible for one tier, I'd probably use mocks a lot more so I wasn't dependent on other's code.

As for the DAO layer of AppFuse, I'll probably always keep that as integration tests that talk to a real database. I've found quite a few issues b/w the different databases that I wouldn't have found if I'd mocked the database. BTW, aren't you supposed to only mock your own APIs?

My plan for AppFuse 1.7: no mocks in DAO tests, mock DAOs in Manager tests, mock Managers in Action/Controller tests and use JSP tests (i.e. Canoo WebTest or jWebUnit) tests to verify full integration.

Posted by Matt Raible on October 07, 2004 at 02:44 PM MDT #

jkjk

Posted by 59.144.12.41 on June 09, 2006 at 11:57 AM MDT #

Post a Comment:
  • HTML Syntax: Allowed