Raible Designs

Mock Shmocks!

Let me start this post by saying that I've never used Mock Objects in my JUnit tests. Simon digs it, crazybob is an advocate and so is the gang that wrote Open Source Programming. In fact, the OSP book says that your unit tests should only take a couple of seconds to run! A couple of seconds!? Mine aren't that fast, which definitely makes me think I should look at Mocks to try to speed things up.

However, there is a method to my madness. Almost all my tests depend on a database connection and use DBUnit to populate the database before running. This can be time consuming, but it's not too bad for what I'm getting: real tests that verify the environment, not just the code. I just ran all tests in the application - 8 minutes 7 seconds - for 165 JUnit/Cactus tests and ~25 WebTest (JSP) tests.

If I did use Mocks, I wouldn't have discovered all the bugs in my code this week. When migrating from MySQL to DB2, I found that some things worked on MySQL, but they didn't on DB2. I modified the code, doing some tweaks to make Hibernate happy, and voila - now everything works on DB2 and MySQL. Furthermore, all my errors occurred when running my app in the container, and when I used simple JUnit tests (on my Managers) to test the same logic, it worked.

Who knows, I'm definitely learning new stuff everyday. Maybe it's possible that Mocks could do all this, but testing the production environment seems pretty important to me.

Posted in Java at Jan 21 2004, 06:34:28 PM MST 5 Comments

Comments:

!Hi Matt, Mock objects allow you to unit test at the finest possible level and develop method by method, in <g>isolation</g> from the other methods or the environment. Testing in isolation as many advantages : - you can test a method even if all the code you depend on has not yet been written. - you have the ability to write focused tests without potential side effects from other objects being called from the method under test. If your test fails, you know exactly where it comes from. - your tests run fast. What you are doing with DBUnit is <g>integration testing</g> or <g>functional testing</g>. These strategies are not exclusive with unit testing in isolation, they are complementary, but their goals are just different. IMHO, your experience with appfuse and DB2 is is a good demonstration because it shows that : 1. your persistence layer isn't tested in isolation (because you're not using mocks :p) 2. you know that this layer works because you have integration tests / functional tests (Canoo) working with MySQL 3. if you have problems with another db for example, you have to do some debug and/or trace because you don't know if it comes from your DAO layer, the OR mapping framework you're using... 4. (optional) and if you if you hadn't known your application is working with MySQL ? This can be perfectly acceptatble but actually I think that all testing strategies (unit testing, integration testing, functional testing, performance testing, ...) can help to make a continuous integration process more robust. But defining a global testing strategy depends on your requirements, on your design, on the criticity of your application... I love DBUnit. But I love mock objects too. I just use them for different purposes.

Posted by Pascal Thivent on January 21, 2004 at 09:06 PM MST #

You should always test the db access layer with a real database. Mock anything else that sits on top of that layer. Just read the damn book Matt ;-) Ara.

Posted by Ara Abrahamian on January 22, 2004 at 06:28 AM MST #

Matt - you're missing a bit of the point of a *unit* test. Test only one thing at a time for units. Your end-to-end testing is very valuable and should be there, but it is testing a lot of layers at once. Use mocks for isolating business logic so you don't have to do the database setup, for example. Keep the database dependent code out of the business layer - this is one thing that true unit/mock testing will help you architect.

Posted by Erik Hatcher on January 22, 2004 at 06:54 AM MST #

It's not either/or Unit/Integration Tests. It's both/and. "If I did use Mocks, I wouldn't have discovered all the bugs in my code this week. When migrating from MySQL to DB2, I found that some things worked on MySQL, but they didn't on DB2. ...all my errors occurred when running my app in the container, and when I used simple JUnit tests (on my Managers) to test the same logic, it worked." Which is why you need INTEGRATION tests. The code still needs to be verified in its target execution environment. And there are some things which may not be possible to test with unit tests. "... I just ran all tests in the application - 8 minutes 7 seconds - for 165 JUnit/Cactus tests and ~25 WebTest (JSP) tests." Which is one reason you need UNIT tests. It simply takes too long to run all those Integration tests. This increases the risk that they won't get run at all. It also means means they can't be run as frequently. This in turn increases the risk that they won't get run when they need to or that it will increase the time between the introduction of an error and its identification. For example, we have a JDBC framework which we reuse across many different applications. As of this writing we have 159 unit tests for this Project, which take less than ONE SECOND to run! When you start seeing orders of magnitude difference in development time, it's no longer a matter of degree, the development process is qualitatively different. Another reason why you need unit tests is that you can't test everything with Integration testing, either. Our unit testing standard is 100% branch coverage. Which means not only every it/then/else/select, but also every thrown Exception, every catch block, every exception and error path. This is especially important for reusable JDBC frameworks, since there's no real underlying database to which we can connect, and because we must verify all exception handling and other error paths before we release it internally to our other developers. Mock testing makes this possible. We have been using the MockObjects package, and that has worked well for us. In the past few months we've started using the EasyMock framework. WHile it takes some getting used to, it's saving us a lot of time over using MockObjects, and we can use it for all mock testing, even for application-specific interfaces which would be too time-consuming or fragile (due to ongoing changes) to mock out in code. You can leverage unit testing for your integration testing also. Having unit tests means you DON'T need integration tests for the things you can unit test. You just need the additional integration tests for the "integrations:" where the units collaborate or interact with the environment. Having fewer integration tests means they will run fast, which means ... Thanx - Xris

Posted by Chris Kreussling on January 27, 2004 at 01:30 PM MST #

Matt, I'm not a big fan of cactus. If your testing in the container, your testing more of the containers code than yours. I prefer tools like:
-Mockrunner for testing custom tags
-Strutstestcase for testing actions

As for the business logic and data access. I really like hibernate. If you have the choice, hibernate works with POJOs which are a dream to unit test your BL with. Hibernate on Hypersonic is fast enough for me.

Also, please remember the difference between MockObjects and Stubs. Stubs just replace a level/section of your architecture with a faked out version that returns dummy/known data that the caller can test with. A MockObject tests the caller of that layer/section from the INSIDE. Please read: http://www.connextra.com/aboutUs/mockobjects.pdf

Posted by Paul Kilroy on January 30, 2004 at 09:58 PM MST #

10+ YEARS

TWITTER

RECENT ENTRIES

Tag Cloud

Mock Shmocks!