Unit Test Frameworks | O'Reilly Media

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Most people who write software have at least some experience with unit testing. If you have ever written a few lines of throwaway code just to try something out, you've built a unit test. On the other end of the software spectrum, many large-scale applications have huge batteries of test cases that are repeatedly run and added to throughout the development process. Unit tests are useful at all levels of programming.

What are unit test frameworks and how are they used? Simply stated, they are software tools to support writing and running unit tests, including a foundation on which to build tests and the functionality to execute the tests and report their results. They are not solely tools for testing; they can also be used as development tools on a par with preprocessors and debuggers. Unit test frameworks can contribute to almost every stage of software development, including software architecture and design, code implementation and debugging, performance optimization, and quality assurance.

Unit tests usually are developed concurrently with production code, but are not built into the final software product. The relationship of unit tests to production code is shown in Figure 1-1.

Figure 1-1: Production application and unit test framework

An application is built from software objects linked together. The unit tests use the application's objects, but exist inside the unit test framework. This approach has a number of nice aspects. The production code is not cluttered up with built-in unit tests. The size of the compiled application tends to be kept smaller for the same reason. The tests can be run separately from the application, so the objects can be tested in isolation.

A single unit test should test a particular behavior within the production code. Its success or failure validates a single unit of code. Well-written tests set up an environment or scenario that is independent of any other conditions, then perform a distinct action and check a definite result. These tests should avoid dependencies on the results of other tests (called

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Unit test frameworks are a key element of Test Driven Development (TDD), also known as "test-first programming." TDD is one of the most significant and widely used practices in Extreme Programming (XP) and other Agile Development methodologies. Test frameworks achieve their maximum utility when used to enable TDD, although they still are useful when TDD is not followed. This book concentrates on unit test frameworks as a family of tools, rather than specifically on TDD, but the two topics are closely related.

The key rule of TDD can be summarized as "test twice, code once," by analogy to the carpenter's rule of "measure twice, cut once." "Test twice, code once" refers to the three-step procedure involved in any code change:

Write a test of the new code and see it fail.
Write the new code, doing "the simplest thing that could possibly work."
See the test succeed, and refactor the code.

These three basic steps are the TDD cycle .

Step 1 is to write a test, run it, and verify the resulting failure. The failure is important because it validates that the test fails as expected. It is often tempting to skip running the test and seeing the failure. Don't.

In Step 2, code is written to make the test succeed. A wise guideline is doing "the simplest thing that could possibly work." This may be a completely trivial implementation, such as having the new code return a constant value or copying and pasting code from one place to another. It doesn't have to be pretty; it just has to pass the test. The temptation in this step is to do a little extra work and make some additional code change not directly related to passing the test. Again, don't do this.

In Step 3, the test succeeds, verifying both the new code and its test. At this point, the new code may be refactored. Refactoring is a software engineering concept defined as "behavior-preserving transformation." More formally, refactoring is the process of transforming code to improve its internal design without changing its external functionality. Within the TDD cycle, refactoring starts with the inelegant code that was written to pass the unit test and improves it by removing duplication or other ugliness. Since the unit test is in place, the details of how the code is implemented can be altered with confidence.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Unit test frameworks are valuable when used for automated software testing as part of a quality assurance (QA) process. In many software development groups, the QA process starts when new code is submitted, built, and unit tested. Often, the unit tests include not only programmer tests, but also acceptance tests designed or written by the QA team. If all the unit tests succeed, the code is provisionally accepted and sent to a QA engineer for inspection and testing.

Running the full suite of unit tests as the first step in QA has many benefits. Most importantly, the tests ensure that the code is solid the moment it has left the developers' hands. No human intervention is required to run the tests and evaluate the results. Either they all succeed, or there is a failure. Such Boolean (true/false) results are ideal because an automated system can understand them. The success of the unit tests confirms that the developers' assumptions are valid, and that the low-level functionality is working correctly at a level of scrutiny that functional tests can never achieve. When numerous developers are making changes at once, the unit tests provide confidence that nobody's changes caused someone else's code to break. Furthermore, unit tests help to provide accountability. Knowing exactly which test fails usually makes it apparent whose change broke things. "Breaking the build" once meant submitting code that caused a compile to fail, but now often refers to causing a unit test failure as well. Many teams employ heinous punishments (such as making the responsible developer buy donuts or beer for coworkers) to remind everyone that breaking the build is a serious offense. The failure of a unit test clearly places a high priority on fixing the problem. If TDD is followed rigorously, the code should never be left in a state in which a unit test fails.

Unit testing doesn't replace all other types of testing. It is entirely possible to develop thoroughly unit-tested, completely bulletproof code that is lacking in usability and performance. Stress testing, performance testing, and usability testing usually are separate considerations from unit testing. QA effort is still necessary to try out the completed application, decide whether it performs acceptably in real-world conditions, observe how things work outside of a controlled development environment, and otherwise apply human judgment. There are elements of software functionality for which it is difficult or impossible to write good unit tests. These include GUI "look and feel," responses to system events, interaction with distributed application components, and many other possibilities. Sometimes unit tests can be written to simulate these types of situations, but ultimately, there is no substitute for reality or for a user's objective feedback.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Writing simple tests comes naturally to most programmers. The classic beginner exercise of writing a three-line program that prints "Hello world!" is a basic unit test of the development language and environment. Find a software shop with no unit test framework in place (if such a prehistoric place could possibly exist), and you may see developers writing their own little "toy programs" or "test utilities" to try out new code. The sad thing about this approach is that the toy programs are thrown away once the developer is done with them. Later, when something breaks, someone has to laboriously debug the production code, without benefit of the developer's test.

Another common low-level testing technique is to build tests into the production code with ASSERT macros. In debug builds, the macro tests a condition and sends a message if it fails. In release builds, the macro is defined to be empty, so no test code is included. This allows a developer to sprinkle assertions throughout the code, reporting any condition that is worthy of someone's attention. Asserts can be a useful thing to have in your software toolbox, but far less so than true unit testing. For an assert to be evaluated, the production code must be run to the point where it is defined. It's not convenient for automated testing, since an automated system doesn't know how to cause a particular assert to fire. Failures don't leave the developer with a clear path to correct the problem. Fixing a failure is no guarantee that the same problem will not happen again under different circumstances. Reliance on testing with this type of assert is unlikely to produce high-quality software. It is a forerunner to formal unit testing, which uses test asserts contained within well-defined tests, rather than placed randomly in the production code.

Just as many developers take the initiative and write test programs to try out small pieces of code, it's common to find developers putting together basic, home-grown unit test frameworks that take care of their testing needs. As demonstrated in the Chapter 2, a test framework can be just a few lines of code to run unit tests and report the results. Even a very simple framework can be the foundation for thorough testing of complex applications.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Software concepts are best explained by example. In this tutorial, you will set up a simple unit test framework and use it to help build a basic application. Following the primary rule of TDD, every change to the code is preceded by a unit test.

Why build our own test framework, instead of starting with one of the xUnits? The xUnit test frameworks are powerful tools. They not only support writing unit tests, running them, and reporting the results, but also include test classes, helper code, test runners, and utilities. Such features minimize the amount of code required to write a unit test and maximize your ability to test complex code. They include much more than the minimum needed to build unit tests.

The core functionality of running tests and reporting the results is fundamentally simple. Developers working in cross-platform environments, using older compilers or uncommon languages or needing closer control over how unit tests and their results are handled may not be able to use the xUnits or want to invest the time to set them up. The proliferation of very basic unit test frameworks available online demonstrates the popular belief that "simpler is better" when it comes to unit test frameworks. Most importantly, creating your own framework clearly demonstrates how unit tests work and how straightforward the unit test framework concept really is.

The example code is given in Java. Appendix A contains the C++ version. The code can be found on the CD accompanying this book in the directory /examples/chapter2. Consider entering the code in this chapter by hand as if you were coding it from scratch. It's an illuminating exercise that will help you to understand how quick and easy it is to set up and start using a unit test framework.

This tutorial assumes that you have a Java runtime environment and compiler installed. Sun's javac compiler is recommended, as is the GNU gcj Java compiler. Versions of both compilers are readily available for most platforms.

The step-by-step procedures given here assume that you are compiling and running the code from the command line. If you are using a graphical Integrated Development Environment (IDE), the details of how you build and run the example code will differ.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This book presents an increasingly complex series of code examples to illustrate unit test framework usage. The examples fit into an overall system concept. Your mission, should you choose to accept it, is to build a system for managing a library full of books. Books will have the attributes you might expect, such as title and author. Users of the system will need to be able to perform a variety of library operations: adding new books, searching for books, checking them in and out of the library, and so forth.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

For the first example, we will create a representation of a book and its title. Since we'll do test-first development, we need to set up a unit test framework prior to writing any code for the book class. This test framework serves both as the foundation for the example's unit tests and also as an illustration of just how simple a functional test framework can be. Building it is Step 0.

The subsequent steps are the usual three steps in the TDD cycle. Step 1 is to write a unit test to verify that a book has been created. At first, the unit test will fail, because the functionality to create a book does not yet exist. Step 2 is to build the functionality to create a book. In Step 3, the test succeeds, proving that the functionality works and providing an example of how to use it.

The unit test framework initially is built on a single class, UnitTest , shown in Figure 2-1.

Figure 2-1: The class UnitTest

The source code for UnitTest is given in Example 2-1.

Example 2-1. The base class UnitTest

UnitTest.java
public abstract class UnitTest {

   protected static int num_test_success = 0;

   public abstract void runTest( ) throws Exception;

   protected void assertTrue( boolean condition, String msg ) 
      throws Exception {
      if (!condition)
         throw new Exception(msg);
      num_test_success++;
   }

}

The class UnitTest is abstract because its purpose is to be the parent class for actual unit tests. It contains a static integer member, num_test_success, which keeps track of the number of successful tests. Descendant classes override the method runTest( ) to run actual tests. The method assertTrue() tests a condition. If the condition is TRUE, the successful test counter is incremented. If it is FALSE, an Exception is thrown containing a message string associated with the condition.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

For the second example, we'll add additional functionality to the library application. The new features will allow us to create a library, add a book to it, and get a book from it. Along the way, we also will add a few features to the unit test framework.

Consider the minimum new code that will provide what is necessary. Creating a library is easy. We can instantiate an empty class called Library and be done. Adding a book to the library is a feature with a little more to think through. We have a class Book, and the fact that we can add a Book to a Library suggests how a Library should work. A Library contains Books. The ability to get a book from the library reinforces this idea.

Let's create a unit test that adds a Book to a Library and then gets the Book back out again, verifying that the Library contains the Book.

The class LibraryTest is the initial unit test for the Library class. Its implementation is shown in Example 2-6.

Example 2-6. Initial version of LibraryTest

LibraryTest.java
public class LibraryTest extends UnitTest {

   public void runTest( ) throws Exception {
      Library library = new Library( );
      Book expectedBook = new Book( "Dune" );
      library.addBook( expectedBook );
      Book actualBook = library.getBook( "Dune" );
      assertTrue( actualBook.title.equals("Dune"), "got book" );
   }

}

The test creates a Library and a Book, adds the Book to the Library, then gets the Book from the Library and asserts that the expected Book was found.

Additional code must be added to TestRunner to run LibraryTest, as shown in Example 2-7.

Example 2-7. TestRunner modified to run LibraryTest

TestRunner.java
public class TestRunner {

   public static void main(String[] args) {
      TestRunner tester = new TestRunner( );
   }

   public TestRunner( ) {
      try {
         BookTest test = new BookTest( );
         test.runTest( );
         LibraryTest test2 = new LibraryTest( );
         test2.runTest( );
         System.out.println("SUCCESS!");
      }
      catch (Exception e) {
         System.out.println("FAILURE!");
         e.printStackTrace( );
      }
      System.out.println( UnitTest.getNumSuccess( )
         + " tests completed successfully" );
   }
}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Kent Beck published a unit test framework for the Smalltalk language in 1999. The architecture of SmalltalkUnit (or SUnit) represents a sweet spot, an ideal balance between simplicity and utility. Later, Erich Gamma ported SUnit to Java, creating JUnit. JUnit in turn begat CppUnit, NUnit, PyUnit, XMLUnit, and ports to many other languages. A dizzying array of unit test frameworks built on the same model now exists. These frameworks are known as the xUnit family of tools. All are free, open source software.

Some of the most popular xUnit test frameworks are listed next, with brief summaries of their target language and testing domain. This is just a sample of the many xUnit-derived test tools.

JUnit: The reference implementation of xUnit, JUnit is by far the most widely used and extended unit test framework. It is implemented in and used with Java and is covered in Chapter 6 of this book.
CppUnit: The C++ port of JUnit, it closely follows the JUnit model. This is covered in Chapter 7 of this book.
NUnit: The xUnit for .NET. Rather than being a direct port of JUnit, it has a .NET-specific implementation that generally follows the xUnit model. It is written in C# and can be used to test any .NET language, including C#, VB.Net, J#, and Managed C++. It is covered in Chapter 8 of this book.
PyUnit: The Python version of xUnit. It is included as a standard component of Python 2.1, and is covered in Chapter 9 of this book.
SUnit: Also known as SmalltalkUnit, this is the original xUnit, and the basis of the xUnit architecture. It is written in and used with the Smalltalk language.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Some of the most popular xUnit test frameworks are listed next, with brief summaries of their target language and testing domain. This is just a sample of the many xUnit-derived test tools.

JUnit: The reference implementation of xUnit, JUnit is by far the most widely used and extended unit test framework. It is implemented in and used with Java and is covered in Chapter 6 of this book.
CppUnit: The C++ port of JUnit, it closely follows the JUnit model. This is covered in Chapter 7 of this book.
NUnit: The xUnit for .NET. Rather than being a direct port of JUnit, it has a .NET-specific implementation that generally follows the xUnit model. It is written in C# and can be used to test any .NET language, including C#, VB.Net, J#, and Managed C++. It is covered in Chapter 8 of this book.
PyUnit: The Python version of xUnit. It is included as a standard component of Python 2.1, and is covered in Chapter 9 of this book.
SUnit: Also known as SmalltalkUnit, this is the original xUnit, and the basis of the xUnit architecture. It is written in and used with the Smalltalk language.
vbUnit: vbUnit is xUnit for Visual Basic (VB). It is written in VB and supports building unit tests in VB and COM development.
utPLSQL: utPLSQL is xUnit for Oracle's PL/SQL language. It is written in and used with PL/SQL.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Beyond the xUnits themselves, many add-on tools are available that extend the functionality of existing unit test frameworks into specialized domains, rather than acting as standalone tools. A representative set of popular extensions is listed here.

XMLUnit: An xUnit extension to support XML testing. Versions exist as extensions to both JUnit and NUnit. This is covered in Chapter 10 of this book.
JUnitPerf: A JUnit extension that supports writing code performance and scalability tests. It is written in and used with Java.
Cactus: A JUnit extension for unit testing server-side code such as servlets, JSPs, or EJBs. It is written in and used with Java.
JFCUnit: A JUnit extension that supports writing GUI tests for Java Swing applications. It is written in and used with Java.
NUnitForms: An NUnit extension that supports GUI tests of Windows Forms applications. It is written in C# and can be used with any .NET language.
HTMLUnit: An extension to JUnit that tests web-based applications. It simulates a web browser, and is oriented towards writing tests that deal with HTML pages.
HTTPUnit: Another JUnit extension that tests web-based applications. It is oriented towards writing tests that deal with HTTP request and response objects.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The xUnits all have the same basic architecture. This section describes the xUnit fundamentals, using JUnit as the reference example, since it is the most widely used of the xUnits. The other xUnits vary in their implementation details, but follow the same pattern and generally contain the same key classes and concepts. The key classes are TestCase, TestRunner, TestFixture, TestSuite, and TestResult.

The architecture diagrams in this section leave out some methods and other design details for clarity and represent the generic xUnit design, not that of JUnit.

xUnit's most elemental class is TestCase, the base class for a unit test. It is shown in Figure 3-1.

Figure 3-1: The abstract class TestCase, the parent of all xUnit unit tests

All unit tests are inherited from TestCase. To create a unit test, define a test class that is descended from TestCase and add a test method to it. Example 3-1 shows the unit test BookTest .

Example 3-1. BookTest, a test built on TestCase

BookTest.java
import junit.framework.*;

public class BookTest extends TestCase {

   public void testConstructBook( ) {
      Book book = new Book("Dune");
      assertTrue( book.getTitle( ).equals("Dune") );
   }

}

The test method testConstructBook() uses assertTrue() to check the value of the Book's title. Test conditions always are evaluated by the framework's assert methods. If a condition evaluates to TRUE, the framework increments the successful test counter. If it is FALSE, a test failure has occurred and the framework records the details, including the failure's location in the code. After a failure, the framework skips the rest of the code in the test method, since the test result is already known.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The classes TestCase , TestRunner, TestFixture, TestSuite, and TestResult represent the core of the xUnit architecture. To understand what they do is to understand how xUnit works. Figure 3-6 shows how they all fit together.

Figure 3-6: Core classes of the xUnit test framework architecture

The test classes created in this chapter and the classes they interact with are shown in Figure 3-7.

Figure 3-7: The test classes LibraryTests, LibraryTest, and BookTest

LibraryTests is a TestSuite containing BookTest and LibraryTest. LibraryTest is a TestFixture, and BookTest is a TestCase. Conceptually, TextTestRunner runs LibraryTests, which runs BookTest and LibraryTest, which in turn run their test methods.

This concludes the discussion of the generic xUnit architecture. Chapter 6 through Chapter 10 describe the specific architectural details of some popular versions of xUnit, with usage examples for each one.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The previous chapters present a simple unit test framework and the fundamentals of xUnit. The unit test framework's architecture is important to understand, but not something you have to think about often. Most of your time should be spent writing unit tests, implementing production code to make the tests pass, or refactoring. This chapter includes examples of common patterns used when writing unit tests, as well as related tips on unit test development.

The code examples in this chapter are unit tests of additional virtual library functionality, including looking up books by author and title, looking up multiple books by one author, and removing books from the library. The Library and Book code to implement the new features is given at the end of the chapter.

The code examples shown so far use plain asserts . These are the most generic type of test assertion, which take a Boolean condition that must evaluate to TRUE for the test to succeed. A plain assert, the unit test for the Library method removeBook( ), is shown in Example 4-1.

Example 4-1. Test method testRemoveBook( ) using a plain assert

LibraryTest.java
   public void testRemoveBook( ) {
      library.removeBook( "Dune" );
      Book book = library.getBook( "Dune" );
      assertTrue( book == null );
   }

If the method removeBook( ) is stubbed out, the test fails. The following test results report the failure:

> java junit.textui.TestRunner LibraryTests
.....F.
Time: 0.06
There was 1 failure:
1) testRemoveBook(LibraryTest)junit.framework.AssertionFailedError
     at LibraryTest.testRemoveBook(LibraryTest.java:32)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

FAILURES!!!
Tests run: 6,  Failures: 1,  Errors: 0

Although the line of code where the failure occurred is shown, the output does not describe the specific cause of the failure. It often is helpful to add an informative message to the assertion. The xUnits generally have two versions of every assert method, one of which takes a message parameter describing the assert. Example 4-2 shows the test method using an assert with a message.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The code examples shown so far use plain asserts . These are the most generic type of test assertion, which take a Boolean condition that must evaluate to TRUE for the test to succeed. A plain assert, the unit test for the Library method removeBook( ), is shown in Example 4-1.

Example 4-1. Test method testRemoveBook( ) using a plain assert

LibraryTest.java
   public void testRemoveBook( ) {
      library.removeBook( "Dune" );
      Book book = library.getBook( "Dune" );
      assertTrue( book == null );
   }

If the method removeBook( ) is stubbed out, the test fails. The following test results report the failure:

> java junit.textui.TestRunner LibraryTests
.....F.
Time: 0.06
There was 1 failure:
1) testRemoveBook(LibraryTest)junit.framework.AssertionFailedError
     at LibraryTest.testRemoveBook(LibraryTest.java:32)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

FAILURES!!!
Tests run: 6,  Failures: 1,  Errors: 0

Although the line of code where the failure occurred is shown, the output does not describe the specific cause of the failure. It often is helpful to add an informative message to the assertion. The xUnits generally have two versions of every assert method, one of which takes a message parameter describing the assert. Example 4-2 shows the test method using an assert with a message.

Example 4-2. Test method using an assert with a message

LibraryTest.java
   public void testRemoveBook( ) {
      library.removeBook( "Dune" );
      Book book = library.getBook( "Dune" );
      assertTrue( "book is not removed", book == null );
   }

With the additional message, the rest results provide better information about the cause of the test failure:

1) testRemoveBook(LibraryTest)junit.framework.AssertionFailedError: 
book is not removed

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The basic assert methods cover only a few common cases. It's often useful to extend them to cover additional test conditions and data types. Custom assert methods save test coding effort and make the test code more readable.

So far, the Library tests check a Book's title attribute to verify the expected Book object, as shown in Example 4-3 in the test method

testGetBooks(
)

.

Example 4-3. Test comparing two Books using their title attributes

LibraryTest.java

   public void testGetBooks( ) {
      Book book = library.getBook( "Dune" );
      assertTrue( book.getTitle( ).equals( "Dune" ) );
      book = library.getBook( "Solaris" );
      assertTrue( book.getTitle( ).equals( "Solaris" ) );
   }

To be really sure that the test Book is correct, the tests should also check the Book's author, but this means adding extra asserts to each test. It's clearly useful to have an assert method that compares an expected Book to the actual Book, checking all of the attributes. This new assert method is easy to implement by building on the generic assertTrue() method, as shown in Example 4-4.

Example 4-4. Custom assert method to compare Books

BookTest.java

public class BookTest extends TestCase {

   public static void assertEquals( Book expected, Book actual ) {
      assertTrue(expected.getTitle( ).equals( actual.getTitle( ) )
              && expected.getAuthor( ).equals( actual.getAuthor( ) ));
   }
}

The assert method assertEquals() takes expected and actual Book objects to compare. It succeeds if the title and author attributes of the two Books are equal. Example 4-5 shows how it is used.

Example 4-5. Using the custom assert method

LibraryTest.java
public class LibraryTest extends TestCase {

   private Library library;
   private Book book1, book2;

   public void setUp( ) {
      library = new Library( );
      book1 = new Book("Dune", "Frank Herbert");
      book2 = new Book("Solaris", "Stanislaw Lem");
      library.addBook(

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

A useful rule of thumb is that a test method should only contain a single test assertion. The idea is that a test method should only test one behavior; if there is more than one assert condition, multiple things are being tested. When there is more than one condition to test, then a test fixture should be set up, and each condition placed in a separate test method.

The xUnits tend to enforce this rule when handling test assertion failures. A test method returns as soon as a failure occurs, skipping any additional code. Running the rest of the test is unnecessary, since the result (failure) is known.

Practically speaking, test methods containing several assertions are not always a terrible thing. Tests may have conditions that can only be combined into one expression with unnecessary complication of the code. The testGetBooks( ) method in the previous section verifies that the Library contains two Books, which is most clearly expressed as two separate asserts, although they could be combined into one compound condition. A single behavior can have several side effects that you should check with separate assertions. So, it's not a problem when a test method contains several asserts, as long as the test method is only testing a single behavior.

However, a test method with many asserts is a clear indicator that a single test is doing too much. Example 4-6 shows a test method with this problem.

Example 4-6. Poorly written unit test that tests multiple behaviors

LibraryTest.java
   public void testLookupBooksByAuthor( ) {
      // Add two books by same author
      Book book3 = new Book( "Cosmos", "Carl Sagan" );
      Book book4 = new Book( "Contact", "Carl Sagan" );
      library.addBook( book3 );
      library.addBook( book4 );
      // Look up books by title and author
      Book book = library.getBook( "Cosmos", "Carl Sagan" );
      BookTest.assertEquals( book3, book );
      book = library.getBook( "Contact", "Carl Sagan" );
      BookTest.assertEquals( book4, book );
      // Look up both books by author
      Vector books = library.getBooks( "Carl Sagan" );
      assertEquals( "two books not found", 2, books.size( ) );
      book = (Book)books.elementAt(0);
      BookTest.assertEquals( book3, book );
      book = (Book)books.elementAt(1);
      BookTest.assertEquals( book4, book );
   }

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

It is important to test the error-handling behavior of production code in addition to its normal behavior. Such tests generate an error and assert that the error is handled as expected. In other words, an expected error produces a unit test success.

The canonical example of a unit test that checks expected error handling is one that tests whether an expected exception is thrown, as shown in Example 4-8.

Example 4-8. Unit test for expected exception

LibraryTest.java
   public void testRemoveNonexistentBook( ) {
      try {
         library.removeBook( "Nonexistent" );
         fail( "Expected exception not thrown" );
      } catch (Exception e) {}
   }

The expected error behavior is that an exception is thrown when the removeBook( ) method is called for a nonexistent Book. If the exception is thrown, the unit test succeeds. If it is not thrown, fail() is called. The fail( ) method is another useful variation on the basic assert method. It is equivalent to assertTrue(false), but it reads better.

Since the removeBook( ) method now throws an exception, the testRemoveBook( ) unit test should be updated, as shown in Example 4-9.

Example 4-9. Unit test that fails when an exception is thrown

LibraryTest.java
   public void testRemoveBook( ) {
      try {
         library.removeBook( "Dune" );
      } catch (Exception e) {
         fail( e.getMessage( ) );
      }
      Book book = library.getBook( "Dune" );
      assertNull( "book is not removed", book );
   }

This example uses fail( ) to cause the test to fail when an unexpected exception is thrown. The exception's message attribute is used as the assert message.

The same general pattern is followed to test expected error behavior that is not represented by an exception: the test fails if the error is not seen and succeeds if it is. Example 4-10 shows a unit test that attempts to get a nonexistent Book from the Library

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Every behavior should be covered by a unit test, but every method doesn't need its own unit test. Many developers don't test get and set methods, because a method that does nothing but get or set an attribute value is so simple that it is considered immune to failure. Tests of such methods are correspondingly trivial, as shown in the test of getTitle( ) in Example 4-11.

Example 4-11. Trivial test of getTitle( ) method

BookTest.java
   public void testGetTitle( ) {
      Book book = new Book( "Solaris", "Stanislaw Lem" );
      assertEquals( "Solaris", book.getTitle( ) );
   }

If a get or set method produces any side effects or otherwise has nontrivial functionality, it should be tested. For example, with lazy initialization, a get method may compute an attribute value before returning it — behavior that deserves a unit test.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

A topic of much discussion within the unit testing community is how to test protected or private methods. Since access to such methods is restricted, writing unit tests for them is not straightforward.

Some developers deal with this quandary by simply ignoring protected or private methods and testing only the public interfaces. It's argued that most of an object's behavior is reflected in its public methods. The behavior of the protected methods can be inferred by the exposed behavior.

There are some drawbacks to this approach. If there are private methods that contain complex functionality, they will not be tested directly. There is a tendency to make everything public so that it is testable. Some behaviors that should be private might be exposed.

It is possible to access and test protected and private methods, depending on the specifics of how a language defines and enforces object access permissions. In C++, making the test class a friend of the production class allows it to access protected interfaces:

class Library {
#ifdef TEST
   friend class LibraryTest;
#endif
}

This introduces a reference to the test code into the production code, which is not good. Preprocessor directives such as #ifdef TEST can omit such references when the production code is built.

In Java, a simple technique that allows test classes to access protected and private methods is to declare the methods as package scope and place the test classes in the same package as the production classes. The next section, "Test Code Organization," shows how to arrange Java code this way.

For Java developers who are not satisfied with the direct approach, the Java Reflection API is a tricky way to overcome access protection. The JUnit extension "JUnit-addons" includes a class named PrivateAccessor that uses this approach to access protected or private attributes and methods.

The truly hardcore can follow the examples given here to write their own code that subverts access protection. In Example 4-12, the values of all of

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

As a project grows in size, organizing the files containing production and test code becomes an issue. Although keeping the test and production code in the same directory is the simplest solution, it is better to have a clean separation between the two categories of code. This strategy helps avoid build complications that occur when a directory contains some code that should be linked into the production application, and some that should not. Including the test code in the delivered application is undesirable because it unnecessarily increases the size of the delivery, and also because the tests may expose behavior or design details that the developer meant to keep "under the hood."

Organizing the code is a language-specific concern. In Java, the directory path to a source file parallels its package membership. The need to test protected interfaces means that unit tests should belong to the same package as the production classes they test, so they must have the same directory path. This can be done by creating separate but parallel hierarchies for the production and test code.

Figure 4-1 shows how the source code for the final version of the virtual library application is organized. There are three Java packages, com.utf.library, com.utf.library.gui, and com.utf.library.xml.

Figure 4-1: Organization of production and test code

The production and test code are located in separate directories, src and test, which are located under the project's top level DEVROOT directory. For example, the production class Library resides in the directory src/com/utf/library, and the test class LibraryTest is in test/com/utf/library. The test classes' package names parallel the production classes' package names, so the test classes can access and test protected behavior of the production code. Since the code is in separate directory trees, it is simple to build and run only the production or test code as desired.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Applications often use interfaces to external objects such as databases, web servers, network services, or hardware devices. Sometimes you must write and test code to interface with objects before they are actually available. Even when the external object is available in the development environment, using it in testing may involve lots of time-consuming, fragile set-up effort, such as loading test data, running services, or placing hardware in a known state. Mock objects are a way of dealing elegantly with this type of situation.

A mock object is a simulation of a real object. Mocks implement the interface of the real object and behave identically with it, to the extent necessary for testing. Mocks also validate that the code that uses them does so correctly. To pass the mock's validation, other objects must call the correct methods, with the expected parameters, in the expected order. A test object that simply stands in for a real object without providing such verification is not a mock; it is a stub.

Databases are commonly mocked objects. Code that interfaces to a database clearly is important to test. To be tested realistically, the code must be able to perform database operations such as opening and closing connections, reading and writing data, and performing transactions. However, running a live database in the development environment can be a pain. Tests often require that the database is in a specific state or that it contains a specific set of test data. If multiple developers run tests simultaneously, their database operations may interfere.

Mocking the database makes having an actual database unnecessary for testing. The mock has the same interface as the actual database object and the same behavior from the perspective of the client software, but it doesn't need to actually contain anything but a minimal implementation and possibly some test data. Once the database mock is created, it becomes much simpler to write tests that assume that the database is in various states. Testing becomes faster and easier without the overhead of interfacing with an actual database engine.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Just like regular classes, abstract classes and interfaces should have their own unit tests. Designing such tests is not straightforward, because these object types cannot be directly instantiated. We'd also like to ensure that every descendant of an abstract class passes the parent object's tests. The AbstractTest pattern is the answer.

An AbstractTest is itself abstract, like the tested object. It contains an abstract factory method, which produces an instance of the object to test. It also contains the test methods for the abstract class. They resemble ordinary unit test methods, but test instances of the abstract class created by the factory method.

To test a concrete class that is descended from the abstract class, the unit test is subclassed from the AbstractTest. Its factory method returns an instance of the concrete class. When the concrete unit test is run, the AbstractTest is run as well. So, the AbstractTest tests every concrete implementation of the abstract class.

Let's create an AbstractTest for the interface DBConnection . We'll add the method isOpen( ) to it, as shown in Example 4-18.

Example 4-18. The interface DBConnection

DBConnection.java
public interface DBConnection {
   void connect( );
   void close( );
   boolean isOpen( );
   Book selectBook( String title, String author );
}

The AbstractTest should test the behavior of the interface to make sure that any concrete implementation of it is correct. Tests of the isOpen( ) method should verify that it returns TRUE after

connect(
)

is called, and FALSE after close() is called. The AbstractTest class AbstractDBConnectionTestCase , shown in Example 4-19, provides these tests.

Example 4-19. The AbstractTest class AbstractDBConnectionTestCase

AbstractDBConnectionTestCase.java
import junit.framework.*;

public abstract class AbstractDBConnectionTestCase extends TestCase {

   public abstract DBConnection

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Like the mock object, unit testing for performance is its own significant topic. Software performance often is neglected at the unit testing level, and is only taken into consideration during functional testing. However, performance-oriented unit tests are powerful tools, especially for applications that require specific performance goals be met. It's been reported that Apple's Safari browser was developed in an environment that automatically ran performance tests on any code that was checked in. The code was rejected if it did not meet or exceed the speed standards of previous versions. Thus, the unit tests ensured that the code's performance is continuously improving.

When a piece of code has a performance problem, it is very useful to first write a test that reveals the problem. This performance test not only lets you know when the code has achieved the desired performance, but also acts as a "canary in the coal mine" that indicates if the performance degrades again.

Tools intended specifically for performance-oriented unit testing are available, such as JUnitPerf. However, it is not difficult to develop performance tests within any unit test framework. This section gives an example of a unit test that tests the speed of retrieving a Book from a Library.

The initial question when writing a performance test is this: what is the performance criterion that the test must meet to pass? Usually, this is expressed in terms of the amount of time that a certain action may take. If the action takes too long, the criterion has not been met, and the test fails.

The Library class developed so far has a very poorly performing algorithm to get a Book. It serially reads through the collection of Books, doing string comparisons on each one until the desired Book is found. This awful lookup stratagem is ideal for demonstrating a performance test that fails initially, but succeeds after a little refactoring. Example 4-22 shows the unit test class LibraryPerfTest .

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|

Table of Contents