This chapter is from the book
This chapter is from the book
This chapter is from the book
Test Spy
How do we implement Behavior Verification?
How can we verify logic independently when it has indirect outputs to other software components?
We use a Test Double to capture the indirect output calls made to another component by the SUT for later verification by the test.
In many circumstances, the environment or context in which the SUT operates very much influences the behavior of the SUT. To get adequate visibility of the indirect outputs of the SUT, we may have to replace some of the context with something we can use to capture these outputs of the SUT.
Use of a Test Spy is a simple and intuitive way to implement Behavior Verification via an observation point that exposes the indirect outputs of the SUT so they can be verified.
How It Works
Before we exercise the SUT, we install a Test Spy as a stand-in for a DOC used by the SUT. The Test Spy is designed to act as an observation point by recording the method calls made to it by the SUT as it is exercised. During the result verification phase, the test compares the actual values passed to the Test Spy by the SUT with the values expected by the test.
When to Use It
A key indication for using a Test Spy is having an Untested Requirement (see Production Bugs) caused by an inability to observe the side effects of invoking methods on the SUT. Test Spies are a natural and intuitive way to extend the existing tests to cover these indirect outputs because the calls to the Assertion Methods are invoked by the test after the SUT has been exercised just like in “normal” tests. The Test Spy merely acts as the observation point that gives the Test Method access to the values recorded during the SUT execution.
We should use a Test Spy in the following circumstances:
- We are verifying the indirect outputs of the SUT and we cannot predict the values of all attributes of the interactions with the SUT ahead of time.
- We want the assertions to be visible in the test and we don’t think the way in which the Mock Object expectations are established is sufficiently intent-revealing.
- We want the assertions to be visible in the test and we don’t think the way in which the Mock Object expectations are established is sufficiently intent-revealing.
- Our test requires test-specific equality (so we cannot use the standard definition of equality as implemented in the SUT) and we are using tools that generate the Mock Object but do not give us control over the Assertion Methods being called.
- A failed assertion cannot be reported effectively back to the Test Runner. This might occur if the SUT is running inside a container that catches all exceptions and makes it difficult to report the results or if the logic of the SUT runs in a different thread or process from the test that invokes it. (Both of these cases really beg refactoring to allow us to test the SUT logic directly, but that is the subject of another chapter.)
- We would like to have access to all the outgoing calls of the SUT before making any assertions on them.
If none of these criteria apply, we may want to consider using a Mock Object. If we are trying to address Untested Code (see Production Bugs) by controlling the indirect inputs of the SUT, a simple Test Stub may be all we need.
Unlike a Mock Object, a Test Spy does not fail the test at the first deviation from the expected behavior. Thus our tests will be able to include more detailed diagnostic information in the Assertion Message based on information gathered after a Mock Object would have failed the test. At the point of test failure, however, only the information within the Test Method itself is available to be used in the calls to the Assertion Methods. If we need to include information that is accessible only while the SUT is being exercised, either we must explicitly capture it within our Test Spy or we must use a Mock Object.
Of course, we won’t be able to use any Test Doubles unless the SUT implements some form of substitutable dependency.
Implementation Notes
The Test Spy itself can be built as a Hard-Coded Test Double or as a Configurable Test Double. Because detailed examples appear in the discussion of those patterns, only a quick summary is provided here. Likewise, we can use any of the substitutable dependency patterns to install the Test Spy before we exercise the SUT.
The key characteristic in how a test uses a Test Spy relates to the fact that assertions are made from within the Test Method. Therefore, the test must recover the indirect outputs captured by the Test Spy before it can make its assertions, which can be done in several ways.
Variation: Retrieval Interface
We can define the Test Spy as a separate class with a Retrieval Interface that exposes the recorded information. The Test Method installs the Test Spy instead of the normal DOC as part of the fixture setup phase of the test. After the test has exercised the SUT, it uses the Retrieval Interface to retrieve the actual indirect outputs of the SUT from the Test Spy and then calls Assertion Methods with those outputs as arguments.
Variation: Self Shunt
We can collapse the Test Spy and the Testcase Class into a single object called a Self Shunt. The Test Method installs itself, the Testcase Object , as the DOC into the SUT. Whenever the SUT delegates to the DOC, it is actually calling methods on the Testcase Object, which implements the methods by saving the actual values into instance variables that can be accessed by the Test Method. The methods could also make assertions in the Test Spy methods, in which case the Self Shunt is a variation on a Mock Object rather than a Test Spy. In statically typed languages, the Testcase Class must implement the outgoing interface (the observation point) on which the SUT depends so that the Testcase Class is type-compatible with the variables that are used to hold the DOC.
Variation: Inner Test Double
A popular way to implement the Test Spy as a Hard-Coded Test Double is to code it as an anonymous inner class or block closure within the Test Method and to have this class or block save the actual values into instance or local variables that are accessible by the Test Method. This variation is really another way to implement a Self Shunt (see Hard-Coded Test Double).
Variation: Indirect Output Registry
Yet another possibility is to have the Test Spy store the actual parameters in a well-known place where the Test Method can access them. For example, the Test Spy could save those values in a file or in a Registry [PEAA] object.
Motivating Example
The following test verifies the basic functionality of removing a flight but does not verify the indirect outputs of the SUT—namely, the fact that the SUT is expected to log each time a flight is removed along with the date/time and username of the requester.
public void testRemoveFlight() throws Exception {
// setup
FlightDto expectedFlightDto = createARegisteredFlight();
FlightManagementFacade facade = new FlightManagementFacadeImpl();
// exercise
facade.removeFlight(expectedFlightDto.getFlightNumber());
// verify
assertFalse("flight should not exist after being removed",
facade.flightExists( expectedFlightDto.
getFlightNumber()));
}
Refactoring Notes
We can add verification of indirect outputs to existing tests using a Replace Dependency with Test Double refactoring. It involves adding code to the fixture setup logic of the tests to create the Test Spy, configuring the Test Spy with any values it needs to return, and installing it. At the end of the test, we add assertions comparing the expected method names and arguments of the indirect outputs with the actual values retrieved from the Test Spy using the Retrieval Interface.
Example: Test Spy
In this improved version of the test, logSpy is our Test Spy. The statement facade.setAuditLog(logSpy) installs the Test Spy using the Setter Injection pattern (see Dependency Injection). The methods getDate, getActionCode, and so on are the Retrieval Interface used to access the actual arguments of the call to the logger.
public void testRemoveFlightLogging_recordingTestStub()
throws Exception {
// fixture setup
FlightDto expectedFlightDto = createAnUnregFlight();
FlightManagementFacade facade = new FlightManagementFacadeImpl();
// Test Double setup
AuditLogSpy logSpy = new AuditLogSpy();
facade.setAuditLog(logSpy);
// exercise
facade.removeFlight(expectedFlightDto.getFlightNumber());
// verify
assertFalse("flight still exists after being removed",
facade.flightExists( expectedFlightDto.
getFlightNumber()));
assertEquals("number of calls", 1,
logSpy.getNumberOfCalls());
assertEquals("action code",
Helper.REMOVE_FLIGHT_ACTION_CODE,
logSpy.getActionCode());
assertEquals("date", helper.getTodaysDateWithoutTime(),
logSpy.getDate());
assertEquals("user", Helper.TEST_USER_NAME,
logSpy.getUser());
assertEquals("detail",
expectedFlightDto.getFlightNumber(),
logSpy.getDetail());
}
This test depends on the following definition of the Test Spy:
public class AuditLogSpy implements AuditLog {
// Fields into which we record actual usage information
private Date date;
private String user;
private String actionCode;
private Object detail;
private int numberOfCalls = 0;
// Recording implementation of real AuditLog interface
public void logMessage(Date date,
String user,
String actionCode,
Object detail) {
this.date = date;
this.user = user;
this.actionCode = actionCode;
this.detail = detail;
numberOfCalls++;
}
// Retrieval Interface
public int getNumberOfCalls() {
return numberOfCalls;
}
public Date getDate() {
return date;
}
public String getUser() {
return user;
}
public String getActionCode() {
return actionCode;
}
public Object getDetail() {
return detail;
}
}
Of course, we could have implemented the Retrieval Interface by making the various fields of our spy public and thereby avoided the need for accessor methods. Please refer to the examples in Hard-Coded Test Double for other implementation options.