- Testing Your Patience
- Extreme Testing
- An Example Using Test:: Modules
- Testing Legacy Code
- A Final Encouragement
This chapter is from the book
This chapter is from the book
This chapter is from the book
3.3 An Example Using Test:: Modules
Let's put what we've learned to use in developing an actual application. Say that we want to create a module that can limit the possible indices of an array, a bounds checker if you will. Perl's arrays won't normally do that,6 so we need a mechanism that intercepts the day-to-day activities of an array and checks the indices being used, throwing an exception if they're outside a specified range. Fortunately, such a mechanism exists in Perl; it's called tieing, and pretty powerful it is too.
Because our module will work by letting us tie an array to it, we'll call it Tie::Array::Bounded. We start by letting h2xs do the rote work of creating a new module:
% h2xs -AXn Tie::Array::Bounded Writing Tie/Array/Bounded/Bounded.pm Writing Tie/Array/Bounded/Makefile.PL Writing Tie/Array/Bounded/README Writing Tie/Array/Bounded/test.pl Writing Tie/Array/Bounded/Changes Writing Tie/Array/Bounded/MANIFEST
That saved a lot of time! h2xs comes with perl, so you already have it. Don't be put off by the name: h2xs was originally intended for creating perl extensions from C header files, a more or less obsolete purpose now, but by dint of copious interface extension, h2xs now enjoys a new lease on life for creating modules. (In Section 8.2.4, I'll look at a more modern alternative to h2xs.)
Don't be confused by the fact that the file Bounded.pm is in the directory Tie/Array/Bounded. It may look like there's an extra directory in there but the hierarchy that h2xs created is really just to help keep your sources straight. Everything you create will be in the bottom directory, so we could cd there. For instant gratification we can create a Makefile the way we would with any CPAN module:
% cd Tie/Array/Bounded % perl Makefile.PL Checking if your kit is complete... Looks good Writing Makefile for Tie::Array::Bounded
and now we can even run a test:
% make test cp Bounded.pm blib/lib/Tie/Array/Bounded.pm PERL_DL_NONLAZY=1 /usr/local/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.6.1/i386-linux -I/usr/lib/perl5/5.6.1 test.pl 1..1 ok 1
It even passes! This is courtesy of the file test.pl that h2xs created for us, which contains a basic test that the module skeleton created by h2xs passes. This is very good for building our confidence. Unfortunately, test.pl is not the best way to create tests. We'll see why when we improve on it by moving test.pl into a subdirectory called "t" and rebuilding the Makefile before rerunning "make test":
% mkdir t % mv test.pl t/01load.t % perl Makefile.PL Writing Makefile for Tie::Array::Bounded % make test PERL_DL_NONLAZY=1 /usr/local/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.6.1/i386-linux -I/usr/lib/perl5/5.6.1 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t t/01load....ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.30 cusr + 0.05 csys = 0.35 CPU)
The big difference: "make test" knows that it should run Test::Harness over the .t files in the t subdirectory, thereby giving us a summary of the results. There's only one file in there at the moment, but we can create more if we want instead of having to pack every test into test.pl.
At this point you might want to update the MANIFEST file to remove the line for test.pl now that we have removed that file.
If you're using Perl 5.8.0 or later, then your h2xs has been modernized to create the test in t/1.t; furthermore, it will use Test::More.7 But if you have a prior version of Perl, you'll find the test.pl file we just moved uses the deprecated Test module, so let's start from scratch and replace the contents of t/01load.t as follows:
#!/usr/bin/perl
use strict;
use warnings;
use Test::More tests => 1;
use blib;
BEGIN { use_ok("Tie::Array::Bounded") }
The use blib statement causes Perl to search in parent directories for a blib directory that contains the Tie/Array/Bounded.pm module created by make. Although we'll usually run our tests by typing "make test" in the parent directory, this structure for a .t file allows us to run tests individually, which will be helpful when isolating failures.
Running this test either stand-alone ("./01load.t") or with "make test" produces the same output as before (plus a note from use blib about where it found the blib directory), so let's move on and add some code to Bounded.pm. First, delete some code that h2xs put there; we're not going to export anything, and our code will work on earlier versions of Perl 5, so remove code until the executable part of Bounded.pm looks like this:
package Tie::Array::Bounded; use strict; use warnings; our $VERSION = '0.01'; 1;
Now it's time to add subroutines to implement tieing. tie is how to make possessed variables with Perl: Literally anything can happen behind the scenes when the user does the most innocuous thing. A simple expression like $world_peace++ could end up launching a wave of nuclear missiles, if $world_peace happens to be tied to Mutually::Assured::Destruction. (See, you can even use Perl to make covert political statements.)
We need a TIEARRAY subroutine; perltie tells us so. So let's add an empty one to Bounded.pm:
sub TIEARRAY
{
}
and add a test to look for it in 01 load.t:
use Test::More tests => 2;
use blib;
BEGIN { use_ok("Tie::Array::Bounded") }
can_ok("Tie::Array::Bounded", "TIEARRAY");
Running "make test" copies the new Bounded.pm into blib and produces:
% make test cp Bounded.pm blib/lib/Tie/Array/Bounded.pm PERL_DL_NONLAZY=1 /usr/local/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.6.1/i386-linux -I/usr/lib/perl5/5.6.1 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t t/01load....Using /home/peter/perl_Medic/Tie/Array/Bounded/blib t/01load....ok All tests successful. Files=1, Tests=2, 0 wallclock secs ( 0.29 cusr + 0.02 csys = 0.31 CPU)
We have just doubled our number of regression tests!
It may seem as though we're taking ridiculously small steps here. A subroutine that doesn't do anything? What's the point in testing for that? Actually, the first time I ran that test, it failed: I had inadvertently gone into overwrite mode in the editor and made a typo in the routine name. The point in testing every little thing is to build your confidence in the code and catch even the dumbest errors right away.
So let's continue. We should decide on an interface for this module; let's say that when we tie an array we must specify an upper bound for the array indices, and optionally a lower bound. If the user employs an index out of this range, the program will die. For the sake of having small test files, we'll create a new one for this test and call it 02tie.t:
#!/usr/bin/perl use strict; use warnings; use Test::More tests => 1; use blib; use Tie::Array::Bounded; my $obj = tie my @array, "Tie::Array::Bounded"; isa_ok($obj, "Tie::Array::Bounded");
So far, this just tests that the underlying object from the tie is or inherits from Tie::Array::Bounded. Run this test before you even add any code to TIEARRAY to make sure that it does indeed fail:
% ./02tie.t 1..1 Using /home/peter/perl_Medic/Tie/Array/Bounded/t/../blib not ok 1 - The object isa Tie::Array::Bounded # Failed test (./02tie.t at line 10) # The object isn't defined # Looks like you failed 1 tests of 1.
We're not checking that the module can be used or that it has a TIEARRAY method; we already did those things in 01load.t. Now we know that the test routine is working properly. Let's make a near-minimal version of TIEARRAY that will satisfy this test:
sub TIEARRAY
{
my $class = shift;
my ($upper, $lower);
return bless { upper => $upper,
lower => $lower,
array => []
}, $class;
}
Now the test passes. Should we test that the object is a hashref with keys upper, lower, and so on? No—that's part of the private implementation of the object and users, including tests, have no right peeking in there.
Well, it doesn't really do to have a bounded array type if the user doesn't specify any bounds. A default lower bound of 0 is obvious because most bounded arrays will start from there anyway and be limited in how many elements they can contain. It doesn't make sense to have a default upper bound because no guess could be better than any other. We want this module to die if the user doesn't specify an upper bound (italicized code):
sub TIEARRAY
{
my ($class, %arg) = @_;
my ($upper, $lower) = @arg{qw(upper lower)};
$lower ||= 0;
croak "No upper bound for array" unless $upper;
return bless { upper => $upper,
lower => $lower,
array => []
}, $class;
}
Note that when we want to die in a module, the proper routine to use is croak(). This results in an error message that identifies the calling line of the code, and not the current line, as the source of the error. This allows the user to locate the place in their program where they made a mistake. croak() comes from the Carp Module, so we added a use Carp statement to Bounded.pm (not shown).
Note also that we set the lower bound to a default of 0. True, if the user didn't specify a lower bound, $lower would be undefined and hence evaluate to 0 in a numeric context. But it's wise to expose our defaults explicitly, and this also avoids warnings about using an uninitialized value. Modify 02tie.t to say:
use Test::More tests => 1;
use Test::Exception;
use blib;
use Tie::Array::Bounded;
dies_ok { tie my @array, "Tie::Array::Bounded" }
"Croak with no bound specified";
If you're running 02tie.t as a stand-alone test, remember to run make in the parent directory after modifying Bounded.pm so that Bounded.pm gets copied into the blib tree.
Great! Now let's add back in the test that we can create a real object when we tie with the proper calling sequence:
my $obj;
lives_ok { $obj = tie my @array, "Tie::Array::Bounded",
upper => 42
} "Tied array okay";
isa_ok($obj, "Tie::Array::Bounded");
and increase the number of tests to 3. (Notice that there is no comma after the block of code that's the first argument to dies_ok and lives_ok.)
All this testing has gotten us in a pedantic frame of mind. The user shouldn't be allowed to specify an array bound that is negative or not an integer. Let's add a statement to TIEARRAY (in italics):
sub TIEARRAY
{
my ($class, %arg) = @_;
my ($upper, $lower) = @arg{qw(upper lower)};
$lower ||= 0;
croak "No upper bound for array" unless $upper;
/\D/ and croak "Array bound must be integer"
for ($upper, $lower);
return bless { upper => $upper,
lower => $lower,
array => []
}, $class;
}
and, of course, test it:
throws_ok { tie my @array, "Tie::Array::Bounded", upper => -1 }
qr/must be integer/, "Non-integral bound fails";
Now we're not only checking that the code dies, but that it dies with a message matching a particular pattern.
We're really on a roll here! Why don't we batten down the hatches on this interface and let the user know if they gave us an argument we're not expecting:
sub TIEARRAY
{
my ($class, %arg) = @_;
my ($upper, $lower) = delete @arg{qw(upper lower)};
croak "Illegal arguments in tie" if %arg;
croak "No upper bound for array" unless $upper;
$lower ||= 0;
/\D/ and croak "Array bound must be integer"
for ($upper, $lower);
return bless { upper => $upper,
lower => $lower,
array => []
}, $class;
}
and the test:
throws_ok { tie my @array, "Tie::Array::Bounded", frogs => 10 }
qr/Illegal arguments/, "Illegal argument fails";
The succinctness of our approach depends on the underappreciated hash slice and the delete() function. Hash slices [GUTTMAN98] are a way to get multiple elements from a hash with a single expression, and the delete() function removes those elements while returning their values. Therefore, anything left in the hash must be illegal.
We're nearly done with the pickiness. There's one final test we should apply. Have you guessed what it is? We should make sure that the user doesn't enter a lower bound that's higher than the upper one. Can you imagine what the implementation of bounded arrays would do if we didn't check for this? I can't, because I haven't written it yet, but it might be ugly. Let's head that off at the pass right now:
sub TIEARRAY
{
my ($class, %arg) = @_;
my ($upper, $lower) = delete @arg{qw(upper lower)};
croak "Illegal arguments in tie" if %arg;
$lower ||= 0;
croak "No upper bound for array" unless $upper;
/\D/ and croak "Array bound must be integer"
for ($upper, $lower);
croak "Upper bound < lower bound" if $upper < $lower;
return bless { upper => $upper,
lower => $lower,
array => []
}, $class;
}
and the new test goes at the end of 02tie.t (italicized):
Example 3.2. Final Version of 02tie.t
#!/usr/bin/perl
use strict;
use warnings;
use Test::More tests => 6;
use Test::Exception;
use blib;
use Tie::Array::Bounded;
dies_ok { tie my @array, "Tie::Array::Bounded" }
"Croak with no bound specified";
my $obj;
lives_ok { $obj = tie my @array, "Tie::Array::Bounded",
upper => 42 }
"Tied array okay";
isa_ok($obj, "Tie::Array::Bounded");
throws_ok { tie my @array, "Tie::Array::Bounded", upper => -1 }
qr/must be integer/, "Non-integral bound fails";
throws_ok { tie my @array, "Tie::Array::Bounded", frogs => 10 }
qr/Illegal arguments/, "Illegal argument fails";
throws_ok { tie my @array, "Tie::Array::Bounded",
lower => 2, upper => 1 }
qr/Upper bound < lower/, "Wrong bound order fails";
Whoopee! We're nearly there. Now we need to make the tied array behave properly, so let's start a new test file for that, called 03use.t:
#!/usr/bin/perl
use strict;
use warnings;
use Test::More tests => 1;
use Test::Exception;
use blib;
use Tie::Array::Bounded;
my @array;
tie @array, "Tie::Array::Bounded", upper => 5;
lives_ok { $array[0] = 42 } "Store works";
As before, let's ensure that the test fails before we add the code to implement it:
% t/03use.t 1..1 Using /home/peter/perl_Medic/Tie/Array/Bounded/blib not ok 1 - Store works # Failed test (t/03use.t at line 13) # died: Can't locate object method "STORE" via package "Tie::Array::Bounded" (perhaps you forgot to load "Tie::Array::Bounded"?) at t/03use.t line 13. # Looks like you failed 1 tests of 1.
How about that. The test even told us what routine we need to write. perltie tells us what it should do. So let's add to Bounded.pm:
sub STORE
{
my ($self, $index, $value) = @_;
$self->_bound_check($index);
$self->{array}[$index] = $value;
}
sub _bound_check
{
my ($self, $index) = @_;
my ($upper, $lower) = @{$self}{qw(upper lower)};
croak "Index $index out of range [$lower, $upper]"
if $index < $lower || $index > $upper;
}
We've abstracted the bounds checking into a method of its own in anticipation of needing it again. Now 03use.t passes, and we can add another test to make sure that the value we stored in the array can be retrieved:
is($array[0], 42, "Fetch works");
You might think this would fail for want of the FETCH method, but in fact:
ok 1 - Store works Can't locate object method "FETCHSIZE" via package "Tie::Array::Bounded" (perhaps you forgot to load "Tie::Array::Bounded"?) at t/03use.t line 14. # Looks like you planned 2 tests but only ran 1. # Looks like your test died just after 1.
Back to perltie to find out what FETCHSIZE is supposed to do: return the size of the array. Easy enough:
sub FETCHSIZE
{
my $self = shift;
scalar @{$self->{array}};
}
Now the test does indeed fail for want of FETCH, so we'll add that:
sub FETCH
{
my ($self, $index) = @_;
$self->_bound_check($index);
$self->{array}[$index];
}
Finally we are back in the anodyne land of complete test success. Time to add more tests:
throws_ok { $array[6] = "dog" } qr/out of range/,
"Bounds exception";
is_deeply(\@array, [ 42 ], "Array contents correct");
These work immediately. But an ugly truth emerges when we try another simple array operation:
lives_ok { push @array, 17 } "Push works";
This results in:
not ok 5 - Push works # Failed test (t/03use.t at line 19) # died: Can't locate object method "PUSH" via package "Tie::Array::Bounded" (perhaps you forgot to load "Tie::Array::Bounded"?) at t/03use.t line 19. # Looks like you failed 1 tests of 5.
Inspecting perltie reveals that PUSH is one of several methods it looks like we're going to have to write. Do we really have to write them all? Can't we be lazier than that?
Yes, we can.8 The Tie::Array core module defines PUSH and friends in terms of a handful of methods we have to write: FETCH, STORE, FETCHSIZE, and STORESIZE. The only one we haven't done yet is STORESIZE:
sub STORESIZE
{
my ($self, $size) = @_;
$self->_bound_check($size-1);
$#{$self->{array}} = $size - 1;
}
We need to add near the top of Bounded.pm:
use base qw(Tie::Array);
to inherit all that array method goodness.
This is a big step to take, and if we didn't have canned tests, we might wonder what sort of unknown havoc could be wrought upon our module by a new base class if we misused it. However, our test suite allows us to determine that, in fact, nothing has broken.
Now we can add to 01load.t the methods FETCH, STORE, FETCHSIZE, and STORESIZE in the can_ok test:
Example 3.3. Final Version of 01load.t
#!/usr/bin/perl
use strict;
use warnings;
use Test::More tests => 2;
use blib;
BEGIN { use_ok("Tie::Array::Bounded") }
can_ok("Tie::Array::Bounded", qw(TIEARRAY STORE FETCH STORESIZE
FETCHSIZE));
Because our tests pass, let's add as many more as we can to test all the boundary conditions we can think of, leaving us with a final 03use.t file of:
Example 3.4. Final Version of 03use.t
#!/usr/bin/perl
use strict;
use warnings;
use Test::More tests => 15;
use Test::Exception;
use blib;
use Tie::Array::Bounded;
my $RANGE_EXCEP = qr/out of range/;
my @array;
tie @array, "Tie::Array::Bounded", upper => 5;
lives_ok { $array[0] = 42 } "Store works";
is($array[0], 42, "Fetch works");
throws_ok { $array[6] = "dog" } $RANGE_EXCEP,
"Bounds exception";
is_deeply(\@array, [ 42 ], "Array contents correct");
lives_ok { push @array, 17 } "Push works";
is($array[1], 17, "Second array element correct");
lives_ok { push @array, 2, 3 } "Push multiple elements works";
is_deeply(\@array, [ 42, 17, 2, 3 ], "Array contents correct");
lives_ok { splice(@array, 4, 0, qw(apple banana)) }
"Splice works";
is_deeply(\@array, [ 42, 17, 2, 3, 'apple', 'banana' ],
"Array contents correct");
throws_ok { push @array, "excessive" } $RANGE_EXCEP,
"Push bounds exception";
is(scalar @array, 6, "Size of array correct");
tie @array, "Tie::Array::Bounded", lower => 3, upper => 6;
throws_ok { $array[1] = "too small" } $RANGE_EXCEP,
"Lower bound check failure";
lives_ok { @array[3..6] = 3..6 } "Slice assignment works";
throws_ok { push @array, "too big" } $RANGE_EXCEP,
"Push bounds exception";
Tests are real programs, too. Because we test for the same exception repeatedly, we put its recognition pattern in a variable to be lazy.
Bounded.pm, although not exactly a model of efficiency (our internal array contains unnecessary space allocated to the first $lower elements that will never be used), is now due for documenting, and h2xs filled out some POD stubs already. We'll flesh it out to the final version you can see in the Appendix. I'll go into documentation more in Chapter 10.
Now we create 04pod.t to test that the POD is formatted correctly:
Example 3.5. Final Version of 04pod.t
#!/usr/bin/perl
use strict;
use warnings;
use Test::Pod tests => 1;
use blib;
use Tie::Array::Bounded;
pod_ok($INC{"Tie/Array/Bounded.pm"});
There's just a little trick there to allow us to run this test from any directory, since all the others can be run with the current working directory set to either the parent directory or the t directory. We load the module itself and then get Perl to tell us where it found the file by looking it up in the %INC hash, which tracks such things (see its entry in perlvar).
With a final "make test", we're done:
Files=4, Tests=24, 2 wallclock secs ( 1.58 cusr + 0.24 csys = 1.82 CPU)
We have a whole 24 tests at our fingertips ready to be repeated any time we want.
You can get more help on how to use these modules from the module Test::Tutorial, which despite the module appellation contains no code, only documentation.
With only a bit more work, this module could have been submitted to CPAN. See [TREGAR02] for full instructions.