Short introduction to the basics of Perl testing and some resources for further reading. Includes basics of Test::Simple, Test::More, using Inline for multi-language testing, testing coverage, testing tests.
2. Cultured Perl
Perl's “test culture” is part of the language.
Test::* modules easily added to Perl code.
Smoke testing provided as part of CPAN.
Reporter accumulates results from installers.
Language structure, modules work together.
3. Where there's smoke, there's Perl.
CPAN::Reporter send back success/fail.
CPAN testers: variety of platforms, daily reports.
cpan-testers & cpan-testers-discuss mailing lists.
http://cpantesters.org/
Community effort at testing is unique to Perl.
4. Excellent Reference
Perl Testing: A developer's Notebook,
O'Reilly Press
Wonderful example of a good howto-book.
Includes modules, ways to use them with variations.
Good cover-to-cover read.
5. Perl is easy to test
Test::* modules do most of the work.
Use the wheels, don't re-invent:
Test::Simple, Test::More, Test::Deep,
Test::Builder, Test::Coverage, Object::Exercise
Perl adds introspection.
6. Onestop shopping
Perl as a great glue language.
Use perl to test other programs.
Have other programs output TAP.
Combined with “inline” you can test almost anything!
7. Test your tests!
Devel::Coverage have you tested all of the code?
Test::Builder roll your own tests.
Build you own re-usable test components:
Test::Harness::Straps
Test::Builder::Tester
10. Its all “ok”
use Test::Simple tests => 2;
# ok( boolean, message );
ok 1 == 1, “Count: '1' (1)”;
ok 2 == 1, “Count: '2' (1)”;
Output like:
ok 1 Count: '1' (1)
not ok 2 Count: '2' (1)
11. Its all “ok”
use Test::Simple tests => 2;
# ok( boolean, message );
ok 1, 'It passed!';
Output like:
1..2
ok 1 It passed!
# Looks like you planned 2 tests but ran 1.
14. Isolate the failure
What to fix:
ok -e $path , “Existing: '$path'”;
ok -d _ , “Directory: '$path'”;
ok -r _ , “Readable: '$path'”;
ok -w _ , “Writeable '$path'”;
18. Good general format
Include what you found, what you expect.
You'll need them both with “not ok”:
ok $got eq $want, 'Name: '$got' ($want)”;
not ok 99 – Name: '' (Jow Bloe)
not ok 99 – Name: '1 Main St.' (Jow Bloe)
not ok 99 – Name: 'Jow Bloe' ()
not ok 99 – Name: 'Jow Bloe' (User::Name=0x1A...
21. Flexable count
Set count with “plan”.
Compute from test input:
plan tests => 42;
plan tests => 3 * keys $config;
plan tests => 2 * values $query_output;
Test count varys?
Skip plan.
23. Notes show up with “prove v”
note “read_config from '$config_path'”;
my $path = read_config 'tmp_dir';
ok -e $path, “Exists: '$path' (tmp_dir)”;
$ prove v;
# read_config from './etc/pass1.conf'
not ok 1 – Exists: '/var/tmp/foobar ' (tmp_dir)
24. Diagnostics show why tests failed
Show up without “-v”.
ok … or diag “Oopsie...”;
ok grep /blah/, @test_input, 'Found blah'
or diag “Missing 'blah':”, @test_input;
ok $dbh, “Connected”
or diag “Failed connect: '$dsn'”;
25. Explain shows exactly what failed
“explain” shows nested structure.
Use with “note” to show setup details.
With “diag” shows extra info on failure.
my $dbh = DBI->connect( @argz );
ok $dbh, 'Database connected'
or diag 'Connect args:', explain @argz;
26. Stringy “ok”
like, not_like use regexen.
saves “=~” syntax in the tests:
like $stuff, qr/milk/, “Got: 'milk'?”
or diag “Have: '$stuff' instead”;
not ok 1 Got: 'milk'?
# Have 'very new cheese' instead
38. Testing Structures
“like” syntax with nested structure:
use Test::Deep;
cmp_deeply $found, $expect, $message;
Great for testing parser or grammar outcome.
39. Devel::Cover: what didn't you check?
All of the else blocks?
All of the “||=” assignments?
All of the “... or die ...” branches?
Devel::Cover bookkeeps running code.
Reports what you didn't test.
Tells you what test to write next.
40. Similar to NYTProf:
Run your program & summarize the results:
$ cover -test;
or
$ perl -MDevel::Cover ./t/foo.t;
$ cover;
Running “cover” generates the report.
41. Introspection simplifies testing
Want to test database connect failure.
Can't assume SQL::Lite.
Flat file databases are messy.
Guarantee that something fails?
You could write an operation that should fail.
Then again, it might not...
49. Devel::Cover & Mocking
The biggest reason for mock anything:
Force an outcome to test a branch.
Iterate:
Test with Devel::Cover.
See what still needs testing.
Mock object/method/function forces the branch.
52. Make the links
Path below ./lib.
Replace slashes with dashes.
Add a leading “01”.
Symlink them all to a generic baseline test.
Symlinks look like:
01WcurveUtil.t generic01t→
53. use FindBin::libs;
use Test::More;
use File::Basename qw( basename );
my $madness = basename $0, '.t'; # 01-WCurve-Util
$madness =~ s{^ 01-}{}x; # WCurve-Util
$madness =~ s{ W+ }{::}gx; # WCUrve::Util
if( use_ok $madness )
{
# check for correct package argument.
ok $madness->can( 'VERSION' ), “$maddness has a method”;
ok $a = $madness->VERSION, “$madness is '$a'”;
}
done_testing;
./t/generic01t
59. Load fixed data
Flat file -> arrayrefs
“insert” as method, [1] as return.
Load the data with:
$sth->$exercise( @data )
Get “ok” message for each of the data record.
60. Roll your own: Test::Builder
Mirrors Test::More with a singleton object.
my $test = Test::Builder->new;
$test->ok( $boolean, $success_message )
or $test->diag( $more_info );
Spread single test across multiple modules.
65. Example: Testing your C code
use Inline C;
use Test::Simple tests => 1;
my $a = 12 + 34;
my $b = add( 12, 34 );
ok $a == $b, "Add: '$b' ($a)";
__END__
__C__
int add(int x, int y)
{
return x + y;
}
68. Test Anything Protcol
Inline Language Support Modules add languages.
Inline supports:
C, C++, Java, Python, Ruby, Tcl,
Assembler, Basic, Guile, Befunge, Octave,
Awk, BC, TT (Template Toolkit), WebChat,
and (of course) Perl.
69. Really: Any Thing
Say you had a quad-rotor UAV...
You'd write a Perl API for it, wouldn't you?
UAV::Pilot::ARDrone, for example.
But then you'd have to test it...
70. First: Define the test actions
my @actionz =
(
[
takeoff => 10
],
[
wave => 8000
],
[
flip_left => 5000
],
[
land => 5000,
send => $expect
],
);
plan tests => 2 + @actionz;
Basic set of actions:
Takeoff, wobble, flip,
and land.
Final “send” validates
end-of-channel.
80. “pass” shows what happens
# Prepare:
# [
# 'land',
# 5000,
# 'send',
# '123'
# ]
ok 2 - Execute: 'takeoff' ()
ok 3 - Execute: 'wave' ()
ok 4 - Execute: 'flip_left' ()
ok 5 - Execute: 'land' (send)
81. “pass” shows what happens
# Prepare:
# [
# 'land',
# 5000,
# 'send',
# '123'
# ]
ok 2 - Execute: 'takeoff' ()
ok 3 - Execute: 'wave' ()
ok 4 - Execute: 'flip_left' ()
ok 5 - Execute: 'land' (send)
ok 6 - Recv completed '123' (123)
82. Cultured Perls
The Developer's Notebook is a great resource.
POD for Test::More == wheels you don't re-invent.
POD for prove == make life faster, easier.
cpantesters.org: test results for modules by platform.
Stay cultured, be testy: use perl.