A compare and contrast of Continuous Integration testing tools that can be used for Perl projects and where they all fall short. Also looking at what an ideal solution could look like.

1. Continuous Integration
Tools in Perl
Michael Peters
Plus Three, LP
YAPC::NA 10
Pittsburgh, PA

2. What is CI?
●Build and test all the time
●As soon as you can
●As loud as you can
●On as many platforms as you need
●On as many different configurations as you need

3. What is CI?
●Home grown (bash + email)
●CruiseControl
●Buildbot
●Tinderbox
●CABIE
●Smolder
Hudson, TeamCity, Continuum, Cerberus, ControlTier,
●
OpenMake Mojo, lots of others

4. Take a drink

5. CruiseControl
●http://cruisecontrol.sourceforge.net/
●Java based
●Very well known, "industry standard"
●Lots of plugins for lots of things
● SCM
● Notification
● Build management (make, Ant, Maven, rake)
● Lots more

6. CruiseControl
●Web UI
● View old builds/tests
● See what CruiseControl is doing right now
● Is it checking out code?
● Is it building?
●Build Loop Daemon
● Separate process to do the heavy lifting

7. CruiseControl

8. CruiseControl
Limitations and Complaints
● Not Perl (Java)
● Run loop is on a single machine
● They are working on distributed
● Format is not a standard
● Yes it's XML, but...
● Whatever JUnit spits out

9. CruiseControl
Limitations and Complaints
● Not Perl (Java)
● Run loop is on a single machine
● They are working on distributed
● Format is not a standard
● Yes it's XML, but...
● Whatever JUnit spits out
● All Java and all XML

10. Take a drink

11. Buildbot
●http://buildbot.net/trac
●Python based
●Designed to be distributed
● master/slave architecture
● nice for OSS projects and volunteers
●Tracks some metrics over time
● warnings
● link checks
● compile time
● etc

12. Buildbot
●Lots of notification channels
● Email
● IRC bot (real time insults)
● GTk app
●Lots of built-in support for CSMs
● CVS, svn, perforce, Bonsai, git, Mercurial,
Bazaar,
●Web UI
●CLI

13. Buildbot

14. Buildbot
Limitations and Complaints
● Not Perl (Python)
● Sometimes too heavy
● Ugggggly
● It's complicated
● Configuration is all Python
● Format is not standard
● Whatever your test spits out and returns
● Developer's can't customize notifications for
themselves
● No detailed breakdown of test failures

15. Take a drink

16. Tinderbox
●http://www.mozilla.org/projects/tinderbox
●Perl based
●Mozilla Project
● currently being rewritten
● used for Firefox, Thunderbird, etc
● Integrates with Bugzilla
● Only supports CVS
●Add comments to build runs
●Currently being rewritten
● Seems to use Buildbot underneath

17. Tinderbox

18. Tinderbox
Limitations and Complaints
● No packages, no releases (grab from CVS)
● Not much support
● from Mozilla
● nor the Perl community
● Just does CVS
● Developers can't customize their notifications
● Kind of ugly

19. Tinderbox
Limitations and Complaints
● Format is not standard
append 3:
object count 10 = 10 OK
array count 13 = 13 OK
0: 3=3 (0x84f9540) c: 4 OK
1: 0=0 (0x84f9510) c: 1 OK
2: 1=1 (0x84f9520) c: 1 OK
3: 2=2 (0x84f9530) c: 1 OK
4: 3=3 (0x84f9540) c: 4 OK
5: 4=4 (0x84f9550) c: 1 OK
6: 3=3 (0x84f9540) c: 4 OK
7: 5=5 (0x84f9560) c: 1 OK
8: 6=6 (0x84f9570) c: 1 OK
9: 7=7 (0x84f9580) c: 1 OK
10: 8=8 (0x84f9590) c: 1 OK
11: 9=9 (0x84f95e8) c: 1 OK
12: 3=3 (0x84f9540) c: 4 OK

20. Take a drink

21. CABIE
●http://cabie.tigris.org/
●Perl based
●Web UI and CLI tools
●Job daemon
● scheduled build jobs
● job priority
● job triggers
●Data stored in MySQL
●Email notification

22. CABIE

23. CABIE
Limitations and Complaints
● No installer
● Format is not standard
● Whatever text your build/test spit out and
return
● It's ugly too
● No community support

24. Take a drink

25. Smolder
●http://github.com/mpeters/smolder/tree/master
●Perl based
●Web UI
●Some CLI tools
●CPAN installable
●Public and Private
● understands projects
● understands developers
●Being used for Parrot VM project

26. Smolder
●View detailed reports about exactly what failed
● Ajax/on-demand
● Much easier to find what went wrong
●All reports are submitted TAP Archives
●All data stored in SQLite
●Notification channels
● Email
● Atom
●Tags
●Trend reporting

27. Smolder

28. Smolder

29. Smolder
Limitations and Complaints
● Fair number of dependecies
● 44 deps
● 132 deps of deps of deps...
● HTTP::Server::Simple isn't very reliable
● Hard coded notification options (only email and
Atom)
● No plugins for extra features
● No integration with other systems

30. Smolder
Limitations and Complaints
● No integrated build system
● simple bash script + cron
● SmokeRunner::Multi
● Module::Build::Smolder
● No notifications of build failures

31. Take a drink

32. My Dream Date
●Smolder and Buildbot sitting in a tree...
●Separate, but integrated build loop and reporting
●Build status and detailed test reporting
●Optionally distributed for multiple platforms and
parallel builds
●Standard formats
● TAP
● Something for the build notifications

33. My Dream Date
Pluggable architecture
●
● Notifications (email, twitter, building alarm system)
● SCM support
● Common build systems (make, Makefile.PL,
Build.PL, Ant, Maven, rake)
●Expose the TAP metadata for plugins to use
● Link to bug reports
● Link to source code or specs
● visual diffs
●Installable via CPAN
●Packagable for distros

34. Continuous Integration
Tools in Perl
Michael Peters
Plus Three, LP
¿?
All comics borrowed from
Toothpaste For Dinner
http://www.toothpastefordinner.com/