Introduction to modern logging in perl via log4perl. Presented at July 2015 Sydney Perl Mongers

1. Getting Modern with
Logging via Log4perl
Dean Hamstead
Sydney PM
July 2015
dean@bytefoundry.com.au

2. Logging
Despite how much time we spend greping and
tailing logs (or both), logging is:
● Always overlooked
● Rarely done well
○ Inconsistent message formats (“started foo”, “foo
ended”)
○ Unhelpful messages (“Error 0x1234 occurred”)
○ Stacktraces???
○ Organic growth
● Reinvented over and over again

3. Why are we acting
against our own
self-interest?

4. Perl Logging 101.
● warn() and die(),
● collected via cron, apache etc.
● Similarly, print STDERR $foo
Better than nothing? Chaos in apache as
everything goes to the fallback error.log

5. Perl Logging 202.
● open(my $log,’>>’, ‘foo.log’)
● print $log “my message”
● $debug && print $log “debugging message”
For extra marks:
● sub log { print $log localtime, @_,”n” }
● log(“my message”)
● sub debug {} etc...

6. Perl Logging 203
● package MyApp::Log;
● use MyApp::Log qw/ log /;
● log(“my message”)
or
● $log = MyApp::Log->new();
● $log->message(“my message”)
(Then horrible stuff starts happening like
passing the $log object around)

8. Decisions, Decisions
Log::Log4perl
● Inspired by Log4j
● Modular
● Its own config file
● Optionally in-line
configured
● Available via CPAN
or your distro
packaging
Log::Dispatch
● It’s own design
● Module
● In-line configured
● Available via CPAN
or your distro
packaging

9. Also check out...
● Log::Any - which tries to let you not care as
much
● Log::Contextual
● Log::Agent - looks perfectly suitable too
● Sys::Syslog - it may be enough for you
● Apache2::Log - may also be just fine

10. Let’s just stick with Log4perl...

11. Installation
# The cpanm way
cpanm Log::Log4perl
# The Debian/Ubuntu/Mint way, where we say log and perl twice
# as a sort of incantation. Don’t say it in the mirror though.
apt-get install liblog-log4perl-perl
# The Fedora/RHEL/CentOS way
yum install ‘perl(Log::Log4perl)’
# The FreeBSD way
pkg install p5-log-log4perl

12. Basic Usage
use Log::Log4perl;
Log::Log4perl->init('my.log4perl.conf');
# defaults to pwd
$logger = Log::Log4perl->get_logger('My::Instance');
# Call the above over and over, you’ll always get the
# same object. i.e. it’s a Singleton
$logger->info('Foo');
$logger->fatal('Bar');
etc...

13. Basic Usage ++
Have Log4perl watch the config, reload and
reconfigure automagically on the fly:
Log::Log4perl->init_and_watch()
Or wait for HUP signal with:
Log::Log4perl->init_and_watch($conf_file, 'HUP');

14. Logging Levels
Lifted from Log::Log4perl pod:
There are six predefined log levels: FATAL, ERROR, WARN, INFO, DEBUG,
and TRACE (in descending priority). Your configured logging level has to at
least match (>=) the priority of the logging message.
If your configured logging level is WARN, then messages logged with info(),
debug(), and trace() will be suppressed. fatal(), error() and warn() will make
their way through, because their priority is higher or equal than the configured
setting.

15. Logging Levels (cont.)
The 6 basic logging levels have corresponding methods:
$logger->trace("..."); # Log a trace message
$logger->debug("..."); # Log a debug message
$logger->info("..."); # Log a info message
$logger->warn("..."); # Log a warn message
$logger->error("..."); # Log a error message
$logger->fatal("..."); # Log a fatal message

16. Logging Levels (cont.)
Because, why have just one way?
use Log::Log4perl::Level;
$logger->log($TRACE, "...");
$logger->log($DEBUG, "...");
$logger->log($INFO, "...");
$logger->log($WARN, "...");
$logger->log($ERROR, "...");
$logger->log($FATAL, "...");
This is actually useful for cleanly & concisely varying the log level based on
some logic appropriate to your program.

17. Austerity is a good thing!
This is expensive and useless if errors aren’t logged:
$logger->error("Erroneous array: @super_long_array");
So we can check before leaping into the cold murky waters:
if($logger->is_error()) {
$logger->error("Erroneous array: @super_long_array");
}
Here is the whole family:
$logger->is_trace() # True if trace messages would go through
$logger->is_debug() # True if debug messages would go through
$logger->is_info() # True if info messages would go through
$logger->is_warn() # True if warn messages would go through
$logger->is_error() # True if error messages would go through
$logger->is_fatal() # True if fatal messages would go through
This example lifted from Log::Log4perl pod

18. Not true in Log4perl!

19. Forget:
$logger->fatal($err) && die($err);
Use:
$logger->logdie($err);
Instead of:
warn($message);
$logger->warn($message);
Try:
$logger->logwarn($message);
Shortcuts!

20. Also you get for free:
● Log event time and date
● System hostname
● Pid of process
● Line number where called
● Package/Class caller
● Chomping, multi-line alignment
● OS-independant newline
● Milliseconds since program started
● Milliseconds since last log event
● Much much more!

21. Log4perl <3 Carp
Functions that, in addition to logging, also pass
the stringified message to their companions in
the Carp package:
$logger->logcarp(); # warn w/ 1-level stack trace
$logger->logcluck(); # warn w/ full stack trace
$logger->logcroak(); # die w/ 1-level stack trace
$logger->logconfess(); # die w/ full stack trace

22. Configuration: log4perl.conf
● By default, use a standalone config file
● Softens the observer problem
● Handy with config. management systems
and development. Install the file appropriate
to the environment
● Keep your own personal library of log4perl
configs. Copy them in place as needed.

23. A Bit of Theory
● As we have seen,
Log4perl provides
standard methods
● In the config file, select the minimum log
level then direct it to an Appender
● An Appender is basically an output
● An Appender can optionally apply a Filter
● Most Appenders allow you to specify a
layout for your log messages

24. Basic File Output
############################################################
# A simple root logger with a Log::Log4perl::Appender::File
# file appender in Perl.
############################################################
log4perl.rootLogger=DEBUG, LOGFILE
log4perl.appender.LOGFILE=Log::Log4perl::Appender::File
log4perl.appender.LOGFILE.filename=/var/log/myapp/myapp.log
log4perl.appender.LOGFILE.mode=append
log4perl.appender.LOGFILE.layout=PatternLayout
log4perl.appender.LOGFILE.layout.ConversionPattern=[%d] %F %L %c - %m%n

25. Basic Screen Output
############################################################
# A simple root logger with a Log::Log4perl::Appender::File
# file appender in Perl.
############################################################
log4perl.rootLogger=DEBUG, LOGFILE
log4perl.appender.LOGFILE=Log::Log4perl::Appender::Screen
log4perl.appender.LOGFILE.layout=PatternLayout
log4perl.appender.LOGFILE.layout.ConversionPattern=[%d] %F %L %c - %m%n

26. Other Appenders
● DBI - make your logs the DBA’s problem
● String - because perl
● RRDs - graphs impress management
● ScreenColoredLevels - like Screen but with colors
● Socket - why not?
3rd Party...
● SMTP - Why not flood your inbox?
● Gearman, RabbitMQ - flood your SOA
● Chunk::Store::S3 - because AWS solves everything
● ElasticSearch(::Bulk) - because no one ever got fired for
using ELK
● Journald - shove it into Lennart Poettering’s monster

27. When to log?

28. ● Program starts, stops, awakens or sleeps
● Opening files, sockets etc.
● Before and after retrieving URL’s
● Done reading or calculating an important
value
● Before and after decisions
● When exceptions are caught
● Log more details when something bad
happens (insomuch as we can anticipate it)
My Rules of Thumb. Log when...

29. How much logging in Production?
What’s happening when no one’s watching...

30. Do not log when...
● You’re a discrete general purpose module
● Instead, “throw exceptions” (i.e. die() or carp()) and make the caller play
catch. Let them log if needed.
my $obj = Foo->new();
eval { $obj->action() };
if ($@) { # handle $@
● Or, return null and provide an error inspection method or variable
$obj->action() or die(‘Bad thing: ‘, $obj->errormsg);
$obj->action() or die(‘Bad thing: ‘, $Foo::errormsg);
● Keep it simple. Write shy modules and avoid side effects.

31. Using Log4perl in a larger program
● Frameworks tend to have it inbuilt or via
plugin: Catalyst, Net::Server for example.
Profit!
● Don’t pass the $logger object around,
remember that Log4perl is a Singleton!

32. Example 1
package MyApp::UtilityMethods;
use Log::Log4perl;
# Log::Log4perl::init() called in main::
my $log =
Log::Log4perl->get_logger(__PACKAGE__);
sub action {
$log->debug(q/Running Action/);
}

33. Example 2
package MyApp::Some::Base;
use Log::Log4perl; # Log::Log4perl::init() is called in main::
sub log { return $self->{_log} }
sub new {
my ( $p, @a ) = @_;
my $c = ref($p) || $p;
my $self = bless {}, $c;
$self->{_log} = Log::Log4perl->get_logger(ref $self);
$self->log->debug( q|I'm here| );
return $self
}

34. Example 2 (cont.)
package MyApp::Some::Thing;
use parent qw/ MyApp::Some::Base /;
sub action {
my $self = shift;
$self->log->debug(q| Running Action |);
}

35. Planning
● Have clear guidelines for severity levels

36. Planning (cont.)
● Let Log4perl take care of metadata line the
timestamp, hostname, pid, package name
etc.
$log->debug(‘Program: starting up at ’
. localtime()
$log->debug(‘starting up’);

37. Planning (cont.)
● Design consistent messages that are easily parsed
(regex match) and tokenized
$log->debug(q|starting up|);
$log->debug(q|got arguments: |.join(q|,|,@args));
$log->debug(q|shutting down|);
$log->debug(q|action: starting|);
$log->debug(q|action: init arguments: |.join(q|,|@args));
$log->debug(q|action: shutdown|);
You’ll be super grateful for this when you inevitably start
looking at ElasticSearch

38. Questions?
Thanks for listening