2. Why monitoring?
• Because you want to find out problems before your customers
find them!
• This is not about finding bugs in your app, more about finding
deployment problems.
• Have to be notified about any problems, including OS, hardware
and Web server problems.
3. Nagios
• My favorite tool for monitoring.
• Been around since 1999 (was called Netsaint back in the days).
• Open source (C and PERL code).
• Works fine on Linux and OS X (use MacPorts to install on OS
X).
4. Nagios
• Can monitor many things with the default plugins : HTTP, SNMP,
disk space, load, etc.
• Can do remote checks (checks runs on the "client", essential for
things like disk space or load).
• Tons of plugins on Nagios Exchange, including for RDMBS like
Oracle or MySQL.
• Easy to write plugins, be it in PERL, Bash, C or even Java.
5. Nagios HTTP check
• You can use the default "check_http" plugin.
• Will check its state based on content's response, timeout, or
HTTP response code (404, 500, etc.)
• Can post data (might be useful for REST services).
• Can specify a specific HTTP method (HEAD, OPTIONS, TRACE,
PUT, DELETE, POST, GET).
• Can even check if your SSL certificate is still valid!
7. Java-specific checks
• Checks with check_http are fine... but won't find some problems
like "Out of memory" errors.
• You can use JMX for that!
8. Java-specific checks
• With JMX, you can monitor things like heap memory usage,
number of threads, number of loaded classes and CPU usage.
• Works fine on Java 5 and 6.
• Easy to enable.
9. Enabling JMX in your app
bash-3.2# cp /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/management/
jmxremote.password.template /Library/WebObjects/jmxremote.password
bash-3.2# chown appserver /Library/WebObjects/jmxremote.password
bash-3.2# chmod 600 /Library/WebObjects/jmxremote.password
In your apps launch arguments:
-Dcom.sun.management.jmxremote.port=XXXX
-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=/Library/WebObjects/jmxremote.password
10. WO-specific checks
• With WO 5.4, the stats available from WOStats are also available
by JMX.
• It give you access to data like memory usage, average session
memory, and average requests per session.
• You need to enable JMX + add one line in your Application class
constructor.
12. Enabling WO JMX in your app
import com.webobjects.appserver.WOStatisticsStore;
public Application() {
...
registerMBean((Object)statisticsStore(), getJMXDomain(), WOStatisticsStore.class.getName());
}
13. check_jmx
• check_jmx is a Nagios plugin to connect to JMX-enabled apps.
• Written in Java, source code is available.
• Sadly, it can only check results of type number or string, no
hashmap support.
15. Finding EOF deadlocks
• Create two DirectActions : one that use EOF, the other pure
JDBC.
• Make sure the DA returns the same result everytime.
• Call the DA with check_http, and validate the result.
• If the JDBC check is good, but not the EOF one, good chance
that you have a EOF deadlock.
16. Checking wotaskd config
• Nagios plugin that I wrote to check for various settings, based on
the /admin direct actions available in Wonder's JavaMonitor.
• Useful to check settings, like if auto recover or refuse new
sessions is off, number of deaths have reached a certain level or
that the app is not running.
• Work in progress, will release it in late September.
17. Graphing
• Would be nice to graph some data, like memory usage.
• Lot of tools can do this (Cacti, SNMP tools, etc.), but you can do
it with Nagios too.
• Graphing and performance data are useful when stress loading
your application with JMeter.
• Look at PNP4Nagios or NagiosGraph if you want graphing.
