1. pg proctab
Accessing System Stats in PostgreSQL
Mark Wong markwkm@postgresql.org
Gabrielle Roth gorthx@gmail.com
PGWest Seattle (JDCon) 2009
Oct 16-18, 2009
3. Soooo . . .
You can query the PostgreSQL system catalog tables (e.g.
pg stat activity, pg stat all tables,
pg stat all indexes) to find out which queries are taking a long
time, which indexes are being scanned an unreasonable number of
times, etc.
5. What if you want to know about the OS?
pg proctab provides a collection of four C stored functions:
◮ pg cputime
◮ pg loadavg
◮ pg memusage
◮ pg proctab
6. What can do you with pg proctab?
◮ Query operating system process table
◮ Query operating system statistics
◮ Processor time
◮ Load averages
◮ Memory usage
◮ Without escaping out to a shell!
◮ ...plus generate reports about timeslices
7. pg cputime() Example
SELECT *
FROM pg_cputime();
user | nice | system | idle | iowait
--------+--------+--------+------------+--------
681317 | 109924 | 395481 | 1466101128 | 462661
(1 row)
8. pg cputime() Column Description
From Linux kernel source code at
Documentation/filesystems/proc.txt:
user: normal processes executing in user mode
nice: niced processes executing in user mode
system: processes executing in kernel mode
idle: processes twiddling thumbs
iowait: waiting for I/O to complete
10. pg loadavg() Column Description
load1: load average of last minute
load5: load average of last 5 minutes
load15: load average of last 15 minutes
last pid: last pid running
12. pg memusage() Column Description
Paraphrased from Linux kernel source code at
Documentation/filesystems/proc.txt:
memused: Total physical RAM used
memfree: Total physical RAM not used
memshared: Not used, always 0. (I don’t remember why. . . )
membuffers: Temporary storage for raw disk blocks
memcached: In-memory cache for files read from disk
swapused: Total swap space used
swapfree: Memory evicted from RAM that is now temporary on
disk
swapcached: Memory that was swapped out, now swapped in but
still in swap
15. pg proctab() Partial Column Description
Everything from the operating system such as /proc/<pid>/stat,
/proc/<pid>/io and /proc/<pid>/cmdline as well as data
from PostgreSQL system catalog such as pg stat activity table
are available but we’ll only cover some of the fields here:
Informative:
◮ pid
◮ comm - filename of the executable
◮ fullcomm (/proc/<pid>/cmdline)
◮ uid
◮ username
Processor:
◮ utime - user mode jiffies
◮ stime - kernel mode jiffies
...
16. pg proctab() Partial Column Description (cont.)
Memory:
◮ vsize - virtual memory size
◮ rss - resident set memory size
I/O:
◮ syscr - number of read I/O operations
◮ syscw - number of write I/O operations
◮ reads - number of bytes which this process really did cause to
be fetched from the storage layer
◮ writes - number of bytes which this process really did cause to
be sent from the storage layer
◮ cwrites - number of bytes which this process caused to not
happen, by truncating pagecache
22. Take a snapshot before running the query
i ps_procstat-snap.sql
BEGIN
ps_snap_stats
---------------
1
(1 row)
COMMIT
23. Execute the query
Don’t focus too much on the actual query, the idea is that is you
want to collect statistics for a single query:
SELECT nation,
o_year,
Sum(amount) AS sum_profit
FROM (SELECT n_name AS nation,
Extract(YEAR FROM o_orderdate) AS o_year,
l_extendedprice * (1 - l_discount) - ps_supplycost * l_quantity AS amount
FROM part,
supplier,
lineitem,
partsupp,
orders,
nation
WHERE s_suppkey = l_suppkey
AND ps_suppkey = l_suppkey
AND ps_partkey = l_partkey
AND p_partkey = l_partkey
AND o_orderkey = l_orderkey
AND s_nationkey = n_nationkey
AND p_name LIKE ’%white%’) AS profit
GROUP BY nation,
o_year
ORDER BY nation,
o_year DESC;
24. Take a snapshot after running the query
i ps_procstat-snap.sql
BEGIN
ps_snap_stats
---------------
2
(1 row)
COMMIT
25. Calculate Processor Utilization
$ ./ps-processor-utilization.sh [pid] [before] [after]
$ ./ps-processor-utilization.sh 4590 1 2
Processor Utilization = 1.00 %
What’s going on (partially):
SELECT stime, utime, stime + utime AS total,
extract(epoch FROM time)
FROM ps_snaps a, ps_procstat b
WHERE pid = ${PID}
AND a.snap = b.snap
AND a.snap = ${SNAP1}
TIMEDIFF=‘echo "scale = 2; (${TIME2} - ${TIME1}) * ${HZ}" | bc -l‘
U=‘echo "scale = 2; (${TOTAL2} - ${TOTAL1}) / ${TIMEDIFF} * 100" | bc -l‘
26. Calculate Disk Utilization
$ ./ps-io-utilization.sh 4590 1 2
Reads = 276981
Writes = 63803
Reads (Bytes) = 2164604928
Writes (Bytes) = 508166144
Cancelled (Bytes) = 36880384
SELECT syscr, syscw, reads, writes, cwrites
FROM ps_snaps a, ps_procstat b
WHERE pid = ${PID}
AND a.snap = b.snap
AND a.snap = ${SNAP1}
TIMEDIFF=‘echo "scale = 2; (${TIME2} - ${TIME1}) * ${HZ}" | bc -l‘
U=‘echo "scale = 2; (${TOTAL2} - ${TOTAL1}) / ${TIMEDIFF} * 100" | bc -l‘
31. Creating Reports: Section 2 - Falling off the right side...
◮ N Tup Upd
◮ N Tup Del
◮ Last Vacuum
◮ Last Autovacuum
◮ Last Analyze
◮ Last Autoanalyze
33. What else can we do with pg proctab?
Enable pg top to monitor remote databases by providing access to
the database system’s operating system process table.
36. . . . the fine print . . .
◮ Linux-only
◮ Developed on 8.3; still works on 8.4
◮ Download it from:
http://git.postgresql.org/gitweb?p=pg_proctab.git
◮ Change it:
git clone
git://git.postgresql.org/git/pg_proctab.git
◮ Patches welcome! We’ll be in the (Sn—H)ackers’ Lounge!
38. License
This work is licensed under a Creative Commons Attribution 3.0
Unported License. To view a copy of this license, (a) visit
http://creativecommons.org/licenses/by/3.0/us/; or, (b)
send a letter to Creative Commons, 171 2nd Street, Suite 300, San
Francisco, California, 94105, USA.