A Case for "Piggyback" Runtime Monitoring

THE FOLLOWING TALK HAS BEEN APPROVED FOR
OPEN-MINDED AUDIENCES

S SCIENTIFIC RESEARCH CONTENT
CONTROVERSIAL IDEAS, CONJECTURES
BACKED BY PRELIMINARY EXPERIMENTAL DATA

R

A Case for "Piggyback"
Runtime Monitoring
Raphaël Tremblay-Lessard and Sylvain Hallé
Laboratoire d'informatique formelle
Université du Québec à Chicoutimi, Canada
Fonds de recherche
sur la nature
et les technologies NSERC
CRSNG

Instrumentation

System

Instrumentation

Trace

System

Instrumentation

Trace

Events

System

Trace
Instrumentation validation

Trace

Events

System

Instrumentation
Runtime monitoring

System

Instrumentation
Runtime monitoring

Overhead

System

We consider a set A of method calls on a
single instance of some class C


C distinguishes between “valid” and
“invalid” sequences of method calls in A


C distinguishes between “valid” and
“invalid” sequences of method calls in A

C is deterministic: for every sequence
m ∈ A*, m is either always valid or
always invalid

( ∧ )→

( ∨∅)→ X

hasNext
or<T>
Iterat

next

next

hasNext

hasNext

hasNext
or<T>
Iterat

next

Close
Read

Open for read

Close

Open for write Write

class MyHouse {

boolean isHome = false;
boolean hasMail = false;

public void carLeaves() {
isHome = false;
}
public void carReturns() {
isHome = true;
}
public void mailman() {
hasMail = true;
}
public void pickMail() {
if (hasMail && !isHome)
hasMail = false;
else Do Something Bad;
}
}

There exists a minimal DFA M = 〈Q,q₀,δ〉
such that ℒ(M) = all valid sequences of
method calls in A

1 Let q = q₀
2 Wait for method call m ∈ A
3 Let q = δ(q, m)
4 If q = then raise error
5 Else goto 2

MyProgram:

m1.carLeaves();

m2.mailman();

m1.pickMail();

. . .

aspect Monitor (MyHouse m) {
MyProgram: int state = 0;

pointcut before carLeaves() {
if (state == 0)
state = 1;
m1.carLeaves(); }
pointcut before carReturns() {
if (state == 1)
state = 0;
m2.mailman(); }
pointcut before mailman() {
if (state == 1)
m1.pickMail(); state = 2;
}
pointcut before pickMail() {
if (state == 2)
. . . state = 1;
}

Non-intrusive...

...but "memoryful" (must persist q)

Lots of events to watch

Is there mail
in the box?

?

Key point

Don't observe
method calls...

Key point

Don't observe ...query the object's
method calls... state directly !
?
?

class MyHouse {
int state = 0;
pointcut before carLeaves() {
if (state == 0)
state = 1;
isHome = false;
}
}
pointcut before carReturns() {
if (state == 1)
isHome = true;
state = 0;
}
}
pointcut before mailman() {
hasMail = true;
if (state == 1)
}
state = 2;
}
hasMail = false;
if (state == 2)
state = 1;
}
}
}

class MyHouse {

isHome = false;
}
isHome = true;
}
hasMail = true;
}
hasMail = false;
if (m.isHome || !m.hasMail)
// Error
}
}
}

class MyHouse {
No state to persist
isHome = false;
} Only one method to
isHome = true; watch
}
hasMail = true; Same effect !
}
hasMail = false;
if (m.isHome || !m.hasMail)
// Error
}
}
}

Key point (again)

The first monitor observes method calls to
deduce the object's current state

The second monitor rather "piggybacks"
on the object's own state

Key point (again)

The first monitor observes method calls to
deduce the object's current state

The second monitor rather "piggybacks"
on the object's own state

PIGGYBACK RUNTIME
MONITORING

On an Iterator i, method i.next() should
only be called...

only be called...

if it immediately
follows i.hasNext()

only be called...

if it immediately if it has more elements
follows i.hasNext() to enumerate

only be called...


history-based

only be called...


history-based state-based

Sequences of
method calls

Object's
state

Wait a minute...

Does it work all
the time?

Let ~ be an equivalence relation between
two sequences m, m' ∈ A* of method
calls

Let ~ be an equivalence relation between
two sequences m, m' ∈ A* of method
calls

m ~ m' ⇔ for every m'' ∈ A*,
both mm'' and m'm'' are
either valid or invalid

[m] = equivalence class of m

Let S = { [m] : m ∈ A*}.

The relation ~ induces a transition
function ω : S × A → S ; namely

ω([m],m) = [mm]

C behaves like a deterministic
state machine O = 〈S,[ε],ω〉

ε a [m12
O b [m12]
M [m10] b
a
[m6] [m9]
a
c a a
b a
[m4] b [m2]
a b
b
c a
[m1]
[m8] [m5]
ε a
b a
a [m3]
b [m7]
[m
[m0] a
b
c [m11] ε

ε a [m12
O b [m12]
M [m10] b
All states of O
a
[m9]
[m6]
are accounted for
a
a a
c
b a
[m4] b [m2]
a b
b
No state of O c a
has two colors [m1]
[m8] [m5]
ε a
b a
a [m3]
b [m7]
[m
[m0] a
b
c [m11] ε

ε a [m12
O b [m12]
M [m10] b
All states of O
a
[m9]
[m6]
are accounted for
a
a a
c
(δ bis total)
a
[m4] b [m2]
a b
b
No state of O c a
has two colors [m1]
[m8] [m5]
ε a
(otherwise O and M b a
disagree on some m)
a [m3]
b [m7]
[m
[m0] a
b
c [m11] ε

ε a [m12
O b [m12]
M [m10] b
All states of O
a There exists a [m ]
are accounted for
a mapping ]
a
[m
a
6
9

c
(δ bis total) H : S→Q
a
[m ] b [m ]
a
such that H is a b 4 2

b
No state of O graph
c a
has two colors [m ]
homomorphism ]
1
[m ] [m
a
8 5

(otherwise O and M 〈S,ω〉 →b〈Q,δ〉 a
ε
disagree on some m) (just take the color!)
a [m ]
3
b [m7]
[m
[m0] a
b
c [m11] ε

2 Fetch s
3 Compute q = δ(H(s), m)
5 Else goto 1

2 Fetch s
3 Compute q = δ(H(s), m)
5 Else goto 1

Memory-less (nothing to persist
between calls)
Only monitor calls that may
lead to

Collection of 13 different monitoring
properties in papers from 2001-2011
(java.util package: the “monitoring canon”)

Collection of 13 different monitoring
properties in papers from 2001-2011
(java.util package: the “monitoring canon”)

Given an Iterator i on a Collection c, i.next()
cannot follow any of c.add(), c.delete(), c.set()
Given an Iterator i, i.remove() should not be called
twice without a call to i.next() in between
Don’t use an InputStreamReader after its
InputStream was closed
...

Source code analysis of the OpenJDK 6
implementation of Java. Of the 13 properties
in the canon:

in the canon:
10 can be piggyback-monitored

in the canon:
Remaining 3 either...

in the canon:
don't distinguish between
valid/invalid calls

in the canon:
don't distinguish between
valid/invalid calls
violate deterministic condition

Wait a minute...

How hard is it to
obtain H and s?

The graph homomorphism
problem

problem

Finding an homomorphism H between
two directed graphs G and G' is NP-
complete.

problem

complete.
(Known result)

problem

complete.
(Known result)

⇒ Checking if a known H “works” is
polynomial

cla
ss
int C
B b x;
... ;

Wait a minute...

cla
ss
flo B
a
A a t w;
cla int
;
ss D d j;
int C ;

B b x;
... ;

Wait a minute...

cla
ss
cla flo A
ss a
int t z;
flo B k;
a
A a t w;
cla int
;
ss D d j;
int C ;

B b x;
... ;

Wait a minute...

cla
ss
cla flo A
ss a
int t z;
flo B k;
a
A a t w;
cla int
;
ss D d j;
int C ;

B b x; cla
ss

... ;
... D

Wait a minute...

cla
ss
cla flo A
ss a
int t z;
flo B k;
a
A a t w;
cla int
;
ss D d j;
int C ;

B b x; cla
ss

... ;
... D

Wait a minute...
How many fields are
involved?

Propert y REMOVE
Traditional version
Given an Iterator i, i.remove() should
not be called twice without a call to
i.next() in between

Propert y REMOVE
Traditional version
i.next() in between

Piggyback version
Given an Iterator i, in every call to
i.remove(), it must hold that i.lastRet
must not be equal to −1

Propert y REMOVE
Traditional version
i.next() in between

Piggyback version
1
Given an Iterator i, in every call to
member field
i.remove(), it must hold that i.lastRet
involved

must not be equal to −1

Propert y SAFEENUM
Traditional version
Given an Iterator i on a Collection c,
i.next() cannot follow any of c.add(),
c.delete(), c.set(), etc.

Propert y SAFEENUM
Traditional version

Piggyback version
Given an Itr i, in every call to i.next(),
it must hold that i.expectedModCount is
equal to i$0.modCount

Propert y SAFEENUM
Traditional version

Piggyback version
2
Given an Itr i, in every call to i.next(),
member fields
it must hold that i.expectedModCount is
involved

equal to i$0.modCount

With the OpenJDK 6, the 10 piggyback
properties involved at most _______
primitive member fields inside a class

2

2
at a level of nesting of at most _______

2
1

2
1

H is simple and shallow

2
1

H is simple and shallow

...but many fields are private and have
no accessor methods ☹

Propert y SAFEF ILE READER
Traditional version

Piggyback version
Given an InputStreamReader r, in every
call to r.read(), it must hold that
r.sd.isOpen is true

Propert y SAFEF ILE READER
Traditional version

Piggyback version
Given an InputStreamReader r, in every
call to r.read(), it must hold that
r.sd.isOpen is true
private

class InputStreamReader {

private InputStream sd;

public boolean isStreamOpen()
{
return sd.isOpen();
}
class InputStream {
...
private boolean isOpen;
}
public boolean isOpen()
{
return isOpen;
}

...

}



{
return sd.isOpen();
} MISSING class InputStream {
...
}
{
return isOpen;
}

...

}



{
return sd.isOpen();
} MISSING class InputStream {
...
}
{
return isOpen;
} MISSING
...

}

cla
ss
cla pri
vat flo A
ss e i at z
flo B nt ;
pri a k;
v A a t w;
pri cla pri ate iint ;
vat ss vat
e
nt
D d j;
ei C D
;
int
nt
xx;
Bb ; cla
ss

... ;
... D

Wait a minute...
What about information
hiding?

“...one begins [to decompose a system] with a
list of difficult design decisions or design
decisions that are likely to change. Each
module is then designed to hide such a
decision from the others.”

― David Lodge Parnas, 1972



{
return sd.isOpen();
}
class InputStream {
...
}
{
return isOpen;
Difficult design decision? }
Likely to change? ...

}

The member fields of C involved in
the homomorphism H should have
the same read access visibilit y as C

design for monitoring


design for monitoring (or just for correct use)


OVERHEAD

Wait a minute...
Is overhead actually
reduced ?

Runtime monitoring of the 10 piggyback
properties on the DaCapo benchmark

Set of open source, real world applications
with non-trivial memory load
Same benchmark used by past monitoring
papers

Benchmark error margin ≈ 3%
Piggyback memory overhead ≈ 0%

Piggyback runtime overhead ≈ 3%

Highlights:
Runtime overhead (%)
Benchmark Piggyback
pmd ≈ 0%
avrora ≈ 1%

Highlights:
Runtime overhead (%)
Benchmark Classical Piggyback
pmd ≈ 123% ≈ 0%
avrora ≈ 118% ≈ 1%

Epilogue
Conclusion and future work

Piggyback runtime monitoring uses an
object's own, existing member fields as an
encoding of the monitor's state

Under some conditions, this encoding is
guaranteed to exist

Stateful properties become stateless
properties ; no data to persist between calls

Emprical evidence on real-world
benchmarks on the OpenJDK 6 reveal
that:

that:
Most properties are piggyback-
compatible

that:
compatible
H involves few variables

that:
compatible
H involves few variables
Runtime overhead is reduced over
classical monitors

Future work and open questions:

Use annotations instead of aspects
Find H automatically
Verify a given H statically
Extend to methods with arguments
Relax hypotheses

Piggyback and classical monitors are
complementary

complementary

The choice of a classical monitor should be
justified, not taken for granted

complementary

The choice of a classical monitor should be
justified, not taken for granted

Is overhead really what matters?

A Case for "Piggyback" Runtime Monitoring

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (10)

More from Sylvain Hallé

More from Sylvain Hallé (20)

A Case for "Piggyback" Runtime Monitoring