1. Model Checking Programs with Java PathFinder Willem Visser < [email_address] > Peter Mehlitz < [email_address] > NASA Ames Research Center

11. JPF Configuration

15. Directory Structure

17. Under the Hood - Search

19. Extending JPF - SearchListener public interface SearchListener { /* got the next state */ void stateAdvanced (Search search); /* state was backtracked one step */ void stateBacktracked (Search search); /* a previously generated state was restored (can be on a completely different path) */ void stateRestored (Search search); /* JPF encountered a property violation */ void propertyViolated (Search search); /* we get this after we enter the search loop, but BEFORE the first forward */ void searchStarted (Search search); /* there was some contraint hit in the search, we back out could have been turned into a property, but usually is an attribute of the search, not the application */ void searchConstraintHit (Search search); /* we're done, either with or without a preceeding error */ void searchFinished (Search search); }

20. Extending JPF - VMListener public interface VMListener { void instructionExecuted (JVM vm); // VM has executed next instruction void threadStarted (JVM vm); / / new Thread entered run() method void threadTerminated (JVM vm); // Thread exited run() method void classLoaded (JVM vm); // new class was loaded void objectCreated (JVM vm); // new object was created void objectReleased (JVM vm); // object was garbage collected void gcBegin (JVM vm); // garbage collection mark phase started void gcEnd (JVM vm); // garbage collection sweep phase terminated void exceptionThrown (JVM vm); // exception was thrown void nextChoice (JVM vm); // choice generator returned new value }

21. Extending JPF - Listener Example public class HeapTracker extends GenericProperty implements VMListener , SearchListener { class PathStat { .. int heapSize = 0; .. } // helper to store additional state info PathStat stat = new PathStat(); Stack pathStats = new Stack(); public boolean check (JVM vm, Object arg) { // GenericProperty return (stat.heapSize <= maxHeapSizeLimit); } public void stateAdvanced (Search search) { // SearchListener if (search.isNewState()) {.. pathStats.push(stat); stat = (PathStat)stat.clone(); .. } public void stateBacktracked (Search search) { // SearchListener .. if (!pathStats.isEmpty()) stat = (PathStat) pathStats.pop(); } public void objectCreated (JVM vm) {.. // VMListener ElementInfo ei = vm.getLastElementInfo(); ..stat.heapSize += ei.getHeapSize(); .. } public void objectReleased (JVM vm) { // VMListener ElementInfo ei = vm.getLastElementInfo(); ..stat.heapSize -= ei.getHeapSize(); .. } ... }

26. MJI - How

29. POR - Scheduling Relevance

31. Choice Generator Motivation

32. JPF Perspective State consists of 2 main components, the state of the JVM and the current and next choice Generator (i.e. the objects encapsulating the choice enumeration that produces new transitions) Transition is the sequence of instructions that leads from one state. There is no context within a transition, it's all in the same thread. There can be multiple transitions leading out of one state Choice is what starts a new transition. This can be a different thread, i.e. scheduling choice, or different “random” data value.

33. Role of Choices In other words, possible existence of Choices is what terminates the last Transition, and selection of a Choice value precedes the next Transition. The first condition corresponds to creating a new ChoiceGenerator, and letting the SystemState know about it. The second condition means to query the next choice value from this ChoiceGenerator (either internally within the JVM, or in an instruction or native method).

35. Regression Tests with JUnit

36. JUnit Example package gov.nasa.jpf.mc; import org.junit.Test; import org.junit.runner.JUnitCore; import gov.nasa.jpf.jvm.TestJPF; public class TestOldClassicJPF extends TestJPF { static final String TEST_CLASS = &quot;gov.nasa.jpf.mc.oldclassic&quot;; public static void main (String[] args) { JUnitCore.main(&quot;gov.nasa.jpf.mc.TestOldClassicJPF&quot;); } @Test public void testDFSearch () { String[] args = { TEST_CLASS }; runJPFDeadlock(args); } @Test public void testBFSHeuristic () { String[] args = { &quot;+search.class=gov.nasa.jpf.search.heuristic.HeuristicSearch&quot;, &quot;+search.heuristic.class=gov.nasa.jpf.search.heuristic.BFSHeuristic&quot;, TEST_CLASS }; runJPFDeadlock(args); } }

39. Stoned Hippies Germany Netherlands 5 10 2 1

40. Stoned Hippies Germany Netherlands 2 5 10 1 2

41. Stoned Hippies Germany Netherlands 3 5 10 1 2

42. Stoned Hippies Germany Netherlands 8 10 1 2 5

43. Stoned Hippies Germany Netherlands 19 1 2 5 10

45. Concrete Execution Path (example) x = 1, y = 0 1 >? 0 x = 1 + 0 = 1 y = 1 – 0 = 1 x = 1 – 1 = 0 0 – 1 >? 0 int x, y; if (x > y) { x = x + y; y = x – y; x = x – y; if (x – y > 0) assert(false); }

46. Symbolic Execution Tree (example) x = X, y = Y int x, y; if (x > y) { x = x + y; y = x – y; x = x – y; if (x – y > 0) assert(false); } X >? Y [ X > Y ] y = X + Y – Y = X [ X > Y ] x = X + Y – X = Y [ X > Y ] Y - X >? 0 [ X <= Y ] END [ X > Y ] x = X + Y [ X > Y, Y – X <= 0 ] END [ X > Y, Y – X > 0 ] END

47. Example class Node { int elem; Node next; Node swapNode() { if (next != null) if (elem > next.elem) { Node t = next; next = t.next; t.next = this; return t; } return this; } } ? null E0 E1 E0 E0 E1 null E0 E1 ? E0 E1 E0 E1 Input list + Constraint Output list E0 > E1 none E0 <= E1 none E0 > E1 E0 > E1 E0 > E1 E1 E0 ? E1 E0 E1 E0 E1 E0 null E0 E1 E0 ? null NullPointerException

52. Algorithm (illustration) consider executing next = t.next; E0 next E1 next t null t E0 next E1 next ? next E0 next E1 t next E0 next E1 next t E0 next E1 next t Precondition: acyclic list E0 E1 next t null next t E0 E1 next ? next next

53. Implementation via Instrumentation program instrumentation counterexample(s)/test suite [heap+constraint+thread scheduling] model checking decision procedure instrumented program correctness specification continue/ backtrack state: original program path condition (data) heap configuration thread scheduling

55. Red-Black Trees (1) The root is BLACK (2) Red nodes can only have black children (3) All paths from a node to its leaves contain the same number of black nodes. Self-balancing Binary Search Trees Java TreeMap Implementation (4) Acyclic (5) Consistent Parents repOk(): conditions (1)-(5)

56. repOk() Fragment boolean repOk(Entry e) { // root has no parent, root is black,… // RedHasOnlyBlackChildren workList = new LinkedList(); workList.add(e); while (!workList.isEmpty()) { Entry current=(Entry)workList.removeFirst(); Entry cl = current.left; Entry cr = current.right; if (current.color == RED) { if(cl != null && cl.color == RED) return false; if(cr != null && cr.color == RED) return false; } if (cl != null) workList.add(cl); if (cr != null) workList.add(cr); } // equal number of black nodes on left and right sub-tree… return true; }

58. Symbolic Execution of repOk() Example public static boolean repOk() { if (root == null ) return true ; if (root.color == RED) return false; … Size 1

60. repOk() x 2 abstract and concrete Symbolic Execution of Code During Lazy Initialization check Abstract repOk() When coverage is achieved, solve the symbolic constraints to create concrete inputs Concretize inputs by symbolic execution of Concrete repOk() over symbolic structures - as with Black-box TIG - Abstract repOk() : Symbolic Structure {true,false,don’t know} Concrete repOk() : Symbolic Structure Concrete Structure

62. White-box TIG: cover branches in deleteEntry(Entry p) /* precondition: p. repOk() */ private void deleteEntry(Entry p) { if (p.left != null && p.right != null ) { Entry s = successor(p); swapPosition(s, p); } Entry replacement = (p.left != null ? p.left : p.right); if (replacement != null ) { replacement.parent = p.parent; if (p.parent == null ) root = replacement; else if (p == p.parent.left) { p.parent.left = replacement; } else p.parent.right = replacement; p.left = p.right = p.parent = null ; if (p.color == BLACK) fixAfterDeletion(replacement); ...

63. Symbolic Execution for white-box TIG if (p.left != null && p.right != null ) { ... Symbolic structure before executing branch Concretize Concrete structure that will cover the code The symbolic structure is used as input to repOk() and lazily executed to obtain the concrete structure Symbolic structure(s) that cover the branch This structure “passes” the abstract repOk()

64. API Based Testing SUT ENV (m,n) m is the seq. length of API calls & n is the number of values used in the parameters of the calls API … put(v) del(v) Evaluate different techniques for selecting test-cases from ENV(m,n) to obtain maximum coverage

65. Framework SUT with minor instrumentation ENV TestListener Abstraction Mapping + State Storage Coverage Manager JPF

66. Environment Skeleton M : sequence length N : parameter values A : abstraction used for (int i = 0; i < M; i++) { int x = Verify.random(N - 1); switch (Verify.random(1)) { case 0: put(x); break; case 1: remove(x); break; } } Verify.ignoreIf(checkStateMatch());

67. Symbolic Environment Skeleton M : sequence length A : abstraction used for (int i = 0; i < M; i++) { SymbolicInteger x = new SymbolicInteger(“X“+i); switch (Verify.random(1)) { case 0: put(x); break; case 1: remove(x); break; } } Verify.ignoreIf(checkStateMatch());

68. Sample Output Test case number 77 for '15,L+R+P-REDroot ': put(0);put(4);put(5);put(1);put(2);put(3);remove(4); Unique ID for the test Branch Number Predicate Values Test-case to achieve above coverage Test case number 7 for '32,L-R-P+RED': X2 (0) == X1 (0) && X2 (0) < X0 (1) && X1 (0) < X0 (1) put( X0 );put( X1 );remove( X2 ); Test case number 7 for '32,L-R-P+RED': put(1);put(0);remove(0); Concrete Symbolic Path Condition with solutions Symbolic TC

69. Subsumption Checking x1 x2 x3 x4 x5 + x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1 x1 x2 x3 x4 x5 + x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1 If only it was this simple!

70. Existential Elimination x1 x2 x3 x4 x5 PC s1 < s2 & s4 > s3 & s4 < s1 & s4 < s5 & s7 < s2 & s7 > s1 s1 s4 s2 s3 s5 +  s1,s2,s3,s4,s5 such that x1 = s1 & x2 = s4 & x3 = s3 & x4 = s5 & x5 = s2 & PC x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1

73. JPF Symbolic Execution - BEFORE Omega Interface Formula satisfiable/unsatisfiable Omega Java Version JPF

74. JPF Symbolic Execution - NOW Generic Decision Procedure Interface Formula satisfiable/unsatisfiable Omega Maryland JPF CVCLite Stanford Yices SRI STP Stanford Collaborator Saswat Anand

79. Results TCAS

80. Results TreeMap STP took > 1 hour

81. State Matching in JPF VM State Matchable + Restorable Stored State (hashed) Matchable compression Collaborator Peter Dillinger Abstract State erase some parts of the state could be lossy Matchable

83. New Architecture VM/Search DefaultBacktracker VM bool int[] Restorer Serializer Serialized StateSet int[] set VM

84. Old Scheme in the New Architecture VM/Search DefaultBacktracker FullStateSet int[] set bool int[] Collapsing (de)Serializer int[] VM

85. New Architecture VM/Search CollapsingRestorer FilteringSerializer DefaultBacktracker VM bool int[] objects This is the default setting for JPF at the moment: vm.backtracker.class = gov.nasa.jpf.jvm.DefaultBacktracker vm.restorer.class = gov.nasa.jpf.jvm.CollapsingRestorer vm.serializer.class = gov.nasa.jpf.filter.FilteringSerializer vm.storage.class = gov.nasa.jpf.jvm.JenkinsStateSet JenkinsStateSet int[] set VM

87. New Architecture Revisited - Abstraction VM/Search CollapsingRestorer AbstractingSerializer DefaultBacktracker VM bool int[] objects This is the setting for the above configuration: vm.backtracker.class = gov.nasa.jpf.jvm.DefaultBacktracker vm.restorer.class = gov.nasa.jpf.jvm.CollapsingRestorer vm.serializer.class = gov.nasa.jpf.abstraction.abstractingSerializer vm.storage.class = gov.nasa.jpf.jvm.JenkinsStateSet JenkinsStateSet int[] set VM