Parity games are a formalism that is used in software verification. The question whether a computer system satisfies a given property can be described as a parity game. The complexity of solving parity games (the problem is in NP and co-NP), makes that het problem is not only interesting from a software verification point of view, but also from a complexity theoretic perspective. As a consequence, in recent years a large number of algorithms and heuristics for solving parity games have been developed. However, in this papers often only the theoretical complexity of the algorithm is considered, or, if experiments are performed they are based on a small set of parity games. In this talk, I introduce a large set of parity games that should lead to better and more complete comparisons of algorithms that have been introduced in the literature. This talk was presented at FSEN 2015. See my blog at http://www.jeroenkeiren.nl/blog/paper-benchmarking-parity-games/ for a more detailed description, and a download of the corresponding paper.

1. Benchmarking Parity Games
FSEN 2014
Jeroen Keiren
Open University of the Netherlands & VU University Amsterdam
24 April 2015
1 / 19

2. Where are parity games used?
Model Checking
Equivalence Checking
Satisfiability/Validity of modal logic
Synthesis
2 / 19

3. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
3 / 19

4. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
3 / 19

5. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
3 / 19

6. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
3 / 19

7. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
3 / 19

8. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
3 / 19

9. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
Winner?
3 / 19

10. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
Winner?
Optimal strategies?
3 / 19

11. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
Winner?
Optimal strategies?
3 / 19

12. Parity Games
V A set of vertices.
→ ⊆ V × V An edge relation.
Ω : V → N A priority mapping.
, Two players (even, odd).
(V , V ) A partition of V .
0
s
0
t
0
u
0
w
1
v
=even
=odd
Winner?
Optimal strategies?
3 / 19

13. Winning Parity Games
Memoryless determinacy
Partition (W , W ) of V
Player has memoryless winning strategy from W , for
∈ { , }
4 / 19

14. Solving Parity Games
Solving a parity game:
Determine partition (W , W )
Complexity:
Problem is in NP ∩ co-NP
Is it in P?
5 / 19

15. Solving Parity Games
Solving a parity game:
Determine partition (W , W )
Complexity:
Problem is in NP ∩ co-NP
Is it in P? Open!
5 / 19

16. Why benchmark?
Complexity + applications ⇒ active research
Algorithms for:
solving
simplifying
reducing
parity games
How to compare new algorithms to existing ones?
6 / 19

17. Existing practice
Only theoretical analysis (big-O)
Class of games that meets upper bound
Random games
(Very) small set of games
Results from different papers not comparable
7 / 19

18. Requirements on Benchmarks
Cover broad range of games:
Different problems
Different structural properties
Games from the literature
8 / 19

19. Contributions
Set of parity games
List of structural properties
Analysis of games w.r.t. these properties
9 / 19

20. Set of parity games
Model checking:
Communication protocols (C)ABP, BRP, SWP
Cache coherence protocol
Two-player board games
Industrial IEEE-1394 link-layer, truck lift
Elevator, Hanoi towers
Equivalence checking: strong-, weak-, branching
bisimulation of communication protocols
Validity/satisfiability of LTL, CTL, CTL*, PDL and
µ-calculus (using MLSolver)
Random games (using PGSolver)
Hard cases (using PGSolver)
10 / 19

21. Structural properties
Some properties known to affect complexity of solving:
Number of vertices and edges (“size”)
Number of priorities
Width measures (tree-width, DAG-width, etc.)
SCCs
New: alternation depth (inspired by modal equation
systems)
And some more. . .
11 / 19

22. Alternation depth
Describe complexity more accurately
Similar to ideas in [Emerson & Lee 1986] for µ-calculus
Three steps (let C ∈ sccs(G))
1. Nesting depth of v in C is #alternations between even
and odd priorities on paths of descending priorities in C
2. Nesting depth of C is max{nestingdepth(v) | v ∈ C}
3. Alternation depth of of a parity game is the maximal
nesting depth of its SCCs
12 / 19

23. Analysis of games w.r.t. structural properties
Vertices vs. edges
101
102
103
104
105
106
107
101
102
103
104
105
106
107
108
Vertices
Edges
modelchecking
equivalence
mlsolver
specialcases
random
13 / 19

24. Analysis of games w.r.t. structural properties
Alternation depth
101
102
103
104
105
106
107
100
101
102
103
104
Vertices
Alternationdepth
modelchecking
equivalence
mlsolver
specialcases
random
14 / 19

25. Analysis of games w.r.t. structural properties
Diameter
101
102
103
104
100
101
102
103
104
BFS Height
Diameter
modelchecking
equivalence
mlsolver
specialcases
random
15 / 19

26. Applications
Used to assess parity game reductions in [Cranen, K &
Willemse 2011,2012]
Subset of generation process used for benchmarks in [K,
Wesselink & Willemse, 2014]
Confirmed observation from [Friedmann & Lange 2009]:
recursive algorithm beats sophisticated algorithms
(unpublished)
16 / 19

27. Summary
I presented:
A set of parity games
Structural properties of parity games
An analysis of the games w.r.t. these properties
17 / 19

28. Open issues
Use structural properties to optimise/design algorithms
Perform large-scale comparison of different algorithms
Extend set of games with other encodings/more examples
Design algorithms for computing more complex structural
properties
18 / 19

29. Please contribute your own games!
jeroenkeiren.nl
github.com/jkeiren/paritygame-generator
19 / 19