Our talk from the 22nd International Symposium on Formal Methods. Full paper: http://www.cs.cmu.edu/~iruchkin/docs/ruchkin18-ipl.pdf Abstract: "Design and verification of modern systems requires diverse models, which often come from a variety of disciplines, and it is challenging to manage their heterogeneity -- especially in the case of cyber-physical systems. To check consistency between models, recent approaches map these models to flexible static abstractions, such as architectural views. This model integration approach, however, comes at a cost of reduced expressiveness because complex behaviors of the models are abstracted away. As a result, it may be impossible to automatically verify important behavioral properties across multiple models, leaving systems vulnerable to subtle bugs. This paper introduces the Integration Property Language (IPL) that improves integration expressiveness using modular verification of properties that depend on detailed behavioral semantics while retaining the ability for static system-wide reasoning. We prove that the verification algorithm is sound and analyze its termination conditions. Furthermore, we perform a case study on a mobile robot to demonstrate IPL is practically useful and evaluate its performance. "

1. IPL: An Integration Property Language
for Multi-Model Cyber-Physical Systems
Ivan Ruchkin, Joshua Sunshine, Grant Iraci, Bradley Schmerl, David Garlan
Institute for Software Research
Carnegie Mellon University
July 15, 2018
22nd International Symposium on Formal Methods

2. 22nd International Symposium on Formal Methods 2
Power-aware mobile robot

3. 3
Diverse CPS models
Power
Planning Control Mechanics
Obstacle dynamics

4. 4
Diverse CPS models and tools
Power
(Matlab)
Planning
(PRISM)
Control
(Simulink)
Mechanics
(Modelica)
Obstacle dynamics
(KeYmaera)

5. 5
Systemic qualities from models
Power
(Matlab)
Planning
(PRISM)
Control
(Simulink)
Mechanics
(Modelica)
Obstacle dynamics
(KeYmaera)
safe?
reliable?
secure?

6. Power-related models
• Linear equations for power
consumption and charging
• Built on experimental data
• Explicit turns, separate from
moving forward
22nd International Symposium on Formal Methods 6

7. Power-related models
• Linear equations for power
consumption and charging
• Built on experimental data
• Explicit turns, separate from
moving forward
• Markov decision process
• Different modeling formalism
• Implicit turns, part of
transitions between locations
22nd International Symposium on Formal Methods 7

8. 22nd International Symposium on Formal Methods 8
End-to-end power safety
Physical reality Regression model Planning model
f( , , )
Assurance argument: if the battery charge is always greater than total_error,
then the robot will not run out of power.
err_reg err_cons err_pltotal_error =

9. 22nd International Symposium on Formal Methods 9
Inconsistency example
Potential inconsistency: different treatment of turns
leads to different estimated energy costs.

10. 22nd International Symposium on Formal Methods 10
Existing approaches
• Super-model
• A single model to represent and reason about all aspects
• One-model-per-aspect
• Each “ground truth” is contained in some model
• Other models query it as needed

11. • Our approach
views & behavioral properties
IPL syntax
verification algorithm
• Evaluation
soundness/termination
case study
22nd International Symposium on Formal Methods 11
Outline

12. • Our approach
views & behavioral properties
IPL syntax
verification algorithm
• Evaluation
soundness/termination
case study
22nd International Symposium on Formal Methods 12
Outline

13. 13
Integration property
Integration property: “the difference in energy estimates should not
be greater than a predefined constant (err_cons)”
Potential inconsistency: different treatment of turns
leads to different estimated energy costs

14. • Separation of
structure (views)
and behavior
(modal properties)
• Logical co-
constraining models
via integration
properties
22nd International Symposium on Formal Methods 14
Our integration approach

15. 22nd International Symposium on Formal Methods 15
Our integration approach
Integration property: “the difference in energy estimates should not
be greater than a predefined constant (err_cons)”

16. • Design principles
• expressiveness
• modularity
• tractability
• Views
• static abstractions of model structures
• reasoned over by SMT solvers
• Behavioral properties
• modal logic formulas to constrain/query model behaviors
• verified by model checkers
22nd International Symposium on Formal Methods 16
Integration Property Language (IPL)

17. • Our approach
views & behavioral properties
IPL syntax
verification algorithm
• Evaluation
soundness/termination
case study
22nd International Symposium on Formal Methods 17
Outline

18. • A power view is designed as a library of atomic tasks
• one view for a given map
• automatic generation is possible
• Tasks can be combined to form missions
• constrained to represent valid sequences of robot motions
• instantiated using an SMT solver
• Each task is a component with property values
• start, end, energy, time, …
• used AADL as a language for views
22nd International Symposium on Formal Methods 18
Views for the power model

19. 22nd International Symposium on Formal Methods 19
Views for the power model
Declarations of atomic
forward-motion tasks
Declarations of atomic
rotation tasks

20. 20
Views for the power model
Taskl1→l2Taskl2→l1Taskl2→l3

21. • Specified using a PCTL-based property language
• checkable by the PRISM model checker
• manual creation
• Encoding of “maximum probability of the robot making the
given straight-turn-straight moves (t1 → t2 → t3)”
“…without skipping the middle location”
“…going through the locations in order…”
22nd International Symposium on Formal Methods 21
Behavioral properties for the planning model
“Maximum probability of…”

22. • Our approach
views & behavioral properties
IPL syntax
verification algorithm
• Evaluation
soundness/termination
case study
22nd International Symposium on Formal Methods 22
Outline

23. 22nd International Symposium on Formal Methods 23
IPL Syntax

24. 22nd International Symposium on Formal Methods 24
Integration property in IPL
“For any three sequential tasks from the power view that
• form a straight-turn-straight, non-intersecting sequence, and
• have sufficient energy,
any execution of the planning model that
• visits points in that sequence in the same order and
• is initialized appropriately (same energy modulo err_cons),
does not run out of power.”

25. • Our approach
views & behavioral properties
IPL syntax
verification algorithm
• Evaluation
soundness/termination
case study
22nd International Symposium on Formal Methods 25
Outline

26. 26
Verification algorithm
IPL verification: views, models ⊨ formula
Formula transformations: to PNF, removal of
quantifiers, abstraction of model subformulas (MS)
Functional abstraction (FA):
MS → uninterpreted f-ns
Saturation with SMT (on views):
find all free var solutions for FA ≠ CA
Model checking (on models):
interpret FA on the above solutions
Final check (on views):
check quantified FA conjoined
with the above interpretations
Constant abstraction (CA):
MS → uninterpreted consts
FA(t1, t2, t3) = 1
CA = 1
≠
CA = 1
FA(t1, t2, t3) = 1
FA(t1, t2, t3) = 1

27. • Our approach
views & behavioral properties
IPL syntax
verification algorithm
• Evaluation
soundness/termination
case study
22nd International Symposium on Formal Methods 28
Outline

28. Theorem (informally): The final check returns that the formula
is valid on the SMT solutions with the model checker
interpretations if and only if the formula is valid (semantically)
Corollary (informal): If IPL verification returns an answer, it is
the correct answer
Termination: Theoretically, the verification is not guaranteed
to terminate (no completeness). In practical conditions (finite
views and no quantification over infinite domains), it terminates
22nd International Symposium on Formal Methods 29
Soundness and termination

29. • Our approach
views & behavioral properties
IPL syntax
verification algorithm
• Evaluation
soundness/termination
case study
22nd International Symposium on Formal Methods 30
Outline

30. • Applied to an existing CPS
• Historic artifacts from a previous phase of the project
• Reviewed various models, focused on power
• Formalized 3 versions of the constraint on err_cons
• “If the regression model is power-safe, then so is the planner”
• “If the planner is power-safe, then so is the regression model”
• “For any mission, either both or neither are power-safe”
• Verified 120+ properties in total
• Variance due to many versions of maps and models
22nd International Symposium on Formal Methods 31
Mobile robot case study

31. • In the planner, no check for battery >= req_energy
Bug: battery := max(battery – req_energy, 0)
Effect: the last transition can be made with insufficient battery
charge
Consequence: some plans are too aggressive and would lead
to running out of power if executed
Fix: checking in the final state that battery > 0
(See other inconsistencies and performance results in the paper)
22nd International Symposium on Formal Methods 32
Discovered inconsistency

32. • What is the role of integration properties in practice?
• important and implicit step in end-to-end safety arguments
• violations of these properties lead to safety bugs
• Can these properties be specified in IPL?
• yes, by combining views and behavioral properties
• Is IPL verification tractable in practice?
• reasonable, but not ideal performance (seconds to days)
• low overhead: 0.74% +- 0.78%
22nd International Symposium on Formal Methods 33
Case study outcomes

33. • Complex and contextual integration properties
• requires understanding of complex relations between models
• integration properties do not transfer to new models
• Effortful additional modeling for views
• require difficult design decisions
• not always possible to automate
• Limited performance of this implementation
• SMT solving is not incremental
• model checking not concurrent
22nd International Symposium on Formal Methods 34
Limitations

34. 22nd International Symposium on Formal Methods 35
Summary