Proactive process adaptation can prevent and mitigate upcoming problems during process execution. Proactive adaptation decisions are based on pre- dictions about how an ongoing process instance will unfold up to its completion. On the one hand, these predictions must have high accuracy, as, for instance, false negative predictions mean that necessary adaptations are missed. On the other hand, these predictions should be produced early during process execution, as this leaves more time for adaptations, which typically have non-negligible latencies. However, there is an important tradeoff between prediction accuracy and earliness. Later predictions typically have a higher accuracy, because more information about the ongoing process instance is available. To address this tradeoff, we use an ensemble of deep learning models that can produce predictions at arbitrary points during process execution and that provides reliability estimates for each prediction. We use these reliability estimates to dynamically determine the earliest prediction with sufficient accuracy, which is used as basis for proactive adaptation. Experimental results indicate that our dynamic approach may offer cost savings of 27% on average when compared to using a static prediction point.

1. Proactive Process
Adaptation using
Deep Learning
Ensembles
Andreas Metzger, Adrian Neubauer,
Philipp Bohn, and Klaus Pohl
CAiSE 2019, Roma, MMXIX
Bocca della Verità
S. Maria in Cosmedin
A. Metzger, A. Neubauer, P. Bohn, and K. Pohl, “Proactive process adaptation using
deep learning ensembles,” in 31st Int’l Conference on Advanced Information
Systems Engineering (CAiSE 2019), Rome, Italy, June 3-7, 2019, ser. LNCS, P.
Giorgini and B. Weber, Eds., vol. 11483. Springer, 2019. [Online]. Available:
https://doi.org/10.1007/978-3-030-21290-2_34

2. Process
completiontCheckpoint j
Process
start
Proactive Process Adaptation “in a Nutshell”
BIOC/FAiSE, Roma, MMXIX 2
Monitor
Predict
Proactive
adaptation
planned /
acceptable situations
= Violation
= Non-
Violation

E.g., Delayed
freight delivery
E.g., Schedule air
instead of land
transport
E.g., Freight
delivery within 2
days
Process
Performance

3. Agenda
1. Problem Statement and State of the Art
2. Deep Learning Approach
3. Experimental Evaluation
4. Summary and Outlook
CAiSE 2019, Roma, MMXIX 3

4. Problem Statement
Accuracy
• False violations  Unnecessary adaptations
• False non-violations  Missed adaptations
Earliness
• Early predictions  More time for adaptations
But: trade-off between accuracy and earliness
BIOC/FAiSE, Roma, MMXIX 4
[Teinemaa et al., 2019]
BPIC 2017BPIC 2012
[Metzger & Neubauer, 2018]
Cargo 2000
Accuracy[MCC]
Accuracy[AUC]

5. State of the Art
Improve earliness of accurate process predictions
• Use additional process data [Teinemaa et al., 2016], [Leontjeva et al., 2015]
• Hyper-parameter optimization [Francescomarino et al., 2016]
• Clustering [Francescomarino et al., 2018]
Early time series classification
• Classify with lowest number of data points [Mori et al., 2018], [Petitjean et al., 2014]
• Classify considering probability threshold [Mori et al., 2017]
Reliability estimates to select earliest prediction
• Class probabilities of random forests [Maggi et al., 2014], [Francescomarino et al., 2016]
Limitations / Gaps
• Probabilities of prediction techniques may be poor reliability estimates [Zhou, 2012]
 Deep learning ensembles to estimate individual prediction error
• No analysis on usefulness for proactive process adaptation
 Experimental analysis of cost savings
CAiSE 2019, Roma, MMXIX 5

6. Agenda
1. Problem Statement and State of the Art
2. Deep Learning Approach
3. Experimental Evaluation
4. Summary and Outlook
CAiSE 2019, Roma, MMXIX 6

7. Dynamically Deciding on Proactive Adaptation
Deep learning ensembles (RNN) to compute reliability estimates
7CAiSE 2019, Roma, MMXIX
Process
monitoring
data at
Checkpoint j
RNN Model 1
RNN Model m
…
Ensemble
Prediction
[Tj = “non-violation”]
Proactive
Process
Adaptation
No Proactive
Process
Adaptation
Prediction Tj
Reliability
estimate j
[Tj =
“violation”]
[j  threshold]
[j < threshold]

8. RNNs as Base Learners
RNN = Recurrent Neural Network
Benefits
• High accuracy [Tax et. al. 2017; Evermann et al. 2017, Metzger & Nebauer, 2018]
• Arbitrary length process instances (without sequence encoding)
• Predictions at any checkpoint
Scalability
• Long training time
 Parallelization
 Hardware speedups
8BIOC/FAiSE, Roma, MMXIX
Hardware type Training
time
CPU 25 min
GPU (Nvidia CuDNN) 8 min
Google TPU (Tensorflow) 2 min

9. Agenda
1. Problem Statement and State of the Art
2. Deep Learning Approach
3. Experimental Evaluation
4. Summary and Outlook
CAiSE 2019, Roma, MMXIX 9

10. Experimental Variables
Dependent: Cost
Independent
Relative adaptation costs 
• Adaptation cost =  · penalty
Reliability threshold 
Adaptation effectiveness 
•  linearly decreasing over process execution
10CAiSE 2019, Roma, MMXIX
Costs =
Adaptation
Costs
Adaptation
Costs
+ Penalty
0
Penalty
Violation
Non-Violationeffective
not
effective
Violation
Non-Violation
j 
threshold
No Proactive
Process Adaptation
Proactive Process
Adaptation
j <
threshold
Tj = “violation”
Tj = “non-violation”
Prediction
Tj

11. Data Sets
Four public data sets with different key characteristics
11CAiSE 2019, Roma, MMXIX
Cargo2000BPIC2012

12. Results
12BIOC/FAiSE, Roma, MMXIX
“Cheap”
adaptation
“Expensive”
adaptation
Static checkpoint
Dynamic approach
No proactive
adaptation

13. Results
Average cost savings
13CAiSE 2019, Roma, MMXIX
9.2%
27.2% 35.8%
15.1%
27%

14. Agenda
1. Problem Statement and State of the Art
2. Deep Learning Approach
3. Experimental Evaluation
4. Summary and Outlook
CAiSE 2019, Roma, MMXIX 14

15. Summary
Cost savings when dynamically balancing earliness and accuracy
Benefits of dynamic deep learning approach
• No need to decide on a fixed checkpoint
• No need for testing phase to compute accurate accuracy
15CAiSE 2019, Roma, MMXIX
Alarm about Delay
Reliability Estimate
Real-World Prototype

16. Outlook
Extension 1: More complex cost models
• Nonlinear costs
• Regression (instead of classification) models
Extension 2: Advanced reliability estimates
• E.g., exploit variance of ensemble [Bosnic & Kononenko, 2008]
16CAiSE 2019, Roma, MMXIX
Research leading to these results has
received funding from the EU’s Horizon
2020 research and innovation programme
under
Objective ICT-15 ‘Big Data PPP:
Large Scale Pilot Actions ‘
http://www.transformingtransport.eu
Thanks!