BPM in the Cloud: Early Research Works and Challenges

NICTA Copyright 2012 From imagination to impact
BPM in the Cloud:
Early Research Works
and Challenges
Dr. Ingo Weber
Senior Researcher
NICTA, Sydney (ATP)
September 2013

Cloud Service Market Growth vs. BPM Market
Gartner (2013)
• The cloud is big, and here to stay

Cloud Service Market Growth vs. BPM Market
Gartner (2013)BPM market estimated at $2.6B in 2013, $7B in 2018
(PRWeb 16/6/2013)
• The cloud is big, and here to stay
• By comparison, BPM is a dwarf

Why BPM in the Cloud?
• BPM as a standalone system may not take off…
– But there are opportunities to use BPM thinking,
techniques, and tools in other domains / applications
• BPM in the cloud will happen
– If something provides value, the industry will build it
– This is an opportunity for the BPM research
community to have practical impact on a promising
area
• Many avenues to explore…
– This talk aims to provide a short overview of where
we are at, what is needed, and some of my works.
4

Cloud Computing Overview
5
Service Model
SaaS
Software as a Service
Examples: CRM, email, online games, vitual
desktops, etc.
PaaS
Platform as a Service
Examples: web server, database, execution
runtime, development tools, etc.
IaaS
Infrastructure as a Service
Examples: virtual machines, storage, load
balancers, network, etc.
Examples in BPM
In-browser modelling, e.g., Signavio, Oryx
Specialized environments, e.g., IFTTT, Effektif
Complete environments, e.g., Intalio|Create, Cordys
(now OpenText)
BPM execution platforms, e.g., Intalio
Workflow tools, e.g., AWS Simple Workflow Service
-
Left-hand side based on Wikipedia article Cloud Computing, 09/2013

SaaS-level BPM for end-users?
• Characteristics needed
– Limit functionality to something small but useful
– User experience
– Simplicity
– Social aspects for easy connection to
colleagues/friends
– Short learning curve: people need to understand the
tool very quickly
• Are we there yet?
– ...or should we try and re-think what BPM in the cloud
can be?
6
Top part inspired by T. Baeyen„s talk at BPM, 08/2013

Example: IFTTT does this quite well
7
www.ifttt.com, 09/2013

Example recipes in IFTTT
8
Whenever a game of
the Boston Red Sox
ends, send me an
email

Example recipes in IFTTT
9
Whenever a game of
the Boston Red Sox
ends, send me an
email
Whenever someone
tags me on a facebook
photo, download that
photo to my dropbox
account

Example: IFTTT
10
If there is a birthday of
a friend in my Google
calendar for today,
post „happy birthday“
on their wall

Are we there yet?
• Questions to drive research directions
– How can BPM benefit from the cloud?
– What should BPM in the cloud look like?
– What can BPM bring to cloud computing practices?
• Next: some of my early works
11

How can BPM benefit from the cloud?
• SaaS
– No installation, very little burden for people to get
started – if the tool fulfils the characteristics while
being useful
– My work: FormSys Process Designer [1,2]
• PaaS / IaaS
– Use cloud resources to enable process execution
– Benefit from elasticity to stay within SLAs, but not
over-pay for over-provisioning
– My work: process-aware auto-scaling of cloud
resources (on the IaaS level) [3]
12

Process-aware auto-scaling of cloud resources
• Assumption
– Executing process model that calls Web services (WSs)
– WSs are fully automated and perform compute-intensive tasks
– Horizontal scaling of the VMs running the WSs is possible
– Complexity of a task can depend on the input
• Example
– Integration logic around proprietary systems (e.g., in COBOL)
– Translation, transformation, aggregation can create high
utilization of resources (CPU, disk I/O, etc.) and slow down
process execution
• Problem
– Say we have external SLAs for the whole process, e.g., to
complete 95% of instances within 300s
– How many resources to schedule for each task, when the
requests change (complexity, frequency)?
13

• Setup of our experiment
14
Elastic Load Balancer 1
Auto Scaling Group 1
Instances for Service 1
ServiceProvider
Service 1 Service 2 Service 3

• Early work: use a fuzzy controller for process-aware auto-scaling
• Process backlog on cloud IaaS: process-aware vs. standard auto-scaling
Load profile (different
scale)
Complexity of the
requests increases,
plateaus, then
decreases, and finally
no new instances are
created

Backlog: in a given
minute, how many
more process
instances are started
than completed?
scale)

scale)
“Cut-offs” show where
time-outs happened,
caused by the ELBs.

What should BPM in the cloud look like?
• SaaS
– My work: FormSys Process Designer (2010-2011) [1,2]
– Metaphor: forms – ideally the ones known by users
– Approach: scripting (not Programming by Demonstration), but
made easy
– For execution & to enable technical editing of end-user-created
process models: translation to BPEL
– Features:
• user-editable forms are linked to Web services
• restricted, yet powerful generic composition language
• immediate automated process verification
• process simulation in the design phase
• automatic code generation: parallelized orchestrations
– Evaluated in a user study
18

FormSys Process Designer - core ideas
• Service: name + icon
– Visualize Data Service
• Messages to / from services
– Example: Sirca, computing GARCH Daily Volatility
19
Form through which
the user usually
controls a service

FormSys Process Designer - core ideas
• Service: name + icon
– Visualize Data Service
• Messages to / from services
– Example: Sirca, computing GARCH Daily Volatility
20
Form through which
the user usually
controls a service
Fields in the form
can be linked to
parts of WSDL
messages (by a
developer)

FormSys Process Designer - example
• Financial data analysis process:
1. Find news data: e.g., news data on the company „BHP‟
2. Find performance data: e.g., on company code „BHP.AX‟
3. Aggregate the performance data, e.g., avg. hourly stock prices
4. Merge datasets: e.g., merge the result data sets from the first
and third steps
5. Visualize dataset: e.g., to see influence of news on prices
21
Control flow
modelling
Data flow
modelling, see
[1,2] for details

FormSys Process Designer - User study [1]
• Ability to solve tasks in FormSys Process Designer
– Tech.: users with technical expertise
– Fin.: users with financial expertise
– Gen.: users with expertise in both
22
0
1
2
3
4
5
6
7
8
Tech. Gen. Fin.
Solved
Not solved
Not attempted

• Times by expertise
23
0:00
0:14
0:28
0:43
0:57
1:12
1:26
Gen. Tech. Fin.
Time(h:mm,successonly)
Expertise

• Times by expertise
24
0:00
0:14
0:28
0:43
0:57
1:12
1:26
Gen. Tech. Fin.
Time(h:mm,successonly)
ExpertiseMean time using
Intalio|bpms

What can BPM bring to cloud computing
practices?
• My work: using process mining in systems operations
(a.k.a. cloud administration / management)
– Disclaimer: started only 2 months ago
• Core question: can a process view help in operations?
1. Understanding how a tool is running
2. Understanding what the best
practice is (manual processes)
3. Detecting when process steps are
supposedly complete, but failed
4. Detecting deviations from
best-practice / usual path
5. Detecting unusual log messages
25

What can BPM bring to cloud computing
practices?
• My work: using process mining in systems operations
(a.k.a. cloud administration / management)
– Disclaimer: started only 2 months ago
• Core question: can a process view help in operations?
1. Understanding how a tool is running
2. Understanding what the best
practice is (manual processes)
3. Detecting when process steps are
supposedly complete, but failed
4. Detecting deviations from
best-practice / usual path
5. Detecting unusual log messages
26
Process discovery
Assertion checks
Conformance checks
Classification
Approaches:

Process Discovery for System Operations
• Challenges:
– How to get the right logs
– How to correlate events from several logs
• Especially in a large-scale distributed system
• What is a process instance
– How to handle concurrent sub-processes
– How to deal with low-level log messages
– How to analyze incomplete log messages
– How to deal with 1000s of log lines per instance
27

Process Discovery – Example: Asgard
• Netflix Asgard is a cloud management tool
– Open source
– For Amazon Web Services (AWS)
– Sunny day example: 1 log source, high quality logs, ...
• Example process: rolling upgrade
– “Rolling”: upgrade one virtual machine (VM) at a time
– Auto-scaling group (ASG): set of VMs from the same
Amazon machine image (AMI), usually behind an
elastic load balancer (ELB)
– Upgrade by replacing AMI for the ASG, then de-
register a VM from the ELB, terminate the VM, wait
for ASG to create a new VM from the new AMI
28

Process Discovery – Asgard logs: 3 lines
[2013-07-18 15:37:31,369] [Task:Pushing ami-ad059597 into group hadoopcluster--
firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135:
{Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast-
2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app
hadoopcluster] Started on thread Task:Pushing ami-ad059597 into group
hadoopcluster--firstASG for app hadoopcluster.
hadoopcluster] Updating launch from hadoopcluster--firstASG-20130718120001
with ami-ad059597 into hadoopcluster--firstASG-20130718153731
hadoopcluster] Create Launch Configuration 'hadoopcluster--firstASG-
20130718153731' with image 'amiad059597'
29

20130718153731' with image 'amiad059597'
30
„Instance ID“

20130718153731' with image 'amiad059597'
31
Payload
AMI ID
ASG ID

Process Discovery – Example: Asgard
32

Process Discovery – Approach
33

Open Challenges
34
Analysis Design Implementation Enactment Monitoring Evaluation
On-demand self
service
Broad network
access
Resource pooling
Rapid elasticity
Measured
service
SaaS
PaaS
IaaS
Private cloud
Community
cloud
Public cloud
Hybrid cloud
BPMlife-cycle (Mendling 2007)
NISTDefinitionofCloudComputing
EssentialCharacteristics
Service
Model
Deployment
Models
• Some dimensions when thinking about solutions
Disclaimer: likely incomplete and needs per-tool analysis

Summary & Outlook
• BPM in the cloud will happen
– If something provides value, the industry will build it
– This is an opportunity for the BPM research
community to have practical impact on a promising
area
• Presented some early work
– Process-aware auto-scaling of cloud resources [3]
– Domain-user BPM [1,2]
– Process view on system operations using mining
• Contact us for more information
– http://ssrg.nicta.com.au/people/?cn=Ingo+Weber
35

References
1. Ingo Weber, Hye young Paik, and Boualem Benatallah. Forms-
based web service composition for domain experts. ACM
Transactions on the Web (TWEB), conditionally accepted, 2013.
2. Ingo Weber, Hye young Paik, Boualem Benatallah. Forms-based
Service Composition. Accepted for publication in ICSOC'11: 9th
International Conference on Service Oriented Computing, short
paper, Paphos, Cyprus, December 2011.
3. Christian Janiesch, Ingo Weber, Michael Menzel, and Jörn
Kuhlenkamp. Optimizing the performance of automated business
processes executed on virtualized resources. In HICSS'14:
Hawaiian International Conference on System Science, January
2014, to appear.
Acknowledgements:
Thanks to Christian Janiesch and Len Bass for their input to the slides.
And thanks to many colleagues for the discussions and feedback.
36

BPM in the Cloud: Early Research Works and Challenges

Recomendados

Recomendados

Más contenido relacionado

Más de Ingo Weber

Más de Ingo Weber (8)

Último

Último (20)

BPM in the Cloud: Early Research Works and Challenges