Designing a Language and System for REST-oriented service composition,as a service

IBM TJ Watson Research Center, Component Systems Group

Designing a Language and System for
REST-oriented service composition,
as a service
Rania Khalaf
rkhalaf@us.ibm.com

2010 SSAIE Summer School, Crete, Greece
© 2010 IBM Corporation

Outline

Developing a REST-friendly flow language, Bite.
Core concepts and design points of the Bite process model
Understanding the Bite runtime engine

Incorporating Bite into a hosted BPM as a Service system
The overall system
Issues and concerns when hosting such a system in the cloud

Conclusion

2 © 2010 IBM Corporation

The Bite language

Creating a lightweight workflow language

Expand the reach of flow oriented development
An attractive high level development model
Large pool of potential consumers

Approach: Web centric, lightweight, flexible
Leverage process-centric composition in SOA to deliver composition directly for the Web

What is the opportunity
Extend use of workflow as a development model of choice for interactive, Web centric
applications
Able to on-ramp to high QoS process engines through automatic model conversion: BPEL,
XPDL


The Bite language

Unified flow model

Targeting two flow usage scenarios:
Data flows, where aggregation of feeds is the main model.
Interactive flows, where a flow drives a set of Web centric interactions

Any combination of the two – we believe it does make
sense to combine data and interaction flows


The Bite language

Highlights

Deep Integration with the Web
Processes as Active Resources
Dynamic Data Types
Extensible Activity Set
Lightweight Process Model
Flat Graph Model
Workflow Scripting:
• Use implies definition for variables
• Convention over configuration
• Reduction of redundancy, indirection, and the need for multiple files (by
comparison with BPEL)
Flexible configurability:
• Values of variables may be set outside the process itself, similar to Java
properties.
• Allows late binding to service URLs, turning paths of the process on/off.

The Bite language

Lightweight and flexible

Lightweight model
Compatible subset of BPEL execution semantics
Forgiving, script like approach – up and running in seconds
Lightweight infrastructure:
Direct deployment to runtime through REST interaction with server
Runs on Web platform: IBM WebSphere sMash, servlet container, etc.
Flexibility: Extensible activities
Allow community defined steps with first class XML support within the language
Defined in Java, Groovy, or any of a set of supported scripting languages
Example: <xquery query=“…”/>, <twitter …/>
Flexibility: Pluggable data model and expression languages
Data and documents, content-type aware: XML, JSON, form-encoded, other mime types
Pluggable scripting capabilities: Javascript, XPath, Groovy…
Short and long running flows, persistence provided transparently


The Bite language

First class support for Web interactions

First class primitives for REST interaction
Web client and server capabilities (GET, PUT, POST)
E-mail, forms
ATOM lifecycle model
Deployed flow seen as a collection of flow instances
New flow instances created as initial request is posted to collection
Flow instances as resources
Support for many flavors of Web applications
Feed aggregation
Form based people flows
Synchronous and asynchronous interactions (HTTP, email)


The Bite language

Interaction support

Well defined interaction activities
Pluggable/adaptable protocol selection:
Incoming:
• REST(HTTP, with any content: XML, JSON, etc)
Outgoing:
• REST(HTTP, with any content), E-mail, etc
• Binding of addresses is inline. Late binding possible through:
- Variables, which may be ‘customized’
Local: Java, scripts, …

Support for asynchronous, human centric interactions
Support for long running sessions

The Bite language

Operational semantics

Basics: Activities and Links
Data links: <receivePOST name="orderRcv"
Carry single data item by value url=“initiateCase" />
Implies control and data dependency
<sendMail name="sendToManager"
Control links address=…/>
<input value=“orderRcv”/>
When a dependency is not associated with
data passing </sendMail>
•E.g. manager approval requirement
<receive-replyGET name=“MgrApproval”>
Have a transition condition, default is ‘or’
<control value=“sendToManager/>
BPEL like execution semantics </receive-replyGET/>
Graph style
<sendMail name="sendToSupplier"
Dead path elimination and associated address="orderRcv_Output.mfrEmail[0]" …>
restrictions Variables
<control value="MgrApproval"/>
Variables: <input value="orderRcv"/>
</sendMail>
Each activity gets default output and
exception variables:
•[activity_name]_Output
•[activity_name]_Exception

The Bite language

Other aspects of the Bite model

Extensible activities
Community defined action types, with dedicated high level XML syntax can be linked as
part of a Splice flow
Optional metadata for human readable description and tool enablement
Simple model with minimal pre-reqs: Does not require writing XML Schema, fancy tools,
etc.
Architected for dynamic content type metadata support, pluggable
expression languages
Architected for Web style dynamic content-type exchange
Multiple data access languages supported: JS, XPath 1.0
Rest-based Management API


The Bite language

APP inspired lifecycle: runtime and deployment

A deployed process as a logical collection whose members are process
instances.
The process as a collection whose URL address corresponds to the start-able
receive of the process
Runtime
“Implicit factory” model: POST to a base URI creates a new process instance.
• A new URI is created and associated with the new instance.
The newly created instance URI is returned to caller in the HTTP Location header.
• It identifies a “process instance resource”,
Subsequent calls are to the process instance URL followed by a segment with the relative URL
of a nested resource (ie: receive activity)
• A request to a URI containing the process id leads to lookup for the instance
Process instance termination is usually implicit when process execution completes

Rest-based Deployment Lifecycle
Currently using just GET (get state) /POST (create instance) /DELETE (explicit termination)
Deployment simply results in placing a file in the file system on the server
Additional management service includes retrieving deployed process URLs,
next available URLs per instance per user, etc.

The Bite language

Relation to BPEL

Started with BPEL
BPEL is powerhouse workflow language and standard, able to handle enterprise situations
We were after a script-like lightweight model to bring workflow to bring workflow to situational
application developers:
• Looked at BPEL sources of complexity, trimmed the syntax and the capabilities
• Added capabilities needed for Web apps
Ideas for what to keep and what to live without?


The Bite language

Relation to BPEL

Started with BPEL
BPEL is powerhouse workflow language and standard, able to handle enterprise situations
We were after a script-like lightweight model to bring workflow to bring workflow to situational
application developers:
• Looked at BPEL sources of complexity, trimmed the syntax and the capabilities
• Added capabilities needed for Web apps

Removed:
Scopes, including fault, compensation and event handlers
Ability to combine structured and flat graph based processes, choosing the flat graph model as more
intuitive. Think back to FDL.
Dependency on WS-* stack, especially requiring a-priori creation of WSDL, Schemas, etc.

Added:
Error links
Shortcut combining data and control link
Ability to handle Web data types natively
REST primitives for interaction
Default variables
13 Inline expressions, bindings. © 2010 IBM Corporation

The Bite language

Bite: Core Language Constructs
Interaction activities Notes

<receiveGET>, <replyGET>, Also *POST. Receiving and replying to messages coming over HTTP.
Contain a relative URL attribute used to match an incoming message.
<receiveReplyGET>
<GET> <PUT> Sending HTTP requests
<POST> <DELETE>
Non-interactive activities

<action> Call local code

<assign> Copy data

<wait> <empty> <terminate> Utility activities

Control primitives

<while> <foreach> Iteration. The only structured activities. Have an implicit ‘counter’
variable
<pick> External choice. Flattened BPEL pick. The output variable contains a
‘message’ and a ‘choice’
<control> Control link.

<input> Data dependency that serves as a control link as well if the value is an
activity name

The Bite language

Feed aggregation

GET FEED GET FEED

Fetch price and Fetch customized
availability updates catalog

Customized catalog
and prices

A document pipeline model


The Bite language

Adding interaction capabilities


The Bite language

Plus primitives to encode business logic

Data flow
Control flow


The Bite language

Feed Aggregator Example

receive

fetchYahooFeed fetchBBCFeed

aggregate

sort

reply


The Bite language

Feed Aggregator Example
<process name="sortSample">
<receivePOST name="rssRcv" url="getFeeds"/>

rssRcv
<GET name="getYahooFeed“
target="'http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/world/rss.xml'">
<control source="rssRcv"/>
</GET>

<GET name="getBBCFeed"
getYahooFeed getBBCFeeed
target="'http://rss.news.yahoo.com/rss/topstories'">
<control source="rssRcv"/>
</GET>

aggregate <aggregate name="aggregate">
<input value="getBBCFeed"/>
<input value="getYahooFeed"/>
</aggregate>
sort
<sort name="sort">
<input value="aggregate"/>
</sort>
rssRply
<replyPOST name="rssRply" url="getFeeds">
<input value="sort"/>
</replyPOST>
</process>


Deploying and Calling the Feed Aggregator Example

Deploy by POSTing the flow file to the deployment service
This places the file in its own directory (under Web-apps for a servlet container).
If the engine is local, deployment can be done by simply copying the file manually

Let’s assume it is now in …/flows/aggregatorSample/index.flow
To call the process, send an HTTP POST to:
http://[host-name]:[port]/bite/runtime/flows/aggregatorSample/getFeeds

runtime base url path to resource starting receive
relative URL


The Bite language

Interactive Flow Scenario

A customers requests a clothing item that needs to be ordered specially. A task flow is
created:
Request by employee to manager that item can be ordered without pre-payment
Employee puts call into designer to place order
Reminders are send via email to employee reminding them weekly to follow-up on order
When item is received, the “complete task” can be activated which sents am email notifying the customer
and cc’ing the employee.


The Bite language



The Bite language

Receive form data
(HTTP post)

Send email

Send back
HTTP
response

Control dependency
Take data
from a prior
activity

Iteration

Choice
based on an
event

Conditional
branching


The Bite language

Interactive Flow Scenario <receivePOST name="orderRcv" url=“initiateCase" />

orderRcv

<sendMail name="sendToMgr“
address="orderRcv_Output.mgrEmail[0] …> sendToMgr
…
</sendMail> <replyPOST name="replyToInitiate" url="initiateCase">
reply …
</replyPOST>
response
FromMgr

responseFromMgr=”yes”
<sendMail name="sendToMfr"
sendToDes address="orderRcv.mfrEmail[0]" …>
<control source="responseFromMgr" …/>
<pick name="pick"> <input value="orderRcv_Output"/>
<choice name="confirmation" url="confirm“/> while !confirmed </sendMail>
<choice name="reminderInterval" for="'P7D'"/> pick1
</pick> confirmation: confirm
reminderInterval: for 7 days

pick1.choice=='confirmation'
pick1.choice=='reminderInterval' <while name="loop"
condition=“$:pick.choice!=‘confirmation’">
replyToEmployee,
remindEmployee
confirm …
</while>

<replyGET name="replyToEmployee" url="confirm">
<control value="pick"
notify Customer
transitionCondition=
“$:pick_Output.choice=='confirmation'"/>
…
</replyGET>


The Bite language


The Bite language

Extensibility in the runtime

Bite extensions are resolved at runtime:
At parse-time:
• Unknown XML elements in the definition are deemed extensions
At runtime, when an extension activity is reached in the navigation
• The Bite extension registery is searched for the extension implementation
module.
• The registry is simply of element name to module mappings.
Upon navigating to an extension activity:
The engine navigator hands control and necessary data to the implementation module.
Once the module completes, it hands control and output data back to the navigator.
The navigator places the data in the activity's output variable, marks the extension activity
complete, and continues.
Macros are also enabled
A macro is a special extension whose implementation is a Bite fragment
The engine fires monitoring events as it executes a flow instance.


The Bite language

Anatomy of the Bite Runtime

...
GET /PUT / POST /
EMail DELETE

[ Activity ]
interpreter manager navigator
handler

...
Process context
XPath / XML
*. flow files
XPath /JSON
Javascript / JSON

Javascript /XML
Navigation Process Expression
state variables evaluator


The Bite language

Process Context

Contains all state information for a particular process instance, such as:
Variable values
Activity states
Pending events queue
Instance identifier

Only one thread is allowed to navigate a process instance at a time.
Events that come while the context is in use are added to a queue.


The Bite language

Process Manager

Process deployment.
Maintains tables of deployed processes.
Maintains a table of instances for each process.
Each instance context consists of tables of tables
containing all instance state.
Message routing.
For each incoming message, it locates the process
and the process’s instance.
It requests the navigator handle the message
(incoming event), passing it the message and the
process context using a thread from the manager’s
thread pool.

…

…
Handles external invocation requests from a
process
Uses a thread from its thread pool to
perform external invocation


The Bite language

Process Navigator

Contains generic process navigation
logic.
Given a process, a process context,
and an event queue:
It executes as many activities of the process
and as many events in from the queue as it
can, in a single thread.

It returns if:
the process has completed, or
there are no more activities that can be
executed, and
there are no events in the queue that can be
consumed.


The Bite language

Navigation Example

(3) s
(5) (7)
(1) (2)

(4)
receive receive receive

assign assign assign
receive

receive invoke receive invoke receive invoke
assign

reply reply reply reply reply reply
(6) receive invoke

(8) reply reply


Recap

So far:
Explained Bite’s characteristics and core design points
Showed several example of its usage
Delved into syntax and semantics

A few surprises in the last 3 years
A very small percentage of the processes created have required the use of non-default variables.
Nearly none have required the use of the separate <variable/> element
Nearly no processes use a non-default transition condition
Nearly no processes use error links
The extension mechanism has been widely popular

Next:
Into the cloud we go ☺


… and now, ‘as a Service’

Simple: REST APIs for the Bite Engine:
Deployment service
Monitoring registration service
Also enable security handlers

Complex: Complete BPM system with visual editor, monitoring capabilities, built
in forms integrations


Lightweight BPM as a Service: Business Mashups
Presentation Mashups
• Aggregate visual content with
internal/external services and feeds
Data Mashups
• Combine internal/external feeds into new
feeds
Designer
Business Mashups
• Combine internal/external business
services, information and collaboration
services

…
3rd Party
REST
Activities
Database Services

Bluehouse
Store &
Email Share

Forms
34 Feeds © 2010 IBM Corporation

Our System


Design considerations of such a system

Ideas?


Design considerations of such a system

Some that we highlight include:
Distributed execution
Security concerns
Malicious processes and liability
Social BPM: Leveraging the community


Distributed Execution

Load Balancing, routing, instance migration and persistence
The execution model provides partial support
separation of navigation logic from instance state

Requires complex support for routing, maintaining and replicating state.
In its simplest form, one engine per process with appropriate use of engine
placemeent and routing tables.


Security concerns


Security concerns

Problems and motivation
1. Secure inbound and
One needs to be able to: outbound interactions
with the flow

2.Perform calls
3. Associate activities on behalf of a user
with users/groups

40
40 18-Mar-13 © 2010 IBM Corporation

Security concerns

Motivating example: interview scheduling mashup


Security concerns

Security Challenges for Business Mashups

The environment is not a traditional enterprise setting
(Server side) Mashups attempt to leverage services offered from multiple domains
Different domains use different authentication protocols
Service access is registered to users, not the flow engine
Users have different identities on different domains

Moreover,
Multiple users in a typical (collaborative) business mashup
Simplified user and development experiences are critical to maintaining the mashup
value proposition
Hard-coding authentication is not an option
Storing credentials in the mashup server is not appropriate


Security concerns

Approach for Business Mashups Security

Bite Integration
Language/Model Extension:
• A way to declare security in a flow (Syntax)
• Managing and propagating multiple security contexts in a flow
Security Handler
• Decides if/when to contact the Secure Authentication Services (SAS)
• Handles communication with SAS

Secure Authentication Service (SAS)
Authentication & Authorization (explicitly ask user for permission)
Follows and implements OAuth (extended with attributes)
Assumption: lives in a trusted environment

43
43 18-Mar-13 © 2010 IBM Corporation

Security concerns

Architecture Overview
Multi-protocol
Language and engine support architecture
extensions

Interactive user
authentication with
trusted SAS


Security concerns

Requesting user credentials: Synchronous vs. asynchronous

Asynchronous is directly supported as stated before
Notification receiver information is used to contact appropriate user, direct to SAS for authentication

Synchronous is trickier:
Must have an open HTTP connection to get back to the user
Assumption is that an “open receive” is outstanding and a “reply” activity is available that has not
been executed
Moreover, it must “belong” to the right user: must have a “matching user” value
• if not available, follow asynch. redirect procedure, if provided, or fault is thrown
Special processing is provided for that reply activity:
• Enters “awaiting redirection” state
• HTTP response is sent back with redirection to SAS
• When SAS authentication is completed, HTTP response from SAS redirects user to
process instance


Security concerns

User Authentication Sequence

Authentication step: after providing
credentials user must authorize service
access

Authentication dialog on SAS redirect:
user selects link, authentication form
pops up


Security concerns

Secure Authentication Service (SAS) Operation

Extended OAuth
SAS supports different
security mechanisms for
calling out to services

SAS exposes itself using an
OAuth interface
Extension headers have been
defined to support each
authentication types

SAS operates in a secure
and trusted area within the
company network
Trusted area/party
Or operated by a trusted 3rd
party


Malicious Processes and Liability



The Challenge of Deploying Processes in the Cloud

Processes, in general, may have modeling problems
Processes in the cloud are especially vulnerable:
Users are charged according to how much resources they consume, so need to avoid surprise
charges
Cloud providers have to manage the resource pool
Cloud providers are (usually) liable for DoS or other attacks initiated from hardware they own and
operate, even if the software that causes this his written by an external party.



Techniques to Address Cloud Process Deployment Challenges

It may not be possible to detect and reject/suspend all cases, but some mitigation
techniques include:
Restrictions in the process language
E.g.: BPEL and Bite have DPE and cannot deadlock itself

Static Checking of (1) the model, (2) expressions in the process, (3) extension
activities
At design time by the tool
At deployment time by the process engine or extension deployment module

Runtime Clamping
Monitoring of resources consumed by the process and checking against thresholds
Monitoring of URLs called by process owned by the same user to detect potential DoS


Social BPM: Leveraging the Community


Social Enablement of a Workflow System

Aim to enable social production and Collaboration in a
workflow system focusing on three areas
Extension Activities
Workflow Models
Workflow Instances

Two key aspects:
Method and Technical Enablement
Business and Security Aspects



End Users Interact with Flow Instance

End User
End User
End User

Workflow Instance



Recruiting Scenario

Design
Career Fair
Workflow

Ted:
Brainshare
Inc. employee



Recruiting Scenario

Design Invite Dean to
Career Fair Collaborate
Workflow on Design

Ted:
Brainshare
Inc. employee



Recruiting Scenario
How about a
student brochure
design contest?
Also, resumes are
Design Invite Dean to in LotusLive Files.
Workflow on Design Collaborate

Ted:
Dean of
Brainshare
McGrath
Inc. employee
University



Recruiting Scenario
How about a
student brochure
design contest?
Also, resumes are

Ted:
Dean of
Brainshare
McGrath
Inc. employee
University

Update Flow.
Look for Files
Extension to
Get Resumes

Earlier that year… Create and Share
Extension Activity for
LotusLive Files Service

Carol: LotusLive
Developer


Recruiting Scenario
How about a
student brochure
design contest?
Also, resumes are

Ted:
Dean of
Brainshare
McGrath
Inc. employee
University

Look for Files Find and Add
Extension to Carol’s
Get Resumes Extension.
Update Flow.


Carol: LotusLive
Developer


Recruiting Scenario
How about a
student brochure
design contest?
Also, resumes are

Ted:
Dean of
Brainshare
McGrath
Inc. employee
University

Look for Files Find and Add
Extension to Carol’s Great!

Get Resumes Extension. Review
Update Flow.


Carol: LotusLive
Developer


Development and Deployment Life-Cycle

1. Community of developers easily build and share custom extension
activities
2. Process Designers easily include community extension activities into
flow applications
3. Flow applications are deployed in a secure, multi-user execution
environment
4. Sharing and collaboration are enabled for flow models and instances
5. Flow applications execute in a hosted environment



Creating, Sharing, and Using Extension Activities

• Hosted Platform - Nothing to install
• Example Scenarios
• Forms based approval (e.g. vacation request)
• Collaborative web-based ad-hoc workflows

• Main Components
• Flows Editor and Flows Runtime
• Forms Editor and Forms Runtime
• Catalog and Repository of Extension Activities

Workflow • Workflow Model and Instance Sharing

Designer

Discover and use extension activities

IT Developer
Development Environment
of choice

Catalog of Extension Activities


Some of the extensions we’ve incorporated so fars

Customized extensions for verticals:
We integrated the system in IBM’s LotusLive collaboration platform
As a result, created extensions for integrating into their collaboration services, such as:
• File sharing
• Lotus Activity lifecycle

Community provided extensions
Teams across IBM provided new extensions for services they needed:
• Twitter, Google Calendar, Lotus Forms, MQ.



Recall: Our BPM as a Service System



Sharing Workflows

Workflow Models
Access controls are enforced on different workflow models
A per-user dashboard shows models the user may view and/or edit.
Enable sharing created workflow models with others

Workflow Instances
Design time access controls are possible on all of a flow’s entry points.
For instances, leverage Bite’s process lifecycle model:
• A deployed workflow model becomes accessible at a base URL which acts
as an instance factory.
• Users targetting the base URL create a new instance with a newly created
instance URL
• Subsequent flow instance entry points are relative to the instance URL
By sharing base/instance URls, we enable different users to:
• Kick off a new instance
• Access different entry points on one instance


Designer Shares Flow Model with Others

Workflow
Designer

How about …?



Designer Provides Flow to End Users

Workflow
Designer

Designer provides Process
entry point URL to end
users



End Users Interact with Flow Instance

End Users
Workflow Instance


Summary

A lightweight workflow model for the Web environment
A hosted BPM as a service system for quickly and easily creating and running
Web applications using a flow paradigm.
Created by a mashup of IBM teams ☺:
Developed in internal open source
Contributors:
• Across IBM: Research, Software Group, Global Business Services
• Across geos: US, Japan, Israel, Canada, and China


Videos

• Video 1: Creating and running a process
• Video 2: Extensibility demo


Where can I get more information on …?
Bite:
A version is the flow language in IBM Project Zero (WebSphere sMash) http://projectzero.org
ICSOC 2007
XIME-P Workshop at ACM SIGMOD 2008
Bite and Collaborative Flows
IEEE Internet Computing, Special Issue on Services Mashups 2008
BPM as a service system:
Devoxx 2008 - presentation by Christina Lau, ‘BPMZero’
Composer in LotusLiveLabs (http://lotuslive.com), deployment disabled
IBM BPM BlueWorks, design time only
Detecting Potential for Undesirable Behavior from Business Processes Deployed
in the Cloud
IBM Research Report by Douglas Lovell, to appear.
End-to-End Security
ICSOC 2009 (patent filed)
Extensibility and Sharing
Workshop on Business Process Management and Social Software, BPM 2009
Anything else?
70 rkhalaf@us.ibm.com © 2010 IBM Corporation

Designing a Language and System for REST-oriented service composition,as a service

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Destacado

Destacado (7)

Similar a Designing a Language and System for REST-oriented service composition,as a service

Similar a Designing a Language and System for REST-oriented service composition,as a service (20)

Designing a Language and System for REST-oriented service composition,as a service