Getting Started with OpenSplice and Esper

Getting Started
with
OpenSplice and Esper
Angelo Corsaro, Ph.D. Thomas Bernhardt
Chief Technology Ofﬁcer Chief Technology Ofﬁcer / Funder
OMG DDS Sig Co-Chair EsperTech
PrismTech tom@espertech.com
angelo.corsaro@prismtech.com

Agenda
Welcome & Intro

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
!

! DDS Overview
! CEP Overview
! DDS + CEP in Action
! Q&A

DDS Overview
...from a Stream Processing Perspective

Angelo CORSARO, Ph.D.
Chief Technology Ofﬁcer
OMG DDS Sig Co-Chair
PrismTech
angelo.corsaro@prismtech.com

Stream Processing [1/3]

! Stream Processing is an architectural style for building

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
systems that operate over continuous (theoretically
infinite) streams of data
! Stream Processing is often reified under one of its many
declinations, such as:
! Reactive Systems
! Signal Processing Systems
! Functional Stream Programming
! Data Flow Systems


! The Stream Processing Architecture very

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
naturally models systems reacting to streams of
data and events produced by the external
world, such as the data produced by sensors, a
camera or even the data produced by the
stock exchange.

! Stream Processing Systems usually operate in
real-time over streams and generate in turns
other streams of data providing information on
what is happening or suggesting actions to
perform, such as by stock X, raise alarm Y, or
detected spatial violation, etc.


! Stream Processing Systems are typically

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
modeled as collection of modules
communicating via typed data
channels called usually streams
Filter
! Modules usually play one of the Filter
following roles:
Filter
! Sources: Injecting data into the System
! Filters/Actors: Performing some
computation over sources Sink
! Sinks: Consuming the data produced by Stream
the system source

Deﬁning Streams
! In abstract terms, a stream is an infinite sequence of

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
data samples of a given type T
! Streams can be further classified in continuous and
discrete streams. Sometimes referred as Behaviors/
Signals and Events
! In this presentation we’ll refer to Continuous Streams
as Data Streams and to Discrete Streams as Event
Streams

Data Streams
Data Streams Temp

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
! The value of a Data Stream

is always defined, i.e.
continuous.
! Good examples of a Data

Stream are the value
assumed by a real-world
entity, such as temperature, time
pressure, a price, etc.

Event Streams
Event Streams OverheatAlarm

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
! The value of the stream is

defined at precise point in
time, i.e. it is discrete
! Good examples of Event

Streams are events in the real
world, such a violation of a
regulatory compliance, the
time
temperature higher than a
given value, etc.

[A Stream Perspective]

What is DDS?
! DDS is a high-performance, real-time, highly-

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
available, fully-distributed, messaging technology
that allows you to define data/event streams and
make them dynamically discoverable
! DDS is equipped with a rich set of QoS providing
control on the key temporal and availability
properties of data

DDS Topics [1/2]
“org.opensplice.demo.TTempSensor”

! A Topic defines a stream class/

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
category
! A Topic has associated a user
defined type and QoS
! The Topic name, type and QoS
defines the key functional and
non-functional invariants
! Topics can be discovered or
locally defined struct TempSensor {
long Id; DURABILITY,
float temp; DEADLINE,
float hum;
} PRIORITY,
#pragma keylist TempSensor id …

DDS Topics [2/2]
“org.opensplice.demo.TTempSensor”

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
! DDS Topic types can have
associated keys
! Each unique key-value
identifies a unique sub-
stream of values -- called
Topic Instance
struct TempSensor {
long id; DURABILITY,
float temp; DEADLINE,
float hum;
} PRIORITY,
#pragma keylist TempSensor id …

“Seeing” Streams

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
id =701

struct TempSensor {
id =809 @key long id;
Topic float temp;
float hum;
};
id =977

Instances Instances

QoS Policies

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;

Anatomy of a DDS Application

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Domain val dp = DomainParticipant(0)
Domain
Participant

// Create a Publisher / Subscriber
Publisher p = dp.create_publisher();
Publisher Topic Subscriber
Session Subscriber s = dp.create_subscriber();
// Create a Topic
Topic<TempSensor> t = Gives access to a
dp.create_topic<TempSensor>(“com.myco.TSTopic”) DDS Domain
DataWrter DataReader

Reader/Writers
User Defined for Types
// Create a DataWriter/DataWriter
DataWriter<TempSensor> dw = pub.create_datawriter(t);
DataReader<TempSensor> dr = sub.create_datareader(t);


!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Domain
Participant

val pub = Publisher(dp)
Session val sub = Subscriber(dp)
// Create a Topic
val topic = Topic[TempSensor](dp,
“org.opensplice.demo.TTempSensor”)


Reader/Writers
User Defined for Types Pub/Sub
Abstractions
DataWriter<TempSensor> dw = pub.create_datawriter(t);
DataReader<TempSensor> dr = sub.create_datareader(t);


!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Domain
Participant

val pub = Publisher(dp)
Session val sub = Subscriber(dp)
// Create a Topic
val topic = Topic[TempSensor](dp,
“org.opensplice.demo.TTempSensor”)


Reader/Writers for User Defined for Types
val writer = DataWriter[TempSensor](pub, topic)
val reader = DataReader[TempSensor](sub, topic) Reader/Writer for
application
// Write data defined Topic
val t = new TempSensor ts(101, 25, 40)
writer write ts; Types

Data & Event Streams
! DDS does not provide different types for Data/Event Streams.

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
The difference between the two can be made through the
DataReader API by using properly using read/take operations
! DataReader::read
! Reads the value of the stream w/o removing it from the stream. As a result
multiple read can see the last known value of the stream

! DataReader::take
! Takes the value available on the stream (if any yet) and removes it from the
stream

Content Filtered Topics
! Content Filtered Topics

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
provide a way of defining a Example:
filters over an incoming // Create a Topic (on default domain)
stream associated with a val topic = Topic[TempSensor](“TTempSensor”)
val ftopic =
given topic ContentFilteredTopic[TempSensor](“CFTempSensor”,
topic,
filter,
! Filters are expressed as the params)

“WHERE” clause of an SQL // - filter is a WHERE-like clause, such as:
// “temp > 20 AND hum > 50”
statement // “temp > %0”
// “temp > hum”
// “temp BETWEEN (%0 AND %1)
! Filters can operate on any //
// - params is the list of parameters to pass to the
attribute of the type // filter expression – if any

associated with the topic

Filter Expression Syntax

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
! DDS Filters are condition over
a topic type attributes

! Temporal properties or
causality cannot be
captured via DDS filter
expression

History

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Data older than “n samples ago”
get’s out the window
! DDS provides a way of
controlling data future past
windows through the
History QoS now
The window keeps
the last n data samples

[Putting it All Together]

TempSensor Moving Average
object MovingAverageFilter {
def main(args: Array[String]) {
if (args.length < 2) {

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
println("USAGE:ntMovingAverageFilter <window> <filter-expression>")
}

val topic = Topic[TempSensor]("TTempSensor")
val ftopic = ContentFilteredTopic[TempSensor]("CFTempSensor",topic, args(1))

val rqos = DataReaderQos() <= KeepLastHistory(args(0).toInt)
val reader = DataReader[TempSensor](ftopic, rqos)

reader.reactions += {
case e: DataAvailable[_] => {
var average: Float = 0
val window = e[TempSensor].reader.history
window foreach (average += _.temp)
average = average / window.length
println("+--------------------------------------------------------")
println("Moving Average: " + average)
}
}
}

Product Organization
Commercial Edition
! No Cost Runtime Licenses

!"#$%&'()*+,--.*/%&01234(*5**677*8&'()0*8303%93:
! Your choice of licensing
! LGPL or Commercial
! Subscription or Perpetual
! Complete DDS Implementation
Commercial
Edition
! Comprehensive Developer and
Deployment Support Options with a
range of Service Level Agreements
Commercial Add-Ons
! Individually licensable technologies

Commercial Add-ons
! Rich ecosystem covering tools,
integration, testing, etc.

Key Points So Far

!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
! DDS key abstractions for building stream processing
architectures
! DDS provides some event processing capabilities that
facilitate the development of Stream Processing Filters
! What else can you use for “Stream Processing” in combination
with DDS?
! Let’s have Tom introduce us to the world of CEP and Esper!

!

!

!

#$

%!#

!!
'(

!

*!
+!!
#

,
!
- !-#

!
+!!
#

.

/
#
0#
!

3

3
455 !

# 2
!
6

' #
! 2
7$82

1$9(
!#
#

-

/
!#
#

!
:
!#!-# 2

!!!-#
0#
##2
!#;!
)

#$

4
#$

!
!

! !

#
,
!!

/#
=

#!
'(
)

!
!

#!
'(

0
!
!

/#

!

!
8
,

##,
-
?!@

!

!

A

8

!
B

8
#!
!-
8
#
A

1

!

0CD%

0CD%
%
! !
'!
# ( %
#
:! #
E :! #
! !
%F
'
!- ! ( G 4

!!
E 5 %--53

6
!!#

H IJ
EJK

!#
#!
!#4

#$
2

#! 2
!

- L

!#

#!

!#

#!

! !
5
!

.
!#

-

!
$#
#$

#
!#4

!
!
#
!
+

5

0C
5
C

',
!
#
-!=
(

!#

;
%F
!
!
+!!
!

#!!

#
'

-

!
(
!
-

=
+

0C
!
C

!

-
#
,

M!#
#
5
:
!#!-#
,
!
#

-
!#!-#
#!
!
#
-N
!

%#
#!
%#
#!
[insert into insert_into_def]
select select_list
from stream_def [as name] [, stream_def [as name]] [,...]
[where search_conditions]
[group by grouping_expression_list]
[having grouping_search_conditions]
[output output_specification]
[order by order_by_expression_list]

%!#
select acctId, sum(amount)
from Withdrawal.win:time(1 minute)
group by acctId
having sum(amount) 1000
order by acctId asc

9

!
$
O

P
.!
!
$

!
!
!
!

!
4
C!
-#
-#=
M#
$
.-

-!

#
-

$ !#
2

!
.%

,=

.!
!#
.! 4

#
!

!
$
-!

$ !#

Q
M##,
R!!4!S
!$
!#
! 4

7

!#
.!
.!
$!#

M##,
-
select *
from pattern [w=Withdrawal - Deposit(acctId = w.acctId)]

!

$
!#

w=Withdrawal -
(pin=PinChange(acctId = w.acctId) and
pwd=PasswordChange(acctId = w.acctId)
)

##

,

every (
(w=Withdrawal - Deposit(acctId = w.acctId))
or
(d=Deposit - Withdrawal(acctId = d.acctId))
)

T

!#
.!

.!
$!#

!
!

!#
.-

Getting Started with OpenSplice and Esper

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (6)

Destacado

Destacado (20)

Similar a Getting Started with OpenSplice and Esper

Similar a Getting Started with OpenSplice and Esper (20)

Más de Angelo Corsaro

Más de Angelo Corsaro (20)

Último

Último (20)

Getting Started with OpenSplice and Esper