Más contenido relacionado Similar a I need the data and I need it 1ms ago! (20) Más de Angelo Corsaro (20) I need the data and I need it 1ms ago!1. OpenSplice DDS
Delivering Performance, Openness, and Freedom
I Need Market Data
Angelo Corsaro, Ph.D.
Chief Technology Officer
OMG DDS SIG Co-Chair
angelo.corsaro@prismtech.com
and I need it 1ms Ago!
3. Financial Firms
Trends
‣ Direct connection to exchanges is increasingly chosen
over data aggregators to reduce trading latency
‣ Market data is growing steadily
‣ Milliseconds speedup over competitors can be worth
millions of dollars
Challenges
‣ Process several hundreds-of-thousands of messages,
quickly approaching millions messages, per second
‣ Minimize latency and jitter, while maintaining the
system stability and reliability
Deployed middleware solutions were not designed to
address these challenges!
© 2009, PrismTech. All Rights Reserved
4. Exchanges
Trends
‣ More and more firms connecting directly
‣ Increasing number of sources providing the same quotes
‣ Competitive advantage obtained by reducing latency on data
feeds
Challenges
‣ Ensuring fairness of data distribution
‣ Minimizing latency
‣ Scaling with the number of destinations
Deployed middleware solutions were not
designed to address these challenges!
© 2009, PrismTech. All Rights Reserved
5. OPRA Predictions
OPRA Market Data Rates Forecasts Trends
2000000
Msgs/sec 1807000
‣ One order of magnitude
growth on market data
from 2005 to 2008
1500000
‣ OPRA message rates
predicted to reach ~2M
1000000 msgs/sec by early 2009
701000
‣ Traffic bursts predicted to
further increase by 20%
500000 359000
Challenges
‣ Can your messaging
190000
Jan 6, 2009
Jan 8, 2008
88000 Jan 9, 2007
0
May 16, 2006 infrastructure keep-up
Jan 13, 2005
with this trend?
© 2009, PrismTech. All Rights Reserved
6. Trading with Pareto
‣ Usually 20% of the trades
provides 80% of the gains
Msgs/sec
‣ Under overload condition
how can you ensure you
keep trading at least 20%
of your target?
Vilfredo Pareto, 1848-1923
‣ How can you ensure that
you keep trading the the
Average
right 20% of data?
System Message Rate
Incoming Message Rate
Time
© 2009, PrismTech. All Rights Reserved
8. Addressing Data Distribution Challenges
DDS is standard designed to address the data-distribution challenges across
The OMG DDS Standard a wide class of Defense and Aerospace Applications
‣ Introduced in 2004 to address the Data
Distribution challenges faced by a wide
class of Defense and Aerospace
Applications
‣ Key requirement for the standard were its
ability to deliver very high performance while
seamlessly scaling from embedded to ultra-
large-scale deployments
‣ Today recommended by key administration
worldwide and widely adopted across
several different application domains, such
as, Automated Trading, Simulations, SCADA,
Telemetry, etc.
© 2009, PrismTech. All Rights Reserved
9. The OMG Data Distribution Service (DDS)
DDS v1.2 API Standard
‣ Language Independent, OS and HW architecture Application
independent Object/Relational Mapping
‣ DCPS. Standard API for Data-Centric, Topic- Data Local Reconstruction Layer (DLRL)
Based, Real-Time Publish/Subscribe
Content
‣
Ownership Durability
DLRL. Standard API for creating Object Views out Subscription
of collection of Topics Minimum Profile
Data Centric Publish/Subscribe (DCPS)
DDSI/RTPS v2.1 Wire Protocol Standard
‣ Standard wire protocol allowing interoperability
Real-Time Publish/Subscribe Protocol
DDS Interoperability Wire Protocol
between different implementations of the DDS
standard UDP/IP
‣ Interoperability demonstrated among key DDS
vendors in March 2009
© 2009, PrismTech. All Rights Reserved
10. Scope of Standardization
AMQP OMG DDS
Application
Application
Object/Relational Mapping
Data Local Reconstruction Layer (DLRL)
DDS v1.2
Content
Ownership Durability
Subscription
API (?) Minimum Profile
Data Centric Publish/Subscribe (DCPS)
DDSI v2.1
AMQP v0.10
Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol
Interoperability Wire Protocol DDS Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP UDP/IP
© 2009, PrismTech. All Rights Reserved
11. Scope of Standardization
AMQP OMG DDS
Application
Application
Object/Relational Mapping
Data Local Reconstruction Layer (DLRL)
DDS v1.2
Content
Ownership Durability
Subscription
API (?) Minimum Profile
Data Centric Publish/Subscribe (DCPS)
DDSI v2.1
AMQP v0.10
Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol
Wire Protocol Standard
Interoperability Wire Protocol DDS Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP UDP/IP
© 2009, PrismTech. All Rights Reserved
12. Scope of Standardization
AMQP OMG DDS
Application
Application
Object/Relational Mapping
Data Local Reconstruction Layer (DLRL)
DDS v1.2
Content
Ownership Durability
Subscription
API (?) Minimum Profile
Pub/Sub Standard API
Data Centric Publish/Subscribe (DCPS)
DDSI v2.1
AMQP v0.10
Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol
Wire Protocol Standard
Interoperability Wire Protocol DDS Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP UDP/IP
© 2009, PrismTech. All Rights Reserved
13. Scope of Standardization
AMQP OMG DDS
Application
Application
Object/Relational Mapping Object Relational Mapping
Data Local Reconstruction Layer (DLRL) Standard API
DDS v1.2
Content
Ownership Durability
Subscription
API (?) Minimum Profile
Pub/Sub Standard API
Data Centric Publish/Subscribe (DCPS)
DDSI v2.1
AMQP v0.10
Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol
Wire Protocol Standard
Interoperability Wire Protocol DDS Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP UDP/IP
© 2009, PrismTech. All Rights Reserved
14. Architectural Style
AMQP OMG DDS
Subscriber Subscriber
Publisher Publisher
Broker Broker
Publisher
Broker Subscriber
Publisher
Broker Subscriber
Broker
Broker
Publisher Publisher
Subscriber Subscriber
© 2009, PrismTech. All Rights Reserved
15. Key Differences in Summary
DDS AMQP
‣ Peer-to-Peer Communication ‣ Brokered Communication
Architecture Architecture
‣ Wire Protocol + API Standard ‣ Wire Protocol Standard
‣ Interoperability across DDS implementations ‣ Interoperability across AMQP
implementations
‣ Application Portability across DDS
Implementation
‣ Content-Based Filtering/Quering ‣ Header Based Routing
‣ Designed bottom-up for maximum ‣ Not geared for High-Performance
performance and determinism
© 2009, PrismTech. All Rights Reserved
17. OpenSplice DDS
‣ Product reorganized into Editions
‣ Community Edition
Enterprise Ed.
‣ Compact Edition
‣ Professional Edition
‣ Enterprise Edition Professional Ed.
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Product Editions provide a a growing set of
Enterprise Ed. Compact Ed.
functionality to address the needs of increasingly
more sophisticated users
‣ The Community Edition is Open Source Community Ed.
‣ Compact, Enterprise and Professional Edition are
available only through Commercial Subscriptions
© 2009, PrismTech. All Rights Reserved
18. Community Edition
Features Licensing
‣ OMG DDS v1.2 DCPS ‣ LGPLv3
‣ Minimum Profile
‣ Content Subscription Profile Application
‣ Durability Profile
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Ownership Profile Content
DDS v1.2
Ownership Durability
Subscription
‣ Networking Minimum Profile
‣ DDSI v2.1 Implementation Real-Time Pub/Sub (DCPS)
‣ Real-Time Networking Implementation
DDSI v2.1
Interoperable Wire Protocol
Real-Time Networking
(DDSI)
Networking Technology
UDP/IP
Open Source LGPL Licensed Edition -- The Best Way to Get Started with DDS
© 2009, PrismTech. All Rights Reserved
19. Performance on Commodity HW
Throughput -./0/1/23' 456/'
Latency
#+(+%++' &&++%++'
$+(+%++' "*(%)+' ")(%&!' ""&%,&'&+++%++' Inter-Node Latency
!"#$%&#'
‣ 60 usec
"!+%++'
)"#%*)' "++%++'
)$,%$*'
!+(+%++'
)++%++'
Inter-Core Read-Latency
&'$($)$*+%
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
*!#%,&'
‣ 2 usec
!"#$%
,+(+%++' *++%++'
()(!%(*'
#++%++'
(+(+%++' &")&%+)' Inter-Core Latency
‣ <10 usec
$&"%,,'
$++%++'
&!(,%!)'
&+(+%++'
),,%$#' !++%++'
,*+%&)'
!,*%))'
,($%!+' (("%!"'
&&)%*$' $"%"$' ,+%($' ,++%++' HW:
(+%++'
)' &#' ,(' #!' &()' ($#' $&(' &+(!' (+!)' !+"#'
‣ Dell blade-server
!*$$,(*%-./*%
‣ Dual-core, Dual-CPU, AMD Opteron 2.4 Ghz
Test Scenario OS
‣ Linux 2.6.21-1.3194.fc7
‣ Single Threaded Application (multi-threaded networking service) Network
‣ 8192 bit message batches ‣ Gigabit Ethernet cards
‣ Dell PowerConnect 5324 switch
© 2009, PrismTech. All Rights Reserved
20. OpenSplice DDS Adoption
Defense
‣ Combat Management Systems
‣ Tactical Systems
‣ Drones (land/air)
‣ Communication Systems
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Radar Processing
‣ Submarine Systems
‣ Future Combat Systems
SCADA/Utilities
‣ Industrial Automation
‣ Power Grids (Smart Grids)
‣ Complex Telemetry (e.g. NASA Rocket Launch System)
© 2009, PrismTech. All Rights Reserved
21. OpenSplice DDS Adoption
Transportation
‣ Air Traffic Management/Control
‣ Metropolitan Traffic Management
‣ Underground Metropolitan Transportation
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Financial Services
‣ Automated Trading
‣ Risk Management
‣ Option Trading
© 2009, PrismTech. All Rights Reserved
22. Some OpenSplice DDS Users
!"#$%&'()('*+'##,-+.+/($$,012"0
Experti
réalisée par Bernard M
© 2009, PrismTech. All Rights Reserved
24. As Simple as it Gets
‣ DDS is based around the concept
of a fully distributed Global Data
Space (GDS)
Brokers
Global Data Space
DDS
© 2009, PrismTech. All Rights Reserved
25. As Simple as it Gets
‣ DDS is based around the concept
of a fully distributed Global Data
Space (GDS)
‣ Publishers and Subscribers can Brokers
join and leave the GDS at any time
Global Data Space
DDS
© 2009, PrismTech. All Rights Reserved
26. As Simple as it Gets
‣ DDS is based around the concept
of a fully distributed Global Data
Publisher Subscriber
Space (GDS)
‣ Publishers and Subscribers can Brokers
join and leave the GDS at any time
Subscriber
Publisher
Global Data Space
Publisher Subscriber
DDS
© 2009, PrismTech. All Rights Reserved
27. As Simple as it Gets
‣ DDS is based around the concept
of a fully distributed Global Data
Publisher Subscriber
Space (GDS)
‣ Publishers and Subscribers can Brokers
join and leave the GDS at any time
Subscriber
‣ Publishers and Subscribers Publisher
Global Data Space
express their intent to produce/
consume specific type of data,
e.g., Topics
Publisher Subscriber
DDS
© 2009, PrismTech. All Rights Reserved
28. As Simple as it Gets
‣ DDS is based around the concept
of a fully distributed Global Data
Publisher Subscriber
Space (GDS)
‣ Publishers and Subscribers can Brokers
join and leave the GDS at any time
Subscriber
‣ Publishers and Subscribers Publisher
Global Data Space
express their intent to produce/
consume specific type of data,
e.g., Topics
Publisher Subscriber
DDS
© 2009, PrismTech. All Rights Reserved
29. As Simple as it Gets
‣ DDS is based around the concept
of a fully distributed Global Data
Publisher Subscriber
Space (GDS)
‣ Publishers and Subscribers can Brokers
join and leave the GDS at any time
Subscriber
‣ Publishers and Subscribers Publisher Global Data Space
express their intent to produce/
consume specific type of data,
e.g., Topics
Subscriber
‣ Data flows from Publisher to Publisher
Subscribers DDS
© 2009, PrismTech. All Rights Reserved
30. As Simple as it Gets
‣ DDS is based around the concept
of a fully distributed Global Data
Publisher Subscriber
Space (GDS)
‣ Publishers and Subscribers can Brokers
join and leave the GDS at any time
Subscriber
‣ Publishers and Subscribers Publisher Global Data Space
express their intent to produce/
consume specific type of data,
e.g., Topics
Subscriber
‣ Data flows from Publisher to Publisher
Subscribers DDS
© 2009, PrismTech. All Rights Reserved
32. DDS Topics
{Circle, Square, Triangle}
Topic
‣ Unit of information exchanged between
Publisher and Subscribers.
‣ An association between a unique name, a
type and a QoS setting {ShapeType} {...}
Topic Type.
struct ShapeType {
‣ Type describing the data associated with one long x;
or more Topics long y;
‣ A Topic type can have a key represented by an long shapesize;
string color;
arbitrary number of attributes };
‣ Expressed in IDL (or XML) #pragma keylist ShapeType color
© 2009, PrismTech. All Rights Reserved
33. DDS Topics
Circle
struct ShapeType {
long x;
long y;
long shapesize; Topic QoS
string color;
};
#pragma keylist ShapeType color
© 2009, PrismTech. All Rights Reserved
34. DDS Topics
Triangle
struct ShapeType {
long x;
long y;
long shapesize; Topic QoS
string color;
};
#pragma keylist ShapeType color
© 2009, PrismTech. All Rights Reserved
35. DDS Topic Instances and Samples
Topic Instances
‣ Each key value identifies a unique Topic Instance,
‣ Topic’s instance lifetime can be explicitly
managed in DDS
Topic Samples
‣ The values assumed by a Topic Instance
over time are referred as Instance Sample
struct ShapeType {
long x;
long y;
long shapesize;
string color;
};
#pragma keylist ShapeType color
© 2009, PrismTech. All Rights Reserved
37. Content Filtering
X0 X1
X0 <= X <= X1
‣ DDS allows to specify content-
filtered Topics for which a subset
of SQL92 is used to express the
filter condition
Y0
‣ Content filters can be applied on
the entire content of the Topic Type Y0 <= Y <= Y1
‣ Content filters are applied by DDS
Y1
each time a new sample is
produced/delivered
© 2009, PrismTech. All Rights Reserved
38. Local Queries
‣ Subscribed Topics can be seen locally as “Tables”
‣ A subset of SQL92 can be used for performing queries on X0
multiple topics as well as natural joins
‣ Queries are Circle Topic
performed under Y0
color x y shapesize Y0
user control and red 57 62 50
provide a result that blue 90 85 50
Y0 <= Y <= Y1
depends on the yellow 30 25 50
current snapshot of SELECT * FROM ShapeType s
WHERE s.x > 25 AND s.y < 55 Y1
the system, e.g.,
samples currently color x y shapesize
available yellow 30 25 50
© 2009, PrismTech. All Rights Reserved
40. Anatomy of a DDS Application
Topic
TopicTopic Circle)
(e.g.
Samples
Samples
Instances Samples
Instances
Instances
1 21 62
1 21 62
1 22 62
1 22 62
1 23 63
1 23 63
DataReader
DataReader
blue 21 62 80 blue 23 66 80 blue 27 70 80
2 20 61 2 19 60
DataReader
DataReader 2 20 61 2 19 60
orange 21 62 80 orange 21 62 80 DataWriter
DataWriter
DataReader
DataReader struct ShapeType {
3 25 70 3 25 71
25 3 25 74 3 26 77
3 26 77
DataWriter
DataWriter
long x; 3 25 70 3 25 71
25 3 25 74
long y; green 97 75 50 green 94 71 50
long shapesize;
string color;
};
#pragma keylist ShapeType color
© 2009, PrismTech. All Rights Reserved
41. Anatomy of a DDS Application
Topic
TopicTopic Circle)
(e.g.
Samples
Samples
Instances Samples
Instances
Instances
1 21 62
1 21 62
1 22 62
1 22 62
1 23 63
1 23 63
DataReader
DataReader
blue 21 62 80 blue 23 66 80 blue 27 70 80
2 20 61 2 19 60
DataReader
DataReader 2 20 61 2 19 60
orange 21 62 80 orange 21 62 80 DataWriter
DataWriter
DataReader
DataReader struct ShapeType {
3 25 70 3 25 71
25 3 25 74 3 26 77
3 26 77
DataWriter
DataWriter
long x; 3 25 70 3 25 71
25 3 25 74
long y; green 97 75 50 green 94 71 50
long shapesize;
string color;
Arrows };
#pragma keylist ShapeType color
show
structural
relationship
s, not data-
flows Partition
© 2009, PrismTech. All Rights Reserved
42. Anatomy of a DDS Application
Topic
TopicTopic Circle)
(e.g.
Samples
Samples
Instances Samples
Instances
Instances
1 21 62
1 21 62
1 22 62
1 22 62
1 23 63
1 23 63
DataReader
DataReader
blue 21 62 80 blue 23 66 80 blue 27 70 80
2 20 61 2 19 60
DataReader
DataReader 2 20 61 2 19 60
orange 21 62 80 orange 21 62 80 DataWriter
DataWriter
DataReader
DataReader struct ShapeType {
3 25 70 3 25 71
25 3 25 74 3 26 77
3 26 77
DataWriter
DataWriter
long x; 3 25 70 3 25 71
25 3 25 74
long y; green 97 75 50 green 94 71 50
long shapesize;
string color;
Arrows };
#pragma keylist ShapeType color
show
structural
relationship
s, not data- Subscriber Publisher
flows Partition
Domain Participant
Domain
© 2009, PrismTech. All Rights Reserved
44. The Market Data Topic
Type Declaration Topic Instantiation
struct StockQuote {
char<4> symbol; // Create Topic with Default QoS
char<4> exchange; dds::Topic<StockQuote> topic(“stock_quote”);
char<64> name;
float quote; // Create Content Filtered Topic
dds::ContentFilteredTopic cfTopic =
}; topic.create_content_filtered(expr,
#pragma keylist StockQuote symbol params);
symbol name exchange quote
AAPL Apple Inc. NASD 165.37
GOOG Google Inc. NASD 663.97
MSFT Microsoft Corp. NASD 33.73
© 2009, PrismTech. All Rights Reserved
45. Filtering Market Data
Apple
164.0 Microsoft
40
161.5
30
MSFT > $35
159.0
20
AAPL < $157 |156.5
AAPL > $162 10
154.0
Untitled 1 Untitled 20 Untitled 39 Untitled 58 Untitled 77 Untitled 96
0
AAPL MSFT
‣ DDS continuous queries can be used for filtering the stream of data produced for a
specific stock symbol.
© 2009, PrismTech. All Rights Reserved
46. Writing Market Data
Creating a Writer
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Writer with Default Qos
dds::Writer<StockQuote> writer(topic);
© 2009, PrismTech. All Rights Reserved
47. Writing Market Data
Creating a Writer
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Writer with Default Qos
dds::Writer<StockQuote> writer(topic);
Writing Data
StockQuote quote = {
“AAPL”, “NASD”,
163.37, “Apple Inc.”
};
writer.write(quote);
© 2009, PrismTech. All Rights Reserved
48. Writing Market Data
Creating a Writer
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Writer with Default Qos
dds::Writer<StockQuote> writer(topic);
Writing Data Fastest Way of Writing Data
StockQuote quote = {
StockQuote quote = {
“AAPL”, “NASD”,
“AAPL”, “NASD”,
163.37, “Apple Inc.”
163.37, “Apple Inc.” };
};
dds::InstanceWriter iwriter =
writer.write(quote); writer.register_instance(quote);
iwriter.write(quote);
© 2009, PrismTech. All Rights Reserved
50. Reading Samples
‣ Read iterates over the
available sample instances
1 1 1 2 1 3 1 4
‣ Samples are not removed
2 1 2 2 2 3
from the local cache as
DataReader
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
3 4 3 5
result of a read
3 1 3 2 3 3
‣ Read samples can be read
Topic again, by accessing the
Samples Read Samples not Read cache with the proper
options (more later)
DataReader Cache
© 2009, PrismTech. All Rights Reserved
51. Reading Samples
‣ Read iterates over the
available sample instances
1 1 1 2 1 3 1 4
‣ Samples are not removed
2 1 2 2 2 3
from the local cache as
DataReader
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
3 1 3 2 3 3 3 4 3 5
result of a read
‣ Read samples can be read
Topic
again, by accessing the
Samples Read Samples not Read cache with the proper
options (more later)
DataReader Cache
© 2009, PrismTech. All Rights Reserved
52. Reading Samples
‣ Read iterates over the
available sample instances
1 1 1 2 1 3 1 4
‣ Samples are not removed
2 1 2 2 2 3
from the local cache as
DataReader
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
3 1 3 2 3 3 3 4 3 5
result of a read
‣ Read samples can be read
Topic
again, by accessing the
Samples Read Samples not Read cache with the proper
options (more later)
DataReader Cache
© 2009, PrismTech. All Rights Reserved
53. Taking Samples
‣ Take iterates over the
available sample instances
1 1 1 2 1 3 1 4
‣ Taken Samples are
2 1 2 2 2 3
removed from the local
DataReader
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
3 4 3 5
cache as result of a take
3 1 3 2 3 3
‣
Topic
Samples not Taken
DataReader Cache
© 2009, PrismTech. All Rights Reserved
54. Taking Samples
‣ Take iterates over the
available sample instances
1 2 1 3 1 4
‣ Taken Samples are
2 2 2 3
removed from the local
DataReader
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
3 2 3 3 3 4 3 5
cache as result of a take
Topic
Samples not Taken
DataReader Cache
© 2009, PrismTech. All Rights Reserved
55. Taking Samples
‣ Take iterates over the
available sample instances
1 3 1 4
‣ Taken Samples are
2 3
removed from the local
DataReader
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
3 3 3 4 3 5
cache as result of a take
Topic
Samples not Taken
DataReader Cache
© 2009, PrismTech. All Rights Reserved
56. Reading Data
Creating a Reader
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qos
dds::Reader<StockQuote> reader(topic);
© 2009, PrismTech. All Rights Reserved
57. Reading Data
Creating a Reader
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qos
dds::Reader<StockQuote> reader(topic);
Reading Data
StockQuoteSeq data;
dds::SampleInfoSeq info;
reader.read(data, info);
© 2009, PrismTech. All Rights Reserved
58. Reading Data
Creating a Reader
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qos
dds::Reader<StockQuote> reader(topic);
Reading a Specific Instance
Reading Data
StockQuote quote;
quote.symbol = “APPL”;
StockQuoteSeq data; StockQuoteSeq data;
dds::SampleInfoSeq info; dds::SampleInfoSeq info;
reader.read(data, info); dds::InstanceReader ireader =
reader.instance(quote);
ireader.read(data, info);
© 2009, PrismTech. All Rights Reserved
59. Taking Data
Creating a Reader
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qos
dds::Reader<StockQuote> reader(topic);
© 2009, PrismTech. All Rights Reserved
60. Taking Data
Creating a Reader
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qos
dds::Reader<StockQuote> reader(topic);
Taking Data
StockQuoteSeq data;
dds::SampleInfoSeq info;
reader.take(data, info);
© 2009, PrismTech. All Rights Reserved
61. Taking Data
Creating a Reader
// Create Topic with Default QoS
dds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qos
dds::Reader<StockQuote> reader(topic);
Taking a Specific Instance
Taking Data
StockQuote quote;
quote.symbol = “APPL”;
StockQuoteSeq data; StockQuoteSeq data;
dds::SampleInfoSeq info; dds::SampleInfoSeq info;
reader.take(data, info); dds::InstanceReader ireader =
reader.instance(quote);
ireader.take(data, info);
© 2009, PrismTech. All Rights Reserved
62. Sample, Instance and View States
History Depth = 2
‣ Along with data samples, DataReaders are provided
1 1
with state information allowing to detect relevant
2 2
transitions in the life-cycle of data as well as data
3 writers
DataReader
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
SampleInfo ‣ Sample State (READ | NOT_READ): Determines
wether a sample has already been read by this
1 1 1 2
DataReader or not.
2 2 2 3
‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED).
3 1
Determines wether (1) writer exist for the specific
Samples instance, or (2) no matched writers are currently
Topic available, or (3) the instance has been disposed
DataReader Cache
‣ View State (NEW, NOT_NEW). Determines wether
this is the first sample of a new (or re-born) instance
© 2009, PrismTech. All Rights Reserved
63. Would you let me know...
DDS provides three main mechanism for exchanging information with the
application
‣ Polling. The application polls from time to time for new data or status changes.
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
The interval might depend on the kind of applications as well as data
‣ WaitSets. The application registers a WaitSet with DDS and waits (i.e. is
suspended) until one of the specified events has happened.
‣ Listeners. The application registers a listener with a specific DDS entity to be
notified when relevant events occur, such as state changes or
© 2009, PrismTech. All Rights Reserved
64. Reading with Waitsets
// Functor handling new data arrival 1
struct MarketDataHandler {
void operator()(MarketDataReader& reader);
};
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
© 2009, PrismTech. All Rights Reserved
65. Reading with Waitsets
// Functor handling new data arrival 1
struct MarketDataHandler {
void operator()(MarketDataReader& reader);
};
// Create & Register Condition 2
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
MarketDataHandler functor;
dds::ReadCondition cond =
reader.create_read_condition(functor);
dds::WaitSet ws;
ws.attach(cond);
© 2009, PrismTech. All Rights Reserved
66. Reading with Waitsets
// Functor handling new data arrival 1
struct MarketDataHandler {
void operator()(MarketDataReader& reader);
};
// Create & Register Condition 2
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
MarketDataHandler functor;
dds::ReadCondition cond =
reader.create_read_condition(functor);
dds::WaitSet ws;
ws.attach(cond);
// Wait & Handle 3’
dds::ConditionVector cvec = ws.wait();
for (int i = 0; i < cvec.size(); ++i)
cvec[i].execute();
© 2009, PrismTech. All Rights Reserved
67. Reading with Waitsets
// Functor handling new data arrival 1
struct MarketDataHandler {
void operator()(MarketDataReader& reader);
};
// Create & Register Condition 2
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
MarketDataHandler functor;
dds::ReadCondition cond =
reader.create_read_condition(functor);
dds::WaitSet ws;
ws.attach(cond);
// Wait & Handle 3’ // Dispatch 3”
dds::ConditionVector cvec = ws.wait(); ws.dispatch();
for (int i = 0; i < cvec.size(); ++i)
cvec[i].execute();
© 2009, PrismTech. All Rights Reserved
68. Reading with Listeners
// Functor handling new data arrival 1
class MarketDataListener {
public:
void handle_data(MarketDataReader& reader);
};
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
© 2009, PrismTech. All Rights Reserved
69. Reading with Listeners
// Functor handling new data arrival 1
class MarketDataListener {
public:
void handle_data(MarketDataReader& reader);
};
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
2
// Functor handling new data arrival
MarketDataListener mdl;
dds::connection cnt =
reader.on_data_available_signal_connect(
std::bind(&MarketDataListener::handle_data,
&mdl, _1));
© 2009, PrismTech. All Rights Reserved
71. QoS Model
‣ QoS-Policies are used to control relevant
properties of OpenSplice DDS entities, Type Matching
QoS matching
such as: QoS QoS QoS QoS QoS QoS QoS
‣ Temporal Properties
‣ Priority
Topic
Name
Publisher Subscriber
‣ Durability ... DataWriter writes Type reads DataReader
...
...
‣ Availability DomainParticipant writes Type reads DataReader DomainParticipant
DataWriter
‣ ... Name
‣ Some QoS-Policies are matched based on
Topic
QoS QoS QoS
a Request vs. Offered Model thus QoS-
enforcement
‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible
‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability
© 2009, PrismTech. All Rights Reserved
72. QoS Policies
QoS Policy Applicability RxO Modifiable Type Matching
QoS matching
DURABILITY T, DR, DW Y N Data Availability QoS QoS QoS QoS QoS QoS QoS
DURABILITY SERVICE T, DW N N
Topic
Name
Publisher Subscriber
LIFESPAN T, DW - Y ... DataWriter writes Type reads DataReader
...
T, DR, DW N N ...
HISTORY
DomainParticipant DataWriter writes Type reads DataReader DomainParticipant
PRESENTATION P, S Y N Data Delivery Name
Topic
RELIABILITY T, DR, DW Y N
PARTITION P, S N Y QoS QoS QoS
DESTINATION ORDER T, DR, DW Y N
OWNERSHIP T, DR, DW Y N ‣ Rich set of QoS allow to configure
OWNERSHIP DW - Y
STRENGTH several different aspects of data
DEADLINE T, DR, DW Y Y Data Timeliness availability, delivery and timeliness
LATENCY BUDGET T, DR, DW Y Y
TRANSPORT PRIORITY T, DW - Y
‣ QoS can be used to control and
optimize network as well as
TIME BASED FILTER DR - Y Resources
computing resource
RESOURCE LIMITS T, DR, DW N N
USER_DATA DP, DR, DW N Y Configuration
TOPIC_DATA T N Y
GROUP_DATA P, S N Y
© 2009, PrismTech. All Rights Reserved
73. Mapping QoS Which properties does QoS controls?
TimeBasedFilter Deadline
Data
Throughput
Latency
LatencyBudget TransportPriority
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Control over Latency/Throughput tradeoff
Control over data latency
Control over data priority
© 2009, PrismTech. All Rights Reserved
74. Mapping QoS Which properties does QoS controls?
TimeBasedFilter Deadline History
Data Data
Throughput Lifespan Durability
Latency Availability
Ownership
LatencyBudget TransportPriority Ownership
Strength
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Control over Latency/Throughput tradeoff
Control over data queueing
Control over data latency
Control over data persistency
Control over data priority
Control over data sources hot-swap
© 2009, PrismTech. All Rights Reserved
75. Mapping QoS Which properties does QoS controls?
TimeBasedFilter Deadline History
Data Data
Throughput Lifespan Durability
Latency Availability
Ownership
LatencyBudget TransportPriority Ownership
Strength
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Control over Latency/Throughput tradeoff
Control over data queueing
Control over data latency
Control over data persistency
Control over data priority
Control over data sources hot-swap
Reliability
OpenSplice DDS provides programmatic
Destination
QoS-driven support for configuring the most
Presentation Data Delivery
Order important properties of data distribution!
Control over data distribution reliability
Control over data ordering
Control over presentation
© 2009, PrismTech. All Rights Reserved
77. History QoS
History Depth = 5
1 1 1 2 1 3 1 4 1 5
DataReader 2 1 2 2 2 3 2 4 2 5
3 1 3 2 3 3 3 4 3 5
1
// Create Reader QoS Topic
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
dds::DataReaderQos drqos;
drqos.set_keep_last(N);
// Create Reader
‣ The History QoS Controls the number of
dds::DataReader<MarketData> reader(topic, drqos); samples-per-instance that will be stored by
the middleware on behalf of a Reader
2 ‣ Keep Last K. The History QoS can be set so
// Create Reader QoS to always have the latest K samples
dds::DataReaderQos drqos;
drqos.set_keep_all(); ‣ Keep All. The History QoS can be set so
// Create Reader keep all samples produced by the writer and
dds::DataReader<MarketData> reader(topic, drqos);
not yet taken, until resource limits are not
reached
© 2009, PrismTech. All Rights Reserved
78. Transport Priority QoS
Make sure that important data gets the right priority Shared Single Copy per Node Shared
Pack Across Topics/Applications
Memory Optimal Unmarshaling Memory
1
// Create Writer QoS
dds::DataWriterQos dwqos;
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
dwqos.set_priority(MID_PRIO); OpenSplice DDS OpenSplice DDS
// Create Writer Binding
Pre-emptive Network Scheduler
Binding
dds::DataWriter<MarketData> writer(topic, dwqos); Networking Priority Scheduler
Networking
Data Urgency Traffic Pacing
// Write Data
writer.write(data); Network Channels
// Change Priority Priority Bands
dwqos.set_priority(HIGH_PRIO); Traffic Shaping
writer.writer(data)
© 2009, PrismTech. All Rights Reserved
79. Ownership QoS
Availability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer
‣ Writer strength is used to coordinate replicated writers
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
StockQuote
symbol: "MSFT"
StockQuote
StockQuote
name: "Microsoft Corp."
StockQuote
symbol: "GOOG"
symbol: "GOOG"
exchange: "NASD""GOOG"
symbol:
name: "Google Inc."
quote: 33.73
name: "Google Inc."
name: "Google Inc."
exchange: "NASD"
exchange: "NASD"
exchange: "NASD"
StockQuote
StockQuote
StockQuote
quote: 663.97
W1
quote: 663.97"AAPL"
symbol: "AAPL"
quote: 663.97
symbol: "AAPL"
symbol:
name: "Apple Inc."
name: "Apple Inc."
name: "Apple Inc."
exchange: "NASD"
exchange: "NASD"
exchange: "NASD"
R1
quote: 165.37
quote: 165.37
quote: 165.37
STRENGTH=3
StockQuote
symbol: "AAPL"
StockQuote Inc."
name: "Apple
symbol: "GOOG""NASD"
exchange:
name: quote: 165.37
"Google Inc."
StockQuote "NASD"
exchange:
symbol: "MSFT"
quote: 663.97
name: "Microsoft Corp."
exchange: "NASD"
quote: 33.73
W1’ R2
STRENGTH=2 StockQuote
symbol: "AAPL"
StockQuote Inc."
name: "Apple
symbol: "GOOG""NASD"
exchange:
name: quote: 165.37
"Google Inc."
StockQuote "NASD"
exchange:
symbol: "MSFT"
quote: 663.97
name: "Microsoft Corp."
exchange: "NASD"
quote: 33.73
W2’’ R3
STRENGTH=1
© 2009, PrismTech. All Rights Reserved
80. Ownership QoS
Availability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer
‣ Writer strength is used to coordinate replicated writers
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
StockQuote
symbol: "MSFT"
StockQuote
StockQuote
name: "Microsoft Corp."
StockQuote
symbol: "GOOG"
symbol: "GOOG"
exchange: "NASD""GOOG"
symbol:
name: "Google Inc."
quote: 33.73
name: "Google Inc."
name: "Google Inc."
exchange: "NASD"
exchange: "NASD"
exchange: "NASD"
quote: 663.97
W1
quote: 663.97
quote: 663.97
StockQuote
symbol: "AAPL"
R1
STRENGTH=3 name: "Apple Inc."
exchange: "NASD"
quote: 165.37
StockQuote
symbol: "GOOG"
name: "Google Inc."
StockQuote "NASD"
exchange:
symbol: "MSFT"
quote: 663.97
name: "Microsoft Corp."
exchange: "NASD"
quote: 33.73
W1’ R2
StockQuote
symbol: "AAPL"
name: "Apple Inc."
exchange: "NASD"
STRENGTH=2
quote: 165.37
StockQuote
symbol: "GOOG"
name: "Google Inc."
StockQuote "NASD"
exchange:
symbol: "MSFT"
quote: 663.97
name: "Microsoft Corp."
exchange: "NASD"
quote: 33.73
W2’’
StockQuote
symbol: "AAPL"
name: "Apple Inc."
exchange: "NASD"
quote: 165.37
R3
STRENGTH=1
© 2009, PrismTech. All Rights Reserved