DDS, JMS, REST APIs
Data-Centric Publish/Subscribe
Pluggable Discovery
Reliability, Serialization, Transport
Persistence Service
Monitoring, Logging, Replay
Connext Messaging adds:
- Request/reply
- Guaranteed messaging
- JMS API
- Persistence
- Additional transports
- Security
- Future: REST API
It is built on top of Connext DDS for data distribution.
<XML>
3. DDS: The Interoperability Standard for
Mission Critical Systems
• Data Distribution Service from OMG
• OMG: world’s largest systems
software standards org
– 470+ members
– UML, DDS, CORBA, more
• No vendor lock-in
– ~12 implementations
– Proven interoperability
– Choice of middleware vendors
– Choice of subsystem vendors
4. Navy Embraces OA
• Next-generation of the
U.S. Navy Aegis Weapon
System, by Lockheed
• Highly distributed system
including radar, weapons,
displays and controls
• Standards-based, high-
performance middleware
avoids vendor lock-in and
future-proofs the
architectural design
5. Government Adopts Standards
• Dominant in military
– DISA: DISR mandated
– Navy: Open Architecture,
FORCEnet
– Air Force, Navy and DISA: NESI
– Army, OSD: UCS
– NATO, UK MOD, South Korea,
many more
• Many other applications
– Air traffic control, industrial
automation, transportation,
medical
• Hundreds of active programs
– Multiple interoperable
implementations
9. Agenda
• DDS standard
– Motivation
– Applicability
• RTI Connext
– Overview
– What’s new
• RTI’s Infrastructure Community (IC) model
– RTI DDS free as Open Community Source
– Low-cost commercial licenses and support
10. Trend: Increasing System Scale
Drivers
• Increased situational
awareness, information sharing
• Responsiveness, timeliness,
speed of command
System of
systems
Implications
• More CPUs, nodes, apps, data
• More developers, teams,
suppliers, integration
11. Cost Constrains Scale
Limits Information Sharing & Connectivity
O(n2)
Time & cost of
integration,
maintenance
and upgrades
System Scale and Age
12. Challenges
• Business
– Cost reduction: lifecycle and procurement
Development, integration, maintenance, upgrades
Legacy system integration and modernization
– Agility: timely fielding of new capabilities
• Technical
– Scalability: information volume
– Reliability and availability
13. Solution
• Interoperability
– Easily share data across (sub)systems
– Minimize need for integration
– Goal: “plug and play”
• Modularity
– Decompose systems into interoperable
components
– Components can evolve and be
competed independently
– Reduces complexity
• Decentralization
– Systems of systems are inherently
distributed, span organizations
– Eliminates performance bottlenecks
14. Achieving Interoperability
• Communicate via explicit, well-defined interfaces
– Independent of:
• Implementation
• Programming language
• Platform (operating system, CPU type)
• Physical location, proximity (edge, data center, cloud, same server)
– Standard network protocol
• Integration requires no knowledge of internals
– No need for reverse engineering, code inspection
• A lingua franca
– Integration effort is O(1) to O(n), not O(n2)
• Components are
evolvable, interchangeable, swappable, shareable, reus
able
• Foundation of SOA, OA
15. DDS Enables Interoperability
Independently developed and loosely coupled
software modules, applications, subsystems and systems
S/W S/W S/W S/W
Data Distribution Service
16. DDS Overview
• Standardizes: Cross-vendor source portability
– Means to define domain- and
system-specific interfaces
– Network protocol
– Programming interface Standard API Schema
• Designed for mission-critical
systems, systems of systems DDS Implementation
Data centricity, QoS
– Decentralized
– Scalability Standard Protocol
– Availability
– Disparate Quality of Service
requirements Cross-vendor interoperability
17. DDS Applicability
Composition, Systems of Systems Decomposition
Mission Vehicle
Mapping
Planning Comms
Data Distribution Service Data Distribution Service
Increase: • Enable reuse, sharing,
• Information sharing competition
• Responsiveness • Simplify maintenance, upgrades
– Decouples components
– Eliminates complexity
18. Deployment
Integration Bus Native Interoperability Interface
Disparate Disparate
Component Component
Natively Natively
DDS or other protocol
Interoperable Interoperable
Adapter Adapter Component Component
API
DDS Library DDS Library DDS Library DDS Library
DDS-RTPS Wire Interoperability Protocol
• Alternative to traditional Enterprise Service Bus (ESB)
• Open: integration logic not specific to an ESB implementation
• Decentralized for high scalability and availability
19. Paradigm:
Data-Centric Publish/Subscribe
Subscribe
Publish Line
Fligh
Dest Arv
t
UA 567 SFO 7:32
Lin Fligh
AA 432 LAX 9:15 Squawk
e t
Lon 1234 UA 567
Squawk Lat Alt
g
7654 AA 432
1234 37.4 -122.0 500.0
7654 40.7 -74.0 250.0
Virtual Global Data Space
• Components:
– Publish the data they produce
– Subscribe to the data they consume
• DDS middleware:
– Routes data between matching publishers and subscribers
– Maintains consistent shared state
20. DDS Data Centricity
Component Component Component Optional
Persistence
DDS DDS DDS
Like a database
• Simple integration
– Data and schema are discoverable
– Producers and consumers are decoupled
– New components don’t impact existing ones
– Standard, application-independent CRUD operations: Create, Read, Update, Delete
– …not specific to data source, type of processing/algorithms, or business processes
• Fosters information sharing
• Single source of truth
– Robust
– Temporal decoupling
– Automatic state synchronization
21. DDS Data Centricity
Component Component Component Optional
Persistence
DDS DDS DDS
Unlike a database
• Event driven for real-time and low-latency processing
• Decentralized
– Highly scalable: no performance bottlenecks or expensive servers
– No single point of failure: non-stop availability
– Peer-to-peer communication: low latency
• Data cached locally for instant access
22. DDS Quality of Service
Reliable, 2 Hz,
Reliable, Western U.S.
100 Hz Fligh
Line Dest Arv
t
Reliable
UA 567 SFO 7:32
Lin Fligh
AA 432 LAX 9:15 Squawk
e t
Lon Best Effort,
Squawk Lat Alt 1234 UA 567
g 1 Hz, SAN area
7654 AA 432
1234 37.4 -122.0 500.0
Best Effort, 0.2 Hz, 7654 40.7 -74.0 250.0
UA flights
• Each component specifies its QoS capabilities and requirements
– Data volatility: Durability, History, Lifespan
– Data delivery: Reliability, Time based filter, Content filter, Deadline
– High availability: Liveliness, Ownership, Ownership Strength
• DDS implements and enforces contracts
23. DDS QoS Benefits
Reliable, 2 Hz,
Reliable, Western U.S.
100 Hz Fligh
Line Dest Arv
t
Reliable
UA 567 SFO 7:32
Lin Fligh
AA 432 LAX 9:15 Squawk
e t
Lon Best Effort,
Squawk Lat Alt 1234 UA 567
g 1 Hz, SAN area
7654 AA 432
1234 37.4 -122.0 500.0
Best Effort, 0.2 Hz, 7654 40.7 -74.0 250.0
UA flights
• Reduces complexity and • Efficiently scales with data volumes
associated lifecycle costs – Only required data is distributed, delivered
– Decoupling: publishers don’t need – Reduces network and processor overhead
to know subscribers’ requirements • Fault tolerance
– Disparate subscribers almost
always have different requirements – Redundancy management
– Moves logic from applications to – Components notified if QoS not satisfied or
DDS middleware connectivity lost
– Can take remedial action
24. Support for Mission-Critical Systems
• Autonomous operation
– Automatic discovery of applications
– Requires no system admin or
centralized infrastructure
• Non-stop: no single point of failure
• QoS provides visibility into real-time
behavior and system health
• Embeddable
• TRL 9 implementations
25. Integration Approaches
Point-to-point, application-centric
Tightly coupled: complex, stovepipe, brittle
Integration logic embedded in apps
High lifecycle costs: development, integration, maintenance & upgrades
Poor information sharing, robustness (state maintained in each app)
Centralized
ESB
Poor scalability and performance
Database Single point of failure: slow failover
Broker Ill-suited for autonomous systems
ESB integration logic is vendor-
specific, can be complex within ESB
DDS
Database’s simplicity, information sharing & robustness
Highly scalable, available, performant
Suitable for critical, autonomous, real-time, embedded
Integration logic is vendor independent
26. Asset Tracking
Legacy design:
• 12,000 tracks
• 11 servers with 88 cores
• Poor reliability and uptime
• 1.5M SLOC
• 2-8 years to develop
• Custom, proprietary
DDS design:
• 250,000 tracks
• 80% of a single core
• Full redundancy
• 50k SLOC
• Proof of concept in under a week
• 100% standards based
27. DDS Summary
• Enables interoperability and
portability by standardizing: Cross-vendor source portability
– Interface definitions
– Network protocol
– Programming interface
• Applicable to system composition and Standard API Schema
decomposition
• Data-centric publish/subscribe
DDS Implementation
– Simplifies integration Data centricity, QoS
– Fosters information sharing
– Improves robustness, particularly as
systems scale Standard Protocol
• Satisfies scalability, reliability and QoS
requirements of mission-critical
system
Cross-vendor interoperability
28. RTI Connext
Reduce the time, cost and risk of system development and integration
with the most proven, mature and comprehensive DDS solution
29. RTI Connext Product Family
Disparate
New and Upgraded Apps with Native Interoperability Apps/Systems
API: Full DDS DDS++ & JMS DDS Subset Adapters
Connext Connext Connext Connext
DDS Messaging Micro Integrator
DDS-RTPS Interoperability
Administration Recording
Monitoring Replay
System Viz Logging
Connext Tools
30. Connext DDS
• World’s leading DDS implementation
• 70+% market share
• 500+ customers
• 315+ university and IR&D projects
• 350,000+ deployed copies
• Most versions are TRL 9
• Includes library source code
• DDS core is free as Open Community Source
31. Example Customers
U.S. Defense International Defense Automotive
Boeing – AWACS Base10 – RoboScout Audi
Boeing – B-1B Cassidian – GCS VW
General Atomics – GCS QinetiQ – test & evaluation
Medical
Lockheed Martin – Aegis QinetiQ – vehicle integration
DocBox
Northrop Grumman – CLIP Rheinmetall – camp protection
Mevion
Raytheon – DDG 1000 Saab – naval CMS
Varian
Raytheon – SSDS Samsung –naval CMS
Raytheon – LPD-17 Ultra Electronics – OA platform Industrial Control
Nikon – semi mfg
NASA Transportation
Schneider - PLCs
Constellation launch control ATLANTIDA – ATM
Siemens Wind Power
Human Robotic Systems City of Tokyo – Highway
Robonaut Wi-Tronix – asset tracking Simulation
CAE – flight simulator
Science IT
Force Technology – tugboat
ESO – Telescope Paremus – cloud platform
Max Planck – nuclear fusion PIMCO – bond trading Telecom
Schilling – UUV Xuenn - sports betting Harmonic – video
IPC – VoIP
32
32. Application Code
<XML>
Data Types
Interface Dynamically
Generated Custom Pre-defined
Compiler defined (API)
Interface Definitions APIs – event-driven, polled & SQL query
• IDL
• XML / XSD DDS: C, C++, C#, Java, Ada*
• WSDL
Data-Centric Publish/Subscribe
Local API & remote
Quality of Svc
Monitoring*
API & file-based
Pluggable Discovery <XML>
Automatic History
Peer-to-peer Discovery Cache
File-based*
Custom Reliability • Serialization • DDS-RTPS Wire Protocol
ucast & mcast
ucast & mcast
Memory
Custom
Shared
UDPv4
UDPv6
TCP
Pluggable Transport Interface
Operating System and Network Stack
Windows, Linux, Unix, embedded, RTOS *Add-on
34. New with 5.0: Type Extensibility
• DDS Topics are strongly typed struct Track {
long id; //@key
– Safety float range;
float bearing;
– Wire efficiency }
• New XTypes interface allows types to
be extended Compatible
– Dynamically or not?
– While retaining type safety
• Subtype relationships match struct AirTrack {
long id; //@key
float range;
publications and subscriptions float bearing;
float elevation;
– Subscriptions to Track match }
publications of AirTrack
35. New Experimental Feature:
XML-Based Application Creation
• Use XML for complete system definition
– Data Types
– Quality of Service settings
– Topics
– All Entities: DomainParticipants, Publishers, Subscribers,
DataWriters and DataReaders.
• Benefits
– Share system configuration
– Reduce code size and errors
– Simplify & Speedup development
– Let developers focus on application the behavior
– Prototype and execute without writing any code!!
36. Replaces Configuration and Setup Code
Create
Participant
Register type
Static Create Topic
Create
Create Publisher
Subscriber
Create Writer Create Reader
Do the
Behavioral important
work
37. New Experimental Feature:
Connext Prototyper
• Command-line tool included with XML
Application Creation SDK
• Try out scenarios directly from their XML
descriptions, without writing any code!
– Directly see applications execute on the network
– See impact of QoS, Topic Designs, Data Type
definition…
– Specify data-values/ranges and publication rates
per topic
38. Connext Messaging:
Built on Connext DDS
• Additional enterprise integration patterns
– Request/reply
– Guaranteed messaging
• Persistence service
• Java Message Service (JMS) API
• Additional transports
– Secure (TLS, DTLS)
– WAN
• Future:
– New security plugins
– REST API
39. Application Code
<XML>
Data Types
Dynamically
Generated Custom Pre-defined
Interface defined (API)
Compiler
Request/reply, Guaranteed Messaging
Interface Definitions APIs – event-driven, polled & SQL query
• IDL
• XML / XSD DDS: C, C++, C#, Java, Ada* JMS
• WSDL
Data-Centric Publish/Subscribe
Local API & remote
Quality of Svc
Monitoring*
API & file-based
Pluggable Discovery <XML>
Automatic History
Peer-to-peer Discovery Cache
File-based*
Custom Reliability • Serialization • DDS-RTPS Wire Protocol
TLS & DTLS
ucast & mcast
ucast & mcast
Memory
Custom
Shared
UDPv4
UDPv6
WAN
(SSL)
TCP
Pluggable Transport Interface
Operating System and Network Stack
Windows, Linux, Unix, embedded, RTOS *Optional
40. New with 5.0: Integration Patterns
• Request/Reply: • Guaranteed Messaging:
Process commands Ensure action
– Remote method invocation – Persistent publications
– Request/async reply – Durable subscribers
– Request/multiple-reply – Application
– Request/multiple repliers acknowledgement
Reply
Replier A
Publisher Subscriber
Message Message
Durable
Requester Replier B Subscriber
Message
Request
Replier C Message
41. New with 5.0: Enhanced Scalability
• Optimized writer-side filtering
– Unlimited remote readers
– Greatly optimized protocol*
• Ultra-scalable request-reply Subscriber
– 1000s of clients Temperature
Update Turbine 1
Publisher Subscriber
Publisher
Temperature Temperature
…
Updates Update Turbine 2
Subscriber
Temperature
Update Turbine N
* Still DDS compliant, backwards compatible
42. Persistence Service
Publisher Subscriber
Domain
Subscriber
Persistence
Service
• Data availability, even if publisher fails
• Reduced load and memory for data writers
43. OpenSSL for TLS and DTLS
• OpenSSL: dominant industry solution for TLS
• Point-to-point: TLS over TCP, DTLS over UDP
• Integrity, authentication and encryption
– Authentication: X.509 based
– Encryption: wide selection
• FIPS 140 Level 1 certified
• Applied at the DDS DomainParticipant level
44. Future Security Support
• Finer grained access control
– Topics
– Read/write
• Multicast encryption
– One encryption per sample
– …not per subscriber
• Built-in key distribution
45. OMG Secure DDS Submission
certificates application component App.
App.
App.
Authentication Secure DDS Key
Plug-in middleware Management
? Plug-in Other
Other
Access Control Other
DDS
DDS
Plug-in DDS Entities Cryptography DDS
System
System
Plug-in System
Auditing Protocol Data
Plug-in Engine cache
Secure Transport
TAG Encrypted Data
Network
46. Future: REST Interface
Preview Available Now
• Web interface to DDS
• Access DDS from any
application, platform or language that
can invoke a Web Service DDS-RTPS
– Web applications, e.g., Google Maps
– JavaScript, Flash, Perl, PHP, Python, CGI
scripts
• Lightweight
– Clients do not need to link or load HTTP
special libraries
• Basis for RTI submission to OMG
Web-Enabled DDS standard
49. Platform Support
• Mainstream 32/64-bit processors
• 16-bit micro-controllers with 32-bit integer
support
• Operating system support
– Windows, Linux
– VxWorks, VxWorks Cert, VxWorks 653, FreeRTOS
– Preview: iOS, Android
• Source code included for easy porting to other
platforms
• Platform plug-in API
50. Compatibility
• Interoperable with general-purpose edition
– RTPS 2.1 compliant
• Directly compatible with
– rtiddsgen
– Wireshark
– Routing Service
• Compatible when using dynamic discovery or via
Routing Service
– Spreadsheet Add-in for Microsoft Excel
– Real-Time Connect
– Recording Service
51. Connext Integrator
Compose Systems of Systems
• Adapts non-DDS applications to DDS (e.g.,
legacy)
• Transforms between incompatible data models
to achieve interoperability
• Bridges DDS systems
– Across networks – LAN and WAN
– Across security domains
55. Recording
• Applications: • Record high-rate data arriving in
– Future analysis and debugging real-time
– Regulatory compliance • >15,000 messages/updates per
– Replay for testing and simulation second per disk
purposes • Non-intrusive
Publisher Subscriber
Domain
Publisher Subscriber
Status Control
Topic Topic
Recorder File DB
56. Playback
• Real-time playback of data
– Captured by RTI Recorder
– From any number of topics
DDS
• Applications:
– Debugging – replay what happened
– Live post-mission analysis, e.g., UAV
– Replay for simulation and training Playback
Data
• Non-intrusive Log
– Just another publisher
– Transparent to subscribers
57. RTI Analyzer
Detailed system topology display
Displays QoS settings
Helps debugging Writer to
Reader connections
Helps to Track and tune QoS
parameters
Helps in diagnosing unusual
behavior
58. Application Monitoring Features
• Detailed statistics on traffic,
errors, and resource usage
• Detailed system topology display
• Configurable alerts and thresholds
• Helps to Track and tune
performance
• Helps in diagnosing unusual
behavior
59. Admin Console
• Centralized console for infrastructure
• Dashboard summary of all running services
• Live status updates
• Live distributed logging
• Real-time statistics
• System Performance statistics (CPU and memory)
• Remote administration of Connext Integrator
• GUI interfaces for sending remote commands
• Retrieving and updating current configurations
• Built-in XML editor for modifying configuration files
• Built on top of Eclipse
61. RTI Connext Add-on Products
• Limited bandwidth transport
• CORBA compatibility kit
• Ada 2005 language support
• Add-on for National Instruments LabView
62. Limited-Bandwidth Transport
• Designed specifically for wireless or satellite comms
• Limited-Bandwidth RTPS Transport Plug-in
– Increases data to packet ratio
• Compression Transport Plug-in
– Further optimizes data to packet ratio
• Quasi-static Discovery Plug-in
– Reduces meta-data traffic
– Minimizes system set-up and reconnection times
• Simulator
– Packet drops, limited bandwidth
64. Infrastructure Community
Business Model
• Facilitates adoption of a DDS software
infrastructure and interoperability interface
• …across development projects and
organizations
• Community members get RTI DDS for free
– Open Community Source license
– Modifiable
– Redistributable
65. What Is an Infrastructure Community?
• Any community sharing software
– Seeking a common or interoperable
software infrastructure
– Across projects, divisions, companies,
programs
• Examples
– Software supply chains
– Enterprises or corporate divisions
– Government or industry standards
communities (FACE, UCS, COE, ICE)
– Large projects
• “Everyone you care about”
66. Infrastructure Community Model
• Free core RTI DDS for entire IC
– Full source & binary for latest RTI DDS
• Windows & Linux pre-built binaries
• Share source & binaries within IC (original and modified)
– Royalty free
– Optional support on a time and materials basis
• Low-cost commercial product for projects
– Subscriptions start at $1,000 per developer (U.S.)
– Adds tools, advanced functionality, platforms, transports
– Warranty & bounded support costs
– Still no royalties or deployment fees
68. Summary
• DDS delivers the interoperability, scalability and
availability required to cost-effectively integrate
mission-critical systems of systems
• RTI provides the most proven, mature and
comprehensive DDS solution
• RTI’s Open Community Source makes it easy to
realize DDS’ benefits by facilitating sharing of
infrastructure across organizations
Notas del editor
This chart illustrates the impact of system scale on the time and cost required for integration. Integration time goes up exponentially as the number of applications and system components increases. With two applications, there are only up to two data flows to handle, from A to B and B to A. With five applications, there are 20 potential connections in a system. With 25 applications, there are 600 potential connections. With 100 applications, which is not unusual in a system of systems, there are nearly 10,000 potential connections. That’s why integration and upgrades can take years.Costs also increase over time as the systems themselves become larger, more complex and less maintainable.
Interoperability reduces the integration burden:Components that are directly interoperable don’t need to be integratedComponents that provide a well-defined interoperability interface can be easily composed without having to do any reverse engineering or code inspection
Interoperability reduces the integration burden:Components that are directly interoperable don’t need to be integratedComponents that provide a well-defined interoperability interface can be easily composed without having to do any reverse engineering or code inspection
Composition – compose systems of systemsDecomposition – decompose systems/subsystems/applications into modules, monolithic open architecture
-Goal is obtain writers and readers needed to do the important work-The process is clearly defined
Scalability enhancements include the following.Enhanced writer-side filtering, now supporting unlimited remote readersGreatly reduced gap and heartbeat messagesOptimized writer-side filtering that is independent of the number of remote readers
This is the OMG Secure DDS architecture. The DDS security design allows plugins for all key functionsAll vendors will implement interoperable plugins (the spec includes wire & plugin API)Security specialists can build advanced pluginsThe proposed security architecture will be configurable via newly added QoS polices, while remaining open and extensible via “plug-in” APIs so that customers can integrate with pre-existing Identity Management Mechanisms, Access Control Policy repositories, or cryptographic libraries which might be program or project specific. TAn interoperable implementation of the plug-ins will also be part of the OMG standard. The plug-ins can be enabled or not. RTI is going to implement standard plug-ins, so that interoperability with other vendors is maintained. When security is enabled, customers can use the interoperable plug-ins as the base level security or their own custom based ones. The standard plug-ins will provide authentication, access control, encryption, key management and data tagging capabilities. High security systems may want to implement custom plugins. We're working with people like Tresys to develop some higher security and more complex plug-ins for systems that require more than the typical off-the-shelf interoperability security.
The baseline configuration provides a minimal feature set.The problem is that the required minimal feature set is application-dependent.The solution is to offer a user-configurable feature set.Note RTPS is not in the baseline configuration, enabling efficient intra-process communication.
Additional features and QoS (beyond the baseline configuration) are selectively enabled either at compile-time or run-time. This is done to minimize the library size.The optional feature sets are the main theme of the next EAR.QoS management includes getting/setting QoS and changing default QoS.Entity management includes getting, deleting and ignoring entities. This includes getting remote entity information. Note the baseline configuration only supports deleting all contained entities at once.
This slide lists the discovery protocols supported by the baseline configuration.The protocols are plug-in components.Dynamic and quasi-static discovery are supported by the current EAR.