The Industrial Internet is an emerging communication infrastructure that connects people, data, and machines to enable access and control of mechanical devices in unprecedented ways. It connects machines embedded with sensors and sophisticated software to other machines (and end users) to extract data, make sense of it, and find meaning where it did not exist before. Machines--from jet engines to gas turbines to medical scanners--connected via the Industrial Internet have the analytical intelligence to self-diagnose and self-correct, so they can deliver the right information to the right people at the right time (and in real-time). Despite the promise of the Industrial Internet, however, supporting the end-to-end quality-of-service (QoS) requirements is hard. This talk will discuss a number of technical issues emerging in this context, including: Precise auto-scaling of resources with a system-wide focus. Flexible optimization algorithms to balance real-time constraints with cost and other goals. Improved fault-tolerance fail-over to support real-time requirements. Data provisioning and load balancing algorithms that rely on physical properties of computations. It will also explore how the OMG Data Distribution Service (DDS) provides key building blocks needed to create a dependable and elastic software infrastructure for the Industrial Internet.

1. Elastic Software Infrastructure to
Support the Industrial Internet
Douglas C. Schmidt
d.schmidt@vanderbilt.edu
Institute for Software
Integrated Systems
Vanderbilt University
Nashville, TN
RTI Webinar Series, October 23rd, 2013

2. Outline of Presentation
• Context & terminology
• Prior R&D progress
• Current R&D trends
& challenges
• A promising solution
• Concluding remarks
2

3. Outline of Presentation
• Context & terminology
• Prior R&D progress
• Current R&D trends
& challenges
• A promising solution
• Concluding remarks
3

4. Overview of the Industrial Internet
• The Industrial Internet is a term coined by GE that refers to the integration
of complex physical machinery with networked sensors & software
4
en.wikipedia.org/wiki/Industrial_Internet

5. Overview of the Industrial Internet
• The Industrial Internet is a term coined by GE that refers to the integration
of complex physical machinery with networked sensors & software
• It combines fields such as machine learning, big data, the Internet of things,
& machine-to-machine communication to
5

6. Overview of the Industrial Internet
• The Industrial Internet is a term coined by GE that refers to the integration
of complex physical machinery with networked sensors & software
• It combines fields such as machine learning, big data, the Internet of things,
& machine-to-machine communication to
• Connect machines embedded
with sensors to other
machines (& end users)
6

7. Overview of the Industrial Internet
• The Industrial Internet is a term coined by GE that refers to the integration
of complex physical machinery with networked sensors & software
• It combines fields such as machine learning, big data, the Internet of things,
& machine-to-machine communication to
• Connect machines embedded
with sensors to other
machines (& end users)
• Enable access & control
of mechanical devices
7

8. Overview of the Industrial Internet
• The Industrial Internet is a term coined by GE that refers to the integration
of complex physical machinery with networked sensors & software
• It combines fields such as machine learning, big data, the Internet of things,
& machine-to-machine communication to
• Connect machines embedded
with sensors to other
machines (& end users)
• Enable access & control
of mechanical devices
• Extract data from these
devices, make sense of it, &
deliver the right information
to the right people at the
right time (& in real-time)
8

9. Overview of the Industrial Internet
• The Industrial Internet is a term coined by GE that refers to the integration
of complex physical machinery with networked sensors & software
• It combines fields such as machine learning, big data, the Internet of things,
& machine-to-machine communication to
• Connect machines embedded
with sensors to other
machines (& end users)
• Enable access & control
of mechanical devices
• Extract data from these
devices, make sense of it, &
deliver the right information
to the right people at the
right time (& in real-time)
• Derive some form of value in
terms of improved utility, &
cost savings
9

10. Overview of the Industrial Internet
• The Industrial Internet is a term coined by GE that refers to the integration
of complex physical machinery with networked sensors & software
• It combines fields such as machine learning, big data, the Internet of things,
& machine-to-machine communication to
• Connect machines embedded
with sensors to other
machines (& end users)
• Enable access & control
of mechanical devices
• Extract data from these
devices, make sense of it, &
deliver the right information
to the right people at the
right time (& in real-time)
• Derive some form of value in
terms of improved utility, &
cost savings
10
At the heart of the Industrial Internet are cyber-physical systems & clouds

11. Overview of Cyber-Physical Systems
• A cyber-physical system
(CPS) features a tight
coordination between the
system’s computational
& physical elements
11
en.wikipedia.org/wiki/Cyber-physical_system

12. Overview of Cyber-Physical Systems
• A cyber-physical system
(CPS) features a tight
coordination between the
system’s computational
& physical elements
• CPSs increasingly use
networked processing
elements to control
physical, chemical, or
biological processes or
devices
12
www.ge.com/stories/industrial-internet has other apt examples

13. Overview of Cyber-Physical Systems
• A cyber-physical system
(CPS) features a tight
coordination between the
system’s computational
& physical elements
• CPSs increasingly use
networked processing
elements to control
physical, chemical, or
bi-ological processes or
devices
• In CPSs the ―right answer‖
delivered too late becomes
the ―wrong answer‖
• i.e., dependability has
a temporal dimension
(& increasingly a
security dimension)
13
This talk focuses on distributed CPSs rather than standalone CPSs

14. Overview of Cloud Computing
• Cloud computing is a model for enabling
ubiquitous, convenient, on-demand
network access to a shared pool
of configurable computing
resources
• e.g., networks, servers,
Measured
service
storage, applications, &
services
On-demand
self-service
14
Resource
pooling
Rapid
elasticity
Broad network
access

15. Overview of Cloud Computing
• Cloud computing is a model for enabling
ubiquitous, convenient, on-demand
network access to a shared pool
of configurable computing
resources
• These resources can be
Measured
service
rapidly provisioned &
released with minimal
management effort or
service provider interaction
On-demand
self-service
Resource
pooling
Rapid
elasticity
Broad network
access
15
csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf

16. Overview of Cloud Computing
• Cloud computing is a model for enabling
ubiquitous, convenient, on-demand
network access to a shared pool
of configurable computing
resources
• These resources can be
rapidly provisioned &
released with minimal
management effort or
service provider interaction
• Cloud offerings enable
―economies of scale‖ via
multi-tenancy & elasticity
• e.g., run atop shared
(often virtualized) data
access, storage, hardware,
software, middleware, etc.
16

17. Overview of Cloud Computing
• Cloud computing is a model for enabling
ubiquitous, convenient, on-demand
network access to a shared pool
of configurable computing
resources
• These resources can be
rapidly provisioned &
released with minimal
management effort or
service provider interaction
• Cloud offerings enable
―economies of scale‖ via
multi-tenancy & elasticity
• Cloud services don’t require users to know of the configuration & physical
location of the computing & communication infrastructure delivering services
• Similar to traditional utilities, such as power grids, water, sewer,
as well as datacom/telecom service providers
17

18. Overview of Cloud Computing
• Cloud computing is a model for enabling
ubiquitous, convenient, on-demand
network access to a shared pool
of configurable computing
resources
• These resources can be
rapidly provisioned &
released with minimal
management effort or
service provider interaction
• Cloud offerings enable
―economies of scale‖ via
multi-tenancy & elasticity
• Cloud services don’t require users to know of the configuration & physical
location of the computing & communication infrastructure delivering services
• Similar to traditional utilities, such as power grids, water, sewer,
as well as datacom/telecom service providers
18
Some implementations of cloud computing may be at odds with CPS needs..

19. Outline of Presentation
• Context & terminology
• Prior R&D progress
• Current R&D trends
& challenges
• A promising solution
• Concluding remarks
19

20. Prior R&D Progress for Cyber-Physical Systems
From this design paradigm…
Nav
Air
Frame
WTS
AP
FLIR
SPLnner
IFF
Cyclic Exec
The designs of legacy CPSs tend to be:
• Stovepiped
• Proprietary
• Brittle & non-adaptive
• Expensive to develop & evolve
• Vulnerable
20
Problem: Small changes can break nearly anything & everything

21. Prior R&D Progress for Cyber-Physical Systems
…and this operational paradigm…
Utility “Curve”
Utility
Real-time QoS requirements for legacy
CPSs:
• Ensure predictable end-to-end QoS,
e.g.,
• Bound latency, jitter, & footprint
• Bound priority inversions
• Allocate & manage resources
statically & avoid sharing
“Broken”
“Works”
Resources
“Hard” Requirements
21
Problem: Lack of any resource can break nearly everything

22. Prior R&D Progress for Cyber-Physical Systems
…and this operational paradigm…
Real-time QoS requirements for legacy
CPSs:
• Ensure predictable end-to-end QoS,
e.g.,
• Bound latency, jitter, & footprint
• Bound priority inversions
• Allocate & manage resources
statically & avoid sharing
22
This is not at all what we think of as a computing cloud!

23. Prior R&D Progress for Cyber-Physical Systems
…to this design paradigm…
Air
Frame
AP
Event
Channel
PLanner
Nav
WTS
Replication
Service
IFF
FLIR
Information Backbone
The designs of today’s leading-edge CPSs
tend to be more:
•
Layered & componentized
•
Standards- & COTS-based
•
Robust to failures & adaptive to
operating conditions
•
Cost effective to evolve & retarget
23
Result: changing requirements & environments can be handled more flexibly

24. Prior R&D Progress for Cyber-Physical Systems
…and this operational paradigm…
Utility
Desired
Utility
Curve
“Working
Range”
Resources
• Ensure acceptable end-to-end QoS, e.g.,
“Softer” Requirements
• Minimize latency, jitter, & footprint
• Minimize priority inversions
• Resources are allocated/managed dynamically & can be shared
24
Result: better support for operations with scarce/contended resources

25. Prior R&D Progress for Cyber-Physical Systems
…and this operational paradigm…
• Ensure acceptable end-to-end QoS, e.g.,
• Minimize latency, jitter, & footprint
• Minimize priority inversions
• Resources are allocated/managed dynamically & can be shared
25
Some CPS operating platforms have much in common with computing clouds

26. Prior R&D Progress for Cyber-Physical Systems
…and this operational paradigm…
• Ensure acceptable end-to-end QoS, e.g.,
• Minimize latency, jitter, & footprint
• Minimize priority inversions
• Resources are allocated/managed dynamically & can be shared
26
See www.dre.vanderbilt.edu/~schmidt/JSS-DRM.pdf for more info

27. New Challenge: Ultra-Large-Scale Cyber-Physical Systems
Key problem space challenges
• Dynamic behavior
• Transient overloads
• Time-critical tasks
• Context-specific requirements
• Resource conflicts
• Interdependence of (sub)systems
• Integration with legacy
(sub)systems
Key solution space challenges
• Enormous accidental & inherent
complexities
• Continuous evolution & change
• Highly heterogeneous platform,
language, & tool environments
27

28. New Challenge: Ultra-Large-Scale Cyber-Physical Systems
Key problem space challenges
• Dynamic behavior
• Transient overloads
• Time-critical tasks
• Context-specific requirements
• Resource conflicts
• Interdependence of (sub)systems
• Integration with legacy
(sub)systems
Key solution space challenges
• Enormous accidental & inherent
complexities
• Continuous evolution & change
• Highly heterogeneous platform,
language, & tool environments
Mapping problem space requirements28 solution space artifacts is very hard!
to

29. New Challenge: Ultra-Large-Scale Cyber-Physical Systems
Key problem space challenges
• Dynamic behavior
• Transient overloads
• Time-critical tasks
• Context-specific requirements
• Resource conflicts
• Interdependence of (sub)systems
• Integration with legacy
(sub)systems
Key solution space challenges
• Enormous accidental & inherent
complexities
• Continuous evolution & change
• Highly heterogeneous platform,
language, & tool environments
29
See www.dre.vanderbilt.edu/~schmidt/PDF/FOME-HCDS-paper.pdf for more

30. New Challenge: Ultra-Large-Scale Cyber-Physical Systems
Key problem space challenges
• Dynamic behavior
• Transient overloads
• Time-critical tasks
• Context-specific requirements
• Resource conflicts
• Interdependence of (sub)systems
• Integration with legacy
(sub)systems
Key solution space challenges
• Enormous accidental & inherent
complexities
• Continuous evolution & change
• Highly heterogeneous platform,
language, & tool environments
30
Ultra-Large-Scale CPSs are well beyond scope of today’s computing clouds

31. New Challenge: Ultra-Large-Scale Cyber-Physical Systems
Key problem space challenges
• Dynamic behavior
• Transient overloads
• Time-critical tasks
• Context-specific requirements
• Resource conflicts
• Interdependence of (sub)systems
• Integration with legacy
(sub)systems
Key solution space challenges
• Enormous accidental & inherent
complexities
• Continuous evolution & change
• Highly heterogeneous platform,
language, & tool environments
31

32. New Challenge: Ultra-Large-Scale Cyber-Physical Systems
Key problem space challenges
• Dynamic behavior
―Gentlemen, we
• Transient overloads
have
• Time-critical tasks run out of
money. It is time
• Context-specific requirements
to start
• Resource conflicts thinking.‖
• Interdependence of (sub)systems
• Integration with legacy
(sub)systems
Key solution space challenges
• Enormous accidental & inherent
complexities
• Continuous evolution & change
• Highly heterogeneous platform,
language, & tool environments
32
en.wikiquote.org/wiki/Talk:Winston_Churchill

33. Outline of Presentation
• Context & terminology
• Prior R&D progress
• Current R&D trends &
challenges
• A promising solution
• Concluding remarks
33

34. Convenient Trend: Elastic Hardware Platforms
•
―Elastic hardware‖ based on
multi-core & distributed-core
architectures now available
at reasonable prices
34
en.wikipedia.org/wiki/Elasticity_(cloud_computing) has more info

35. Convenient Trend: Elastic Hardware Platforms
•
•
―Elastic hardware‖ based on
multi-core & distributed-core
architectures now available
at reasonable prices
Elastic hardware has potential
to substantially accelerate
performance by parallelizing
application work-loads & autoscaling data processing at
runtime
– Goal is to add/utilize more
hardware without changing
application business logic or
configurations
35

36. Convenient Trend: Elastic Hardware Platforms
•
•
•
―Elastic hardware‖ based on
multi-core & distributed-core
architectures now available
at reasonable prices
Elastic hardware has potential
to substantially accelerate
performance by parallelizing
application work-loads & autoscaling data processing at
runtime
Current focus of elastic
hardware is largely on web
hosting applications in public
cloud environments
36

37. Convenient Trend: Elastic Hardware Platforms
•
•
•
―Elastic hardware‖ based on
multi-core & distributed-core
architectures now available
at reasonable prices
Elastic hardware has potential
to substantially accelerate
performance by parallelizing
application work-loads & autoscaling data processing at
runtime
Current focus of elastic
hardware is largely on web
hosting applications in public
cloud environments
37
Elastic hardware is necessary—but not sufficient—for elastic CPS applications

38. Impediments to Applying Elastic Hardware for CPSs
•
Inadequate programming
models
ISR Processing
– Complicated & obtrusive APIs
– Can’t use hardware predictably
& scalably
38
SCADA Systems
Air Traffic Mgmt
Aerospace

39. Impediments to Applying Elastic Hardware for CPSs
•
•
Inadequate programming
models
Inadequate knowledge of
real-time, concurrency, &
networking
ISR Processing
– e.g., high probability of race
conditions, deadlocks, priority
inversion, & missed deadlines
39
SCADA Systems
Air Traffic Mgmt
Aerospace

40. Impediments to Applying Elastic Hardware for CPSs
•
•
•
Inadequate programming
models
Inadequate knowledge of
real-time, concurrency, &
networking
Inadequate mechanisms to
transition seamlessly from
multi- to distributed-core
environments
ISR Processing
40
SCADA Systems
Air Traffic Mgmt
Aerospace

41. Impediments to Applying Elastic Hardware for CPSs
•
•
•
•
Inadequate programming
models
Inadequate knowledge of
real-time, concurrency, &
networking
Inadequate mechanisms to
transition seamlessly from
multi- to distributed-core
environments
Inadequate quality-of-service
(QoS) support at scale
ISR Processing
– e.g., lack of system-wide control over
key QoS impacting resource usage &
end-to-end data deliver semantics
41
SCADA Systems
Air Traffic Mgmt
Aerospace

42. Impediments to Applying Elastic Hardware for CPSs
•
Inadequate programming
models
ISR Processing
SCADA Systems
Air Traffic Mgmt
Aerospace
– Complicated & obtrusive APIs
– Can’t use hardware predictably
& scalably
•
Inadequate knowledge of real-time,
concurrency, & networking
– e.g., high probability of race
conditions, deadlocks, priority
inversion, & missed deadlines
•
•
Inadequate mechanisms to transition
seamlessly from multi- to distributedcore environments
Inadequate quality-of-service (QoS)
support at scale
– e.g., lack of system-wide control over
key QoS impacting resource usage &
end-to-end data deliver semantics
42
Some impediments affect many types of systems, some mostly affect CPSs

43. Key Research Challenges for Elastic CPSs
1. Precise auto-scaling of
ISR Processing
SCADA Systems
Air Traffic Mgmt
Aerospace
resources with a systemwide end-to-end focus
2. Flexible optimization
algorithms to balance realtime constraints with cost &
other goals
3. Improved fault-tolerance
fail-over that supports realtime requirements
4. Data provisioning & load
balancing algorithms that
consider physical properties
of computations & storage
43
Meeting these challenges requires rethinking some cloud computing tenets

44. Key Research Challenges for Elastic CPSs
1. Precise auto-scaling of
ISR Processing
resources with a systemwide end-to-end focus
– State-of-the-art in autoscaling algo-rithms manage
services in isolation
• CPSs require autoscaling algo-rithms to
operate on end-to-end
task chains
SCADA Systems
Air Traffic Mgmt
CPU utilization
44
Aerospace

45. Key Research Challenges for Elastic CPSs
1. Precise auto-scaling of
ISR Processing
resources with a systemwide end-to-end focus
– State-of-the-art in autoscaling algo-rithms manage
services in isolation
– Physical stability &
safety properties may
require exceedingly
complex analyses
• e.g., reachability of
hybrid cyber-physical
states
SCADA Systems
Air Traffic Mgmt
CPU utilization
45
Aerospace

46. Key Research Challenges for Elastic CPSs
algorithms to balance realtime constraints with cost &
other goals
– CPS deployments must be
schedulable on all resources
acquired from cloud providers
to ensure real-time response
times, while optimizing desired
objective functions
• e.g., minimizing costs
ISR Processing
SCADA Systems
Air Traffic Mgmt
Aerospace
Multi-dimensional Resource Management
Cost
2. Flexible optimization
46

47. Key Research Challenges for Elastic CPSs
algorithms to balance realtime constraints with cost &
other goals
– CPS deployments must be
schedulable on all resources
acquired from cloud providers
to ensure real-time response
times, while optimizing desired
objective functions
– Principled means are needed to
co-schedule and/or per-form
admission control & eviction of
mixed-criticality task sets
deployed on cloud resources
ISR Processing
SCADA Systems
Air Traffic Mgmt
Aerospace
Multi-dimensional Resource Management
Cost
2. Flexible optimization
47

48. Key Research Challenges for Elastic CPSs
3. Improved fault-tolerance
ISR Processing
fail-over that supports
real-time requirements
– Some cloud platforms
tolerate faults for
provisioned re-sources
• This is insufficient for
CPSs where realtime fault-tolerance
of end-to-end task
chains must be met
simultaneously
48
SCADA Systems
Air Traffic Mgmt
Aerospace

49. Key Research Challenges for Elastic CPSs
3. Improved fault-tolerance
ISR Processing
fail-over that supports
real-time requirements
– Some cloud platforms
tolerate faults for
provisioned re-sources
– Reasoning about the
consequences of faults
is an important open
re-search area due to
the complex & stochastic
nature of many CPSs
49
SCADA Systems
Air Traffic Mgmt
Aerospace

50. Key Research Challenges for Elastic CPSs
4. Data provisioning & load
balancing algorithms that
consider physical properties
of computations & storage
– CPSs generate load on
a computing cloud
due to physical stimuli
ISR Processing
cache
affinity
geographic
associations
50
SCADA Systems
Air Traffic Mgmt
Aerospace
social
network
linkages
power
consumption

51. Key Research Challenges for Elastic CPSs
4. Data provisioning & load
balancing algorithms that
consider physical properties
of computations & storage
– CPSs generate load on
a computing cloud
due to physical stimuli
– To build more scalable &
high-performance CPSs,
algorithms & techniques
are needed to
• Exploit physical
characteristics
of data & computation
ISR Processing
cache
affinity
geographic
associations
51
SCADA Systems
Air Traffic Mgmt
Aerospace
social
network
linkages
power
consumption

52. Key Research Challenges for Elastic CPSs
4. Data provisioning & load
balancing algorithms that
consider physical properties
of computations & storage
– CPSs generate load on
a computing cloud
due to physical stimuli
– To build more scalable &
high-performance CPSs,
algorithms & techniques
are needed to
• Exploit physical
characteristics
of data & computation
• Improve the distribution
of work in a computing
cloud
ISR Processing
cache
affinity
geographic
associations
52
SCADA Systems
Air Traffic Mgmt
Aerospace
social
network
linkages
power
consumption

53. Key Research Challenges for Elastic CPSs
4. Data provisioning & load
balancing algorithms that
consider physical properties
of computations & storage
– CPSs generate load on
a computing cloud
due to physical stimuli
– To build more scalable &
high-performance CPSs,
algorithms & techniques
are needed to
• Exploit physical
characteristics
of data & computation
• Improve the distribution
of work in a computing
cloud
ISR Processing
cache
affinity
geographic
associations
SCADA Systems
Air Traffic Mgmt
Aerospace
social
network
linkages
power
consumption
53
We need a holistic solution that provides an elastic CPS software infrastructure

54. Outline of Presentation
• Context & terminology
• Prior R&D progress
• Current R&D trends &
challenges
• A promising solution
• Concluding remarks
54

55. Requirements for Elastic CPS Software Infrastructure
•
Flexibility – Loosely coupled
components that can be
analyzed, replaced, reused,
distributed, & parallelized
dependably
ISR Processing
SCADA Systems
Air Traffic Mgmt
Dynamic
Discovery
Load
Balancing
Dependability
Middle
ware
Low Latency
55
Data Distribution
Aerospace

56. Requirements for Elastic CPS Software Infrastructure
•
Flexibility – Loosely coupled
components that can be
analyzed, replaced, reused,
distributed, & parallelized
dependably
•
Adaptability – Provide APIs
that adapt to existing code,
rather than always having to
adapt code to an API
ISR Processing
SCADA Systems
Air Traffic Mgmt
Dynamic
Discovery
Load
Balancing
Dependability
Middle
ware
Low Latency
56
Data Distribution
Aerospace

57. Requirements for Elastic CPS Software Infrastructure
•
Flexibility – Loosely coupled
components that can be
analyzed, replaced, reused,
distributed, & parallelized
dependably
•
Adaptability – Provide APIs
that adapt to existing code,
rather than always having to
adapt code to an API
•
ISR Processing
SCADA Systems
Air Traffic Mgmt
Dynamic
Discovery
Load
Balancing
Dependability
Middle
ware
Uniformity – Seamless
(ideally standards-based) support
for multi-core & distributed-core
Low Latency
57
Data Distribution
Aerospace

58. Requirements for Elastic CPS Software Infrastructure
•
Flexibility – Loosely coupled
components that can be
analyzed, replaced, reused,
distributed, & parallelized
dependably
•
Adaptability – Provide APIs
that adapt to existing code,
rather than always having to
adapt code to an API
•
•
ISR Processing
SCADA Systems
Air Traffic Mgmt
Dynamic
Discovery
Load
Balancing
Dependability
Middle
ware
Uniformity – Seamless
(ideally standards-based) support
for multi-core & distributed-core
Low Latency
Scalability – Static & dynamic
load balancing ensures best &
dependable utilization of available
elastic hardware resources
58
Data Distribution
Aerospace

59. Requirements for Elastic CPS Software Infrastructure
•
Flexibility – Loosely coupled
components that can be
analyzed, replaced, reused,
distributed, & parallelized
dependably
•
Adaptability – Provide APIs
that adapt to existing code,
rather than always having to
adapt code to an API
•
•
Uniformity – Seamless
(ideally standards-based) support
for multi-core & distributed-core
ISR Processing
SCADA Systems
Air Traffic Mgmt
Aerospace
Dynamic
Discovery
Load
Balancing
Dependability
Middle
ware
Low Latency
Data Distribution
Scalability – Static & dynamic
load balancing ensures best &
dependable utilization of available
elastic hardware resources
59
Middleware is a key element of elastic CPS software infrastructure

60. Key Layers of CPS Software Infrastructure
ISR Processing
SCADA Systems
Provide mechanisms to manage end-system
resources, e.g., CPU scheduling, inter-process
communication, memory management, & file systems
Air Traffic Mgmt
Aerospace
Domain-Specific
Services
Common
Middleware Services
Distribution
Middleware
Host Infrastructure
Middleware
Operating Systems &
Protocols
60

61. Key Layers of CPS Software Infrastructure
ISR Processing
SCADA Systems
Encapsulates & enhances native OS mechanisms to
create reusable network programming components
Air Traffic Mgmt
Aerospace
Domain-Specific
Services
Common
Middleware Services
Distribution
Middleware
Host Infrastructure
Middleware
Operating Systems &
Protocols
61

62. Key Layers of CPS Software Infrastructure
ISR Processing
SCADA Systems
Defines higher-level programming models whose
reusable APIs & components automate & extend
native OS capabilities across distribution boundaries
Air Traffic Mgmt
Aerospace
Domain-Specific
Services
Common
Middleware Services
Distribution
Middleware
Host Infrastructure
Middleware
Operating Systems &
Protocols
62

63. Key Layers of CPS Software Infrastructure
ISR Processing
SCADA Systems
Augment distribution middleware by defining higherlevel domain-independent services that focus on
programming ―business logic‖
Air Traffic Mgmt
Aerospace
Domain-Specific
Services
Common
Middleware Services
Distribution
Middleware
Host Infrastructure
Middleware
Operating Systems &
Protocols
63

64. Key Layers of CPS Software Infrastructure
ISR Processing
SCADA Systems
Tailored to requirements of particular domains, such
as SCADA, avionics, aerospace, vehtronics, C4ISR, air
traffic management, integrated healthcare, etc.
Air Traffic Mgmt
Aerospace
Domain-Specific
Services
Common
Middleware Services
Distribution
Middleware
Host Infrastructure
Middleware
Operating Systems &
Protocols
64

65. Promising Elastic CPS Middleware: DDS
•
The OMG Data Distribution
Service (DDS) promotes a
pattern language that yields
loosely coupled, polyglot,
evolvable, scalable, efficient
& dependable CPSs
ISR Processing
SCADA Systems
Air Traffic Mgmt
Aerospace
65
en.wikipedia.org/wiki/Data_Distribution_Service has a good DDS overview

66. Promising Elastic CPS Middleware: DDS
•
The OMG Data Distribution
Service (DDS) promotes a
pattern language that yields
loosely coupled, polyglot,
evolvable, scalable, efficient
& dependable CPSs
– DDS supports relational
& OO information modeling
ISR Processing
• Data-Centric Publish-
Subscribe (DCPS) & Data
Local Reconstruction Layer
(DLRL)
66
SCADA Systems
Air Traffic Mgmt
Aerospace

67. Promising Elastic CPS Middleware: DDS
•
The OMG Data Distribution
Service (DDS) promotes a
pattern language that yields
loosely coupled, polyglot,
evolvable, scalable, efficient
& dependable CPSs
– DDS supports flat,
relational, & OO
information modeling
– DDS global data space
allows apps to read/write
data anonymously &
asynchronously, decoupled
in space & time
ISR Processing
SCADA Systems
Topic
Data
Reader
Domain
Participant
Subscriber
Global
Data
Space
67
Data
Writer
Air Traffic Mgmt
Aerospace
Topic
Data
Writer
Publisher
Data
Reader
Data
Reader
Subscriber

68. Promising Elastic CPS Middleware: DDS
•
The OMG Data Distribution
Service (DDS) promotes a
pattern language that yields
loosely coupled, polyglot,
evolvable, scalable, efficient
& dependable CPSs
– DDS supports flat,
relational, & OO
information modeling
– DDS global data space
allows apps to read/write
data anonymously &
asynchronously, decoupled
in space & time
– DDS pub/sub model allows
apps to produce/consume
information into/from the
global data space
ISR Processing
SCADA Systems
Topic
Data
Reader
Domain
Participant
Subscriber
Global
Data
Space
68
Data
Writer
Air Traffic Mgmt
Aerospace
Topic
Data
Writer
Publisher
Data
Reader
Data
Reader
Subscriber

69. Promising Elastic CPS Middleware: DDS
•
The OMG Data Distribution
Service (DDS) promotes a
pattern language that yields
loosely coupled, polyglot,
evolvable, scalable, efficient
& dependable CPSs
– DDS supports flat,
relational, & OO
information modeling
– DDS global data space
allows apps to read/write
data anonymously &
asynchronously, decoupled
in space & time
– DDS pub/sub model allows
apps to produce/consume
information into/from the
global data space
ISR Processing
SCADA Systems
Topic
Data
Reader
Domain
Participant
Subscriber
Data
Writer
Air Traffic Mgmt
Aerospace
Topic
Data
Writer
Publisher
Data
Reader
Data
Reader
Subscriber
Global
Data
Space
69
DDS mainly provides distribution middleware & common middleware services

70. Promising Elastic CPS Middleware: DDS
•
DDS controls resource usage,
end-to-end data delivery, &
data availability via a rich set
of QoS policies, e.g.:
– Batching
– Priority
– Deadline
– Data Durability
– Redundancy
– Data History
ISR Processing
70
SCADA Systems
Air Traffic Mgmt
Aerospace

71. Promising Elastic CPS Middleware: DDS
•
DDS controls resource usage,
end-to-end data delivery, &
data availability via a rich set
of QoS policies, e.g.:
– Batching
– Priority
– Deadline
– Data Durability
– Redundancy
– Data History
ISR Processing
71
SCADA Systems
Air Traffic Mgmt
Aerospace

72. Promising Elastic CPS Middleware: DDS
•
DDS controls resource usage,
end-to-end data delivery, &
data availability via a rich set
of QoS policies, e.g.:
– Batching
– Priority
Data
Writer
– Deadline
R
– Data Durability
– Redundancy
Publisher
– Data History
ISR Processing
SCADA Systems
HISTORY
Air Traffic Mgmt
Aerospace
RESOURCE
LIMITS
Topic
R
S1
Data
Reader
R
S2
S3
S4
S5
Subscriber
Subscri
S6
S7
X
S7
ber
LATENCY
S7
S6
S5
S4
S3
S2
S1
COHERENCY
RELIABILITY
72
www.dre.vanderbilt.edu/~schmidt/PDF/CrossTalk-2008-final.pdf

73. Promising Elastic CPS Middleware: DDS
•
DDS controls resource usage,
end-to-end data delivery, &
data availability via a rich set
of QoS policies, e.g.:
– Batching
– Priority
– Deadline
– Data Durability
– Redundancy
– Data History
ISR Processing
SCADA Systems
Topic
Data
Reader
Air Traffic Mgmt
Aerospace
Topic
Requested
Requested
Requested
QoS
QoS
QoS
Subscriber
Domain
Participant
Offered
Data
Reader
Offered
Offered
QoS
QoS
QoS
Subscriber
Domain
Participant
73
DDS’s request/offered (RxO) model matches QoS policies between pub & sub

74. Promising Elastic CPS Middleware: DDS
•
•
DDS controls resource usage,
end-to-end data delivery, &
data availability via a rich set
of QoS policies:
– Batching
– Priority
– Deadline
– Data Durability
– Redundancy
– Data History
Bridges are available
across technologies to
expose relevant data to
heterogeneous network
protocols, without
imposing changes into
existing legacy systems
ISR Processing
74
SCADA Systems
Air Traffic Mgmt
Aerospace

75. Promising Elastic CPS Middleware: DDS
•
DDS is an OMG standard that
itself is based on many
associated open standards
ISR Processing
75
SCADA Systems
Air Traffic Mgmt
Aerospace

76. Promising Elastic CPS Middleware: DDS
•
•
DDS is an OMG standard that
itself is based on many
associated open standards
Key DDS implementations
are now available in opensource form
• Many opportunities
for researchers to
influence DDS
standard &
implementations
ISR Processing
SCADA Systems
Air Traffic Mgmt
Aerospace
76
See www.dre.vanderbilt.edu/~schmidt/PDF/DDS-WAN.pdf for recent paper

77. Promising Elastic CPS Middleware: DDS
•
•
•
DDS is an OMG standard that
itself is based on many
associated open standards
Key DDS implementations
are now available in opensource form
• Many opportunities
for researchers to
influence DDS
standard &
implementations
DDS is used in many CPS
research projects
& production systems
ISR Processing
77
SCADA Systems
Air Traffic Mgmt
Aerospace

78. Promising Elastic CPS Middleware: DDS
•
•
•
•
DDS is an OMG standard that
itself is based on many
associated open standards
Key DDS implementations
are now available in opensource form
• Many opportunities
for researchers to
influence DDS
standard &
implementations
DDS is used in many CPS
research projects
& production systems
portals.omg.org/dds provides
more info on DDS activities &
projects
ISR Processing
78
SCADA Systems
Air Traffic Mgmt
Aerospace

79. Outline of Presentation
• Context & terminology
• Prior R&D progress
• Current R&D trends &
challenges
• A promising solution
• Concluding remarks
79

80. Concluding Remarks
•
Despite advances in elastic hardware, deploying
CPSs in cloud environments is hard without
adequate support from elastic software infrastructure
– It’s unlikely that public clouds will work for
mission-critical Industrial Internet applications
80
Key characteristics of computing clouds for CPS are multi-tenancy & elasticity

81. Concluding Remarks
•
•
Despite advances in elastic hardware, deploying
CPSs in cloud environments is hard without
adequate support from elastic software infrastructure
Standards-based DDS middleware provides
key open-source building-blocks to create a
dependable elastic CPS software infrastructure
81
There are many hard research challenges remaining

82. Concluding Remarks
•
•
•
Despite advances in elastic hardware, deploying
CPSs in cloud environments is hard without
adequate support from elastic software infrastructure
Standards-based DDS middleware provides
key open-source building-blocks to create a
dependable elastic CPS software infrastructure
There are many hard research challenges
remaining
82
www.industrialinternet.com/blog/three-qs-professor-douglas-schmidt/

83. Concluding Remarks
•
•
•
Despite advances in elastic hardware, deploying
CPSs in cloud environments is hard without
adequate support from elastic software infrastructure
Standards-based DDS middleware provides
key open-source building-blocks to create a
dependable elastic CPS software infrastructure
There are many hard research challenges
remaining
―Big breakthroughs often
happen when what is
suddenly possible meets
what is desperately
necessary‖
– Thomas Friedman
83
www.coursera.org/course/posa

84. Additional Information
See www.isis.vanderbilt.edu/workshops/cc4cps for info on an NSF workshop
on Computing Clouds for Cyber-Physical Systems (CC4CPS)
• Attended by ~50 researchers
funded by the NSF
• Topics of workshop included
• Role of computing clouds in data
•
collection, integration, analysis,
& mining for CPS
• Roles of computing clouds in
CPS control
• Stability, safety, security, privacy,
& reliability considerations in
integrating cloud computing with
CPS
• Programming models & paradigms
for computing clouds that support
CPS
84
The NSF CC4CPS workshop report will be available later this year

85. Additional Information
Ultra-large-scale (ULS) systems are sociotechnical ecosystems comprised of softwarereliant systems, people, policies, cultures, &
economics that have unprecedented scale:
• # of software & hardware elements
• # of connections & interdependencies
• # of computational elements
• # of purposes & perception of purposes
• # of routine processes & ―emergent
behaviors‖
• # of (overlapping) policy domains &
enforceable mechanisms
• # of people involved in some way
• Amount of data stored, accessed, &
manipulated
www.sei.cmu.edu/uls
• … etc …
85
See blog.sei.cmu.edu for more discussions of software R&D activities

86. Additional Information
NRC Report Critical Code: Software Producibility for Defense (2010)
The report focuses on ensuring the DoD
has the technical capacity & workforce to
design, produce, assure, & evolve innovative
software-reliant systems in a predictable
manner, while effectively managing risk,
cost, schedule, & complexity
Sponsored by Office of the Secretary of Defense (OSD)
with assistance from the National Science Foundation
(NSF), & Office of Naval Research (ONR),
www.nap.edu/openbook.php?record_id=12979&page=R1
86
See blog.sei.cmu.edu for more discussions of software R&D activities

87. Additional Information
• The Institute for Software
Integrated Systems (ISIS)
was established at
Vanderbilt in 1998
• Research at ISIS focuses
on systems with deeply
integrated software that
are networked, embedded,
& cyber-physical
• Key research areas at ISIS:
• Model-Integrated
Computing
• Middleware for distributed real-time & embedded
(DRE) systems
• Model-based engineering of cyber-physical systems
• Wireless sensor networks
• Systems security & privacy
87
www.dre.vanderbilt.edu/~schmidt/ISIS-research.pdf has more info on ISIS