This document discusses two publish/subscribe communication standards for distributed simulations: HLA and DDS. It introduces NCWare, a middleware developed by Nextel Engineering that unifies HLA and DDS. NCWare allows applications to interoperate between HLA and DDS by mapping data between their models and providing a single API. It also improves performance over HLA by using DDS as the underlying transport protocol while maintaining HLA semantics and services. NCWare provides a simple interface and real-time QoS capabilities to enable true distributed simulations using both HLA and DDS standards.

1. How to Develop True Distributed Simulations?
HLA & DDS Interoperability
José Carlos Díaz
Iván Gálvez
Jose M. López-Rodríguez
Nextel Engineering Systems S.L.
Avda. Manoteras 18 – 4º
Madrid, E-28050 (Spain)
(+34) 91 803 38 02
jcdiaz@nexteleng.es, igalvez@nexteleng.es, jmlrodriguez@nexteleng.es
Keywords:
Middleware, Real-Time, Simulation, M&S, HLA, RTI, Interoperability, DDS, Publish-Subscribe
ABSTRACT:
Nowadays there are two publish / subscribe based communications standards suitable for the simulation market:
IEEE HLA (High Level Architecture), developed in origin by the US DMSO and OMG DDS (Data Distribution
Service), developed by a group of companies with support of OMG organization. HLA is focused in interoperability
and reusability of simulations and DDS is a more general standard, focused on real time systems. This paper presents
NCWare, a new software abstraction layer developed by NEXTEL ENGINEERING that combines HLA and DDS
standards providing interoperability between them.
1. Introduction architecture of the simulators in order to distribute the
data between simulator’s subsystems using open
standards, hence enabling interoperability and
The evolution of M&S systems and NCW[1]/NEC[2] reusability at simulator component level.
scenarios require more performance, in order to join One of the latest standards, DDS2 (Dec 2004 [5]),
more complex applications demanding more data at proposes a communication system optimized for real-
faster rates. These scenarios also demand more time, and thus, offering performance to these scenarios.
interconnection mechanisms, in order to interoperate DDS is not aimed to substitute HLA, since DDS
legacy applications with new ones using the latest applications are broader than simulation, but applied to
standards and communication systems. it, can give a new breath of performance and reliability
Since HLA1 standard [4] is broadly used, much work to this field [3].
has been done on it in order to improve performance In order to successfully adapt legacy applications to
and network use [11], but performance is still seen as a new standards, a middleware which unifies them is a
bottleneck. HLA have demonstrated that it is suitable must. A middleware that combine both standards is best
for interoperating simulators of different providers suited for nowadays demands of quick development of
(federates) in federations distributed across multiple new system or the easy updating of already deployed
networks, but it has not been able to open the
1 High-Level Architecture 2 Data Distribution Service

2. systems to new operational of technical requirements. DDS has also a state-propagation built-in model, so
With such middleware the developer could design and when treating data structures with values which only
develop the system only thinking in the features of the change occasionally, they will be transmitted only once
systems, not in the transport mechanisms or in which for every update, son the network overload will be very
target you must deploy the system. low.
Nextel Engineering has developed NCWare software in The standard defines two levels of interfaces. At a
order to comply with former requirements. NCWare is lower level, it defines a Data Centric Publish Subscribe
a real time networking middleware based on the (DCPS) whose goal is to provide an efficient, scalable,
publish/subscribe paradigm which, through a single predictable, and resource aware data distribution
API, unifies DDS and HLA standards. The similarities mechanism. Then, on top of the DCPS, it defines the
between both standards [6] make them match perfectly. Data Local Reconstruction Layer (DLRL), an optional
interface which automates the reconstruction of data,
1.1. The OMG DDS standard.
locally, from updates received, and allows the
The DDS is a publish-subscribe data distribution application to access data as if it was local.
architecture which allows to connect asynchronously
and anonymously very large sets of distributed nodes of Object Model Profile
a communications mesh (domain participants).
Data Local Reconstruction Layer (DLRL)
A domain participant may simultaneously publish and
subscribe to typed data-streams identified by some Ownership Persistence Content-Subscription
names called “topics”.
The importance of DDS relies in its reliable design. Minimum Profile
DDS QoS3 parameters specify the degree of coupling Data Centric Publish-Subscribe (DCPS)
between participants, properties of the overall model
and of the topics themselves. So, DDS defines a Fig. 2 – DDS architecture
communications relationship between publishers and DDS is fundamentally designed to work over unreliable
subscribers which is: transports such as UDP or wireless networks.
• Decoupled in space (nodes can be anywhere) Besides, central servers or special nodes are not
• Decoupled in time (delivery of data may be required, so all the communication is direct between the
immediately after publication or later) nodes, also known as Peer-to-Peer or P2P.
• Decoupled in flow (delivery may be reliable DDS scheme supports both unicast and multicasting IP
and can be done in a fully controlled networks, so latency of data between nodes will be
bandwidth) insignificant.
Since its introduction in 2003, DDS has enjoyed rapid
adoption as a standard for integrating and developing
high-performance real-time systems. It is a mandated
standard for publish-subscribe messaging by the U.S.
Department of Defense (DoD) Information Technology
Standards Registry (DISR). Programs that have
adopted DDS include the U.S. Navy's Open
Architecture Computing Environment (OACE) and
FORCEnet; the U.S. Army's Future Combat Systems
(FCS); and the joint Air Force, Navy and DISA Net-
Centric Enterprise Solutions for Interoperability
(NESI).
Fig. 1 - DDS performs the network tasks for the application
Scalability is increased thanks to the multiple
independent data channels identified by “keys”. This 1.2. The IEEE HLA standard.
allows nodes to subscribe to many (maybe thousands)
of similar data streams with a single subscription. Originally developed by the US Department of
When the data arrives, the middleware can sort it by the Defense, HLA is the prescribed standard for military
key and deliver it for efficient processing. simulation interoperability within the US. It is also the
standard for simulation interoperability within the
3 NATO. The HLA standard has also become a non-
Quality-of-Service

3. military standard through its civil standardization the middleware knows the complete data
through IEEE 1516. model.
o It gives a high level view of the system to all
users, instead of knowing only portions of it.
The underlying HLA architecture is a publish/subscribe
one where elements publish data onto the bus to be • HLA semantics define federations and federates,
picked up by other units that subscribe to that data, the first being isolated areas where federates
commonly referred to as 'federated data'. This allows a (applications) share their data. A federate has to be
system to be distributed, avoiding the bottlenecks of a connected to a federation to send/receive data from
client-server architecture and allowing the system to be other federates being connected to the same
more easily scalable. federation. With the same meaning, applications
using DDS are Entities, and federations are
HLA enables you to reuse existing systems for new
Domains. In the rest of the paper, a simulation will
purposes. You can also mix different programming
be a group of applications in either a domain or a
languages and operating systems. HLA supersedes
federation. The Domain/Federation concept allows
several earlier standards such as DIS and ALSP.
having many simulations at the same time without
One of its main components is the HLA Run Time interfering with each other.
Infrastructure (HLA-RTI), what is used to interoperate
simulator systems together over a WAN4, and, in some • All other HLA features such as instance discovery,
cases, it is even being seen as a way to link the not available on DDS, had to be implemented.
processing elements within individual simulators. 2. NCWare
But unfortunately, the HLA-RTI wasn't designed to NCWare is basically an interoperability middleware
reach the high levels of performance (low latencies as intended to be used not only in M&S environments but
well as deterministic responses are critical issues when also as a data-centric approach in the design and
we are dealing with Real Time systems). So using DDS development of distributed systems architectures, as it
as an underlying transport is a good approach for this used to be in NCW[1]/NEC[2] scenarios.
problem, because its similar structure allows it to work
very well alongside, and in co-operation with, the 2.1. Features
existing HLA standard. 2.1.1. Simplicity
1.3. Joining HLA and DDS One of the main drawbacks of communication systems
5
In order to join both standards to be used in is often the complexity of its API. One key objective of
simulation, NEXTEL Engineering decided to develop NCWare was to implement a very simple API. The
NCWare software keeping the HLA semantics. This user, who simply wants to send and/or receive data
approach implies the following: from the data model, shouldn’t have to care about
ports, addresses, servers, etc. The user just should have
• Data is organized, as HLA defines, in objects, and to tell the system: hey, I have the following type of data
interactions. Objects are composed of attributes, available, if anyone wants it. Or, it would tell the
and interactions of parameters. system: Hello, I need data of the following type. If
• All data being used in a simulation is defined in a anyone is sending it, please send it also to me.
data model, independent for applications and This is done by the Declaration of publications or
runtime middleware. This feature has many subscriptions. In the former, the user tells the system
advantages: that we might send some data of a type. In the later, the
o The design of the systems is centered in the user tells the system he wants data of a type.
data, not in objects or procedures. In this way After the declarations, the publisher just has to generate
it is easy to decouple features between new data and send it, and all the subscribers of this type
differentes applications and therefore to reuse of data will receive it.
them in other systems.
Experienced HLA users might need using more HLA
o It defines a clear interface for applications. features, such as synchronization, save and restore
Each application knows their publish- points, ownership and instance discovery. All those
subscribe interface with the middleware, and features are available to the user through simple
declaration of reception events, and notifications.
4
Wide Area Network
5
HLA and DDS

4. 2.1.2. Real-Time and QoS • DDS-compliant, real time, fault tolerant,
physical layer independent, peer-to-peer
DDS standard was conceived for Real-Time data
communications for fully distributed
transfer. DDS offers many QoS features starting from
embedded systems
time deadlines and transmission modes: reliable, best-
effort, which can be used in both unicast and multicast HLA or Visual RT Simulation
network modes. simulation or Visual
application application
HLA transmission modes were only reliable and best NCWare SIM
effort. NCWare RTI NCWare SIM
RTI RTI
Mäk Pitch
DDS can also define deadline times, which define the NCWare Core
NCWare Core
maximum reception time for a single data, separation
times, which define the minimum reception time. COTS HLA RTI Distributed systems
applications and embedded
NCWare allows the configuration of these QoS features applications
NCWare RTI
to be done in the Data Model, and therefore all entities
agree of using them. NCWare Core NCWare Core
2.1.3. Interoperability between different HLA’s Fig.3 - Combinations of the layers/components
RTI. of NCWare components
Although HLA Standard was defined, only was defined 2.2.1. NCWare Core
the services for developers but not the network
transport protocol. Thus, two applications using RTIs NCWare Core is the lowest level abstraction layer of
from different providers are not capable to NCWare. It uses the concept of Connectors, which
communicate between each other. The work done on allow the user to establish the communications very
HLA standard has been to define a new dynamic link easily with different DDS vendors.
compatible API which solves all problems. With this Its main objectives are the following:
new API it will be possible to use any of the different
RTI versions without the need of adapting to them. • Join different DDS vendors on a single, very
simple to use API.
But, since many HLA applications have been
developed so far, the incompatibility problem is still • Offer the necessary methods to adjust the QoS for
there. When developing a federation it had to be your applications and also allows you to fulfill
decided which RTI will be used. One of the objectives them in a simple configuration file.
of NCWare was to allow the user to postpone this • A real-time communication system, capable to
decision, and to be able at deployment time to use any send an receive data with QoS features,
of the available RTIs.
2.2.2. NCWare RTI
NCWare allows using many RTIs from different
vendors without the need of adapting the applications. NCWare RTI is an HLA Run Time Infrastucture
Therefore, an application will not depend on the RTI designed for real time performance in large federations.
used, and it will be available to communicate with other It provides an IEEE 1516 HLA API for virtual and live
HLA federates. simulations. The use of OMG DDS standard as
transport protocol offers all the functionality of a HLA
2.2. Architecture overview RTI with the features, performance, power and
It has an n-tier architecture in order to cover three reliability of DDS.
different scenarios: Its main technical features include:
• HLA-compliant federation of simulation - HLA RTI designed for real time performance
systems, enabling the interoperability of HLA in distributed applications (QoS enabled)
federates, using different RTIs available in the
M&S market, such as MÄK High - HLA RTI implementing IEEE 1516-2000
Performance RTI, Pitch pRTI y RTI NG Pro, - RTPS wire protocol as transport mechanism
among other RTI which implements DMSO for HLA as defined by OMG DDS
HLA 1.3 and/or IEEE 1516 API specifications specifications.
• Real time communications for distributed non - Full HLA services support (federation
HLA simulation systems through its own management, declaration management, object
DDS-based transport protocol

5. management, ownership management) as • Helps the developers to map between data models
required in virtual & live simulations. used in simulations. This mapping is one of the key
elements which allow simulations interoperability.
2.2.3. NCWare Sim
Since each simulation has its own data model,
NCWare Sim is the highest level abstraction layer of objects and attributes, which never match
NCWare. It offers a simulation-oriented layer similar to perfectly, some mapping between them need to be
Core’s one but allowing the users to access to several done. But many problems arise on mapping:
HLA services through different RTI vendors or DDS
o One class on one data model may map to one or
performance and fine grained QoS through NCWare
many classes on the other data model and
Core layer.
viceversa. This affects not only to data
NCWare Sim offers a single publish-susbcribe API, transmission and reception, but also to
independent of HLA and DDS. Therefore, the discovery, creation and deletion.
developer can use NCWare Sim as a real time
o Data might be defined in using different
middleware for the distribution of data within a
measurement units. A conversion to translate
simulator. The developer choose the standard to use for
units might be also necessary on each
the integration (HLA or DDS) with other systems
transmission.
through configuration files.
• Filters, allowing only relevant data to go to other
simulations. The filtering feature allows a
significant increase of performance as well as
security.
• Joins not only user data, but also HLA services.
The use of NCWare Gateway greatly expands the
possibilities of interconnection. The rest of the section
will describe many of these possibilities, starting from
the simplest to more complex scenarios.
Fig.4 – NCWare SIM interfaces with DDS & HLA
It allows the developer:
2.2.5. NCWare Performance Tool
- interfacing the same way with any COTS
HLA compliant RTI; A critical issue concerning to any communications
library is reliability and performance.
- accessing to high value features of QoS
(persistence, ownership, filtering, fault It is not possible to say that a library is better than
tolerance) another without comparing them. So it’s essential to
have testing & benchmarking tools to measure
- Modeling simulation data into HLA and DSS parameters of efficiency, performance and reliability.
100% compliant formats, within a Graphical
User’s Interface. NCWare Performance Tool allows:
- Mapping HLA object data models into OMG - To get performance comparisons between
DDS IDL. different versions of DDS implementations
(RTI DDS, TAO DDS or future ones)
Furthermore, NCWare is the underlying technology for
NEXTEL’s SimWare framework of M&S tools (see - To get performance comparisons between the
[9]) existent OS (Windows, Linux, etc.)
2.2.4. NCWare Gateway - To get performance comparisons between
available COTS HLA RTI (Pitch, Mäk,
Jointly with the NCWare, an application called DMSO, NCWare RTI)
NCWare Gateway was developed. Its objective is to
join two different simulations. NCWare Gateway - To evaluate performance cost for each
simply resends all data received on one simulation to NCWare layer (NCWare Core, NCWare SIM,
the other and viceversa. NCWare RTI)
NCWare Gateway performs the following tasks: 3. Scenarios of application
• Joins two simulations 3.1. DDS pure simulation

6. The basic scenario of application would be a simulation 3.4 Sharing data
using only DDS. The main advantage of this simulation
Now, consider the following scenario: you developed
is its high performance.
an application which generates some data. Since you
used NCWare, you might be using it on HLA
NCWare Core federations or DDS domains. The good thing about it is
Simulation
DDS
Module 2 that this application can send data to one or many
DDS simulations, and therefore sharing its data. An example
NCWare SIM
SIMULAT of it could be a weather simulator, which might be used
ION
on a single simulator, or could be used on a complex
NCWare Core
network consisting of many simulations connected by
DDS
Simulation
Module 1 the NCWare Gateway. Weather data will be shared
among all simulations.
Fig. 5 – DDS pure simulation scenario 3.5. Other scenarios
Applications can exchange data at much faster rates Some other scenarios would include the possibility of
than in a HLA simulation. testing all applications in a single simulation, and then
distributing them as wanted, or joining many HLA
federations which use different RTI implementations.
3.2 DDS simulation acting as an HLA federate
4. Applications
An evolution of this scenario would be to attach it to an
NCWare and NCWare Gateway are part of SimWare
external HLA federation. For example, a real-time
architecture. Within this architecture, many other
driving simulator (DDS) could be attached to a
federation including many other vehicles and people. elements interconnected by the NCWare, such as those
described in [8], [9] and development tools such as
SimDeveloper [10] have been produced and have been
Simulation Simulation Federate
Module 1 Module 2 1 successfully used in some scenarios. More are available
Federate Federate
2 3 at [9].
NCWare SIM
NCWare
5. Conclusion and further work
NCWare Core NCWare Core
Gateway
DDS DDS DDS HLA
This paper explained the key features of NCWare,
which make it an ideal for interconnecting simulations.
The single API of NCWare allows true independency
of communication system used.
DDS SIMULATION HLA FEDERATION
The paper also showed some example scenarios of
interconnection, as well as some real applications in
which NCWare has been successfully used.
Fig. 6 – Mixed DDS-HLA scenario
Future work should expand the NCWare Gateway
This attachment can be easily done using the NCWare
capabilities, including interoperability with:
Gateway. The main advantages of this connection are
that the driving simulator is still executed at real-time, - standards such as XMPP6, AMQP7 and JMS8
and that the internal data is kept inside the DDS
- synchronous messaging systems based in web
simulation. The gateway is filtering the data and
services (SOAP, REST, XML RPC, etc)
sending only to the HLA federation the relevant data
(for example vehicle type, position and speed). On the - traditional client-server technologies like CORBA
HLA side, only the object ‘Car’ is seen.
- In-memory databases (e.g. Oracle TimesTen)
3.3. Reusing legacy applications on DDS simulations
A third scenario presents the reusability of current
applications. A great effort has been done developing
6
HLA applications. DDS implies a great performance Extensible Messaging and Presence Protocol. See
improvement. Migrating an application to DDS should IETF RFCs 3920, 3921, 3922 and 3923
not imply adapting and rebuilding the application to it. 7
Advanced Message Queuing Protocol. See
Instead of that, the use of the NCWare Gateway can be
www.amqp.org
used to reuse our HLA application in a DDS simulation
8
without having to change a single line. Java Message Service

7. - enterprise messaging middleware like BEA’s
MessageQ™, IBM’s MQSeries™ and another
proprietary pubsub systems like Tibco’s
Rendezvous™
6. References
[1] Alberts, Gartska, Stein: “Network Centric
Warfare” CCRP, 1999.
[2] Dr. Barry Sowerbutts: “Network Enabled
Capability: Delivering Network Enabled
Capability Now”, Roke Manor Research
www.roke.co.uk.
[3] Gerardo Pardo-Castellote, Ph.D. and Brett
Murphy: “New Networking Standard from OMG
Will Simplify Distributed Simulation
Development”.
[4] IEEE Standard for Modeling and Simulation
(M&S) High Level Architecture (HLA)---
Framework and Rules, IEEE Std 1516-2000.
[5] OMG. Data Distribution Service for Real-time
Systems Specification (DDS) 04-12-02, December
2004. Available at www.omg.org.
[6] Rajive Joshi, Ph.D. Gerardo-Pardo Castellote,
Ph.D. A Comparison and Mapping of Data
Distribution Service and High-Level Architecture.
www.rti.com
[7] Gerardo Pardo-Castellote, Ph.D. DDS Spec Outfits
Publish-Subscribe Technology for the GIG
[8] Carlos Rodriguez, Ignacio Seisdedos. MSOE:
Basics for development of Mission and Simulation
Support Software. 06E-SIW-22
[9] Jose María López, Ignacio Seisdedos, Pedro del
Barrio. Using Open software architectures for
developing simulation systems : a Way to the
interoperability of simulations at subsystem level,
September 2008
[10] Fernando Cartechini, SIMDEVELOPER: An M&S
development tool to boost interoperability and
reusability. 06E-SIW-20
[11] Joe Sorroche, Jerry Szulinski. Bandwidth
reduction techniques used in DIS exercises. 04E-
SIW-025
[12] Dan CHEN, Stephen John TURNER, Boon Ping
GAN, Wentong CAI, Malcolm Yoke Hean LOW,
Junhu WEI. Management of Simulation Cloning
in HLA based Distributed Simulations. 04E-SIW-
010