This presentation introduces OpenSplice Cache, a thin layer over OpenSplice DDS that adds traditional <key> caching functionalities.

1. OpenSplice DDS
Delivering Performance, Openness, and Freedom
The Design and Implementation
Angelo Corsaro, Ph.D.
Product Strategy Manager
OMG RTESS and DDS SIG Co-Chair
angelo.corsaro@prismtech.com
of a Data Caching Extension for
the OMG DDS

2. Why Caching?
Agenda
OpenSplice Cache Explained
Standardization
Concluding Remarks

3. Challenges
Three-Tier Systems
‣ The number of clients growing
exponentially due to increased
adoption of e-*
‣ Amount of data that needs to
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
be accessed growing due
increased digitalization
‣ Computing can be scaled-out
adding cheap HW at the Logic-
Tier, but data access does not
scale-out!
Relevance
‣ SaaS,
‣ Cloud Computing
‣ XTP
‣ WWW
© 2009, PrismTech. All Rights Reserved

4. Challenges
Sense and Respond Systems
‣ Relevant state is either stored in a DBMS, or replicated wherever needed
‣ The DBMS approach leads to inevitable bottlenecks
‣ The fully replicated data, if not dealt properly, might induce excessive
resource usage, or worse waste of resources
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Relevance
‣ C2, C4I Systems
‣ ATC, ATM
‣ Vetronics
‣ DCS
© 2009, PrismTech. All Rights Reserved

5. Challenges
High Performance Computing
‣ Huge amount of data (exceeding the
capacity of a single machine)
needed for carrying on a specific
computation
‣ Certain Data-points could be re-
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
computed to save memory, but this
takes time, e.g., re-computing a
financial instrument
‣ Accessing data all at once, or from
a centralized data server does not
scale, or is too expensive
Relevance
‣ Financial Services
‣ Simulations
© 2009, PrismTech. All Rights Reserved

6. The Key Idea & Why it Works
Idea Main Memory
‣ Cache data close to where it is needed and
keep it there for as long as it is needed, or
as long as you can...
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Why it works
‣ Locality of References. Programs tends to L2
reuse data they have used recently
‣ Temporal Locality. Recently accessed data L1
items are likely to be accessed again in the
near future
‣ Spatial Locality. Items whose addresses
are close to each other tend to be
referenced close together in time
© 2009, PrismTech. All Rights Reserved

7. Cache
Webster Definition (1976)
‣ a safe place for hiding or storing things.
Webster Definition (2008)
‣ (a): a hiding place especially for concealing and preserving provisions or
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
implements (b): a secure place of storage.
‣ something hidden or stored in a cache.
‣ a computer memory with very short access time used for storage of
frequently or recently used instructions or data —called also cache
memory.
Etymology (1797)
‣ French, from cacher to press, hide, from Vulgar Latin coacticare to press
together, from Latin coactare.
© 2009, PrismTech. All Rights Reserved

8. Why Caching?
Agenda
OpenSplice Cache Explained
Standardization
Concluding Remarks

9. OpenSplice Cache in Brief
‣ Fully Distributed Caching OpenSplice Cache
Technology
‣
Key Value
Supports SQL Queries on OpenSplice Cache Key
Key
Value
Disk
Value
OpenSplice Cache
local Cache Key Value
Key Value
Key Value
‣
Key Value
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Key Value
Support for Continuous Key
Key
Disk
Value
Value
Key
Disk
Value
Key Value
SQL Queries on the whole
distributed Cache
‣
OpenSplice DDS
High-Performance, ultra-
low-latency OpenSplice Cache
OpenSplice Cache
communication engine
Key Value
based on OpenSplice DDS Key Value Key Value
Disk
‣
Key Value Key Value
Disk
Fully exploits Data Locality Key
Key
Value
Value
Key Value
© 2009, PrismTech. All Rights Reserved

10. OpenSplice Cache Architecture
‣ Fully exploits the
power of OpenSplice Application
MDE
DDS Real-Time
Pub/Sub for Tuner Caching API
maintaining cache PowerTools OpenSplice Cache
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
coherence
Content
‣ Exposes a simple
DDS v1.2
Ownership Persistence
Subscription
DBMS
API for storing and Minimum Profile
Connectors
retrieving items to/ Real-Time Pub/Sub(DCPS)
from the cache
DDSI v2.1
‣ Can be used by
SOAP Real-Time Interoperable Wire
Secure Networking
Networking Protocol (DDSI)
applications in Connectors Networking Technology
conjunction with UDP/IP
OpenSplice DDS
© 2009, PrismTech. All Rights Reserved

11. OpenSplice Cache Core API
‣ Simple and concise core API that keeps simple most commonly used operations
‣ Extended API providing
‣ Rendezvous
‣ Advanced Cache Configuration
‣ Cached Data QoS
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Cache Replacement Strategy Customization
OpenSplice Cache
Write
Queries
+ write(key, value) Key Value
Key Value + query(sql): view
Read Disk
Key Value + add_continuous_query(sql,
Key Value listener)
+ read(key): value
+ take(key): value
Notifications
+ add_miss_handler(handler)
+ add_insert_handler(handler)
© 2009, PrismTech. All Rights Reserved

12. A Performance Driven Architecture
© 2008, PrismTech. All Rights Reserved

13. Performances Driven Architecture
‣ Cache performances are summarized by the following equation:
Average Data Access = HitTime + MissRate x MissPenalty
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Thus to optimize the Cache performances it is necessary to
‣ Reduce Hit Time
‣ Reduce Miss Rate
‣ Reduce Miss Penalty
OpenSplice Cache has been designed to ensure that all these
factors are optimized
© 2009, PrismTech. All Rights Reserved

14. Hit Time Minimization
‣ The hit-time is the time taken to access data when it is in the cache.
‣ OpenSplice Cache has two different hit time, one local and one remote,
depending on wether the data is locally available, or available somewhere on
the network
Local Hit-Time
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Accessing an object locally is a OpenSplice Cache
matter of micro-seconds
Remote Hit-Time
‣ OpenSplice Cache, relies on Read Key Value
OpenSplice DDS to resolve remotely Key Value
a hit. Disk
+ read(key): value Key Value
‣ As such, this hit time, is close to a + take(key): value Key Value
message roundtrip on OpenSplice
DDS, which depending on the size is
measured in tens of microseconds
© 2009, PrismTech. All Rights Reserved

15. Local and Remote Hits
OpenSplice Cache
OpenSplice Cache
OpenSplice Cache Disk
Local Hit
read(‘a’): ‘alpha’ ‘a’ ‘alpha’
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Disk
Disk
OpenSplice DDS
OpenSplice Cache
OpenSplice Cache
‘b’ ‘beta’
Disk
Disk
© 2009, PrismTech. All Rights Reserved

16. Local and Remote Hits
OpenSplice Cache
OpenSplice Cache
OpenSplice Cache Disk
Local Hit
read(‘a’): ‘alpha’ ‘a’ ‘alpha’
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Disk
Remote Hit Disk
read(‘b’): ‘beta’
OpenSplice DDS
OpenSplice Cache
OpenSplice Cache
‘b’ ‘beta’
Disk
Disk
© 2009, PrismTech. All Rights Reserved

17. Local and Remote Hits
OpenSplice Cache
OpenSplice Cache
OpenSplice Cache Disk
Local Hit
read(‘a’): ‘alpha’ ‘a’ ‘alpha’
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Disk
Remote Hit Disk
read(‘b’): ‘beta’
Miss
read(‘c’): miss handler OpenSplice DDS
OpenSplice Cache
OpenSplice Cache
‘b’ ‘beta’
Disk
Disk
© 2009, PrismTech. All Rights Reserved

18. Miss Rate Minimization
Cache misses can be classified in:
‣ Compulsory. The first access to a data item not in the cache. These are
also called Cold Start Misses or First Reference Misses.
‣ Capacity. If the cache cannot contain all the data needed by an
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
application during its execution, capacity miss will occur because of
blocks being discarded and later retrieved.
‣ Conflict. Depending on how the cache is organized internally, some data
items might compete for resources, and lead to replacement
OpenSplice Cache minimize architecturally all of these misses.
© 2009, PrismTech. All Rights Reserved

19. Pre-Fetching &
‣ Compulsory Misses can be reduced OpenSplice Cache
by trying to guess what an application
will use next
‣ Temporal and Spatial correlation Key Value
among the components of a Working Set
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Key Value
Disk
distributed system can help in Prefetche Key Value
predicting the use d Data Key Value
‣ OpenSplice Cache, exploits unused
space in a local cache instance to
eagerly pre-load data items -- thus
reducing compulsory misses
‣ An API is also provided to the
application to influence and customize
the pre-fetching
© 2009, PrismTech. All Rights Reserved

20. Pre-Fetching
OpenSplice Cache
OpenSplice Cache
OpenSplice Cache Disk
write(‘s’): ‘sigma’ ‘a’ ‘alpha’
‘s’ ‘sigma’
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Disk
Disk
OpenSplice DDS
OpenSplice Cache
OpenSplice Cache
‘b’ ‘beta’
Disk
Disk
© 2009, PrismTech. All Rights Reserved

21. Pre-Fetching
OpenSplice Cache
OpenSplice Cache
OpenSplice Cache Disk
‘s’ ‘sigma’
write(‘s’): ‘sigma’ ‘a’ ‘alpha’
‘s’ ‘sigma’
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Disk
Disk
‘s’ ‘sigma’
OpenSplice DDS
OpenSplice Cache
OpenSplice Cache
‘b’ ‘beta’
Disk
Disk
‘s’ ‘sigma’
‘s’ ‘sigma’
© 2009, PrismTech. All Rights Reserved

22. Pre-Fetching
OpenSplice Cache
OpenSplice Cache
OpenSplice Cache Disk
‘s’ ‘sigma’
write(‘s’): ‘sigma’ ‘a’ ‘alpha’
‘s’ ‘sigma’
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Disk
Disk
‘s’ ‘sigma’
OpenSplice DDS
OpenSplice Cache
OpenSplice Cache
‘b’ ‘beta’
Disk
Disk
‘s’ ‘sigma’
‘s’ ‘sigma’
Local Hit
read(‘s’): ‘sigma’
© 2009, PrismTech. All Rights Reserved

23. Remote Cache Hit & Capacity Miss
‣ We’ve seen that OpenSplice Cache OpenSplice Cache
supports the concept of local and
remote hits
Read Key Value
‣ When a resource constrained Key Value
Disk
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
cannot allocate the whole memory + read(key): value Key Value
+ take(key): value Key Value
needed for its working set locally, it
can still exploit caching done in
more powerful nodes to improve
its over all performance
‣ Thus, OpenSplice Cache through
exploiting resources available,
somewhere else, it is practically
possible to limit, if not eliminate the
capacity miss
© 2009, PrismTech. All Rights Reserved

24. Cache Organization & Conflict Misses
‣ By default OpenSplice Cache behaves as a fully OpenSplice Cache
associative Cache managed by an LRU
replacement policy
Key Value
‣ The default choice is motivated by the optimality of Key Value
Disk
LRU as a replacement strategy
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Key Value
Key Value
‣ An API is provided to customize the replacement
strategy so to fit a specific application need/
behavior
‣ This allows for OpenSplice Cache to always
minimize the conflict misses, and ensure minimal
cache miss rate -- thus best performance
© 2009, PrismTech. All Rights Reserved

25. Putting it all Together
OpenSplice Cache
LRU
Key Value
OpenSplice Cache
OpenSplice Cache Key Value
Disk
LRU
Key Value Assoc.
Key Value
Key Value
Key Value
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Working Set Key Value
Key Value Disk
Disk
Key Value
Key Value
Prefetched Data Key Value
Key Value
OpenSplice DDS
OpenSplice Cache
OpenSplice Cache
Custom
Random
Key Value
Key Value Key Value
Disk
Key Value Key Value
Disk
Key Value Key Value
Key Value
© 2009, PrismTech. All Rights Reserved

26. Architectural Styles
© 2008, PrismTech. All Rights Reserved

27. Symmetric
‣ All nodes have similar
OpenSplice Cache
resources
‣ Caches instances on all Key Value
OpenSplice Cache
nodes have similar sizes OpenSplice Cache Key
Key
Value
Disk
Value
‣ The resulting system is really
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Key Value
Key Value
Key Value
peer-to-peer, and every node, Key Value
Disk
Key
Key
Value
Disk
Value
Key Value
can potentially serve remote Key Value
Key Value
hits
OpenSplice DDS
OpenSplice Cache OpenSplice Cache
Key Value
Key Value Key Value
Disk
Key Value Key Value
Disk
Key Value Key Value
Key Value
© 2009, PrismTech. All Rights Reserved

28. Asymmetric
‣ Some nodes have very Load-Balanced/Partitioned
Super Caches
limited resources
‣ As a consequence
OpenSplice Cache
OpenSplice Cache
Cache instances have
Key Value
OpenSplice Cache
different sized on Key Value Key Value
OpenSplice Cache
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Key Value Key Value
various nodes Key Value Key
Key
Disk
Value
Value
Key
Key
Value
Disk
Value
‣ The system, will rely on
Key Value
Disk
Key Value Key Value
Key Value
Key Value
Key Value Key Value
a set of “super-caches” Key Value
Key
Key
Value
Value
Disk
to resolved remote-hits OpenSplice Cache
Key
Key
Value
Value
‣ Super-Caches can
Key Value
Key Value
Key Value
OpenSplice DDS
Disk
exploit partitioning as Key Value OpenSplice Cache
well as load-balancing OpenSplice Cache
to ensure scalability OpenSplice Cache
Key
Key
Value
Value
Key Value
Disk Key Value
Key Value Key Value
Disk
Key Value
Key Value Key Value
Key Value
Disk Key Value
Key Value Key Value
Key Value
© 2009, PrismTech. All Rights Reserved

29. Why Caching?
Agenda
OpenSplice Cache Explained
Standardization
Concluding Remarks

30. Standardization
‣ No formal Caching Technology standard exist to date
‣ JCache is a JSR, but there not seem to be real interested
toward converging over a standard
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Caching functionalities extend DDS in a very useful way
‣ Possible Standardization venue:
‣ Standardize a Caching API that builds over DDS
‣ Extend the DDS QoS to be able to have “Cache Semantics”
for Subscriber/Reader or Topics
© 2009, PrismTech. All Rights Reserved

31. Why Caching?
Agenda
OpenSplice Cache Explained
Standardization
Concluding Remarks

32. Concluding Remarks
‣ OpenSplice Cache is an innovative caching technology whose
architecture ensures maximum performance by construction
‣ It leverages OpenSplice DDS high performance messaging
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
middleware for implementing its distributed consistency protocol,
and deliver ultra-low hit-times
‣ Architecturally supports equally well symmetric system
deployments as well as highly asymmetric systems, facilitating the
creation of load-balanced/partitioned super caches
OpenSplice Cache provides best performance and
architectural flexibility!
© 2009, PrismTech. All Rights Reserved

33. Online Resources
http://www.opensplice.com/
http://twitter.com/acorsaro/
emailto:openslicedds@prismtech.com
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
http://bit.ly/1Sreg http://dds4u.blogspot.com
http://www.dds-forum.org
http://www.youtube.com/OpenSpliceTube http://portals.omg.org/dds