The Live Integration Framework aims to provide a unified view of information stored across heterogeneous data stores by using topic maps to semantically merge the data sources. It allows read-only access to integrated data without modifying the original systems. The framework uses a mapping file to define how data from different sources like relational databases are translated into topic map constructs. It is currently implemented for MySQL integration but the architecture supports integrating other data stores and technologies in the future.
Live Integration Framework Architecture for Merging Heterogeneous Data
1. Live Integration Framework
Live Integration Framework
Christian Haß and Sven Krosse
TopicMapsLab
University of Leipzig
topicmapslab.de
2. Live Integration Framework
The Problem
● Increasing amount of information
● Information stored in various heterogeneous data stores
● Redundancy
● Retrieve all information about a specific subject often difficult
topicmapslab.de
3. Live Integration Framework
Why Topic Maps?
● Not exactly a new idea
● Subject centric presentation
● Merging of different data sources
● Adding additional semantic information
topicmapslab.de
4. Live Integration Framework
Our Goal
● Support various storage technologies
● “Never touch a running system”
● Full read access
● “Live” view on the data
topicmapslab.de
5. Live Integration Framework
Requirements
● Structured data
● Subjects need identities → Problem
topicmapslab.de
6. Live Integration Framework
Architecture
● Implemented as MaJorToM store
● Technology specific implementation
and mapping definition
MaJorToM
Mapping
JLI Store File
Data
Store
topicmapslab.de
7. Live Integration Framework
Architecture
MaJorToM MaJorToM
Mapping Mapping
JLI Store File JLI Store File
Data Data
Store Store
topicmapslab.de
8. Live Integration Framework
Architecture
Virtual Merging Layer
(Hatana)
MaJorToM MaJorToM
Mapping Mapping
JLI Store File JLI Store File
Data Data
Store Store
topicmapslab.de
9. Live Integration Framework
Architecture
Virtual Merging Layer
(Hatana)
MaJorToM MaJorToM
Mapping Mapping
JLI Store File JLI Store File
Data Data
Store Store
topicmapslab.de
10. Live Integration Framework
Current State
● Work in progress
● Implementation only for RDBMS so far
● Only MySQL
● Still some features to implement
topicmapslab.de
11. Live Integration Framework
Current State
● Topics from single tables
● Unique key required
● Attributes to characteristics and Identifier
● Specify queries in mapping
● Associations from any combination of keys
● Specify types, scopes and datatypes
● Association reifier
topicmapslab.de
12. Live Integration Framework
Future Work
● Support other storage technologies
● Improve mapping for RDBMS
● Support different RDBMS types (Postgres, Oracle, etc.)
● Allow limited write access
● Support temporary caching
topicmapslab.de