Keynote given at RCIS 2019, 13th International Conference on Research Challenges in Information Science, Brussels, Belgium, May 2019 ABSTRACT: The efficient management and retrieval of information via dedicated devices and data structures has been investigated since the early days of Vannevar Bush's seminal article As We May Think introducing the Memex. However, nowadays information is usually fragmented across different media types, devices as well as digital and physical environments, and we are often struggling to retrieve specific information. We discuss three main issues to be addressed when developing solutions for managing information in these co-called cross-media information spaces. First, we have a look at an extensible cross-media linking solution based on the resource-selector-link (RSL) hypermedia metamodel where information can be integrated across applications, devices as well as digital and physical information environments. We then outline some of the limitations of existing digital document formats which are often just a simulation of paper documents and their affordances on desktop computers, and discuss more flexible document representations for cross-media information spaces. Further, new forms of human-information interaction and cross-media user interfaces—including some recent work on dynamic data physicalisation—are discussed. A number of research artefacts are used to illustrate different aspects of the presented data-centric approach for cross-media information spaces and architectures. Last but not least, we provide an outlook on how the embedding of the presented concepts at the level of an operating system might ultimately lead to new possibilities for cross-media information management and innovative forms of human-information interaction. Paper: https://www.academia.edu/40139780/Towards_Cross_Media_Information_Spaces_and_Architectures

1. 2 December 2005
Towards Cross-Media Information Spaces
and Architectures
Prof. Beat Signer
beatsigner.com
Web & Information Systems Engineering Lab
Department of Computer Science
Vrije Universiteit Brussel

2. "As We May Think" (1945)
 Memex
 memory extender
 information retrieval
 associative trails
 trail blazers
information overload

3. 01001000100
0101001100010010100010010
0100100010010001000100001001000
100010001100001001000110010101010110
1100010010100010010010010100010101011001
101001100001100 101010100
0010100 100 110101010101
00100 01010101000
1001 1111111111
111 0000000010
0 11111111
0101001011
010101010
111010101
001010010110
Cross-Media Information Spaces

4. Fragmentation of information
across different applications,
devices and digital & physical
information environments

5. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

6. Data-centric approach for
cross-media information
solutions based on a
hypermedia metamodel

7. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

8. RSL Metamodel and Link Service
Beat Signer and Moira C. Norrie,
As We May Link: A General Metamodel
for Hypermedia Systems, Proceedings of
ER 2007, November 2007

9. EdFest: Paper-Digital Guide
Global Information Systems Group
ETH Zurich

10. PimVis: Personal Information Management
Sandra Trullemans and Audrey Sanctorum

11. PimVis Document View
Audrey Sanctorum

12. PimVis Focus View
Audrey Sanctorum

13. Open Cross-Media Linking
local visual plug-ins
external visual plug-ins
communicate
Cross-Media Link Browser & Editor
Ahmed A.O. Tayeh

14. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

15. What is Wrong with Digital Documents?
Most people don't understand the logic of the concept:
"What You See Is What You Get" is based on printing
the document out ("get" means "get WHEN YOU
PRINT IT OUT"). And that means a metaphysical shift:
a document can only consist of what can be printed!
[...] No overlays [...] – PAPER UNDER GLASS.
Ted Nelson, Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009
Existing document formats often based on
the simulation of paper affordances
on desktop computers

16. What is Wrong with Slideware?
 Simulation of physical slides
 limited space due to slide concept
 linear navigation
 monolithic documents
 difficult to reuse content and embed rich media types

17. MindXpres Presentation Platform
 Flexible representation
of presentations
 use of structural RSL links
 separation of content and
structure
 Extensible platform
 content-based approach
 cross-media content reuse
 non-linear navigation and
zoomable user interface
 associative linking
 rich media types
Reinout Roels

18. MindXpres Document Representation

19. video

20. Different context-dependent
adaptive overlay structures on top
of (parts of) content
Clear separation between content
and structure enables cross-media
content reuse (transclusion)

21. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

22. PimVis: Paper-Digital User Interface
Sandra Trullemans and Audrey Sanctorum

23. ArtVis: Tangible User Interface
Bram Moerman

24. Real-Time Cross-Media Data Exploration
dynamic data
physicalisation

25. Adaptive Distributed Hybrid User Interfaces
Audrey Sanctorum and Beat Signer,
A Unifying Reference Framework and
Model for Adaptive Distributed Hybrid
User Interfaces, Proceedings of
RCIS 2019, May 2019
Audrey Sanctorum
eSPACE reference
framework and model

26. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

27. Unified Cross-Media Information Spaces

28. Common underlying
representation of cross-media
links and structures via
RSL hypermedia metamodel

29. "Operating System" of the Future
RSL-based Hypermedia Engine
…
Legacy Applications
File System Driver
Hypermedia-enabled Applications
RCIS2019.docx
RCIS2019.docx

30. References
 B. Signer, Fundamental Concepts for
Interactive Paper and Cross-Media Information
Spaces, Second Edition, ISBN 978-3-8370-2713-6,
August 2017
 B. Signer and M.C. Norrie, As We May Link: A General
Metamodel for Hypermedia Systems, Proceedings of ER
2007, 26th International Conference on Conceptual
Modeling, Auckland, New Zealand, November 2007
 http://beatsigner.com/publications/signer_ER2007.pdf

31. References ...
 B. Signer, What is Wrong with Digital
Documents? A Conceptual Model for Structural Cross-
Media Content Composition and Reuse, Proceedings of
the 29th International Conference on Conceptual Mod-
eling (ER 2010), Vancouver, Canada, November 2010
 http://beatsigner.com/publications/signer_ER2010.pdf
 R. Roels and B. Signer, A Conceptual Framework and
Content Model for Next Generation Presentation
Solutions, Proceedings of the ACM on Human-Computer
Interaction (PACMHCI), 2019
 http://beatsigner.com/publications/sanctorum_RCIS2019.pdf

32. References ...
 S. Trullemans, A. Sanctorum and B. Signer,
PimVis: Exploring and Re-finding Documents in Cross-
Media Information Spaces, Proceedings of AVI 2016,
International Working Conference on Advanced Visual
Interfaces, Bari, Italy, June 2016
 http://beatsigner.com/publications/trullemans_AVI2016.pdf
 B. Dumas, B. Moerman, S. Trullemans and B. Signer,
ArtVis: Combining Advanced Visualisation and Tangible
Interaction for the Exploration, Analysis and Browsing of
Digital Artwork Collections, Proceedings of AVI 2014,
Como, Italy, May 2014
 http://beatsigner.com/publications/dumas_AVI2014.pdf

33. References ...
 B. Signer and T.J. Curtin, Tangible Holograms:
Towards Mobile Physical Augmentation of Virtual
Objects, Technical Report WISE Lab, WISE-2017-01,
March 2017
 http://beatsigner.com/publications/signer_arXiv2017.pdf
 A.A.O Tayeh and B. Signer, A Dynamically Extensible
Open Cross-Document Link Service, Proceedings of
WISE 2015, 16th International Conference on Web
Information Systems Engineering, Miami, USA,
November, 2015
 http://beatsigner.com/publications/tayeh_WISE2015.pdf

34. References ...
 A. Sanctorum and B. Signer, A Unifying Reference
Framework and Model for Adaptive Distributed Hybrid
User Interfaces, Proceedings of RCIS 2019, 13th
International Conference on Research Challenges in
Information Science, Brussels, Belgium, May 2019
 https://beatsigner.com/publications/sanctorum_RCIS2019.pdf