1. Tailoring the Adaptive Augmented Reality (A²R) Museum Visit:
Identifying Cultural Heritage Professionals Motivations and Needs
The 11th IEEE International Symposium on Mixed and Augmented Reality
Museums and Theater Session
Atlanta, Georgia, USA
Wednesday, November 7th 2012
Dr Areti DAMALA, Dr Nenad STOJANOVIC
Centre d’Etude et de Recherche en Informatique et Communications (CEDRIC) Conservatoire National
des Arts et Métiers (CNAM), Paris, France &
Forschungszentrum Informatik Karlsruhe (FZI), Karlsruhe, Germany

2. How can we improve the AR museum visiting experience ?
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

3. Plan of the presentation
The ARtSENSE project
Adaptive Augmented Reality: Main components
Theoretical background
System Architecture
Design challenges within a Cultural Heritage context
A multi-layered and participatory design approach
Identifying main motivations and needs
Conclusions and directions for future work
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

4. Augmented Reality Supported adaptive and
personalized Experience in a museum based on
processing real-time Sensor Events
European Union co-funded project
36 months duration
www.artsense.eu
See, hear, feel, experience!
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

5. Consortium Partners
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

6. Cultural Heritage Partners
An Art and Design museum
The National Museum of Decorative Arts, Madrid, Spain (MNAD)
A History of Science and Technology museum
The National Museum of the History of Science and Technology,
Paris, France (MAM)
A Contemporary Gallery and New Media Arts Center
Foundation for Art and Creative Technology, Liverpool, UK (FACT)
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

7. From AR to A²R: The ARtSENSE approach
Visual Audio Bio
1. Visual Monitoring and Augmentation
2. Acoustic Monitoring and Augmentation
3. Affective Monitoring and Augmentation
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

8. From AR to A²R: A Theoretical Framework
“Human users need to be treated as centers, immersed in AR
interfaces, rather than as viewers, seeing interfaces or graphical
overlays”
Guidelines
1. Center the human participant
2. Consider the idea of sound as much as the idea of sight
3. Acknowledge that human participants move, think and exist
simultaneously
[Pedersen, 2009]
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

9. From AR to A²R
Adaptive Augmented Reality =
Visual + Acoustic + Affective Monitoring and
Augmentation
Vision bears the heaviest responsibility among all
senses. In a CH context: Linking the signifier with
the signified Stedelijk AR(t): Jan Rothuizen
Fine Arts Museum, Rennes, France Street Museum, Museum of London, UK
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

10. From AR to A²R
Adaptive Augmented Reality = Visual + Acoustic + Affective
Monitoring and Augmentation
Musée des arts et métiers, Paris, France [Kaghat et al., 2011]
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

11. From AR to A²R
Adaptive Augmented Reality = Visual + Acoustic + Affective
Monitoring and Augmentation
Cognitive Engagement
Attention
Knowledge Acquisition
Emotion
Activation/Stimulation
Positive Affect
Negative Affect
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

12. ARtSENSE: System Architecture Overview
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

13. Design Challenges
For the CH professionals
Understand the potential of the technology
Difficult to express needs as AR scenarios
Understand how it works
Apply the AR potential in the design phase
Emerging technologies = inedited design challenges
No established design guidelines
No standardized AR displays
Conceive and create content for the AR application
Evaluate and test the learning materials to be delivered
&
AR and A²R challenges
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

14. Design challenges: Creating the third space…
Questions and issues to resolve
What is the potential of AR for the museum visit?
What is the look and feel of AR displays?
How can A²R can be exploited?
Which are the possible narratives and scenarios that can be
developed?
Early design process, open to:
Technical partners
CH professionals
New Media Artists
Museum visitors
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

15. Design challenges: Creating the third space…
A Multi-layered, participatory design approach
Museum visitors
On site, inobtrusive observations of museum visits
Participatory observations of guided visits
Inobtrusive observations of creative workshops
CH professionals
Brainstorming sessions
Interviews
Testing of several AR displays in the museum environment and gesture interaction
tests
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

16. Design challenges: Creating the third space…
Technical partners
Immersion in the sensitive museum ecology
Collaboration exercises among technical and CH partners
New Media Artists
Collaboration of the Consortium with the International Artists’
collective Manifest.AR
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

17. Rendering visible the invisible
Observing and comprehending artifacts from
multiple viewpoints
MNAD, Spain
FACT, UK
MAM, France
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

18. Identifying motivations and needs: MNAD,
Madrid, Spain
• Museo Nacional de Artes Decorativas, Madrid, Spain
• 3000 objects in the permanent exhibition, ca 30.000 visitors
per year
• Spanish decorative arts: pottery, glass, textiles, carpets,
jewellery, furniture, oriental collections (especially Chinese),
avant-garde european and contemporary design
• Representative of History of Art, Design and Decorative Art
museums
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

19. Motivations and needs: MNAD, Madrid, Spain
1. Observing artefacts from multiple viewpoints
2. Open hidden compartments
3. Contextualize
4. Facilitate the comparison of objects dispersed in different
collections
5. Linking with non exposed objects
6. Linking with other objects of the collection
(Compare “parallels”)
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

20. MNAD Selected Exhibit
Jorge 3D audio kitchen
Late 18th century tiled kitchen from Valencia
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

21. Gaze Tracking
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

22. 3D Audio spatialization
The woman is speaking
directly to me. Her voice
comes from her head.
Position 1 Position 2 Position 3
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

23. Motivations and needs: MAM, Paris,
France
• Musée des arts et métiers, Paris, France
• 4.000 objects displayed, approximately 250.000 visitors per
year
• A University museum
• The French National Museum of the History of Science and
Technology
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

24. Identifying motivations and needs: MAM,
France, Paris
1. Reveal and assist visitors visualize complex physical
phenomena
2. Seeing the machines working again
3. Creating sound-scapes, reconstructing sounds
(intangible cultural heritage)
4. Historical context surrounding the invention
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

25. Activation of Right Hemisphere
Negative affect/avoidance
Activation of Left Hemisphere
Positive affect/approach
MAM Selected Exhibit
The Lavoisier Laboratory and the synthesis of water (1743-1794)
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

26. Motivations and needs: FACT, Liverpool,
UK
• Leading organisation for the
development, support and exhibition
of film, video, new and emerging
media (established in 1989)
• 300.000 entries per year
• Non-existence of a permanent,
physical exhibition
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

27. Motivations and needs: FACT, Liverpool,
UK
1. Encourage different interpretations of the
idea of an artifact
2. Present non-static works (video, narratives,
interactives)
3. Expose inner workings of a building
4. Provide access to “denied” areas
5. Annotation of the architectural building as a
public and social place
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

28. FACT selected artifact
VIP Signature Pillar
Artists and auteurs, filmakers and talks
A growing, ongoing archive
A “hidden” story is revealed that is also the
history of events and programme
FACT history, major events, activities
This rich media content including moving image
Physiological data analysis based on moving image
media experience
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

29. FACT, Liverpool, UK: Artwork commission
to Manifest.AR
1. Implication of the International Artists Collective Manifest.AR
2. Artistic creation and technological innovation
3. A dedicated exhibition opening in June 2013 featuring eight mobile AR
projects
4. More lightweight approach, “technology demistification”
http://manifestarblog.wordpress.com/manifest-ar-fact/
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

30. Conclusions
• From “designing for” to “designing with”
• One step back, two steps forward?
• First impression = life circle of conceptual modeling and
prototyping longer
• Better comprehension of technology = better approach
towards the development of interactive features
• Better comprehension of the needs of CH professionals =
pushing technology to the limits in a meaningful way
• A more enjoyable experience for all project stakeholders
• Museum professionals learning and inspired from each other
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

31. The potential of A²R for Art and Cultural
Heritage
• Visual monitoring and augmentation as important as audio and affective
monitoring and augmentation
• Physiological (bio) sensors: monitoring of the affective impact of the AR
visit
• How to interact once the interaction surface disappears?
• Gaining a better understanding of a visitor’s gaze upon a work of art (eye-
tracking and creation of heat-maps)
• Learning about the physiology of museum visiting
• Content adaptation according to the visitors’ emotions and states
• Can A²R assist us for the shaping of a rewarding museum visit?
• What can we learn about the interrelations among emotion, cognition and
learning?
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

32. Thank you for your attention!
Questions?
www.artsense.eu
ISMAR ARtSENSE Contacts
areti.damala@cnam.fr
nstojano@fzi.de
stojanov@fzi.de
sascha.voth@iosb.fraunhofer.de
Dr Areti Damala, Dr Nenad Stojanovic
Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs