Designing Interactions at the Nexus of the Physical and Virtual World

Ars Electronica Futurelab Research & Innovation Group

Definition of what I am NOT talking about

com·put·er game (plural com·put·er games)

noun

Definition:

game played on computer: a game in the form of computer software, run
on a personal computer or games machine and played by one or more
people using a keyboard, mouse, control pad, or joystick.

Microsoft Encarta


Most Computer games nevertheless …

… use a handful of standardized interfaces

... are build on a very limited number of concepts

… ignore context and environment


The reason is obvious

This is the way how to sell things efficiently.


But this is what you might miss …


From Immersion …

… to Transmersion?
Transmersion?

Not about making interfaces invisible, but about
transforming the interface itself to the main element
for the player’s connection to the physical and social
environment


An alternative, preliminary Definition of Computer Game …

A computer game is structured, computer-based
playing with goals rules challenge and interaction
goals, rules, challenge,
as key components.

(work in progress)


Shadowgram
Interactive Workshop System 2010

A person stands in front of an illuminated screen and is
photographed. The result is a shadow-image in which a human
silhouette is all that can be seen. This picture is then printed out as a
miniature sticker and applied to a “map” whose topography has been
derived from a cluster of topics like education, environment, health
system and society, whereby the body language of the silhouette, a
speech balloon containing a brief statement, and the respective
thematic cluster coalesce into a message.

Shadowgram Creative Catalyst for Social Brainstorming by Shadow Ars Electronica Futurelab Research & Innovation Group

For instance: the person in front of the illuminated screen covers his/her ears with his/her hands while being
photographed. This silhouette is then allocated to the environment cluster and a speech balloon containing a remark
about noise pollution is added to it. The upshot is a highly creative, graphic mode of “social brainstorming” and thus a
means of identifying problems and bringing out potential solutions.

"Shadowgram" is a work by the Ars Electronica Futurelab (AT).


“Robotinity” is a portmanteau word that blends robotics and humanity and thus succinctly expresses the new RoboLab’s central
theme: emblematic examples culled from fields of art, design and science illustrate just how closely human beings and machines are
already living and working together. These highly charged facts & circumstances will serve up a foretaste of the risks and rewards our
shared future just might have in store…

ROBOTINITY Exhibition Formulation of Social Contextual Ars Electronica Futurelab Research & Innovation Group
Robot Studio

myBOT workshop

“myBOT” is an entertaining way to play with identities. The first step is to take portrait photos. Next, you use a simple program to
process them into a short stop-motion animation sequence. Then you select just the right robot body from a gallery of options and
specify its behavior. The result is a customized myBOT that might be funny or serious, offbeat or nerdy. “myBOT” enables you to
design a robot in a way that’s fun and easy.

MyBOT Make your bot!

Outline of this talk

(St)age
of
Futurelab Gulliver
Fab Lab Partici-
World pation
Ars Research
Electronica Questions

Introduction Case Study Future Work

Back- Q&A
ground
Deep
Geocity
Intention Space CADET

t0

Background

Ars Electronica Timeline ART – TECHNOLOGY – SOCIETY
Ars Electronica assembles and configures the commentary, concepts and
visions of our modern, information-based society, whereby currently
prevailing facts & circumstances are always taken as points of departure
Festival Ars Electronica (since 1979) for speculation about where we’re headed.
Festival for Art, Technology and Society
In short: future manifestations that are presently in the process of
Prix Ars Electronica (since 1987)
emerging are what Ars Electronica is interested in. Attention is never
international Competition for Cyberarts
focused singularly on art, technology or society, but rather on their
complex metamorphoses and interrelationships.
Ars Electronica Center (since 1996)
Museum of the Future

Ars Electronica Futurelab
Laboratory for Future Innovations

New Ars Electronica Center
2004 2005 2007 2009 Extension to 6500m², Opening Jan. 2009

1979 1987 1996
Ars Electronica Gallery in Vienna (Vienna Museumsquarter)

Ars Electronica international Ludwig Boltzmann Research Institute
for Media Art Research and Digital Archives


Center
Ars Electronica is a company
owned by Linz City and an
internationally unique
platform for digital art an
media culture, consisting of Futurelab
four division:

Festival

Prix

The annual budget of Ars Electronica: Is this an institute of Linz City?
balance sheet total 2009: 14,4 Mio € Ars Electronica Linz GmbH (=limited company) is 100% owned by the City of
Funding : 60 % subsidies Linz
22 % own income
18 % 3rd party projects Is AEC managed by tax?
Founding consists of subsidies (mainly City of Linz and Region Upper
Number of employee: Austria), own income (entrance fee museum, festival tickets) and 3rd party
76 full time employee projects developed by Ars Electronica Futurelab
96 part time employee
sum: 116 full time employee equivalent Is this an independent organization?
Ars Electronica is owned by the city but independent within it’s mission
Partner and past examples as a collaborative research statement
Siemens CT, SAP AG , Vodafone Group R&D, KANSAI
TV, HONDA R&D, HAKUHODO, … Ars Electronica Futurelab Research & Innovation Group

Futurelab: A Visual Image for a Research System

Business
FL Production Team
(Interface and Interaction Design, Information
Design, Advanced Sensing Technologies,
Network & Database Technologies etc…)

Experience Design
Media Architecture
Media Performance
Projects

FL Research & Innovation Group
Innovative Research

Basic Research
FL Fundamental Research

Futurelab is one department in Ars Electronica, for conducting research and development. In this collaborative
project, FL Research & Innovation Group will be conducting the research, with working FL production team.

Research Topics of Research & Innovation Group

Persuasive
Technology

Interaction Creative Functional
Ecology Catalyst Aesthetic

ROBOTINITY

The Research & Innovation Group focuses on key issues for socialization of media and technologies.

Right now, we have five research topics and all topics formulate international research hubs in each topic.
These five topics are flexibly connecting and generating fusion models to coin new concepts, through actual working models.


Interaction Ecology

“How to Design “Interaction” in Embodied Media Environment”
Overview
This research topic aims for a holistic and analytic approach to Interaction Design.

Traditional User Centered Design commonly focuses on single user interactions between the user and an interface. To deal
with today’s increasingly complex systems, where centralized control over information on a variety of arbitrary interfaces in
varying detail is essential, this is not enough.

It is inevitable to find new cognitive models and metaphors to get along in these ecologies. They constitute themselves as a
functional set of artifacts, people and the surrounding environment and the rich interaction in between. We identify these as
"Interaction Ecologies".

Research Records and Partners
Vodafone, Siemens, SAP, Microsoft, HP, BMW, CADET (Austrian Governmental Fund)

Functional Aesthetic

“How to Design New Communication Integrating Functions and New Materials"
Overview
This research topic focuses on the new possibilities of functions produced by new materials. Now, media technologies are
intergrated to our daily lives and it needs natural and organic user interfaces.
In this research topic, we search for new forms of organic communication generated by new materials.

Elekit, Johannes Kepler University Linz SWITCH

Robotinity

“How to Coexist with Robots in the Future”
Overview
Robotinity is a study how people coexist with robot. Depending on cultural backgrounds, the meaning of robot and technology
is totally different. What's the robot? What is our humanity? Both has to be considered when we design a robot.

In this research topic, we aim to search for factors how people coexist with robot.

Honda R&D, Murata Manufacturing Company, ATR, Osaka University
Meet ASIMO in Deep Space

Creative Catalyst

“How to Create a Medium and Place for Prompting Creativities”
Overview
People are creative. However, many things are hidden in the world.
How media technologies can prompt new creativities for people? In this research topic, we research methodologies and tools
to enhance creativities.

Hakuhodo, Kansai TV, Smart Technologies, STUDIOLAB (EU-FP7)
Shadowgram

Persuasive Technology

“How to Coexist with Technologies for Sustainable Society and Culture
Overview
It is dedicated to the application-oriented research and development of persuasive technologies and Ambient Devices, aimed
at raising awareness of environmentally sustainable practices at individual and societal level. Persuasion in the psychological
processes are studied believe that lead the addressee to setting changes and resulted in tangible changes in behavior. With
expertise regarding this and may also need social change processes in a positive direction can be controlled.
In this research, we search for social responsibilities how new media is developed for keeping sustainable society.
analysis and personalized system adaptations.

Johannes Kepler University Linz

Outline of this talk

(St)age
of
Futurelab Gulliver
Fab Lab Partici-
World pation
Ars Research
Electronica Questions

Introduction Case Study Future Work

Back- Q&A
ground
Deep
Geocity
Intention Space CADET


Case Study: Gullivers World

Hidden Gullivers
Worlds Box

Gullivers
World

Geocity Fab Lab

Deep
Space


Research Topics

• Digital content creation
• Scene creation and non-linear story telling
• Playful and metaphoric Interaction Design
• Interface and tracking technologies
• Virtual / mixed reality installations
• Collaborative Environment
• Exhibition Design


Constraints and Demands

• Visitors time: 10 to 20 minutes
• Workshops with 60+ minutes
• Single user as well as
• Groups of up to 15+
• 70.000+ visitors / year
• Visitors age: 3 to 90+
• Visitors knowledge: novice to expert
• Integrate topics of broader artistic, social and/or
technological concern


Hidden Worlds of Sound
and Voice (2002)

An interactive audiovisual installation whose central
theme is the magical relationship of speech to the ethereal
medium which conveys it. Participants in this exhibit are
able to "see" each others' voices, made visible in the form
of animated graphic figurations that appear to emerge
from the participants' mouths.

Golan Levin, Zachary Lieberman and Ars Electronica
Futurelab


Gulliver’s Box (2003)

This installation represents the effort to pursue new
approaches how to deal with Mixed Reality content. The
challenge at the core of this project was to position an
innovative medium somewhere between theater, film and
installation. The result is an infrastructure that offers
artists new opportunities to convey audiovisual
information, and one that ought to encourage creatives in
every discipline to work with these new approaches.
Adrian Cheok, Hirokazu Kato and Ars Electronica Futurelab


Gulliver’s Box: Interaction Design


Gulliver’s Box: visitor generated content


Gulliver’s World (2004)

Motivated by the success of Gulliver’s Box, Futurelab
took another long look at the concept and expanded it
in several directions. Important new features designed
into Gulliver’s World are that users are no longer
limited to preset environments and characters;
instead, they are called upon to design the artificial
world and its components themselves. This was
accomplished by the development of intuitive editors
with which the environment can be totally revised and
customized anew each time.
Hirokazu Kato and Ars Electronica Futurelab


Gulliver’s World: Character Design


Gulliver’s World: Interface Design


Gulliver’s World: Introduction Video


Data Distribution and Synchronisation
Gulliver’s World: complex interplay of different installations


Gulliver’s World: “Beholder” as central world simulator


Gulliver’s World: Clay workshop and 3d Scanner


Way from the physical into the virtual world: 12 reference pictures


Way from the physical into the virtual world: separation and calibration


Way from the physical into the virtual world: Visual Hull reconstruction


Way from the physical into the virtual world: Texture recreation


Gulliver’s World: Rotation Editor


Gulliver’s World: Object carriers as physical representation for digital content


Gulliver’s World: Paint terrains and stamp objects


Gulliver’s World: Jigsaw puzzle metaphor for the last missing piece of a world


Gulliver’s World: Greenbox for video recordings of visitors


Gulliver’s World: A transparent box to carry the recorded data inside


Gulliver’s World: Character Editor


Gulliver’s World: Character modification influences abilities, behavior and storyline


Gulliver’s World: The World stage is the central element of the installation


Gulliver’s World: Collaborative virtual environment


Gulliver’s World: Motion analysis influences characters


Gulliver’s World: Stereoscopic Tracking of Boxes


Gulliver’s World: Infrared reflective markers


Gulliver’s World: Exploration through virtual expedition


Gulliver’s World: The shelf reveals the content of the box


Gulliver’s World: Pictures from the virtual world can be sent home


City Puzzle (2006)

An interactive simulation environment visualizes
technological approaches that city planners and
architects of the future will be working with. This
installation is an expanded spin-off of “Gulliver’s
World,” the Ars Electronica Center’s mixed reality
environment, and uses a simple urban planning
model as an example illustrating concrete
application possibilities. Visitors can manipulate
the various scenarios with haptic tools.
Ars Electronica Futurelab


City Puzzle: Characters become buildings


City Puzzle: By placing the characters the visitor starts to build a growing city


City Puzzle: The city comes to life


Gulliver’s World: Additional and improved haptic interfaces


City Puzzle: New interfaces allow the exploration of the virtual world

Hidden Gullivers
Worlds Box

Gullivers
World
Geocity FabLab

Deep
Space


Imagine a future in which you no longer buy gym shoes in a store; instead, you print them out at home, use them, and then toss them
into the recycling bin when you’re done. What now seems like a weird futuristic vision could soon be reality. The idea behind FabLab
(from fabrication) is the capability of downloading objects from the internet and printing them out at home. And not just on paper; as
real stuff! A 3D printer uses a computer model to generate an actual object, and a computer-controlled laser cutter does the high-
precision shaping of any chosen material. Using the internet makes it possible to disseminate designs for articles of clothing or
pieces of furniture just as easily as sharing music and pictures today. What is indeed unimaginable is this technology’s impact on
established industries.

FABLAB AEC Do it your self in the future

The Gallery as central element


Making digital objects physical


Making physical objects digital


Create Digital Content


Workshop and Lectures with Artists


Print your personal iPhone cover


Using LASER cut silhouettes …


… to create little phantasy worlds


Create your plushy toys


Hidden Gullivers
Worlds Box

Gullivers
World
FabLab Geocity

Deep
Space


It’s estimated that planet Earth will have nearly 10 billion inhabitants in 2050, more than two-
thirds of them living in cities. All over the world, metropolitan areas are exploding into new
megacities. Homo sapiens is evolving into homo urbanicus, a being whose hopes—and, all too
often, disappointments—are connected more closely than ever before to the course of “life in the
big city” … “GeoCity” examines global developments and their local impact

GEO CITY Discursive and participatory urban Information Environment

Linz, Upper Austria


Three important Locations

Ars Electronica Center
New City Hall

Old City Hall


New City Hall


1:2000 scale model


Old City Hall


Changing Linz


Interactive surface, graspable physical objects


Designing the environment


SimLinz


Panorama Projection


Digital Booklet & Pen

Booklet & pen

Anoto Technology


Information map


Data Selection


Interactive Table Surface


Discursive Environment


Ongoing Development

SimLinz mobile, 2010


Urfahraner Markt


Life Signs: While your heart beats


Sim Linz


We are Linz


Shadowgram


Wall topography for Linz


Hidden Gullivers
Worlds Box

Gullivers
World
Geocity Deep Space

FabLab


CAVE technology was yesterday. Deep Space is right now. Experience a new
dimension of travel through space and time. Immerse yourself into breathtaking 3D
stereo universes featuring jumbo-format, high-definition images that will
absolutely blow you away!

DEEP SPACE Huge Immersive Environment

Playing Games


Exploring the Universe


Concerts, 3d visualized


Give Lectures


Conduct Research

(August 31, 2010) Honda’s humanoid robot ASIMO will make its public debut in Austria at this year's Ars Electronica Festival.
September 2-8 in Linz, festivalgoers will have the opportunity to experience an impressive demonstration of ASIMO’s extraordinary
capabilities at Deep Space in the Ars Electronica Center. Research & Innovation Group collaborates with Honda R&D for the future
vision of humanoid robot.

ASIMO Project Social Experiment in Deep Space

Game based Interaction Experiments


Multiple Participants interacting with Asimo simultaniously


Triggering “Soundballs” by stepping into Visualization


Tracking systems upgrade


Multicamera System


Stereoscopic Depth Estimation


Playful research that doesn’t just aim to entertain is what the Center for Advances in Digital
Entertainment Technologies is all about. The mission: take the latest breakthroughs in full-
body & gesture tracking, bio-signal sensor systems and 3D technology and utilize them in
projects beyond the realm of the computer game sector.
Ars Electronica Futurelab and the Salzburg University of Applied Sciences’ Department of
Multimedia Technology and Multimedia Art have launched this joint research effort, funded by
the Austrian FFG.

CADET Center for Advances in Digital Entertainment Technologies Ars Electronica Futurelab Research & Innovation Group

The Ars Electronica Futurelab is the only non-university R&D facility to be granted funding
this year by the FWF–Austrian Science Fund’s PEEK arts development program. Beginning in
2011, the Futurelab will collaborate with renowned media artist, choreographer and composer
Klaus Obermaier (AT) on a three-year project that aims to develop interactive dramatic
performances.

PEEK (St)age of Participation Ars Electronica Futurelab Research & Innovation Group

Questions


Contact Address

Roland Haring

Ars-Electronica-Straße 1
4040 Linz, Austria
rolandha@aec.at
research.aec.at


Designing Interactions at the Nexus of the Physical and Virtual World

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (7)

Destacado

Destacado (20)

Similar a Designing Interactions at the Nexus of the Physical and Virtual World

Similar a Designing Interactions at the Nexus of the Physical and Virtual World (20)

Último

Último (20)

Designing Interactions at the Nexus of the Physical and Virtual World