The document discusses methods for prototyping interactive public displays using virtual and miniature simulations. It describes two case studies where developers used different approaches: Developer H used a hybrid method with both virtual and miniature simulations, while Developer V relied primarily on virtual simulation with cognitive visualization techniques. Both uncovered issues during prototyping like visibility and sizing that informed real installation. The document suggests methods that combine virtual simulation, miniature simulation and deployment can help address limitations of any single approach.

1. Virtual Prototyping
using Miniature Model and Visualization
for Interactive Public Displays
Yasuto Nakanishi
Keio University / JST Presto

2. Public Displays
Interaction Design<--->Space Design
• Large displays in public spaces
• Shopping mall / Bending machine / Station, Airport /
Exterior of Building
• Embodied / Spatial Interactions

3. Public Displays
Interaction Design<--->Space Design
• Shinagawa station : one of very large stations in Japan
• 44 plasma displays, however, most people doesn t see them...
• X
InformationDesign <----> Space Design

4. CityCompiler : Overview
Software Twenty-two virtual displays
Design PApplet to show PApplets
Processing
Compile Simulation
Java Monkey
Engine
Google
Sketch Up Space
3D model
(Shinagawa station, Tokyo)
Design
• Iterative Prototyping within Virtual Space
• checking size of the idle, shooting angle, walking speed etc.
• adding virtual sensors in order to let the girl walk with the
same speed as the crowd

5. Public Displays (my works)
Interaction Design<--->Space Design
Tokyo Metropolitan Museum
Street in Sendai NTT Inter Communication Center 2004
of Photography 2007
Gallery in Roppongi 2001 Art Event in Kanda 2005 Museum Of Contemporary Art Tokyo 2010

6. Without CityCompiler (2007)
Miniature Prototyping
• Exhibition at Tokyo Metropolitan Museum of Photography (2007)
• Measuring of the wall at the museum + Making a paper doll
with the ratio that considered a 13 inch display as the wall
• Trial-and-error for tuning the parameters of the motion
graphics : size, alpha value and movement speed of the words
• Only simulating a screen; without neighboring space

7. CityCompiler
Virtual Prototyping
• Re-tuning the size and the moving speed of the words +
Changing the layout and size of the screen in runtime
• The circulation plan was also simulated and determined.
•
virtual simulation real space installation
• Unexpected features : the lustrous screen and floor...

8. tion
irtual Simula
Lim itations in V .
installation
l space
often arise in rea
d situations
•Unexpecte resented m
ight be
ation p
ce o f the inform
•Appearan s.
om expectation
different fr
Software
Design
Processing Simulation Installation
Compile
Java Monkey
Engine
Google
Sketch Up Space
Design

9. tion
irtual Simula
Lim itations in V .
installation
l space
often arise in rea
d situations
•Unexpecte resented m
ight be
ation p
ce o f the inform
•Appearan s.
om expectation
different fr
Simulation of Interactive Public Displays
Hybrid of Virtual Simulation Virtual Simulation
and Miniature Simulation with Cognitive Visualization
Virtual Virtual Virtual
+ +
Miniature Miniature
θ

10. tion
irtual Simula
Lim itations in V .
installation
l space
often arise in rea
d situations
•Unexpecte resented m
ight be
ation p
ce o f the inform
•Appearan s.
om expectation
different fr
• Hybrid of Virtual Simulation and Miniature Simulation
• Switching virtual and physical sensors within the same codes.
• Operating a system within both a virtual space and a miniature
space would help to handle the parameters and revise codes.
lation
<--->Min iature simu
l Simulation
V irtua
(similar to) e(?)
ent int o Real Spac
ulation< --->Deploym
Virtual Sim
Virtual Simulation Miniature Simulation Deployment

11. tion
irtual Simula
Lim itations in V .
installation
l space
often arise in rea
d situations
•Unexpecte resented m
ight be
ation p
ce o f the inform
•Appearan s.
om expectation
different fr
•
Color eyesight
Cognitive Visualization within Virtual Space pink 0.2 - 0.4
blue 0.4 - 0.6
cyan 0.6 - 0.8
Landolt rings
green 0.8 - 1.0
yellow 1.0 - 1.2
orange 1.2 - 1.4
red 1.4 - 1.6
• Display of Landolt rings
• Visualization of the eyesight
required to see them
• Point lights showing the
angle of human vision

12. tion
irtual Simula
Lim itations in V .
installation
l space
often arise in rea
d situations
•Unexpecte resented m
ight be
ation p
ce o f the inform
•Appearan s.
om expectation
different fr
Simulation of Interactive Public Displays
Hybrid of Virtual Simulation Virtual Simulation
and Miniature Simulation with Cognitive Visualization
Virtual Virtual Virtual
+ +
Miniature Miniature
θ
the methods ?
sand cons of
Wha t is the pro

13. Case studies of prototyping methods
Feature1 Impression and visibility depend on the size or layout of text and images
Parameter adjustments or source code corrections are made in line with
Feature2
the spatial layout of the devices that comprise the system
Public Display1 :
A system in which a number of photos were captured
using a smartphone and displayed on a map
Public Display2 :
An interactive video display system in which the size
and location of human shadows are captured by a
camera and used to trigger the display of a video

14. Public Display1 Public Display2
Hybrid of Virtual Simulation テキスト Virtual Simulation
and Miniature Simulation with Cognitive Visualization
Public Display1 Public Display2 Public Display1 Public Display2
Developer H Developer V
?
f th e methods
os and cons o
What is the pr

15. Public Display1 : Developer H
Virtual Space Model Space Real Space
Measurement
Creating a virtual
model
Investigation for
display position
A window facing
street was selected.
Creating a
miniature model
Checking visibility
with a camera figure
A window frame
overlapped with
screen, and
displayed photo
size was too small.
Adding function of
changing the size
of photo and the
application size

16. Public Display1 : Developer H
Virtual Space Model Space Real Space
Measurement
Creating a virtual
model
Investigation for
display position
A window facing
street was selected.
Creating a
miniature model
Checking visibility
with a camera figure
A window frame
overlapped with
screen, and
displayed photo
size was too small.
Adding function of
changing the size
of photo and the
application size

17. Public Display1 : Developer H
Virtual Space Model Space Real Space
Measurement
Creating a virtual
model
Investigation for
display position
A window facing
street was selected.
Creating a
miniature model
Checking visibility
with a camera figure
A window frame
overlapped with
screen, and
displayed photo
size was too small.
Adding function of
changing the size
of photo and the
application size

18. Public Display1 : Developer H
Virtual Space Model Space Real Space
Measurement
Creating a virtual
model
Investigation for
display position
A window facing
street was selected.
Creating a
miniature model
Checking visibility
with a camera figure
A window frame
overlapped with
screen, and
displayed photo
size was too small.
Adding function of
changing the size
of photo and the
application size

19. with a camera figure
A window frame
Public Display1 : Developer H
overlapped with
screen, and
displayed photo
size was too small.
Adding function of
changing the size
of photo and the
application size
The brightness of
the inside and the
outside of the
miniature space
were the same
unlike the real
space.
Adding a LED to the
miniature model
Adding a function
to change the
brightness of
application
Deployment and
installation

20. with a camera figure
A window frame
Public Display1 : Developer H
overlapped with
screen, and
displayed photo
size was too small.
Adding function of
changing the size
of photo and the
application size
The brightness of
the inside and the
outside of the
miniature space
were the same
unlike the real
space.
Adding a LED to the
miniature model
Adding a function
to change the
brightness of
application
Deployment and
installation

21. with a camera figure
A window frame
Public Display1 : Developer H
overlapped with
screen, and
displayed photo
size was too small.
Adding function of
changing the size
of photo and the
application size
The brightness of
the inside and the
outside of the
miniature space
were the same
unlike the real
space.
Adding a LED to the
miniature model
Adding a function
to change the
brightness of
application
Deployment and
installation

22. Public Display1 : Developer V
Virtual Space Real Space
Measurement
Creating a virtual model
Investigation for
display position
Interior Interior Window facing
wall 1 wall 2 the street
Verifying visibility of photos with
Landolt ring
Landolt Landolt Landolt
ring seen ring seen ring seen
from 100cm from 200cm from 600cm
A function of changing the size of photo
was added, and the size was decided.

23. Public Display1 : Developer V
Virtual Space Real Space
Measurement
Creating a virtual model
Investigation for
display position
Interior Interior Window facing
wall 1 wall 2 the street
Verifying visibility of photos with
Landolt ring
Landolt Landolt Landolt
ring seen ring seen ring seen
from 100cm from 200cm from 600cm
A function of changing the size of photo
was added, and the size was decided.

24. Verifying visibility of photos with
Public Display1 : Developer V
Landolt ring
Visual Coneの図画欲
しいな… Landolt Landolt Landolt
ring seen ring seen ring seen
from 100cm from 200cm from 600cm
A function of changing the size of photo
was added, and the size was decided.
Checking how the display enters into a
field of view of a virtual human
Verifiying Verifying Verifying
from the with moving from road
entrance. in the room. 600cm apart
Comparing the size of shown photo
The window facing the
street was selected as
a deployment place. Deployment and
installation

25. Verifying visibility of photos with
Public Display1 : Developer V
Landolt ring
Landolt Landolt Landolt
ring seen ring seen ring seen
from 100cm from 200cm from 600cm
A function of changing the size of photo
was added, and the size was decided.
Checking how the display enters into a
field of view of a virtual human
Verifiying Verifying Verifying
from the with moving from road
entrance. in the room. 600cm apart
Comparing the size of shown photo
The window facing the
street was selected as
a deployment place. Deployment and
installation

26. Public Display2 : Developer H
Virtual Space Model Space Real Space
Investigating
location relations
of a projector, a
camera and figurines
Location hunting
Measurement of
the chosen space
Creating a
miniature model
Deciding the screen
size and the video
composition
Taking a video
Checking the shadow
size and the movie
size with using a
figurine.

27. Public Display2 : Developer H
Virtual Space Model Space Real Space
Investigating
location relations
of a projector, a
camera and figurines
Location hunting
Measurement of
the chosen space
Creating a
miniature model
Deciding the screen
size and the video
composition
Taking a video
Checking the shadow
size and the movie
size with using a
figurine.

28. Public Display2 : Developer H
Virtual Space Model Space Real Space
Investigating
location relations
of a projector, a
camera and figurines
Location hunting
Measurement of
the chosen space
Creating a
miniature model
Deciding the screen
size and the video
composition
Taking a video
Checking the shadow
size and the movie
size with using a
figurine.

29. Public Display2 : Developer H
Virtual Space Model Space Real Space
Investigating
location relations
of a projector, a
camera and figurines
Location hunting
Measurement of
the chosen space
Creating a
miniature model
Deciding the screen
size and the video
composition
Taking a video
Checking the shadow
size and the movie
size with using a
figurine.

30. Deciding the screen
size and the video
Public Display2 : Developer H
composition
Taking a video
Checking the shadow
size and the movie
size with using a
figurine.
it was strange that
the video suddenly
started and stopped
when a shadow was
detected.
Retaking a video
Checking the shadow
size and the movie
size using a
figurine.
A problem of the
mini projector.
Checking the shadow
size and the video
size with a virtual
projector.
The video size was
larger than
expected.
Adding a function
to change the size
and the position of
the movie.

31. Deciding the screen
size and the video
Public Display2 : Developer H
composition
Taking a video
Checking the shadow
size and the movie
size with using a
figurine.
it was strange that
the video suddenly
started and stopped
when a shadow was
detected.
Retaking a video
Checking the shadow
size and the movie
size using a
figurine.
A problem of the
mini projector.
Checking the shadow
size and the video
size with a virtual
projector.
The video size was
larger than
expected.
Adding a function
to change the size
and the position of
the movie.

32. Deciding the screen
size and the video
Public Display2 : Developer H
composition
Taking a video
Checking the shadow
size and the movie
size with using a
figurine.
it was strange that
the video suddenly
started and stopped
when a shadow was
detected.
Retaking a video
Checking the shadow
size and the movie
size using a
figurine.
A problem of the
mini projector.
Checking the shadow
size and the video
size with a virtual
projector.
The video size was
larger than
expected.
Adding a function
to change the size
and the position of
the movie.

33. size and the video
size with a virtual
projector.
Public Display2 : Developer H
The video size was
larger than
expected.
Adding a function
to change the size
and the position of
the movie.
Deployment and
installation
The brightness of
the installation
space was
brighter than
expected.
Running the system
with changing the
room brightness.
Adding a function
to binarize the
camera image.
Deployment and
installation

34. size and the video
size with a virtual
projector.
Public Display2 : Developer H
The video size was
larger than
expected.
Adding a function
to change the size
and the position of
the movie.
Deployment and
installation
The brightness of
the installation
space was
brighter than
expected.
Running the system
with changing the
room brightness.
Adding a function
to binarize the
camera image.
Deployment and
installation

35. size and the video
size with a virtual
projector.
Public Display2 : Developer H
The video size was
larger than
expected.
Adding a function
to change the size
and the position of
the movie.
Deployment and
installation
The brightness of
the installation
space was
brighter than
expected.
Running the system
with changing the
room brightness.
Adding a function
to binarize the
camera image.
Deployment and
installation

36. Public Display2 : Developer V
Virtual Space Real Space
Location hunting
Measurement of
the chosen space
Shooting a video
Creating a virtual model
Locating a virtual camera and
a virtual projector
Verifying visibility of the
application with Landolt ring
Verifying display position
with a virtual human
A projector installed between
the road and the space entered
the field of vision.
Deployment and
installation

37. Public Display2 : Developer V
Virtual Space Real Space
Location hunting
Measurement of
the chosen space
Shooting a video
Creating a virtual model
Locating a virtual camera and
a virtual projector
Verifying visibility of the
application with Landolt ring
Verifying display position
with a virtual human
A projector installed between
the road and the space entered
the field of vision.
Deployment and
installation

38. Public Display2 : Developer V
Virtual Space Real Space
Location hunting
Measurement of
the chosen space
Shooting a video
Creating a virtual model
Locating a virtual camera and
a virtual projector
Verifying visibility of the
application with Landolt ring
Verifying display position
with a virtual human
A projector installed between
the road and the space entered
the field of vision.
Deployment and
installation

39. Public Display2 : Developer V
Virtual Space Real Space
Location hunting
Measurement of
the chosen space
Shooting a video
Creating a virtual model
Locating a virtual camera and
a virtual projector
Verifying visibility of the
application with Landolt ring
Verifying display position
with a virtual human
A projector installed between
the road and the space entered
the field of vision.
Deployment and
installation

40. Public Display1 Public Display2
Hybrid of Virtual Simulation Virtual Simulation
and Miniature Simulation with Cognitive Visualization
Public Display1 Public Display2 Public Display1 Public Display2
Developer H Developer V
switching
ing thods ?
tuning s of the me rs models/scales
find
and conrs
smhe pros
g paramete
makinWhat
g i tovin paramete
objects
models

41. Hybrid of Virtual Simulation Virtual Simulation
Methods
and Miniature Simulation with Visualization
•Preparing miniature •Only PC
materials and tools
•A model only for one screen •Easy to copy virtual model.
is easy, but it costs to make •Easy to locate many
Making a model having several displays or projectors.
Models display or projectors.
•Difficult to know how •Difficult to know how
detailed model should be detailed model should be
made beforehand. made beforehand.
•Direct and Intuitive •Indirect Manipulation (GUI,
Manipulation (TUI) slider)
•Moving plural objects is •Moving plural objects takes
Moving
easy. time.
Objects
•Physical Limitation in •No limitation in deployment
deployment positions or positions or angles
angles

42. Hybrid of Virtual Simulation Virtual Simulation
Methods
and Miniature Simulation with Visualization
•Measurement to know •Changing Parameters
Tuning parameter values Numerically
Parameters •Flexible system, but trial- •Visualization as Guideline in
and-error for on-site tuning Tuning parameters
•Switching simulation spaces •Comparing different device
(different devices, different locations or viewpoints
materials) provided provided opportunities.
Finding opportunities to find •Simple CG might not show
Parameters parameters. delicate difference;
Visualization should be
designed to make up for
this demerit.

43. Hybrid of Virtual Simulation Virtual Simulation
Methods
and Miniature Simulation with Visualization
•It costs to change simulation •Possible to simulate on
spaces various scales in one virtual
•it has value; sometimes model.
true, sometimes not true. •it is easy to copy a virtual
•a timing of changing model model, and it might
spaces would depends on obstruct to switch to a
Switching the experience. space of different
Models / properties on the contrary.
Scales
•The scale of the model •Switch to a model with
should affects thinking different characters
viewpoints. intentionally is good?
•Changing scales of
miniature model needs
different miniature models.

44. Discussion
Virtual Simulation
with Visualization
Hybrid of Virtual Simulation
and Miniature Simulation
Sketches and prototypes are both instantiations of the design concept. However they serve
different purposes, and therefore are concentrated at different stages of the design process.
Sketches dominate the early ideation stages, whereas prototypes are more concentrated at
the later stages where things are converging within the design funnel.
–Bill Buxton, Sketching User Experiences

45. A strategy for prototyping
interactive public displays
Virtual Virtual Virtual
Miniature Miniature
θ
Virtual Simulation Hybrid of Virtual Simulation
with Visualization and Miniature Simulation
Basic software operations Problems arising from differences
between virtual and real space are
and ideal positioning of devices
clarified within a miniature model space.
are investigated in virtual space.
+
Feature addition/code adjustment are
iterated within both spaces.

46. Thanks for you attention.
yasuto.nakanishi@acm.org