The lines between art, design, and information are dissolving as we experience new places and objects. Consider, for example, the organic flow of air traffic over North America at daybreak, the bursts of search query memes spreading around the globe, and the pointillist surge of mobile phone usage on New Year's Eve. Using the new techniques of generative data visualization, a new generation of artist/designers/engineer/scientists are creating gorgeous, dynamic experiences driven by massive sets of data about our own lives. Their work comes to life in architectural spaces, on walls of wood and metal and light and shimmering glass clouds suspended overhead. Of course it must be touched to be appreciated and engaged with, simple gestures launch a thousand images and possibilities. Many of these projects have received international recognition. They are primarily 3D applications that can run in real time, but really can only be appreciated by watching them, as movies. These data movies aim to make information easier to understand while being enjoyable to watch. Surprising insights surface through looking at our 'data life' in new ways, and may compel us to design in different, even better ways.

1. 1

2. Outline
• Background (Joy Mountford)
• Data Visualisation (Java Processing movies of
working applications)
• Ubiquity is here (where is the interface?)

3. About Joy...
• Current: Osher Fellow at Exploratorium
• Recent :Yahoo,VP of Design Innovation and UED
• Properties: Y! Mail, Messenger, Front Page, My
Yahoo, Groups, Photos
• 21 years of international University Design Expo:
touched 4000 students world-wide

4. Honeywell - An interface (Dreyfus)

5. Space Shuttle (1990s)

6. Featuritis leads to poor consumer use (still!)
6

7. Apple: QuickTime controller (1988)
Designing Interaction Bill Modridge (06)

8. Lego Music Composer ’03

9. 9

10. LinkMark 00
10

12. Yahoo front page 2005

13. Personal Information Manager 2006
13

14. Y! personal information manager
14

15. Y local offers relevant information
15

16. Opportunity....
• We can start ‘understanding’ real time data to show
what ‘we’ do/are
• Opportunity now is huge to actually use
information data corporations gather
• Uses are both consumer facing, and for internal
diagnostics, and to deliver useful content and
personalised services

17. McKinsey report 2009
• In next decades the ability to take data, to be
able to understand it, to process it, extract value,
to visualise and communication it, is going to be
an important skill in next decade.
• Not just at professional level, but all levels
• We have essentially free and ubiquitous data
• Scarce resource is ability to understand and
extract value

18. Statistics and graphs
• Long history of graphical communication
• “How to lie with statistics” etc....
• Selection of data parameters/styles depends on
task and who is asking for the answer
• Understanding of data displays is very individual
• What is a billion? (US/UK differ)

19. Data Numbers/Freq Visual Format
Data Visual Presentation User
Raw Data
Streams Style form Task
Data Visual View
Transformations Encodings Transformations

20. Accuracy of visual
techniques
Position
Slope
Angle
Area
Volume
Colour

21. Moore’s Law ‘65

22. Sun position over time
1626

23. Belousav-Z reaction
over time

24. 1856: Mortality causes:
Florence Nightingale

25. XBox Tableau tool (MS)

26. Cone Tree (Xerox)

27. Sense.us
work force professions
Flouride starts around here?
Dentistry
1880 1965 2000

28. Consumers Q +
comment

29. Group comments
collected

30. Two styles data
(Wattenberg)
• Voyager - focuses on visualised
data.
• Actively involved with data to
understand
• Voyeur - focuses on comment
listings
• Investigates others explorations

31. Sense.us: % Dentists
and technicians
Dentist
1850 1940 2000

32. New England report
• Framingham study data, 1948
• 1948 10% obese
• 1985 18% obese
• 2009 40% obese
• Fast food started and then network took over
• Spouse obese increased risk by 37%
• Friend obese increased risk by 171%

35. Understanding Data
• Attraction of charts starts initial interest, but aim
to increase consumer engagement and exploration
• Goal to build user interfaces to support and
encourage involvement
• Visual sense making can be social - as well as
collaborative

36. Exploratorium SF
36

38. Slime on peat: diatom
5,000 magnification

39. Fungal arms:
500 magnification

40. Book source: debate
over techniques

41. Chris Jordan
• 32,000 Barbies 60x80 inch
• Number of elective breast augmentation
• Performed monthly in US 2006
• Graduation gift

45. Art can Educate
• Different visual style hepls understanding
• Different visual styles appeal to folks
• Accidental discovery has big impact

46. Design Innovation Y! team
Ben Clemens
Aaron Koblin
Michael Chang
Doug Fritz
S. Joy Mountford

48. Ben Clemens, Aaron
Koblin, Michael Chang

49. Approach for Data Viz
• Data can attract
• Data can be ‘useful’
• Data can now be real time ‘dynamic’
• Put data around people to change POV
• Present in various forms

50. US flight paths (Koblin)

55. Y! Answers
• Use community to answer Qs
• Created vibrant on-line community pulse of ‘what is
happening’
• System created for accreditation of quality answers
• Potential for bloggers

56. Y!Answers cloud (Chang)

57. Traffic
• Use to replan your driving routes
• Use to dispatch ambulances etc
• Use to architect plan cities

58. Y! Local (Koblin)

59. J.Yamashita ‘07

60. Personalized DNA Art

61. Multi touch Table
(MOTO)

62. FlickR
• Built custom multi-user multi-touch surface
• Gesture connects us to the data
• GeoTagged photos showing what else is happening live

64. FlickR: multi-user touch table
(Chang 07)

65. SMS traffic New Years (NL)
Koblin ’08

66. Mail activity
• 2 hour increments of world-wide market traffic
• Shows relative sizes very clearly
• Showed Ham and Spam
• Located a server data loss from a pulse
• Aggregate view shows things otherwise ‘lost’

67. •Y

68. NYTE data
• Globe encounters show volumes of internet data
between NY and world over 24 hours
• Real time continuous update
• Larger glow implies greater IP flow
• Collaboration: Sensible cities MIT, ATT ,Y!
• MOMA New York show of live data, Feb 08

71. Search query bursts
• Active key word searches (rate of change)
• Use reverse IP look up for geo location-data
address locations
• Activity plotted as a particle system
• Time-lapse queries from geo-locations plotted
• Feasible in ‘real time’ for particular users’
interests

72. Y! search box

73. Real time editorial ‘clicks’
on

74. Data information
• Search moves to discovery (decision)
• Opportunity now to use real time data
• Internet business is about personal delivery of
useful, relevant content/ads dynamically to
consumers

75. Ambient Interfaces

76. Internet Archive (Kaehle)
• Machine readable versions of all out of print books - free
• Tool for librarians, museums and consumers
• Bookscape for image browsing
• Dynamic resampling of image data inside 1 zoomable space

77. 3D interactive Internet Archive
bookscape browser (Chang)

78. Multi-user multi-touch
found poetry

79. Personal tools
• Tagging of limited use with large info sets
• Find all related information
• Search anonymously across those with similar
habits/interests
• Surface want I might want

80. Clustering on content
(D. Fritz ’08)

81. Technical approach
• Tag co-occurrence defines distance algorithm
• Hierarchical clustering algorithm defines
dendrogram
• Recursively search dendrogram to find clusters of
optimal cohesion, while selecting a meaningful
human size
• Hierarchical clustering slower, but gives better
results with smaller data sets

82. Y!Haus lessons from
Data Viz
• Information can be provided in ambient personal
widgets
• Displays can be continuous, both in foreground as
well as background
• Animation gets initial interest, but goal is to
encourage understanding

84. Crowd potential
• Crowd sourcing powerful - not directly social
• Isolated but part of an anonymous group
• Little payment or reinforcement
• People want to contribute to the whole: with and
without anonymity
• Used Mechanical Turk (Amazon)

86. Draw sheep facing left for .02 cents
40 days for 10000 sheep (Koblin)

87. 10,000 sheep on poster

88. 10,000 Cents (Koblin)
created unique $100 bills

90. Consumers
• People want to extend their fame
• Hunger to understand themselves relative to others
• Some happy to participate anonymously, others
publically
• All of us becoming curators of own presence(s)

91. What and where is the
interface
• Ubiquitous computing is here
• Interface transparency arrived
• Art and maker community is forging ahead
• Technologists need to create safe ways of
convincing people to allow access to their
personal data for benefit
• Personal need for selective cloaking

92. Examples of user
‘inclusion’
• Unaware
• Partially shown with no choice
• Shown with choice
• Unclear effects
• Confusion

94. eCloud San Jose airport (Koblin)
Text

95. 95

96. Screens at IAC
• 6-8 15ft high wall interactive displays
• NYU ITP graduate play space, some curated
• IAC headquarters, New York
• Web properties and media and newspaper taking lead
with new media

99. Moeller San Jose airport ’10:
pins on sheets over parking lot
99

103. 103

105. 105

106. 106

108. Fire Fly Dress (Orth)

109. Music Perfume bottles
(MIT ‘01)

110. NYU: moving brooch

111. NYU worn printer

112. CCA: Light sensors cooked into lollipops

114. CCA: Real grass moves video:
grown in sensor filaments
Text

115. Bar scene

116. IR light color changing
Coke bottles

117. • Good Guide (mobile)
• iPhone app
• 70000 products
• Health, enviornmental, social
• Text message
– UPC, Product name, type
117

118. USC Cinematic Arts:
Million story building

119. BarCode Stories (UCLA)
119

120. Barcode phone access (UCLA)

121. EKG Ball games
121

122. Mattel MindFlex 09
122

124. 124

125. Sensors everywhere

127. User confusion
• How do I know what is being sensed?
• How do I know what I can use?
• How do I know what I am ‘in’ or ‘part of?’
• How do I stop being part of it? sometimes?
• Privacy crucial
• Cloak/veils to hide

128. Cloaks or veils?
Privacy
Partially
‘available’

129. We are all in the Design
• Interfaces are transparent and everywhere in the
physical world
• We are our own curators and reporters
• Data is pervasive, but difficult to know how best to
enhance understanding
• Issue: how do I know what I am part of, or not part
of, and then stop being part of it?

130. Reality? = Vegas
130