A presentation about the initial version of the PuReWidgets toolkit for interactive public display applications.

1. The
PuReWidgets
toolkit
for
interac6ve
public
display
applica6ons
Jorge
C.
S.
Cardoso
jorgecardoso@ieee.org
Ubicomp@UMinho
May
30,
2012
1

2. Content
• Mo6va6on
• PuReWidgets
features
• Architecture
• Applica6on
models
• Programming
library
overview
• HelloWorld
2

3. Mo6va6on
Complexity
of
incorpora6ng
interac6ve
features
in
public
display
applica6ons.
Too
many
things
to
answer
when
designing
the
interac6on
for
a
public
display
applica6on:
• What
controls
can
we
use
to
offer
interac6ve
features
to
users?
– No
standard
interac6ve
controls
for
public
displays
• What
interac6on
mechanisms
can
be
used
to
provide
those
features?
– Various
possible
interac6on
mechanisms
(SMS,
BT
naming,
OBEX,
Email,
IM,
mobile
app,
desktop,
QR
codes,
touch,
etc.)
– Different
loca6ons
may
have
different
interac6on
resources
• How
will
those
features
work?
– Desktop
interac6on
paradigm
does
not
generally
apply
– Applica6ons
may
not
always
be
visible
• How
can
the
applica6on
react
to
different
users?
– The
interac6on
environment
is
mul6-­‐user
– We
need
to
iden6fy/differen6ate
users
• How
can
users
tell
what
they
can
do
with
the
applica6on?
– Where
are
the
features
presented?
3

4. Mo6va6on
We
can
solve
all
those
ques6ons
for
ad
hoc
applica6ons,
but
• This
means
wasted
effort
in
developing
applica6ons
– Programmers
must
deal
with
issues
extraneous
to
the
main
applica6on
logic
• Will
certainly
lead
to
inconsistencies
in
the
interac6on
model
– Each
solu6on
will
behave
slightly
differently
– Will
confuse
users
We
need
a
programming
toolkit
for
interac6ve
public
display
applica6ons!
4

5. Toolkit
• The
toolkit
should
provide
a
consistent
model
for
user
interac6on
and
applica6on
development
– But
s6ll
allow
some
diversity
in
solu6ons
5

6. PuReWidgets
• PuReWidgets
tries
to
solve
these
problems.
• To
make
sure
it
covers
a
wide
range
of
applica6ons,
we
looked
at
various
interac6ve
public
display
systems
to
learn
– General
requirements
– Types
of
controls
– Types
of
input
mechanisms
6

7. PuReWidgets
Widget-­‐based
toolkit.
• A
widget
represents
an
interac6ve
feature.
• Is
represented
by
class
in
an
object-­‐oriented
programming
model.
• Applica6ons
instan6ate
widgets,
define
a
callback,
and
receive
interac6on
event
through
the
callback
• Main
features
– Mul6ple,
extensible,
controls
– Independence
of
input
mechanisms
and
modali6es
– Automa6cally
generated
graphical
interfaces
– Asynchronous
interac6on
– Concurrent
interac6on
– Graphical
affordances
– Server
&
client
libraries
7

8. Features
Mul6ple,
extensible,
controls.
• Allows
developers
to
create
rich
interac6ve
applica6ons.
Controls
can
be
extended
with
custom
func6onality.
• We
studied
various
interac6ve
display
systems
to
define
a
set
of
basic
interac6ve
controls,
common
to
the
majority
of
applica6ons:
• Ac6on
bugon
– Trigger
an
ac6on
in
the
applica6on,
e.g.,
play
a
video
• Op6on
selec6on
– Selects
among
a
set
of
op6ons,
e.g.,
to
vote
• Text
entry
– Sends
text
to
the
applica6on,
e.g.,
a
comment,
tag,
search
keyword
• Download
– Receives
a
media
file
from
the
applica6on,
e.g.,
download
poster
• Upload
– Uploads
a
media
file
to
the
applica6on,
e.g.,
a
photo
to
be
displayed
• Check-­‐in
– Says
“I’m
here”
8

9. Features
Independence
of
input
mechanisms
and
modali6es.
• Allows
developers
to
focus
on
the
core
applica6on
func6onality
and
not
on
the
low-­‐level
interac6on
details.
• Allows
applica6ons
to
work
in
heterogeneous
loca6ons,
with
different
mechanism
• Interac6on
can
be
accomplished
with
various
mechanisms
(SMS,
Bluetooth
naming,
QR
codes,
and
automa6cally
generated
graphical
interfaces).
9

10. Features
Automa6cally
generated
graphical
interfaces.
• Provide
a
richer
interac6on
for
desktop
and
mobile
devices.
• PuReWidgets
generates
web-­‐based
interfaces
for
each
applica6on.
• It
also
generates
QR
codes
for
each
applica6on's
feature
10

11. Features
Asynchronous
interac6on.
• Allows
applica6ons
to
receive
input
events
that
were
generated
when
the
applica6on
was
not
listening.
• PuReWidgets
provides
a
persistent
input
queue
for
each
applica6on
• Applica6ons
can
request
past
input
at
any
6me
11

12. Features
Concurrent
interac6on.
• Allows
various
users
to
interact
at
the
same
6me,
while
providing
applica6ons
with
iden6ty
informa6on
that
allows
them
to
differen6ate
users.
• Every
input
event
carries
a
user
id
(if
available)
– Where
possible,
anonymous
interac6ons
are
s6ll
“iden6fied”
(more
on
this
later)
12

13. Features
Graphical
affordances.
• Allows
users
to
immediately
recognize
interac6ve
features
and
input
feedback.
• Controls
have
an
op6onal
graphical
representa6on
that
can
be
used
on
the
public
display
interface.
– Applica6ons
may
not
show
the
features
on
the
public
display,
but
they
will
s6ll
be
available
• Input
feedback
is
also
(op6onally)
displayed
on
the
public
display.
13

14. Features
Server
&
client
libraries.
• Programmers
can
choose
to
use
just
the
server,
just
the
client,
or
both
libraries
together.
• Allows
developers
to
choose
the
best
applica6on
model.
14

15. PuReWidgets
Architecture
15

16. Interac6on
services
• PuReWidgets
service
– Keeps
informa6on
about
every
widget
– Generates
graphical
interfaces
for
desktop,
mobile,
touch
plaporms
• These
allow
anonymous
interac6ons,
but
they
generate
persistent
cookie-­‐based
random
ids.
– Generates
QR
codes
for
every
widget
– Generates
unique
textural
reference
codes
for
data-­‐based
interac6ons
(SMS,
BT
naming,
OBEX,
etc.)
• IO
infrastructure
– (from
Instant
places)
– Provides
low-­‐level
data-­‐based
interac6on
16

17. PuReWidgets
library
• Provides
an
object-­‐oriented
library
of
widgets
• Hides
the
communica6on
details
with
the
PuReWidgets
service
• Provides
other
features,
unrelated
to
interac6on,
such
as
– Applica6on
“place-­‐aware”
storage
• PuReWidgets
provides
a
name-­‐value
server
datastore
for
easy
storage
of
place-­‐specific
seqngs/state
– Skeleton
admin
interface
for
place-­‐owners
• To
configure
place-­‐specific
seqngs
17

18. Local
applica6on
model
18

19. Local
applica6on
model
• Applica6on
logic
on
the
public
display
(player)
– Subject
to
the
schedule
6mings
• May
not
be
able
to
react
(or
update
data
structures)
immediately
when
user
interacts
– But
will
eventually
(when
it
is
displayed
again)
• When
displayed,
the
toolkit
periodically
asks
the
service
for
input
events
– And
triggers
the
appropriate
widget
event
19

20. Remote
applica6on
model
20

21. Remote
applica6on
model
• Applica6on
logic
on
the
server
• Can
react
(or
update
data
structures)
immediately
to
user
input
– Even
if
not
on
the
public
display,
the
reac6on
can
be
visible
on
other
channels
• The
toolkit
periodically
asks
the
service
for
input
events
– And
triggers
the
appropriate
widget
event
21

22. PuReWidgets
Implementa6on
• Google
Appengine
(server)
• Google
Web
Toolkit
–
GWT
(client)
• Overview
of
the
library
22

23. Using
PuReWidgets
• Hello
World
applica6on
(local
model)
• Video
23

24. Ini6al
Evalua6on/Development
process
• Two
applica6ons
developed
so
far
(ignore
the
“graphic
design”,
please)
– Youtube
player,
media
intensive,
complex
graphical
components
(several
panels
that
change
over
6me),
local
applica6on
model
– Everybody
votes,
based
on
display
owner
created
content,
simpler
graphical
component,
combines
remote
and
local
applica6on
models
24

25. Ini6al
Evalua6on/Development
process
• Con6nuous
refinement
– Develop
interac6ve
public
display
applica6ons
– Gain
more
insight
about
the
difficul6es
– Refine
the
toolkit
to
address
the
iden6fied
problems
– Refactor
the
applica6ons
to
include
the
toolkit
changes
25

26. Summary
• PuReWidgets
facilitates
applica6on
development
– Provides
ready-­‐to-­‐use
interac6ve
controls
– Integrates
various
input
mechanisms
via
an
I/O
infrastructure
– Generates
applica6on
interfaces
for
desktop,
mobile,
and
QR
codes
– Provides
client
and
server
applica6on
development
models
26

27. The
end
27