Introduction to the Workshop on Crowd-sourcing at the 1st Heidelberg Laureate Forum

1. Crowd-­‐Sourcing
Workshop
Heidelberg
Laureate
Forum
2013
Dr.
rer.
nat.
Gianluca
DemarCni
gianlucademarCni.net
exascale.info

2. Gianluca
DemarCni
• M.Sc.
at
University
of
Udine,
Italy
• Ph.D.
at
University
of
Hannover,
Germany
– EnCty
Retrieval
• Worked
for
UC
Berkeley
(on
Crowdsourcing),
Yahoo!
Research
(Spain),
L3S
Research
Center
(Germany)
• Post-­‐doc
at
the
eXascale
Infolab,
Uni
Fribourg,
Switzerland.
• Lecturer
for
Social
Compu,ng
in
Fribourg
• Tutorial
on
EnCty
Search
at
ECIR
2012,
on
Crowdsourcing
at
ESWC
2013
and
ISWC
2013
• Research
Interests
– InformaCon
Retrieval,
SemanCc
Web,
Human
ComputaCon
2
demartini@exascale.info
Gianluca
DemarCni

3. Schedule
• Intro
and
Crowdsourcing
Overview
• The
HLF
crowd
on
“Crowdsourcing
your
Research”
– MaYhias
Hagen:
understand
Web
search
– Dongrui
Wu:
make
flights
more
efficient
– Ioannis
Psorakis:
classify
astronomical
images
– Trung
T.
Pham:
recognize
geometric
objects
– Rose
Yu:
anomaly
detecCon
on
social
networks
• Ethics
in
Crowdsourcing
&
Conclusions

4. Are
humans
or
machines
beYer
at
• Music?
• Art?
• ComputaCon?

5. Are
humans
or
computers
beYer?
• We
are
good
at
doing
some
things
• Computers
are
good
at
doing
others
• Machines
have
helped
us
in
MANY
domains
• Can
WE
help
them?

7. Crowdsourcing
• Portmanteau
of
"crowd"
and
"outsourcing,"
first
coined
by
Jeff
Howe
in
a
June
2006
Wired
magazine
arCcle
• [Merriam-­‐Webster]
the
pracCce
of
obtaining
needed
services,
ideas,
or
content
by
soliciCng
contribuCons
from
a
large
group
of
people
and
especially
from
the
online
community
rather
than
from
tradiConal
employees
or
suppliers
7

8. Crowdsourcing
IncenCves
• Paid
Crowdsourcing
– Amazon
MTurk
• Fun
(enjoyment)
– Game
With
a
Purpose
• Community
(belonging,
desire
to
help)
– Wikipedia
8

9. Crowdsourcing
Examples

10. CiCzen
Science
10

11. QuesCon
Answering
Systems
11

12. ParCcipatory
Culture
–
Implicit
12
John
Murrell:
GM
SV
9/17/09
…every
Cme
we
use
a
Google
app
or
service,
we
are
working
on
behalf
of
the
search
sovereign,
creaCng
more
content
for
it
to
index
and
moneCze
or
teaching
it
something
potenCally
useful
about
our
desires,
intenCons
and
behavior.

13. OCR
errors:
reCAPTCHA
13

14. Case-­‐Study:
Amazon
MTurk
• Micro-­‐task
crowdsourcing
marketplace
• On-­‐demand,
scalable,
real-­‐Cme
workforce
• Online
since
2005
(sCll
in
“beta”)
• Currently
the
most
popular
plarorm
• Developer’s
API
as
well
as
GUI
• 1M
workers
(10%
acCve)
14

15. Challenges
and
OpportuniCes
• OpportuniCes
– Get
the
best
out
of
efficient
systems
and
effec8ve
human
intelligence
– Design
new
soluCons
to
old
problems
– Interdisciplinary
effort
• Challenges
– Deal
with
Human
Non-­‐determinisCc
Quality
– Be
ready
for
slow
data
• Ethics

16. Schedule
• Intro
and
Crowdsourcing
Overview
• The
HLF
crowd
on
“Crowdsourcing
your
Research”
– MaYhias
Hagen:
understand
Web
search
– Dongrui
Wu:
make
flights
more
efficient
– Ioannis
Psorakis:
classify
astronomical
images
– Trung
T.
Pham:
recognize
geometric
objects
• Ethics
in
Crowdsourcing
&
Conclusions