Pulse Lab Jakarta Launch Presentation

BAPPENAS

Launch Event
Peluncuran
1 October 2012 / 1 Oktober 2012

Pulse Lab Jakarta Launch / Jakarta, Indonesia / 1 October 2012

Ir. Wismana Adi
Suryabrata
Deputy Minister for Development Funding,
Bappenas


Prof. Dr. Armida S.
Alisjahbana
Minister of National Development
Planning Agency / Head of Bappenas


Mr. El-Mostafa
Benlamlih
United Nations Resident Coordinator


Robert Kirkpatrick
Director, UN Global Pulse


HARNESSING THE POWER OF BIG DATA FOR
GLOBAL DEVELOPMENT

Robert Kirkpatrick

Hyperconnectivity

Airplane Flights Telephone Calls

Internet Traffic Social Networking

Real-time isn’t just faster. It’s different.

Welcome to the age of
BIG DATA

Volume. Velocity. Variety….

Phoenician alphabet invented ca. 1000 B.C.E.

400 million tweets per day….

or, 4600 per second

400 million tweets = 7 terabytes of data…

…..or120,000 file cabinets full of paper

If social networks were countries…

1.3 billion

1.2 billion
955 million
~500 million

309 million

238 million

Jakarta - 9 million tweets every day

PRIVATE SECTOR
Monitor operations…in real time
Track market trends…in real time
Get customer feedback…in real time

GLOBAL DEVELOPMENT
Unemployment?
Food Security?
Public Health?
Education?
Migration?
Disaster Relief?

BIG DATA IN REAL TIME: OPPORTUNITIES

1.  Better early warning: Faster detection of
anomalies in at the onset of a crisis allows rapid,
effective, efficient response.

2.  Real-time awareness: A more accurate and up-to-
date picture of population needs and concerns
informs better design of programs and policies;

3.  Real-time feedback: Understand sooner where
programs and policies are NOT working and
make rapid, adaptive adjustments until progress
is on track.

BIG DATA, PRIVACY, AND HUMAN RIGHTS
Global Pulse privacy guidelines:
•  Never analyze personally identifiable data
Never analyze confidential data
•  Never seek to re-identify individuals

EXAMPLE: AIRTIME CREDIT PURCHASE DATA

SIZE AND FREQUENCY PREDICT HOUSEHOLD INCOME

Higher household
income
Average size of purchase

Lower household
income

Average # of purchases / month

CALLING PATTERNS AND ECONOMIC OPPORTUNITY

Lower Higher
socioeconomic level socioeconomic level

MEN AND WOMEN USE THEIR PHONES DIFFERENTLY
Men: Women:
•  Fewer calls •  More calls
•  Shorter calls •  Longer calls
•  Smaller social network •  Larger social network
•  More work-related calls •  More personal calls

Mobile phone locations predict cholera outbreaks

Using mobile phone movement to assess H1N1 response

TWITTER PREDICTS H1N1 INFLUENZA

GOOGLE SEARCHES FOR SYMPTOMS PREDICT DENGUE

RECENT SEARCHES FOR JOBS, COMMODITY PRICES

GLOBAL PULSE 2011

THREE PROOF OF
CONCEPT PROJECTS

Tracking Commodity Prices
through Online Retail
Advertising

Generating Real-Time Inflation Indices

1 2 3 4 5
Use1scraping1 Connect1to1 Find1individual1 Store1key1item1 Calculate1
technology online1retailers items informa-on1in1 PriceStats1
a1database Indexes

• Date
• Item
• Price
• Descrip-on

ARGENTINA BRAZIL CHILE

COLOMBIA URUGUAY VENEZUELA

URUGUAY

Uruguay Bread
Index
(daily data)

Uruguay
Bread & Cereals CPI
(monthly data)

Online Discussions & Unemployment
Ireland

Online Discussions & Unemployment
United States

Tweets per day about food,
during Ramadan in Indonesia
Start of Ramadan

End of Ramadan

Tweets predict food basket inflation
(rice, chilies, fish, sugar, corn, cooking oil)

Tweets about the
price of rice
(per month)

Official
Food Price Inflation
(monthly from 25 cities)

PULSE LABS
R&D INNOVATION STRATEGY
1.  Partner with governments to establish Pulse Labs
around the world.
2.  Build world-class teams of data scientists,
engineers, and policy experts
3.  Partner with private sector for access to real-time
data and cutting edge technology
4.  Design and conduct exploratory research
5.  Build an open source toolkit for real-time
monitoring
6.  Support broad adoption of useful innovations
7.  Share everything we learn and build

Pulse Lab Network

Pulse Lab Jakarta…………October 2012
Pulse Lab Kampala……….January 2013
Other locations…………...???

Dr. Lukita
Dinarsyah Tuwo
Vice Minister of National Development
Planning Agency / Bappenas


Initial Research Findings
from Pulse Lab Jakarta


Indonesia & Social Media

Indonesia 5th in worldwide Twitter users & volume

90% of Indonesian Internet users are on Facebook

10-25% of Indonesian regularly access Internet,
including via Internet cafes and mobile devices

Language barriers, local slang and Indonesia-specific
sites present new challenges for social media
analytics.

Source: Pulse Lab Jakarta Launch / Jakarta, Indonesia / 1 October 2012

Pulse Lab Jakarta
An innovation lab for real-time, Big Data analysis


Research Agenda
2012-2013

Regions of focus:

•  Jakarta
•  Jogja
•  Makassar (South Sulawesi)
•  Medan (North Sumatera)

Topic of focus:

•  Changes in social welfare, especailly as related to
food prices, fuel prices (transport and cooking) and
employment


Research Agenda
2012-2013

Approaches for Exploration:

•  Social Media & Twitter Analysis
•  Mobile CDR Analysis
•  Rapid Mobile Surveys


Research Questions
2012-2013

How are these topics discussed on Social Media?

•  Which social media sources?
•  How much relevant conversation is there?

How can we monitor how they affect Indonesians?

•  Which analytical approaches?
•  Which technological capabilities?


Pre-Launch Research Topics
Initial Exploration

1 Food Prices
•  Staple foods (sembako), rice (beras), sugar
(gula), eggs (telur)

2 Fuel Prices
•  Vehicle fuel (bensin), oil fuel (BBM), LPG
(elpiji)

3 Unemployment
•  Unemployed (pengangguran), severance pay
(pesangon)


Proof of Concept Research
Initial Exploration

101001
001001
Data Sources
001110 •  Sources: Twitter, Facebook, Blogs, News…
010011
•  Real-time data + 2-year historical archive

Analytical Approaches
•  Identify trending topics in online conversation
•  Assess mood and sentiment for those topics

Results
•  Correlate with official data sources (CPI,
unemployment)
•  Visualize trends & map the results

Research Tool 1
Crimson Hexagon: ForSight


Research Tool 2
SAS Social Media Analytics and SAS Text Miner


The Signals Are Getting Stronger

è Big increase in volume of relevant conversations over 18 months
40000

35000

minyak
(oil)

ketahanan
pangan
(food
security)

30000

budidaya
(cul>va>on)

25000

telur
(eggs)

20000

15000

10000

5000

0

Indonesians are increasingly using social media to discuss basic needs


So Are the Temporal Correlations

è Listening to social conversations provides insight on official data

2.5

2

1.5

1

0.5

0

-‐0.5

-‐1

-‐1.5
Social
Media
Food
Index

-‐2

BAPPENAS
Food
Price
Index

-‐2.5


Overview of Preliminary Findings

Food prices influenced by global events
1 •  Global soy bean shortage
•  Rising prices in Indonesia
•  Online complaints about affording food
•  New tool to detect impact faster?

2 Fuel prices affected by government policy
•  Cuts in subsidies => price rise
•  Social outcry - worry, uncertainty, anger
•  New tool to estimate vulnerability?

3 Social media predicts official statistics
•  New tool to derive price index?

The Way Forward

Building up Pulse Lab capabilities:
•  Build interdisciplinary R&D team that consists of
Government, private sector, academia, NGOs and
development partners.
•  Develop a framework for ensuring data privacy

Developing Partnerships:
•  Secure access to new sources of real-time data,
particularly with mobile carriers.
•  Secure access to state-of-the-art technology for analysis
•  Translate analysis technologies into Bahasa and other local
languages
•  Collaborate with BIG and CSIRO on geo-location projects.

Next Steps

•  Through key partnerships and projects, continue to
explore the usefulness of new digital data sources and
new analytical tools.

•  After gaining 2 years’ of experience with the Pulse Lab
approach, assess whether this approach can contribute
usefully to improvements in social protection, early
warning, and Government Development Planning


Dr. Vivi
Yulaswati, MSC
Director of Social Protection and Welfare /
Bappenas

Social Media Research:
Methods and Preliminary Results

OUTLINE

1. Mining Social Media: Topics & Research Questions

2. Research Tools & Results
•  Crimson Hexagon
•  SAS

3. Discussion


TOPICS FOR EXPLORATION

1. Food Prices
staple food (sembako), rice (beras), sugar (gula),
eggs (telur)

2. Fuel Prices
vehicle fuel (bensin), oil fuel (BBM), LPG (elpiji)


BIG QUESTIONS

•  How are food and fuel prices in Indonesia being
discussed on Social Media?
•  How can we monitor how they affect Indonesians?
•  Can social media serve as a real-time approximation of
official statistics?


Crimson Hexagon: Methods
Step
1
Human
Training

Select

Choose
Deﬁne
Label

Data
Sources
Keywords
Categories
Examples

beras Posi>ve

OR telur
OR gula Neutral
I
love
when
mrice
cooks

If
tofu
p um

eggs
instead
of
tempeh…

goes
higher,
I
can’t

Hope
prices
rmore
ore!

aﬀord
any
ise
m :(

Nega>ve

Irrelevant

èHuman
labels
a
few
examples
to
teach
machine.

Crimson Hexagon: Methods
Step
2
Machine
Classiﬁca9on

Consume
Match
Detect
Examine

Data
Sources
Keywords
Categories
Results

beras Posi>ve
+4%

OR telur
OR gula Neutral
-‐1%

Nega>ve
+3%

Irrelevant
-‐1%

èMachine
analyzes
millions
of
tweets
automa>cally.

Food Prices: Global Influence

23
July
25
-‐
27
July

Global
soy
bean
price
hike.
Indonesian
DPR
urges
government

produc>on
of
tempe
and
tofu
cut
back.
to
reduce
soy
bean

import
duty
and
give

subsidy.

10-‐22
July:
Not
many
men>ons
of
‘tempe’
and
‘tahu’

10
July
–
2
August
2012

Global
soy
bean
shortage.
23
-‐
28
July

‘tahu’
and
‘tempe’
become
Spikes
of
comments
on
tempe

and
tofu

hot
topics
in
Bahasa
tweets

Food Prices: Example Tweets
“nyokap
ngomel-‐ngomel
harga
tempe
tahu
naik
:D”

“Harga
Kedelai
Naik,
Sbecause
the
tdit,
buruan
nonton
sp
:D”
tar

“udeh..
kebanyakan
emes>nya
u
emerintah
Bprice’s
u ono
n
“mom’s
grumbling
ngomong
l P empe
tahu
eri
Subsidi”

“Harga
tempe
hprice
kedelai
nkenapa

k-‐
Nega9ve
subsidy”
ise
(Jul
26,
2012)

“Soy
bean
arga
hise,
Govt.
supposed
to
gmpor?,
rice
r
keburu
naik..,
r mmm,
aik
lagi...”

-‐ edelai
i ive
to
pmemangnya
tanah

Indonesia
ga
isubur
ud-‐-‐
Atnnouncement
with
no
t
en9ment
(Jul
2bean

“disaat
harga
tyou
tntuk
empe
nuch,
alhamdulillah
mama
msoy
8,
2012)

“stop
t..
ahu
alked
edelai?
#curhatpenggemartempe”

an
ktoo
m aik,
go
watch
i s before
the
asih

bikin
price
up
up..,
hmmm,
why
w
t price’s
again...”

“Tempe
ahu
dan
tempe
balado
:D”
e
import
soy
bean?,
is
Indonesian
land

inferCle
to
his
tofu
and
tempe
price
hike,
thanks
God
mom
sCll
can
6,
2012)

“during
t plant
soy?
#tempe-‐lover’s-‐vent-‐out”
no
emo9on
(Jul
2
-‐-‐
Realize
price
rise,

cook
spicy
tahu
and
tempe
:Confused
reac>on
to
price
rise
(Jul
28,
2012)

-‐-‐
D“

-‐-‐
Posi9ve/unaﬀected
by
price
rise
(Jul
30,
2012)

10
July
–
2
August
2012

Global
soy
bean
shortage.

‘tahu’
and
‘tempe’
become

hot
topics
in
Bahasa
tweets

Food Prices: Crisis Passes
23
July
-‐
2
Aug

‘tempe’
and
‘tahu’
hot
debate

14
-‐
21
Aug

Ramadhan
/
Idul
Fitri

10
July
–
25
August
2012

Ramadhan
/
Idul
Fitri

Food
men>ons
not
as
high
as

during
‘tempe’
&
‘tahu’
crisis

Food Prices: The Next Topic
“Tempe
tahu
udh
naik
harga,
skrng
daging
menyusul
+_+“

“Tempe
tahu
price
has
hiked,
now
meat
follows
+_+
”

-‐-‐
Aug
16,
2012

“Dedek
jgn
keseringan
minta
dibua>n
mie
telor
ya.
harga
sembako

“Langsung
belanja,
harga
daging
naik
!
mentok
sekilo
30rb.
#lol
“

naik.
mamah
susah.”

“Let’s
go
shopping
now,
meat
price
hike!
It
reached
Rp
30.000
per

“LiFle
kid,
please
don’t
ask
to
cook
noodle
egg
too
oGen.
The

kg.
#lol
”

staple
food
price’s
hiking.
Mom’s
troubled.”

-‐-‐
Aug
14,
2012

-‐-‐
Aug
16,
2012

14
-‐
21
August

‘daging’
(meat)
and

‘sembako’
(staple
food)

become
main
topics

Food Prices: Social vs Official

Spikes
in
social

media
conversa>ons

about
food
prices…

2

…
are
reﬂected
in

1.5

CPI,
%
change

inﬂa>on
sta>s>cs

1

measured
by
govt.

0.5

0

-‐0.5
February
March
April
May
June
July
August

-‐1

Fuel Prices: Local Event
“BBM
naik
,
harga
makanan
juga
naik
x_x
“

“Fuel
price
has
hiked,
now
food
follows

x_x
”

-‐-‐
Mar
27,
2012

“Waduh
bbm
naik
harga
makanan
naik
saya
bs
agak
kurusan
ntar”

“Whoa..

Fuel
price
rise,
the
food
price
rise,
then
I
will
become

“Bagi
rakyat
kecil
bkn
masalah
u/
mngurangi
pnggunaan
BBM,
tp

slimmer”

efek
domino
dr
knaikan
BBM
(harga
pangan,
trnsportasi
umum

-‐-‐
Mar
26,
2012

naik)
yg
memberatkan”

“For
poor,
reducing
fuel
usage
is
not
the
problem,
but
the
domino

eﬀects
of
fuel
price
rise
(food,
public
transportaCon
price
rise)
is”

-‐-‐
”
Mar
30,
2012

18
March
–
7
April

Fuel
subsidy
cut
plan
and

protests:
‘bbm’,
‘makanan’
&

‘sembako’
trending
topics

Fuel Prices: Timeline
27
Mar:
Massive
protests
against

fuel
price
rise
in
Jakarta.

25
Mar:
Some
DPR
members
agree
to

increase
fuel
price.
Protests
began
31
Mar:
Govt.

several
days
before.
announces
it

22
Feb:
News
of
government
plan
to
will
restore

increase
subsidized
fuel
price
subsidies.

Few
men>ons
of

“Fuel”
on
Twiner

15
Feb
–
15
April
2012

Government
plan
to
reduce

fuel
subsidies
triggered

protests

Comparing Crises
Tweets
about
food

18
Mar
-‐
7
Apr

23
July
-‐
2
Aug

Fuel
subsidy
cut
plan
and

‘tempe’
and
‘tahu’
hot
debate

protests
against
it

Social Protection Programs
‘Social
Protec>on
Program’
–
men>ons
gepng
higher
over
>me,

esp.
near
Jakarta
Governor
elec>on

11
July
‘12:

20
September
‘12:

Jakarta
governor
elec>on,

Jakarta
governor
elec>on,

1st
round
2nd
round

OUTLINE

1. Mining Social Media: Topics & Research Questions

2. Research Tools & Results
•  Crimson Hexagon
•  SAS

3. Discussion

Source:

SAS Analytic Workflow
1)
Over
200,000
new

5)
Explore
results
and
correlate

3)
Capture
sen>ment
with
official
sta>s>cs
to
official

Indonesian
language
and
mood
for
Bahasa

BPS
sta>s>cs
:
Consumer
Price

documents
per
day

Index
(CPI)
for
12
common
foods

Global
Pulse
Sen>ment,
Topic
&

Internet
Relevance
Mood
&
Geography
Interac>ve

Conversa>on
Filter
Influence
Categories

Dashboard

2)
Extract
conversa>ons
4)
Detect
loca>on,
price,

about
rice,
cooking
oil,
availability,
specific

fuel,
employment,
etc.
govt.
programs,
etc.

anxious,

confident,

confused,

hos>le,

sad,

happy

(-”-) ;-) ((+_+)) :@ :( :)

hnp://youtube.com/watch?v=MK-‐aHVfQ3BI

Indonesian Data Density

è Data density across archipelagos of Indonesia

The signals are growing stronger
èBig increase in volume of relevant conversations over 18 months

40000

35000

minyak
(oil)

ketahanan
pangan
(food
security)

30000

budidaya
(cul>va>on)

25000

telur
(eggs)

20000

15000

10000

5000

0

Indonesians are increasingly using social media to discuss basic needs

B
So are the temporal correlations
èListening to social conversations provides insight on official data

2.5

2

1.5

1

0.5

0

-‐0.5

-‐1

-‐1.5
Social
Media
Food
Index

Social
edia
Food
Index

-‐2

BPS
Food
PFood
ndex
I
ndex

BAPPENAS
rice
I Price

-‐2.5

Linear Regression Analysis
Social
Media
Topics
BAPPENAS
Sta>s>cs

0.9%
Agricultural
Cul9va9on

(budidaya
pertanian)

Eggs

-‐0.5%
(telur)

Food
Security

=
0.89
Inﬂa>on
(CPI)

0.1%
(ketahanan
pangan)

Cooking
Oil

-‐0.1%
(minyak
goreng)

Regression
Weights
Correla9on
Coeﬃcient

The
predic>ve
power
of

0
=
no
rela>onship

each
conversa>on
topic

1
=
perfect
correla>on

ADVANTAGES

•  Real-time detection of trends to understand community
concerns & coping mechanisms

•  Detect quick response to social protection plans and
policies

•  Augment and complement official statistics with real
time indicators

Source:

CAVEATS

•  Current social media users do not represent the entire
population
=> can’t measure direct response to poverty issues
•  Twitter content good for immediate, short-term trends
=> does not capture long-term aspirations and intentions
•  Machine classification methodologies need human
supervision
=> are not able to deal with sarcasm, and complex human
communications
•  Correlation ≠ Causation
•  Noise, rumors & reliability

Source:

NEXT STEPS

New applications
–  health
–  natural disasters
–  intention to migrate

New datasets
–  additional social media sources
–  mobile phone call detail records (CDRs)

New research projects ready to start
–  UNICEF (child welfare and education)
–  ILO (female labor condition and employment access)

Source:

Oktorialdi
MA, PhD
Head of PUSDATIN, Bappenas


Pulse Lab Operations &
Roadmap


Innovation Lifecycle
Phase 1 – Research

PHASE 1 – Research Phase

Carry out a series of “Proof of Concept” projects exploring
the utility of various sources of new digital data:

•  Exploratory research on social media, mobile data
(CDRs) and other relevant sources
•  Development or adaptation of technology tools for
analysis
•  Conduct analysis of historical data, and compare
against official statistics, to look for correlations and
proxy indicators
•  Publication and sharing of findings

Phase 2 – Prototype

PHASE 2 – Prototype Phase

If Proof-of-Concept phase proves that there are useful
insights within the new data source, built prototypes for
real-time monitoring:

•  Obtain sustainable access to digital data source in real-
time
•  Develop technology prototype for ongoing analysis
•  Sample from 4 cities: Medan, Jakarta, Jogjakarta,
Makassar


Phase 3 – Implementation

PHASE 3 – Use of Data in Real-Time
Decision Making and Planning

If prototypes are useful to the work of line ministries and
programme planning, mainstream and operationalize the
technique and the tools:

•  Mainstreaming use of new indicator/data source
•  Expand to larger-scale operational data analysis after a
year
•  Operationalize data monitoring software, hardware and
human expertise


YEAR 1 ROADMAP
(2012 – 2013)

² Move to permanent Lab space
² Series of Proof-of-Concept research projects and prototypes
² Forge new partnerships around data, technology and
expertise
² Development of legal and data privacy frameworks
² Organize workshops & meet-ups on Big Data for
Development
² Knowledge sharing & publication of findings from research
projects


Work with Pulse Lab Jakarta
Research Project Partners

•  Explore and test new digital data sources that could
contribute to real time program monitoring, or
complement an existing monitoring system.

•  Test data sources that could serve as proxy
development indicators.


Partner with Pulse Lab Jakarta
Innovation Partners

•  Data Sharing Partners
•  Research Partners
•  Technology Partners
•  Host a Training Session
•  Academic Fellowships
•  Funding Support


Q&A


Innovation
Showcase
CSIRO SAS International

Mercy Corps Microsoft LIFE

Nokia Life PoliticaWave


BAPPENAS


Thank You!


Pulse Lab Jakarta Launch Presentation

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Destacado

Destacado (12)

Similar a Pulse Lab Jakarta Launch Presentation

Similar a Pulse Lab Jakarta Launch Presentation (20)

Más de UN Global Pulse

Más de UN Global Pulse (20)

Último

Último (20)

Pulse Lab Jakarta Launch Presentation