We are living in a time of tremendous convergence, convergence of mobile, cloud and social…This convergence is forcing companies to change. At Nokia, we are changing the way we make decisions, from a manufacturing model to a data driven one. Yet making cultural changes is one of the hardest things to accomplish. In this talk, Amy O’Connor will highlight the journey Nokia is taking to evolve its culture - from building a platform for cultural evolution on top of Hadoop, to the administration of Nokia’s data, to how the company conducts the analysis that is enabling Nokia to compete with data.

1. Using Hadoop to
Change Company Culture
Amy O’Connor
Senior Director, Analytics
1

2. The Amazing Everyday
2

3. The Amazing Everyday
3

4. NOKIA’S HISTORY: 1865 TO Now
4

5. Data
is our newest
raw material
5

6. TRAFFIC
US Government data
suggests worsening
conditions in urban
areas…this year
average commute time
has risen by 9 minutes.
Federal Highway
Commission, Urban
Congestion Report
January 2011 to
March 2011
6

7. TRAFFIC
Global car sales are
growing: there will
be 50% more cars
sold in 10 years as
there are today.
IHS Automotive,
October 2011
7

8. More data usually beats
better algorithms
Anand Rajaramann
8

9. Devices in • Image Sensors
use around • Accelerometers
the world
• Gyroscopes
• Compasses
• Pressure Sensors
Probe points • Microphones
collected
monthly from
• Light Sensors
Nokia alone • Assisted GPS
9

10. 18M 24
80M
10

11. Data Storage & Analysis Landscape
11

12. Data Silos
Traffic Search Consumer
Probes Logs Profile
Ad Places Device
Data Registry Data
12

13. Smart Data
Combining
sets of
behavioral &
contextual data
13

14. A Good Way to Change
Corporate Culture
14

15. Getting Children
to Eat Peas…
Tell them you expect them to
eat their peas.
Reward them with ice cream
if they did.
Explain why it’s good for
them to eat their peas.
Eat your own peas as a good
role model.
Leann Lipps Birch,
Head of Human Development &
Family Studies
Pennsylvania State University
15

16. Getting Children
to Eat Peas…
Put them with children who
love peas.
Change the stories
they tell.
Leann Lipps Birch,
Head of Human Development &
Family Studies
Pennsylvania State University
16

17. Identity Media Care & Marketing
Consumer Profile Products, Transactions Device/User CRM
SSO Songs, Delivery Activation Info
Device Activation Net Promotor
Advertising
Campaigns PC Suite
Ad Inventory
Contacts
Campaign Promotions Location
Navteq Ad Canvas Premium Content
All Probes Data Favorite Routes
3D Imagery Nokia Log Files
Street View
Data Map Tiles
Feature Recognition
Asset Imagery
Device Programs Social Search
NAC, IIA Social UGC Log Files
Windows Phone Universal Share Points of Interest
Panel data Journeys Data
Nokia IT
Equipment Master Event Info
Registrations
Factory Data
Device Updates
17

18. Users
Analytics
Decision- Domain Offline Predictive
Makers Expertise Dashboards Analysis Analytics
Data Domain Expertise, Key Value
Analysts Statistical Skills TeraData Store
Domain Expertise,
Data Oozie Map Pig Hive
Statistical Skills, Reduce
Scientists
Computer Science
Flume HBas
e
HDFS
Developers/ Domain Expertise, Scrib HBas
Applications Computer Science e e
FTP HBas
e
18

19. Collaborative Working Model
Present
Analytics
Offline Predictive
Analyze Dashboards Analysis Analytics
and
Aggregate
Key Value
TeraData Store
Load
Oozie Map Pig Hive
Reduce
Transform
Flume HBas
e
Extract HDFS
Scrib HBas
e e
FTP HBas
Platform e
19

20. Collaborative Working Model
To BI Tools:
Present
DataOS, Customer Analytics
SPSS, AD Hoc
Structured Dashboard Tableau, Reporting
Data Cognos
Offline Predictive
Analyze Hive QL
Dashboards Mahout
MR Analysis Analytics
Rec,
and and Pig Analysis Machine
Engines
Aggregate Queries Job Learning
Key Value
TeraData MR Agg Store Agg
MR
User Create
Load Metadata Job and Job and
Metadata Hive
Catalog Data to Data to
Interfaces Schema
Oracle Teradata
Oozie Map Pig Hive
Monitor Reduce
Create Data Develop Develop Develop
Transform Library of and Model Cleanse Validation Partition
MR Jobs Manage Definition Job Job Job
Transform
Flume HBas
Catalog Define
Monitor
Integrate Integrate e
Define
Extract Data Standard
and
Manage
Historical HDFS
Streaming Custom
Sources ETL
Scrib Data Data ETL
HBas
Data Feed
e e
FTP HBas
Platform Co-located developer clusters, pre-production cluster, product cluster e
Data OS Product Teams and/or DataOS
20

21. Smart Data: Behavioral & Contextual
Analytics
Aggregation
• Update top searches table and
Aggregation table in Oracle
geo activity Dashboards
Offline Predictive
Analysis Analytics
• Merge multiple data sources
• Implement app logic
(e.g.; round up Key Value
Standard ETL
latitude/longitude to TeraData Store
• Clean data, remove bad record
3 decimals)
and health checks
• Partition data by date, hour, type Map
Oozie Pig Hive
• Archive raw data Reduce
Flume HBas
Ad Router e
HDFS
Scrib HBas
NAC e e
FTP HBas
Local Search e
21

22. Merchant Portal Heatmap
22

23. Mapping the World
23 © 2011 Nokia Company Confidential
23

24. Mapping the World
Probe data indicates the location, speed,
heading, time etc. about a mobile device.
Billions of probe records per week.
Covers almost the entire world.
24 © 2011 Nokia Company Confidential
24

25. Probe Density: Urban and Arterial
100% AGR
25

26. 792527719 (Jackson Blvd/Financial Pl)
From Ref, One lane
26

27. 27

28. 24 Hours in Our Analytics Ecosystem
~2TB ingested
350M messages via scribe
>3000 MR jobs
10TB processed
28

29. Spreading the Word
Data Asset Catalog
Smart Data Newsletter Stories
Realtime Dashboards
29

30. The Amazing Everyday
Thanks!
30