This document discusses the need for combining big data and ethnography. It summarizes that big data alone is not sufficient and can lead to issues like bias, incomplete representations, and misinterpretations without context. Ethnography provides thick contextual data through open-ended research methods like observation and interviews. The document advocates a mixed methods approach, using ethnography to understand problems and qualitative insights to complement and validate quantitative big data findings. This leads to more human and reflective understandings that benefit both innovation and optimization.

1. www.azimuthlabs.io
Copyright © 2018
BIG DATA NEEDS
WHY
Presented by Matt Artz & Dr. Uldarico Rex Dumdum
ETHNOGRAPHY

2. www.azimuthlabs.io
Copyright © 2018
AZIMUTH LABS
Matt Artz
M.B.A., M.S.
Matt Artz is the founder and principal researcher at Azimuth Labs, an ethnographic research company
offering product development, organizational culture, and corporate strategy consulting. Using
ethnography, he helps organizations unlock hidden insights with users, employees, data, markets, and
products.
Matt has led research, design, and agile product management efforts to ship and scale web and mobile
products in the enterprise and consumer space. He has experience working in energy, biotechnology,
healthcare, financial services, telecommunications, entertainment, fashion, broadcasting, and nonprofit.

3. www.azimuthlabs.io
Copyright © 2018
MARYWOOD UNIV
Dr. Uldarico Rex Dumdum
M.E., M.S., M.B.A., Ph.D
Dr. Uldarico REX Dumdum is an Associate Professor of Information Systems and Leadership in the
School of Business and Global Innovation at Marywood University.
His research interests include problem formulation in ill-structured complex situations, sensemaking,
requirements engineering, contextual intelligence, innovation, convergence of technology, strategy and
leadership, leadership development, and leadership in face-to-face and virtual settings.

4. www.azimuthlabs.io
Copyright © 2018
“All business is about
making bets on human behavior.”
—Christian Madsbjerg, Red Associates 2014

5. www.azimuthlabs.io
Copyright © 2018
ABOUT BIG DATA?

6. www.azimuthlabs.io
Copyright © 2018
I B M
Big data is a term
applied to data sets
whose size or type
is beyond the
ability of traditional
relational
databases to
capture, manage,
and process the
data with low-
latency.

7. www.azimuthlabs.io
Copyright © 2018
“The worldwide business intelligence and
analytics market would reach $18.3 Billion.”
—Gartner 2017

8. www.azimuthlabs.io
Copyright © 2018
“The global Big Data software
market will be worth $31B this year.”
—Forrester 2018

9. www.azimuthlabs.io
Copyright © 2018
“Only 37% of companies who are trying
to be data-driven have been successful”
—NewVantage Partners 2017

10. www.azimuthlabs.io
Copyright © 2018
2018 Success RatesNewVantage Venture Partners & Statista Survey

11. www.azimuthlabs.io
Copyright © 2018
WHY BIG DATA?

12. www.azimuthlabs.io
Copyright © 2018
UNDERSTANDING INNOVATIONOPTIMIZATION
Identify new markets and
products through richer
understanding.
Enhance decision-making by
leveraging data to create
insights.
Optimize business process
by better understanding the
operations.
Big Data PromisesWhat the Proponents Believe

13. www.azimuthlabs.io
Copyright © 2018
“With enough data, the
numbers speak for themselves.”
—Wired Editor in-Chief, Chris Anderson 2008

14. www.azimuthlabs.io
Copyright © 2018
“79% of enterprise executives agree that
companies that do not embrace big data will
lose their competitive position and could
face extinction.”
—Accenture 2018

15. www.azimuthlabs.io
Copyright © 2018
“83% have pursued big data
projects to seize a competitive edge.”
—Accenture 2018

16. www.azimuthlabs.io
Copyright © 2018
“The often implicit assumption that big data is a substitute for, rather
than a supplement to, traditional data collection and analysis.”
The Parable of Google Flu: Traps in Big Data Analysis
David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani. 2014
DATA HUBRIS

17. www.azimuthlabs.io
Copyright © 2018
- GFT overestimated the prevalence of flu in the 2012–2013 season
and overshot the actual level in 2011–2012 by more than 50%.
- From 21 August 2011 to 1 September 2013, GFT reported overly
high flu prevalence 100 out of 108 weeks.
- The explanation was increased media coverage, however in 2009
Google tried to correct for this problem.
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., R. Kennedy, G. King, and A. Vespignani. 2014
Google Flu TrendsA Case Study in Data Hubris
GOOGLE FLU TRENDS OVERESTIMATION

18. www.azimuthlabs.io
Copyright © 2018
"The ability to truly understand someone
other than you is not something that can be
broken down into 1s and 0s.”
—Christian Madsbjerg, Red Associates 2018

19. www.azimuthlabs.io
Copyright © 2018
- In 2009 Tricia Wang recommended to Nokia that low-income
consumers were ready to pay for more expensive smartphones.
- Nokia claimed her ethnographic data was to small in sample
size, and the power of their big data analytics informed them
otherwise.
- Reduced Nokia’s market share to 4% in 2014, from 35% a
decade ago.
Why Big Data Needs Thick Data
Tricia Wang, 2013
Nokia & SmartphonesA Case Study in Thick Data
NOKIA'S QUANTIFICATION BIAS

20. www.azimuthlabs.io
Copyright © 2018
"What is measurable
isn’t the same as what is valuable.”
—Tricia Wang 2013

21. www.azimuthlabs.io
Copyright © 2018
So we ask, can big data speak?

22. www.azimuthlabs.io
Copyright © 2018
If yes, who does it speak for?

23. www.azimuthlabs.io
Copyright © 2018
Can big data be neutral?

24. www.azimuthlabs.io
Copyright © 2018
We contend big data
alone is not sufficient.

25. www.azimuthlabs.io
Copyright © 2018
BIG DATA PROBLEMS

26. www.azimuthlabs.io
Copyright © 2018
“Technology’s interaction with the social ecology is such that technical
developments frequently have environmental, social, and human
consequences that go far beyond the immediate purposes of the
technical devices and practices themselves”
- Kranzberg, M. ‘Technology and History: Kranzberg’s Laws’. 1986.
TECHNOLOGY IS NOT NETURAL

27. www.azimuthlabs.io
Copyright © 2018
Meta ProblemsThe Culture of Big Data
MYTHOLOGY COMPLETENESS REFLEXIVITYINTERPRETATION
The culture of dig
data is closed and
lacks reflection.
All researchers are
biased, even if it is
quantitative.
The sampling & data
collection does not
represent the whole
There is a prevailing
belief that big data
alone is sufficient.

28. www.azimuthlabs.io
Copyright © 2018
B O Y D & C R A W F O R D , 2 0 1 2
Big data is a
cultural,
technological, and
scholarly
phenomenon that
rests on the
interplay of
technology, analysis,
and mythology.”

29. www.azimuthlabs.io
Copyright © 2018
S A R A H P I N K , 2 0 1 7
Data is always
incomplete.. It does
not tell us enough
contextual
information about
the people involved,
the environments
and contingent
circumstances that
they are in.

30. www.azimuthlabs.io
Copyright © 2018
D A N A H B O Y D , 2 0 1 0
Interpretation is the
hardest part of doing
data analysis. And
no matter how big
your data is, if you
don't understand the
limits of it, if you
don't understand
your own biases,
you will misinterpret
it.

31. www.azimuthlabs.io
Copyright © 2018
L U C Y S U C H M A N , 2 0 1 1
As Levi Strauss
said, ‘we are our
tools’ and thus we
should consider how
the tools participate
in shaping the world
with us as we use
them.

32. www.azimuthlabs.io
Copyright © 2018
“It is still necessary to ask critical questions
about what all this data means, who gets
access to what data, how data analysis is
deployed, and to what ends.”
—Boyd & Crawford 2012

33. www.azimuthlabs.io
Copyright © 2018
We argue big data
does not speak.

34. www.azimuthlabs.io
Copyright © 2018
Nor does big data
speak equally for all.

35. www.azimuthlabs.io
Copyright © 2018
Because big data
is not neutral.

36. www.azimuthlabs.io
Copyright © 2018
WHY ETHNOGRAPHY

37. www.azimuthlabs.io
Copyright © 2018
P R I N C E T O N U N I V
The systematic
collection of
diverse types of
data through
observation,
conversation, and
textual study.

38. www.azimuthlabs.io
Copyright © 2018
“To have impact, numbers need stories.”
—Tricia Wang 2013

39. www.azimuthlabs.io
Copyright © 2018
“Stories Are Just Data With A Soul.”
—Dr. Brené Brown 2010

40. www.azimuthlabs.io
Copyright © 2018
Deeply HumanThe Strengths of Ethnography
A deep exploration of the
human experience to
contextualize data.
THICK DATA
02
The process of openly
exploring our beliefs,
practices and concepts of
knowledge.
REFLEXIVITY
03
Intentionally acknowledges
researcher bias in collection,
analysis, and interpretation.
ACKNOWLEDGED BIAS
01

41. www.azimuthlabs.io
Copyright © 2018
R A T T E N B U R Y & N A F U S 2 0 1 8
The ethnographic
approach to this
issue is to treat
bias as data about
the phenomenon to
be explained—not
as a corrupting
factor to be
eliminated.

42. www.azimuthlabs.io
Copyright © 2018
T R I C I A W A N G 2 0 1 6
Thick data grounds
are business
questions in human
questions... and
rescues the context
loss that comes
from making big
data usable and
leverage the best of
human intelligence.

43. www.azimuthlabs.io
Copyright © 2018
M A R G A R E T M E A D
Scientists have to
learn how to relate
self-knowledge as a
multisensory being
with a unique
personal history as
a member of a
culture at a specific
period to ongoing
experience.

44. www.azimuthlabs.io
Copyright © 2018
OUR APPROACH

45. www.azimuthlabs.io
Copyright © 2018
Mixed MethodsCombine Big Data & Ethnography
Continue to reflect on our
process, and use
research and design to
iterate on it.
Iterate the Process
Data scientists and social scientists practicing ethnography should be
paired with each other in the process of asking question, listening,
defining, reframing, collecting, analyzing, and presenting the data.
The Goal
Acknowledge our
assumptions & limitations
when interpreting data.
Account for Bias
Combine offline with
online data to add depth
and understand context.
Seek the Context
From participants to
researchers, focus on the
people behind the
project.
Focus on People

46. www.azimuthlabs.io
Copyright © 2018
F O R I N N O VAT I O N
Use qualitative studies to first define the problem
space, and define and prototype some possible
solutions. Then use quantitative studies to
validate.
EXPLORATORY SEQUENTIAL DESIGN
F O R O P T I M I Z AT I O N
Use quantitative studies to understand the current
performance and to identify anomalies. Then use
qualitative studies to explain the findings.
EXPLANATORY SEQUENTIAL DESIGN

47. www.azimuthlabs.io
Copyright © 2018
- In 2013 Netflix knew from their quantitative data that people were
watching, but they couldn’t explain what, why, and how.
- They hired Grant McCracken to conduct an ethnography of
consumers to understand these offline decisions.
- He found people are watching good TV with a kind of passionate
and critical engagement, where they second-guess casting
decisions and camera angles and take almost a practitioner’s
pleasure in observing how the thing is crafted, even as they are
caught up by the craft.
- Now Netflix offers the same content continuously instead of
related content, which has proved quantitatively effective.
Netflix & BingingA Case Study in Mixed Methods
NETFLIX SEEKS A MIX METHODS APPROACH

48. www.azimuthlabs.io
Copyright © 2018
Leverage SensemakingUsing Research in the Service of People
2
3
4
5
1
6 YOUR
GREAT
TITLE
LISTEN
REFRAME
STRATEGY
INSIGHTS
RESEARCH
MINING
Strategy
Use the insights to inform a
business strategy.
Insights
Synthesize the patterns to
generate actionable insights.
Reframe
Reframe the business problem
from the customer perspective.
Listen
Listen with empathy to the
business problem.
Mining
Mine the data for patterns that tell
a story.
Research
Conduct the research and collect
the data.

49. www.azimuthlabs.io
Copyright © 2018
“By outsourcing our thinking to Big Data,
our ability to make sense of the world by
careful observation begins to wither, just as
you miss the feel and texture of a new city
by navigating it only with the help of a
GPS.”
—Christian Madsbjerg, Red Associates 2014

50. www.azimuthlabs.io
Copyright © 2018
www.azimuthlabs.io
DISCOVER THE PEOPLE
Behind Your Products, Culture & Strategy