Slides from "Big Data at Human Scale," delivered at the Wharton Web Conference, July 2013.

1. BIG DATA AT HUMAN SCALE.
Matt LeMay, @mattlemay

2. BIG DATA
IS BIG

3. How BIG is it?

4. We have built the capacity to store
more bytes of data than the
Earth has grains of sand.

5. ... about 315 times more.

6. If each bit of data we have the capacity to
store were to represent a star, then
there would be a GALAXY OF
DATA for every person on Earth.

7. The data Walmart generates every hour from
its customer transactions represents 167 times the
information contained in all the books in the United
States Library of Congress.
PWNED

10. The number of bytes
we’ve built the
capacity to store
constitutes only a
TINY FRACTION
of the number of
atoms you have in
your body.

11. ... or the amount of
data stored in your
DNA.

12. In fact, the data storage capacity of the entire
world is less than one percent of the information
stored in the DNA molecules of a single person.

13. as we approach human scale...
...big data seems smaller.

14. ... but it’s bigger than it’s ever been before.

15. =
ALL the data
created until the
year 2003
ALL the data
created every
two days

16. Scale of Data ~3,000 Years Ago:

17. Scale of Data ~300 Years Ago:

18. Scale of Data ~30 Years Ago:

19. Scale of Data ~3 Years Ago:

21. We’ve been writing stuff on walls for 30,000 years...
... and we’re still not entirely what it all means.

22. “BIG DATA” is US*,
in higher resolution.

23. “We’re distracted by a bunch of nonsense.”

24. “Ephemeral thoughts and actions, which were once
lost to time, are now recorded forever.”

25. That record is “BIG DATA.”

29. According to , 43% of all data
gathered on people comes from social media.

31. We overshare compulsively, but we are more
concerned than ever before about our privacy.

32. Privacy vs Permission

33. Privacy = “My data is valuable, and
others want access so that they can spy
on me or sell me stuff I don’t want.”
Permission = “My data is valuable, so
I will explicitly grant others access to it
in specific situations where it is
worthwhile for me to do so.”

34. Privacy is something we need to worry about
when expectations are violated around the
permissions we agree to.

36. Even explicit permission...
... doesn’t override expectation.

37. ... often struggles to square permission with
expectation, at times to their own detriment.

38. weknowwhatyouredoing.com

41. We expect clicks to be private gestures,
and shares to be public gestures.
Facebook’s social reader violated those
expectations.

42. We share who we want to be.
We click who we fear we are.

43. ... and it matters.

44. We share our
information
because we trust
that sharing will
make it more
valuable to us.

45. “The future has an ancient heart.”
- Carlo Levi

46. My data Your data
BIG DATA “MAGIC”
Me You

47. BIG DATA “MAGIC”
“HADOOP!”

48. MAGICKAL RABBITS OF INSIGHT!!11
Me You

49. ... but “BIG DATA” is not magic.

50. “MAGIC BIG DATA TECHNOLOGY”
is a set of tools...
... necessitated by scale.

51. - Tim O’Brien, O’Reilly Strata Conference

52. COUNTING
is not
UNDERSTANDING

53. THE ALGORITHM
WON’T SAVE YOU

54. BIG DATA is only as
good as the questions
we ask of it.

55. ... and many of those questions haven’t changed.

56. Loyalty clubs and targeted coupons are the
oldest trick in the “big data” book.

57. - Andrew Pole,Target

58. Big Data could make advertising and
marketing better.*
(Which will, in turn, hopefully pay for all those nifty services we use to generate all that data.)

59. Twitter Search == BIG Data.

62. *

63. ... but the potential goes beyond advertising.

64. When done right, BIG DATA encourages
you to SHARE MORE, not less.

65. “BIG DATA” is all around us.

66. ...and it doesn’t feel ZOMG WORLD-CHANGING
... because it’s in our cells.

67. Thank you.
Questions?
@MATTLEMAY