Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Strata hadoop world 2015 context computing - jonas keynote - final

962 visualizaciones

Publicado el

Context Computing: My keynote at Strata & Hadoop World 2015.
http://strataconf.com/big-data-conference-ny-2015/

Publicado en: Datos y análisis
  • DEAR FRIEND, I AM MR. MORRIS COULIBALY. I HAVE A LATE CLIENT WHO LEFT THE SUM OF $11. 5 MILLION DOLLARS IN OUR BANK I AM HIS PERSONAL ACCOUNT OFFICER I CONTACTED YOU BECAUSE YOU HAVE THE SAME LAST NAME OR SURNAME WITH THE DECEASED CLIENT AND I CAN PRESENT YOU AS THE BENEFICIARY AND NEXT OF KIN TO THE FUND SINCE YOU BEAR THE SAME SURNAME WITH MY LATE CLIENT. THE FUND WILL BE SHARE AMONG TWO OF US 50% EACH . I WAIT TO HEAR FROM YOU SO THAT I WILL GIVE YOU MORE DETAILS ON HOW THE FUND WILL BE RELEASE AND TRANSFER INTO YOUR BANK ACCOUNT . PLEASE CONTACT ME BACK IF YOU ARE INTERESTED FOR MORE DETAILS BEST REGARDS, . MR. MORRIS COULIBALY
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Strata hadoop world 2015 context computing - jonas keynote - final

  1. 1. © 2015 IBM Corporation Context Computing Strata + Hadoop World 2015 Jeff Jonas, IBM Fellow Chief Scientist, Context Computing http://www.twitter.com/jeffjonas www.jeffjonas.typepad.com
  2. 2. © 2015 IBM Corporation Jeff Jonas IBM Fellow Chief Scientist, Context Computing  Founded Systems Research & Development (SRD) in 1985  Architected, designed, developed roughly 100 systems over the last three decades – Financial services – Defense, intelligence – Manufacturing – Humanitarian efforts  Acquired by IBM in 2005  Currently focused on Context Computing, Sensemaking and Privacy by Design
  3. 3. © 2015 IBM Corporation No Context Newsletter Subscriber
  4. 4. © 2015 IBM Corporation Context “Better understanding something by taking into account the things around it.”
  5. 5. © 2015 IBM Corporation I ducked as the bat flew my way. Another exciting baseball game.
  6. 6. © 2015 IBM Corporation In Context Social Media Influencer Newsletter Subscriber Loyalty Club Member High Value Customer Job Applicant Watch Listed Party
  7. 7. © 2015 IBM Corporation Context Accumulating Context Accumulation Contextualized Observations Observation (Any kind of data from any kind of sensor)
  8. 8. © 2015 IBM Corporation Context Informs Decisioning Context Accumulation Contextualized Observations Observation In Context Decisioning Act Data Finds Data Relevance Finds You Observation (Any kind of data from any kind of sensor)
  9. 9. © 2015 IBM Corporation The Puzzle Metaphor  Imagine an ever-growing pile of puzzle pieces of varying sizes, shapes, colors  What it represents is unknown – there is no picture on hand  Is it one puzzle, 15 puzzles, or 1,500 different puzzles?  Some pieces are duplicates, missing, incomplete or have errors  Some pieces may even be professionally fabricated lies  Until you take the pieces to the table, it is nearly impossible to assess the scene
  10. 10. © 2015 IBM Corporation Puzzling Images: Courtesy Ravensburger © 2011 270 pieces 90% 200 pieces 66% 150 pieces 50% 6 pieces 2% 30 pieces 10% (duplicates)
  11. 11. © 2015 IBM Corporation
  12. 12. © 2015 IBM Corporation
  13. 13. © 2015 IBM Corporation First Discovery
  14. 14. © 2015 IBM Corporation More Data Finds Data
  15. 15. © 2015 IBM Corporation Duplicates in Front Of Your Eyes
  16. 16. © 2015 IBM Corporation First Duplicate Found Here
  17. 17. © 2015 IBM Corporation
  18. 18. © 2015 IBM Corporation Incremental Context – Incremental Discovery 6:40pm START 22min “Hey, this one is a duplicate!” 35min “I think some pieces are missing.” 37min “Looks like a bunch of hillbillies on a porch.” 44min “Hillbillies, playing guitars, sitting on a porch, near a barber sign and a banjo!”
  19. 19. © 2015 IBM Corporation 150 pieces 50%
  20. 20. © 2015 IBM Corporation Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” 2hr “Let’s switch sides, and see if we can make sense of this from different perspectives.” 2hr10m “Wait, there are three … no, four puzzles.” 2hr18m “I think you threw in a few random pieces.”
  21. 21. © 2015 IBM Corporation
  22. 22. © 2015 IBM Corporation How Context Accumulates  With each new observation one asserts: 1) unrelated; 2) related; or 3) connected  Must favor the false negative  New observations sometimes reverse earlier assertions  Some observations produce novel discovery  The emerging picture helps focus collection interests
  23. 23. © 2015 IBM Corporation Big Data [in context]. New Physics. More data: better the predictions – Lower false positives – Lower false negatives More data: bad data good – Suddenly glad your data is not perfect More data: less compute
  24. 24. © 2015 IBM Corporation Big Data Pile of ______ Information In Context
  25. 25. © 2015 IBM Corporation One Essential Form of Context: “Entity Resolution”  Is it 5 people each with 1 account or is it 1 person with 5 accounts?  Is it 20 cases of SARS in 20 cities or one case reported 20 times?  If one cannot count, one cannot estimate vector or velocity (direction, speed).  Without vector and velocity prediction is nearly impossible.
  26. 26. © 2015 IBM Corporation Who is Fang Wong? Fang Wong Top 100 Customer F A Wong Seattle, DOB: 6/12/82 Former Customer @FangWong 2.5M Followers FangWong@Email.com Newsletter Subscriber Fang Wong FangWong@Email.com Marketing Department’s Prospect List
  27. 27. © 2015 IBM Corporation Resolving the Fang Wong Fang Wong Top 100 Customer F A Wong Seattle, DOB: 6/12/82 Former Customer @FangWong 2.5M Followers FangWong@Email.com Newsletter Subscriber Fang Wong FangWong@Email.com Marketing Department’s Prospect List
  28. 28. © 2015 IBM Corporation Resolving the Fang Wong Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber
  29. 29. © 2015 IBM Corporation Graphing the (resolved) Fang Wong Bill Smith Member of the Board Employee Customer Customer Fraudster Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber
  30. 30. © 2015 IBM Corporation Contextualizing Sandy Maden Bill Smith Member of the Board Sandy Maden New Account Employee Lives With Co-signer Former Customer Customer Customer Customer Fraudster Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber
  31. 31. © 2015 IBM Corporation “Entities” Bill Smith Member of the Board Lives With Co-signer Sandy Maden New AccountFormer Customer Employee Customer Customer Customer Fraudster Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber Company Boat Plane Asteroid Car
  32. 32. © 2015 IBM Corporation Asteroid Hunting
  33. 33. © 2015 IBM Corporation Single Detection Image courtesy of: Eva Lilly, Institute of Astronomy, University of Hawaii
  34. 34. © 2015 IBM Corporation From Orphans to Orbits Single Detections (trash) Tracklette Track Orbit Forecasting Named entity: S100ZUtza Single Detection (orphan) Anticipation
  35. 35. © 2015 IBM Corporation http://www.space.com/7854-slam-asteroids-suspected-space-collision.html
  36. 36. © 2015 IBM Corporation "We have directly observed a collision between asteroids for the first time, instead of having to infer that they happened from million-year-old remains." Colin Snodgrass Planetary Scientist Max Planck Institute for Solar System Research
  37. 37. © 2015 IBM Corporation Geospatial Context via “Space Time Boxes”
  38. 38. © 2015 IBM Corporation Detecting Colocation TIME 1 day 1 hour Determine encounter distance and time Space Time Boxes
  39. 39. © 2015 IBM Corporation Computing 600k Asteroid Interactions over 25 Years 4-5 orders of magnitude improvement Initial Analysis Adding 1 New Trajectory Space-Time Box Method 2,880 CPU hours 15 CPU minutes N-body Simulation Method 10,000,000 CPU hours 4,000 CPU hours
  40. 40. © 2015 IBM Corporation Asteroid vs. Asteroid Encounters Encounter Distance Asteroid 1 Size Asteroid 2 Size May 1, 2032 299km 00A9170 2-4km 0008758 4-9km Nov 24, 2016 449km 00P5634 1-2km 0055711 2-5km Jan 11, 2018 449km K08E88J 530-1200m 00N0062 2-4km
  41. 41. © 2015 IBM Corporation June 12th, 2015 Hi Jeff & the gang, I have great news! On Tuesday I happened to observe a close encounter you guys predicted - one 1 km and the other one 2 km in diameter! To my knowledge this is the first case ever of direct observation of a close encounter in the small main belt asteroids. The closest point of encounter unfortunately happened during bright daylight in Hawaii, so I missed it … Cheers! Eva -
  42. 42. © 2015 IBM Corporation Image courtesy of: Eva Lilly, Institute of Astronomy, University of Hawaii
  43. 43. © 2015 IBM Corporation [Theatrical Pause]
  44. 44. © 2015 IBM Corporation Action Red Analytics Green Analytics Blue Analytics Observation Space Old School: Isolated Analytics
  45. 45. © 2015 IBM Corporation Observation Space ActionInformation In Context Next: General Purpose Context Computing Data Finds Data Relevance Finds You Context Computing
  46. 46. © 2015 IBM Corporation Observation Space ActionInformation In Context Data Finds Data Relevance Finds You Context Computing Helping Focusing Human Attention General Purpose • Marketing • Customer service • Fraud detection • Asteroid hunting Simultaneously!
  47. 47. © 2015 IBM Corporation Making Data Work: Recommendations  Widen the observation space  Accumulate context to improve understanding  Deliver significantly higher quality outcomes … everywhere – Life sciences – Financial services – Public safety  Leverage Hadoop/Spark to accelerate innovation
  48. 48. © 2015 IBM Corporation More  Blog: www.jeffjonas.typepad.com  Email: JeffJonas@us.ibm.com  Next: San Francisco, Nov 10-12, @Datapalooza
  49. 49. © 2015 IBM Corporation Context Computing Strata + Hadoop World 2015 Jeff Jonas, IBM Fellow Chief Scientist, Context Computing http://www.twitter.com/jeffjonas www.jeffjonas.typepad.com

×