1. Watson
The IBM Jeopardy! Challenge
Jim De Piante, PMP
Executive Project Manager
IBM Research
© 2011 IBM

2. Video clip 1

3. Jim De Piante, PMP
Executive Project Manager
IBM Research
Watson
The IBM Jeopardy! Challenge
© 2011 IBM

4. Watson wins
February 16, 2011
© 2011 IBM 4

5. At a restaurant in upstate New York
Charles Lickel
Ken Jennings
© 2011 IBM 5

6. Deep Blue
May 11, 1997
© 2011 IBM 6

7. The next Grand Challenge
• Difficult problem
• Broader audience
• Scientific community
• Relevant to clients
© 2011 IBM 7

8. IBM Research pursues the idea
Dave Ferrucci
Paul Horn
© 2011 IBM 8

9. What they were in for
2 hours 3 seconds
Wrong 66% Right 90%
Simple questions Quirky clues
© 2011 IBM 9

10. Dave Ferrucci meets Harry Friedman
Harry Friedman
© 2011 IBM 10

11. Early sparring match
© 2011 IBM 11

12. Sparring matches with champions
Former Jeopardy! champions
Todd Crain
© 2011 IBM 12

13. © 2011 IBM ‹#›

14. Watson’s record
Ken Jennings Brad Rutter
© 2011 IBM 14

15. Watson was a contender
Press conference after the practice match
© 2011 IBM 15

16. Show time
© 2011 IBM 16

17. All there to witness…
© 2011 IBM 17

18. End of round 1
© 2011 IBM 18

19. The result
© 2011 IBM 19

20. Watson
© 2011 IBM 20

21. Watson
The role ofProject
© 2011 IBM
Management

22. • Scope
• Schedule
• Budget
• Risk
• Stakeholders
• Communications
• Change
• Procurement
• Integration
© 2011 IBM

23. Scope
© 2011 IBM

24. Scope
 Win on Jeopardy!
• Against the best of the best
• In early 2011
 Radically new technology
• Architecture
• Software
 HW configuration
 Marketing / Communications
 Scientific research
• 55 sparring matches
• Research papers
 Asset to commercialize
© 2011 IBM 24

25. Unlike any other project
 One of a kind
• Unprecedented
• No “methodology”
• SW development, but not
 Division of responsibilities
• Principal Investigator
• Project Managers
 IBM infrastructure not suited
 Creative use of PM techniques
• Flexibility
• Non-rigorous
© 2011 IBM 25

26. Principal Investigator
 Deep QA software
• Architecture
• Algorithms
• Development
• Integration
 Watson hardware
• Configuration
• Scale-out
• Optimization
 Front end
• Game strategy
• Betting David Ferrucci, PhD
• Watson’s voice
• Game state
© 2011 IBM 26

27. Project Managers
1. Project communications
2. Procurement
3. Stakeholder management
4. Elements of the budget
5. Milestone management
6. Dependencies David Shepler, PMP
7. Risk management
8. Sparring matches
9. Contestant management
10. Negotiations
11. Change management
12. Hardware build-out
13. Move to Hollywood
14. The set at IBM Research
15. Integration of all elements Jim De Piante, PMP
© 2011 IBM 27

28. Schedule
© 2011 IBM

29. One unyielding deadline
 Early 2011
 Public commitment
 IBM Research: Grand Challenge
 Hollywood: Ratings
Jeopardy! Challenge …..……|………………………|……..
© 2011 IBM 29

30. Shelf life of clues
© 2011 IBM 30

31. Budget
© 2011 IBM

32. Budget
 Severely constrained
• Immutable deadline
• Fixed scope
• Financial pressures
 Managed at several levels
 Principal Investigator
• Development resources
• Department travel
 Project Managers
• Hardware
• Other travel
• Procurement
© 2011 IBM 32

33. Risk
© 2011 IBM

34. Key risks
 Technology
 Reputation (brand)
 Schedule
 Perceptions
• Might seem too easy
• If Watson wins
• If Watson loses
• HAL-like associations
 Failure Rep. Holt beats Watson?
• Show might not air
• Watson could lose
 Move Watson to Hollywood Not!
© 2011 IBM 34

35. Perceptions
 Negative perceptions
• Jeopardy! (Sony)
• IBM
 Risk analysis
• Probability: high
• Effect: high
• Position: avoid
 Actions
• Avatar
• Voice
• Publicity
© 2011 IBM 35

36. Move to Hollywood
 Watson damaged in transit
• Probability: low
• Effect: devastating
• Position: mitigate
 Risk analysis
• Likelihood of accident
• Likelihood of damage
 Actions considered
• Second system
• Fly/drive
• Choice of supplier
 Action: change approach
© 2011 IBM 36

37. Stakeholders
© 2011 IBM

38. Stakeholder groups
 IBM Research
 Sony Pictures
 Ogilvy
 Talent
 IBM internal
• Marketing
• Legal
• Procurement
• Industrial Design
• Systems Technology Group
© 2011 IBM 38

39. Yorktown
Heights
Hollywood
© 2011 IBM 39

40. IBM Research
 Creative culture
 Desire for flexibility
 Not used to PM approach
 Worried about perceptions
 Protective of brand
 Communication style
 Contracts
 Science
 IBM
© 2011 IBM 40

41. Sony Pictures
 Creative culture
 Desire for flexibility
 Not used to PM approach
 Worried about perceptions
 Protective of brand
 Communication style
 Contracts
 Entertainment
 Hollywood
© 2011 IBM 41

42. What they had in common
 Creative culture
 Desire for flexibility
 Not used to PM approach
 Worried about perceptions
 Protective of brand
© 2011 IBM 42

43. Where they were different
 Motives for the project
• Science and product revenue
• Entertainment and ad revenue
 Contracts
 Culture
 Communications
• Internally
• Externally
© 2011 IBM 43

44. Communications
© 2011 IBM

45. Communications approach
 Speak several languages
• IBM
• IBM Research
• Hollywood
• Legal
• Procurement
• Marketing
• IP Law
 Use various means
 Update frequently
© 2011 IBM 45

46. Change
© 2011 IBM

47. Approach to change
 Facilitate change
• Know alternatives
• Know costs
 Isolate change
• System: PI
• Everything else: PMs
 Two examples
• Watson’s hand
• Match venue
© 2011 IBM 47

48. The hand of Watson
 Plan
• Watson receives clue
• Watson buzzes electronically
 Rationale for change
• Avoid perception problems
• Level playing field
 Create mechanical hand
• IBM Research facilities
• Working prototype in 3 weeks
 New risk
• Introduced slight delay
• Possibly lose the buzz
© 2011 IBM 48

49. Change of venue
 Plan
• Watson goes to Hollywood
• Wheel-of-Fortune set
 Rationale for change
• Avoid perception problems
• On IBM Research turf
 Create full Jeopardy! set
• Huge undertaking
• Watson stays in Yorktown Heights
• Jeopardy! comes to IBM Research
 Opportunities seized
• Reduced risk of the move
• Showcase IBM Research facility
© 2011 IBM 49

50. Procurement
© 2011 IBM

51. Procurement
 Key PM focus
 Highly specialized needs
 Sole source
 Key areas
• Talent
• Compliance
• Contestants
 Outside the IBM norm
• Processes
• Skills
© 2011 IBM 51

52. Talent
 Challenge
• Role of Alex Trebek
• Short lead time
 New world for IBM
• Specification
• Actor’s guild
 Solution
• Sole source
• Known quantity
Todd Allen Crain
© 2011 IBM 52

53. Compliance and integrity
 Challenge
• Game-show compliance
• Challenge integrity
 New world for IBM
Alex Hendrie
• Specification
• Making payments
 Solution
• Sole source
• Jeopardy’s supplier
© 2011 IBM 53

54. Sparring contestants
 Challenge
• Former Jeopardy! contestants
• Manage the logistics
• Tight scheduling
 Solution
• Considered “cattle call”
• Solicit through Jeopardy!
• Hire as consultants
© 2011 IBM 54

55. Contestant management
 Challenge
• Find the right skills
• Outsource completely
 Solution
• Sole source
• Existing approach Josiane Emorine and Alex Trebek
• Existing supplier
• Explicit specification
• Detailed process definition
• Supplier web site
• Supplier travel agency
• Supplier disbursement
• Extreme empowerment
© 2011 IBM 55

56. Integration
© 2011 IBM

57. Integration
 Crisp boundaries
 Communication
 Blended approach
• Co-location
• Virtual
 Tight linkages
• Researchers
• Sony Pictures
 Milestone driven
• Clear goals
• Clear deadlines
© 2011 IBM 57

58. What next for
Watson
© 2011 IBM
?

59. Does Watson think?
“Does a submarine swim?”
© 2011 IBM 59

60. What does Watson do?
Deep
© 2011 IBM 60

61. The future
© 2011 IBM 61

62. Nuance
ARMONK, NY and BURLINGTON, MA – 17 Feb 2011
IBM and Nuance Communications today announced a
research agreement to explore, develop and
commercialize the Watson computing system's advanced
analytics capabilities in the healthcare industry.
© 2011 IBM 62

63. WellPoint
INDIANAPOLIS and ARMONK, NY – 12 Sep 2011
WellPoint, Inc. (NYSE: WLP), and IBM (NYSE: IBM)
announced an agreement today to create the first
commercial applications of the IBM Watson technology.
© 2011 IBM 63

64. Watson
The IBM Jeopardy! Challenge
Jim De Piante, PMP
Executive Project Manager
IBM Research
© 2011 IBM