Product School Presentation on Building AI Products by Ria Sankar Microsoft PM on 120618

1. www.productschool.com
Part-time Product Management, Coding, Data, Digital
Marketing and Blockchain courses in San Francisco, Silicon
Valley, New York, Santa Monica, Los Angeles, Austin, Boston,
Boulder, Chicago, Denver, Orange County, Seattle, Bellevue,
Toronto, London and Online

2. FREE INVITE
Join 30,000+ Product Managers on

3. +5000
Alumni Graduated
across 14
Campuses
· San Francisco
· Silicon Valley
· New York
· Los Angeles
· Santa Monica
· Orange County
· Austin
· Boston
· Boulder
· Chicago
· Denver
· Seattle
· Toronto (Canada)
· London (UK)

4. COURSES
Product Management
Learn the skills you need to land a product manager job

5. COURSES
Coding for Managers
Build a website and gain the technical knowledge to lead software engineers

6. COURSES
Data Analytics for Managers
Learn the skills to understand web analytics, SQL and machine learning concepts

7. COURSES
Digital Marketing for Managers
Learn how to acquire more users and convert them into clients

8. COURSES
Blockchain for Managers
Learn how to trade cryptocurrencies and build products using the blockchain

9. TONIGHT’S SPEAKER
@SankarRia
www.linkedin.com/in/riasankar/
rias@microsoft.com
Ria Sankar

10. Building AI Products
Ria Sankar
Product Management @Microsoft

11. Why AI?
Source: https://thefuturesagency.com/

12. AGENDA
Source: https://thefuturesagency.com/
• Components of an AI System
• Typical AI Product Lifecyle
• Types of AI Products
• Closing & Discussion

13. Components of an AI System

14. Typical AI Lifecycle

15. AI product lifecycle
Define
SimplifyDeploy
• User & Business Understanding
• Data & Metric Definition
• Data Preparation
• UX Design, Model Evaluation
• Model Deployment
• Ongoing Measurement &
Learning
13
2

16. Define
SimplifyDeploy
• User & Business Understanding
• Data & Metric Definition
1
Define

17. Customer Understanding: Jobs Theory
Seeing tasks from a customer vs. product context
1a
Functional
“Help me wake up
with the best coffee
at consistent
quality”
Social
“Give me a place to
connect with my
friends”
Emotional
“Help me treat
myself
at the end of a long
day”

18. CASE STUDY: Xiaolice
https://www.altexsoft.com/blog/business/chatbots-in-travel-how-to-build-a-bot-that-travelers-will-love/

19. Business Consequences: Building Trust
Use FATE Principles
1b
https://www.microsoft.com/en-us/ai/our-approach-to-ai

20. CASE STUDY: Travel Chatbots (Mezi, Expedia)
https://www.altexsoft.com/blog/business/chatbots-in-travel-how-to-build-a-bot-that-travelers-will-love/

21. Data Understanding: Preventing Bias1c
https://i.imgflip.com/1w3emg.jpg
• Comprehensive test cases
(represent the real world)
• Data stratification
• Diverse workforce (avoid tech
bro AI)
• Unconscious bias – review
model outputs for correlations
to race and gender

22. Case Study:
What Celeb do you look like?
http://www.playbuzz.com/chloep19/what-female-celebrity-do-you-look-like
• Image understanding
• Face types, Colors, Tones
• Emotion understanding

23. Metric Understanding: Problem & Output
Definition
1d
https://i.imgflip.com/1w3emg.jpg
• Instrumentation
• Data Quality
• Primary vs. secondary goals
• Product vs. Feature
• Standard vs. derived
metrics

24. Task & Domain Selection1e
When to use:
• Diversify dataset
• Generate dataset
• Generate labelled data
• Structure NL responses

25. Outcome: Find your niche
Customer
DataBusiness
NicheArticulate AI value in terms of:
• Agility / Performance / Cost
• Growth drivers
• Brand value / Industry Status
• Risk reduction
• Accessibility
• Customer Delight
• Convenience/usability

26. Discussion:
You are Product Manager in
charge of improving YouTube’s
Recommendation Engine.

27. Define
SimplifyDeploy
Simplify
Data Prep, Model Evaluation2

28. Data Preparation
4Cs of data quality
• Correct
• Conforms
• Current
• Consistent
• Consolidated
2a
http://4.bp.blogspot.com/
Why is this important:
• Avoid underfitting, remove bias
• Avoid overfitting, reduce noise

29. Designing AI
5 principles of ethical design
• Humans as Heroes
• Honor Diversity
• Balance EQ and IQ
• Know context
• Evolve over time
2b
https://www.microsoft.com/en-gb/ai/our-approach-to-ai
"The AI tools and services we create must assist humanity and augment our
capabilities."
—Harry Shum, Executive Vice President, AI and Research

30. Model Selection
https://devzum.com/wp-content/uploads/2015/05/best-machine-learning-cheat-sheets3.jpg
https://azure.microsoft.com/en-us/blog/announcing-automated-ml-capability-in-azure-machine-
learning/?WT.mc_id=azure-presentation-lazzeri
Special thanks: https://www.linkedin.com/in/deepakmukunthu/
Other Resources:
https://docs.microsoft.com/en-us/azure/machine-learning/studio/media/algorithm-cheat-sheet/machine-
learning-algorithm-cheat-sheet-small_v_0_6-01.png
2c
Factors influencing model
complexity and data selection:
• Supervised vs.
Unsupervised
• Batch vs. Online
• Classification vs. Predicting
value

31. What model will you choose?
• Classifying cheese into Brie, Mexican,
Parmesan, Mozzarella

32. What model will you choose?
• Analyzing weather patterns to uncover
trends

33. What model will you choose?
• Analyzing weather patterns to FORECAST
the next 10 days

34. What model will you choose?
• To find anomalies in your dataset?
• For spam, fraud filtration?

35. Model Evaluation2d
• Iterative process to find the best model for your scenario (your MVP)
• Balance model performance vs. accuracy
Sample feature recovery rate for a matrix completion algorithm

36. Define
SimplifyDeploy
Deploy
Model Deployment, Ongoing
Learning
3

37. Model Deployment
• Scale data collection
• Scale scenario coverage
• Action movie recos for all users vs.
Movie recos for subset of users
• Scale model
• Check outliers and bias
• Visualize outputs
• Model specific
• Optimizations: Bagging, Boosting
3a
https://cloud.google.com/automl/
https://cloud.withgoogle.com/next18/sf/sessions/session/193072
https://www.youtube.com/watch?v=GbLQE2C181U

38. Ongoing learning: Uber Michelangelo Platform
https://eng.uber.com/michelangelo/
3b

39. Case Study:
Uber Eats

40. Types of AI Systems

41. So where can we use AI?
https://lexfridman.com/stephen-wolfram/

42. AI Progression:
Balancing interactions between humans and AI
https://medium.com/@davidbessis/building-ai-first-products-90d503ccd43a
USER BENEFIT

43. Pre-Engine vs. Engine-Inside vs. Engine-First
https://medium.com/@davidbessis/building-ai-first-products-90d503ccd43a

44. Pre-AI

45. AI Inside
https://www.businessinsider.com/amazon-alexa-voice-recordings-how-to-access-them-2018-5https://www.androidcentral.com/google-assistant

46. AI First
https://www.tesla.com/autopilot
http://www.automotivetechnologies.com/autonomous-self-driving-cars

47. Discussion: How would you build an AI experience
for a restaurant?

48. RECAP: CRISP-DM Methodology
Business &
User
Understanding
Data & Metric
Understanding
Data Prep
Model
Development
Model
Evaluation
Deployment
1. Scope your problem -> find your
niche
2. Build the business case for ML / AI
3. Select your ML model
4. Balance model performance and
accuracy
5. Ensure model relevancye to
changing business needs
6. Human powered vs. machine AI

49. https://cloud.google.com/blog/products/ai-machine-learning/6-must-see-sessions-ai-
and-machine-learning-next-18
Recommended Viewing
https://cloud.withgoogle.com/next18/sf/sessions/session/193072

50. Recommended Viewing
https://www.youtube.com/watch?v=pYW12_TZyWAhttps://www.coursera.org/lecture/big-data-
machine-learning/crisp-dm-icJH6
Brad Smith on AI

51. Q&A

52. Thank you

53. CONTACT INFORMATION
@SankarRia
www.linkedin.com/in/riasankar/
rias@Microsoft.com
Ria Sankar

54. UPCOMING EVENTS
Wednesday, December 12
How to Transition to B2C
vs. B2B Product
Management

55. UPCOMING Product Management Courses
December 8 - February 9
9:00am - 3:30pm
2 Spots left!
Saturdays - Seattle Tues & Thurs - Seattle
January 22 - March 14
6:30pm - 9:00pm
8 Spots left

56. Saturdays - Seattle
6 Spots left
January 26 - March 23
9:30am - 3:30pm
UPCOMING Coding for Managers Courses

57. Mondays & Wednesdays
7 Spots left
UPCOMING Data Analytics for Managers Courses
January 28 - March 25
6:30pm - 9:00pm