Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

AI Orange Belt - Session 3

1.214 visualizaciones

Publicado el

Session 3 of the AI Orange Belt Training Program for business managers and executives.

Publicado en: Educación
  • Inicia sesión para ver los comentarios

AI Orange Belt - Session 3

  1. 1. Build AI in your company management, culture, legal, ethics, governance Orange Belt - Session 3 1
  2. 2. What we have seen so far 1. What is AI 2. What can AI do and what it can’t do 3. How to select a project 4. What are the steps necessary for a first successful ML project 2
  3. 3. AI is not traditional software A totally different lifecycle 3
  4. 4. 4
  5. 5. 5
  6. 6. We saw this last week 01 03 02 06 04 05 Monitoring & Updates Have the right talents & solutions Maintenance Select the right question Choose the performance metric Decide the level of explainability Identify Use the right architecture Have the talents in place Deploy Find the right data Structure annotate data Clean Data Data Decide on an acceptable error Test on the right scope Evaluate Select the right algorithm Tune the model Model 6
  7. 7. In practice, everything is more iterative 7
  8. 8. Plan for today 0 (Base on the case seen in the assignment) What does it feel like to work on a complex ML project ? 1. AI Transformation Playbook 2. AI Maturity of your company 3. Perspectives on costs & roadmap 4. Build vs Buy 5. Legal & ethics considerations 6. Human and AI interactions 8
  9. 9. 1. AI Transformation Playbook How to lead your company into the AI era 9 Source : https://landing.ai/ai-transformation-playbook/
  10. 10. AI Transformation Playbook 1. Execute pilot projects to gain momentum 2. Build an in-house AI team 3. Provide broad AI training 4. Develop an AI strategy 5. Develop internal and external communications 10
  11. 11. 1. Execute pilot project to gain momentum • More important for the initial project to succeed rather than be the most valuable • Show traction within 6-12 months • Can be in-house or outsourced 11
  12. 12. 2. Build an in-house AI team Centralized AI Platform 12
  13. 13. 3. Provide broad AI training Role What they should learn Nb hours of training Executives and senior business leaders - What AI can do for the company ? - AI Strategy - Resource allocation >= 4 hours Leaders of divisions working on AI projects - Set project direction (both technical and business diligence) - Resource allocation - Monitor progress >= 12 hours AI engineer trainees - Build and ship AI software - Gather data - Execute on specific AI projects >= 100 hours 13
  14. 14. How to document yourselves AI for Everyone, by Andrew Ng Deep Learning Specialization 3blue1brown Siraj OpenAI ImportAI https://towardsdatascience.c om/ 14
  15. 15. 4. Develop an AI strategy • Build several difficult AI assets that are broadly aligned with a coherent strategy • Leverage AI to create an advantage specific to your industry sector. • Design strategy aligned with the “Virtuous Cycle of AI” AI plays a role here 15
  16. 16. 4. Develop an AI strategy • Consider creating a data strategy • Strategic data acquisition • Unified data warehouse • Create network effects and platform advantages • In industries with “winner take all” dynamics, AI can be an accelerator • What about more traditional strategy framework ? • AI can allow a low cost strategy • AI can allow a high value product strategy 16
  17. 17. 5. Develop internal and external communications • Investor relations • Government relations • Consumer / user education • Talen / recruitment • Internal communications 17
  18. 18. AI pitfalls to avoid Don’t : Do : - Expect AI to solve everything - Be realistic about what AI can and cannot do, given limitations of technology, data and engineering resources - Hire 2-3 ML engineers and count solely on them to come up with use cases - Pair engineers with business talent and work across cross-functional team to find valuable projects - Expect the AI project to work the first time - Plan for AI development to be an iterative process, with multipe attemps needed to succeed. - Expect traditional planning processes to apply without changes - Work with AI team to establish timeline estimates, milestones, KPIs, etc. - Think you need superstar AI engineers before you can do anything - Keep building the team but get going with the team you already have 18
  19. 19. Some initial steps you can take • Start learning (with this course) • Start brainstorming projets • Hire a few ML/DS people to help • Hire or appoint an AI leader (VP AI, CAIO, …) • Discuss with CEO/Board possibilities of AI Transformation • Will your company be much more valuable and/or more effective if it were good at AI ? 19
  20. 20. Exercise What is the first step in the AI Transformation Playbook for helping your company become good at AI? 20
  21. 21. Exercise Of the following options, which is the most important trait of your first pilot project? A) Succeed and show traction within 6-10 months B) Drive extremely high value for the business C) Be executed by an in-house team D) None of the above 21
  22. 22. Exercise Say you are building the DropBox OCR system, and want to accumulate data for your product through having many users. Which of these represents the “Virtuous circle of AI” for this product? 22
  23. 23. Exercise Why is developing an AI strategy NOT the first step in the AI Transformation Playbook? 23
  24. 24. 2. AI Maturity Assess the readiness of your organisation 24
  25. 25. 25
  26. 26. A startup mindset is always nice 26
  27. 27. 3-steps roadmap to maturity Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY DATA PEOPLE LEGAL&ETHICS PRODUCT 27
  28. 28. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget DATA No infrastructure Data Silos Descriptive analytics PEOPLE No data scientists No education LEGAL&ETHICS No legal compliance No principles or processes PRODUCT Business cases but no development 28
  29. 29. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics PEOPLE No data scientists No education LEGAL&ETHICS No legal compliance No principles or processes PRODUCT Business cases but no development 29
  30. 30. 30
  31. 31. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding value PEOPLE No data scientists No education LEGAL&ETHICS No legal compliance No principles or processes PRODUCT Business cases but no development 31
  32. 32. 32
  33. 33. Data requirements according to business needs 33
  34. 34. Define data acquisition strategy It’s not a one shot 34
  35. 35. Data annotation pipeline To go at scale, you need guidelines, internal and external annotators. Even pre-annotation with machine learning that can be validated DOG CAT 35
  36. 36. Standardise quality check Find the relevant metrics Exemple Sound: signal to noise ratio / cross-talk / silence detection 36
  37. 37. Govern data Different access restrictions, stay compliant 37
  38. 38. 38
  39. 39. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGAL&ETHICS No legal compliance No principles or processes PRODUCT Business cases but no development 39
  40. 40. Example roles • Software Engineer • Build user interface, web & mobile applications, back end operations, … • Machine Learning Engineer • Input (A) to Output (B) • Machine Learning Researcher • Extend state-of-the-art in ML 40
  41. 41. Example roles • Data Scientist • Examine data and provide insights • Make presentation and communicate to team / executives • Data Engineer • Organize Data • Make sure data is saved in an easily accessible, documented and cost effective way • More and more required as the amount of data managed by companies increases • AI Product Manager • Help decide what to build; what’s feasible and valuable 41
  42. 42. Getting started with a small team • 1 Software Engineer, or • 1 Machine Learning Engineer / Data Scientist, or • Nobody by yourself 42
  43. 43. Chief Data/Analytics/Information Officer 43
  44. 44. Structure a team - decentralised 44
  45. 45. Structure a team centralised (SWAT) 45
  46. 46. Structure a team add use case specialists 46
  47. 47. Exercise Suppose you are building a trigger word detection system, and want to hire someone to build a system to map from Inputs A (audio clip) to Outputs B (whether the trigger word was said), using existing AI technology. Out of the list below, which of the following hires would be most suitable for writing this software? 47
  48. 48. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGAL&ETHICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development 48
  49. 49. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGAL&ETHICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development Use of APIs Discrete proof of concepts Pilots AI is core product & core competency 49
  50. 50. Exercise : create your own roadmap Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGAL&ETHICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development Use of APIs Discrete proof of concepts Pilots AI is core product & core competency 50
  51. 51. Exercise Try to identify the critical 3 next steps for your organization, using the matrix 51
  52. 52. 3. AI Management Anticipate costs and timing 52
  53. 53. prices & timing are not at all set in stone.. 53
  54. 54. but we can still assess feasibility 54
  55. 55. but we can still assess feasibility 55
  56. 56. but we can still assess feasibility 56
  57. 57. but we can still assess feasibility 57
  58. 58. Machine Learning is not regular Software 58
  59. 59. Bring back your expected performance requirements 59
  60. 60. 60
  61. 61. 61
  62. 62. To sum up 1. Anticipate the cost drivers (data, accuracy, problem difficulty) 2. There is a big upfront cost to anticipate as opposed to regular projects 3. The cost scales nonlinearily with the accuracy requirements 4. The time it takes to get that accuracy is nonlinear as well 62
  63. 63. Exercise What is the biggest cost of your current idea? What do you think will be the bottleneck? What type of strategy could you devise to anticipate that? 63
  64. 64. 4. Build vs Buy The eternal dilemma 64
  65. 65. 65
  66. 66. With a consultant you don’t know, always look to start with a small proof of concept deliverable to prove to yourself that this consultant knows their stuff. Work with the consultant to come up with a project that is a low hanging fruit. Something that they can deliver on quickly without much development effort (e.g. based on existing code they already have, and data you have already collected). If this first step goes well, then you can confidently move to a bigger project scope. 66
  67. 67. 67
  68. 68. Access to resources (cloud, ML libraries, production systems) Access to talent Proven success stories to get you up and running in no time So why would you do everything yourselves ? Don’t try to run faster than the train Collaborate with Startups! 68
  69. 69. 5. Legal & Ethics What should you be careful about 69
  70. 70. Explainability 70
  71. 71. What does a Convolution sees ? VGG16, convolutional layer 1-1, a few of the 64 filters 71
  72. 72. What does a Convolution sees ? Variation of kernel size Source : 72
  73. 73. Adversarial Examples The left image is predicted with 99.9% confidence as a magpie. 73
  74. 74. Adversarial Examples Machine Learning classifiers today are easily fooled ! 74
  75. 75. Adversarial Examples Machine Learning classifiers today are easily fooled ! 75
  76. 76. Adversarial Examples 76
  77. 77. The problem with explainability https://distill.pub/2019/activation-atlas/ 77
  78. 78. Performance/Explainability tradeoff 78
  79. 79. • Maximise customer satisfaction, • Think about human decision making expectations • Complex or direct simple explanations? • Stay pragmatic - “reasonably necessary” Reasonable explanation 79
  80. 80. Alternative accountability Flagging - Stress test - Auditing 81
  81. 81. Privacy 82
  82. 82. 1. AI systems must be transparent. 2. An AI must have a “deeply rooted” right to the information it is collecting. 3. Consumers must be able to opt out of the system. 4. The data collected and the purpose of the AI must be limited by design. 5. Data must be deleted upon consumer request. Bernhard DebatinProfessor and Director of the Institute for Applied and Professional Ethics Guidelines for ethical privacy 83
  83. 83. 1. Clear consent requests to customer (and easy out) 2. Breach notifications within 72h 3. Right to access all personal data upon request 4. Right to be forgotten in practice 84
  84. 84. Fairness & Bias What is fair anyways? 85
  85. 85. Risks & Safety How to protect from potential dangers 86
  86. 86. 87
  87. 87. 88
  88. 88. 89
  89. 89. 90
  90. 90. ‘build a world-class face recognition AI model’ Specific, focus & measurable 91
  91. 91. ‘build a face recognition service that can detect male/female genders, with pre- defind specific age groups, and these specific subset of races, and ethnicities in the requirements document (which is grounded in a standard taxonomy from a neutral organization such as the United Nations Race and Ethnicity taxonomy)) with at least 90% accuracy on ‘these’ given specific test datasets’ where ‘these’ test datasets were carefully crafted by the offering management team to have an even distribution of all the genders, age groups, specific races, and ethnicities for which the model is supposed do well’ Specific, focus & measurable 92
  92. 92. 93
  93. 93. 94
  94. 94. 95
  95. 95. 96
  96. 96. 97
  97. 97. Foster trust By Element AI 98
  98. 98. To summarise 1. Specify a robust goal 2. Make sure you respect privacy 3. Make it explainable 4. Make it secure 5. Make it transparent 99
  99. 99. Exercise Can you identify the most sensitive data in your organizations? Do you already have a strategy to be compliant? Do you need a big explainability or would you rather get better performance? 100
  100. 100. 6. Towards better HCI Human-Computer interactions have to be reconsidered 101
  101. 101. Human-in-the-loop Design for better collaboration 102
  102. 102. The problem with hype… 103
  103. 103. 104
  104. 104. 105
  105. 105. 106
  106. 106. 107
  107. 107. Human vs Machine • Machines never forget • Crunch numbers and scan fast • Never bored or impatient or tired • People have better emotional nuances + humanity • People have better common sense • People can tackle new tasks easily 108
  108. 108. 3 types of collaboration 109
  109. 109. AI recommends multiple options 110
  110. 110. AI makes the decision 111
  111. 111. AI coaches 112
  112. 112. Keep human psychology in mind Lots of false positive = irritating Introducing without consultation = resentment Most often good decision = overtrust (automation bias) … 113
  113. 113. Exercise Try to imagine the end product of your project, leveraging both AI and humans using the type of collaborations we just talked about What would the user be the most sensitive about ? 114
  114. 114. Conclusion Create your AI implementation strategy 115
  115. 115. Quiz 116

×