According to recent research report by Wall Street Journal, AI project failure rates near 50%, more than 53% terminates at proof of concept level and does not make it to production. Gartner report says that nearly 80% of the analytics projects are not delivering any business value. That means for every 10 projects, only 2 projects are useful to the organization. Let us pause here a moment, rather than looking at what makes AI projects to fail, let’s look at the challenges involved in AI projects and find a solution to overcome these challenges. AI projects are different from traditional software projects. Typical software projects, as shown in Figure 1, consist of well-defined software requirements, high level design, coding, unit testing, system testing, and deployment along with beta testing or field testing. Now, organizations are adopting Agile process instead of traditional V or waterfall model, but still steps mentioned are valid. However, AI and Machine Learning projects’ methodology is different from the above. Our experience working on many AI/ML projects has given us insights on some of the challenges of executing AI projects. Also, we are in regular touch with senior executives and thought leaders from different industries who understand the success formula. The following discussion is based on our practical experience and knowledge gained in the field. Successful execution of AI projects depends on the following factors: 1. Clearly aligned Business Expectations 2. Clarity on Terminologies 3. Meeting Data Requirements 4. Tools and Technology 5. Right Resources 6. Understanding Output Results 7. Project Planning and the Process

1. N-U Sigma U2 Analytics Lab web: www.businessanalyticsr.com email: umesh@mytechnospeak.com Ph: +1 408 757 0093
Umesh Rao Hodeghatta, Ph.D
Chief Data Scientist
Challenges of Executing AI &
Machine Learning Projects
10/18/2020

2. Outline
 AI Project Success Reports
 AI and ML Project Execution Challenges
 AI Project Vs Software Projects
 Best Practices For Executing AI Projects
 Q & A
2

3. Brief Introduction about me
 I have more than 20 years of experience
 I have worked for AT&T Bell Labs, Cisco Systems, McAfee, Wipro
 I have provided AI and ML solutions to some of the leading retail
companies, Human Resource company and healthcare company.
 I have authored Books titled “Business Analytics Using R and “The
InfoSec Handbook: An Information Security”, published by
Springer Apress.
 I have taught at Walden University, Kent State University, Xavier
University (XIMB).
3

4. AI Project Statistics
AI project failure rates near 50%,
More than 53% terminates at proof of concept level and
does not make it to production
Gartner report says that nearly 80% of the analytics projects
are not delivering any business value
4

5. AI Project Execution Challenges
Business Expectations
Data
Tools and Technology
Resources
Results and Outcome
Project Planning and the Process
5

6. Business Expectations
CUSTOMER
SATISFACTION
BUSINESS
DECISIONS
PERFORMANCE
AND ROI
6

7. Leverage AI
Deployment Timeline
Quick deployment
Smooth Integration
Cybersecurity
Value Add
% development efforts
Risks
Agile
7
“What percentage of business decisions are we
making with help from AI?”

8. Data is The KEY!
8

9. AI/ML Project Depends on DATA
9

10. Tools and Technology
Simple Analytics and Data Visualization
Data Plumbing
Data cleaning
Data cleaning utilities
Data Preparation
Reading pdf files and search certain key words?
Data scaling utilities
Feature selection
AI and ML Predictive Analytics
Image recognition
Recommendation Engine
Sentiment Analysis
10

11. Resources
Important roles (not in the order of
importance)
Data Engineer
Data Analyst
Machine Learning
Data Scientist
Software Programer/Coder Programs the solution as per the direction of the Data Scientist
Data Analyst analyses the data using data analysis tools and techniques like
MSBI, Tableau, etc. and finds out patterns and what the data broadly suggests
Data Plumbing Engineer: Capturing the data, collating the data, cleaning up
data, etc.
11

12. Output Results
Wrong interpretation of the results
• Misguided Decisions
• Few false positives or few false negatives can seriously undermine the use of the model
in spite of high accuracy
Selecting Metrics
• Different algorithms have different but relevant metrics
Overfit models may not be relied upon as they throw up different
results on the unseen / new data.
Trust and Ethics
12

13. AI Project Vs Software Project
13

14. Typical Software Project Process
1. Initiation
4. Deployment
2. Planning
3. Execution
1. Requirements
2. Hardware/Software
3. Aligning Business
1. Resources
2. Scheduling
1. Development
2. Testing
3. Deployment
3. Risks Planning
1. Monitoring, Tracking
2. Reassessing/Calibration
1. Beta Testing
2. Closure
14

15. AI Machine Learning Project Process
ASSESS
DATA MODEL
Deploy
Science
Initiation
• Requirements
• Aligning to Business
• Data Availability
• Quality Of Data
• Data vis-à-vis Business
Requirement
• Data Collection/Collation & Cleaning
• Descriptive
• Predictive
• Prescriptive
• NLP, Deep Learning
• Ph.Ds and Coders
• Experiments
• Measure & Validate
• Vary Parameters
• Re-validate
• Errors
• Deploy
• Verify
• Calibrate
15

16. E X E C U T I O N O F A I A N D M L P R O J E C T S
Best Practices

17. AI is Science!!
 Clarity on the data
 Label Data
 Distinguish supervised or unsupervised ML
 Composition of the classes/categories
 Do Not Directly jump into the model building
 Coding without science
 Understand the features and their contribution to the
model
 Business Objectives, Requirements, Features and
Data
17

18. Developing AI Solutions
 Sufficient and reliable training data
 Sample size or Population?
 Training data cover all the relevant and possible results?
 Is the training data in tune with the basic underlying fundamentals of the field under
study?
 Validating Algorithms
 validations carried out using sound principles of validation?
 Results available?
 Results are sufficiently challenged to bring out the weaknesses if any?
 Transparent process of generating the model?
 The process worked in tune with the requirements to get the requisite confidence in
the results?
 No Secrets
18

19. Project Planning
 Agile Stories
 Scrum Meetings
 Sprints
 Weekly discussions
 Weekly updates
 Code reviews or process
 Demos
 Discussion with Business
 Guide Business and Team
19

20. Summary
 W.r.t. Data science project, you don’t know if it’s going to work
 Data science requires an experimental process that allows for
uncertainty
 Businesses and all the resources involved need to clearly
understand the roles and responsibilities of various stakeholders
 Transparency is critical for the success
20

21. References
21
 Hodeghatta, U. R., & Nayak, U. (2016). Business analytics using R-a
practical approach. Apress.
 https://www.wsj.com/articles/ai-project-failure-rates-near-50-but-it-doesnt-
have-to-be-that-way-say-experts-11596810601
 https://www.forbes.com/sites/gilpress/2019/07/19/this-week-in-ai-stats-up-to-50-
failure-rate-in-25-of-enterprises-deploying-ai/#34ea003272ce;
 https://www.forbes.com/sites/gilpress/2020/01/13/ai-stats-news-only-146-of-firms-
have-deployed-ai-capabilities-in-production/#4da0ee526500

22. THANK YOU
E M A I L : U M E S H @ B U S I N E S S A N A L Y T I C S R . C O M
: U M E S H @ M Y T E C H N O S P E A K . C O M
W E B : H T T P : / / W W W . M Y T E C H N O S P E A K . C O M
P H : + 1 4 0 8 7 5 7 0 0 9 3
Q & A