The presentation on Machine Learning telepathy for Shift Right approach of testing was done during #ATAGTR2017, one of the largest global testing conference. All copyright belongs to the author.
Author and presenter : Santhosh GS
2. Agile Testing Alliance Global Testing Retreat 2017
Abstract
Product Released, what next?
Collect Data (SHIFT )
Analyze & Predict (Machine Learning)
Future Readiness with Actions
3. Agile Testing Alliance Global Testing Retreat 2017
• Why are only some clients reporting it?
• Is it a environment issue?
• Is this part of frequently used workflow?
• How Clients are using new feature
How can we Prevent/Answer these types of
Questions ?
Client reports Case.
4. Agile Testing Alliance Global Testing Retreat 2017
Machine learning
Computers can learn on its own without writing logic for
all the cases based on data classification
Shift Right
Approach Says Continuous feedback from customer is
important even after releasing Quality product
5. Agile Testing Alliance Global Testing Retreat 2017
Needed V/s Asked
Shift from KYC to LYC
Know your Customer (KYC) to fulfil
what customer asks
But with Machine Learning enables
Learn your Customer (LYC) to
predict what customer needs
6. Agile Testing Alliance Global Testing Retreat 2017
Dynamic Baseline vs Manual Threshold
Supervised Machine learning Algorithms system
can forecast the performance growth and
Predict probable Future performance
bottlenecks .
Get Ready Before
customer ASKs !!
7. Agile Testing Alliance Global Testing Retreat 2017
Supervised Learning
Select
Interested Data
Create and Train
Model
Predict the
Performance
Insights to
Development
Plan for Next
Release
8. Agile Testing Alliance Global Testing Retreat 2017
UX Improvements With ML
Unsupervised Machine learning
Algorithms system can predict the most
common navigation pattern of the users.
Results help Agile teams improve the client user
User experience on realistic
values
9. Agile Testing Alliance Global Testing Retreat 2017
Intent Vs Reality
Generate usage analytics of new
features and predict future
usage growth with help of
Machine Learning.
Take action!!
10. Agile Testing Alliance Global Testing Retreat 2017
ML Applications for Beta Feedback
Record
Feedback
Convert
to Text
(Google
Speech API)
Analyze
the Text
(Google
Natural
Language
API)
Analyze
Verbs,
Nouns
Generate
report
With Google Machine Learning API
11. Agile Testing Alliance Global Testing Retreat 2017
Popular ML Algorithms
Linear Regression Algorithm
Polynomial prediction Algorithm
Exponential prediction algorithm
Decision tree learning
13. Agile Testing Alliance Global Testing Retreat 2017
Case Study
Following studies are in progress
1. Generate Real time performance predictions
from all the clients – Helps to create dynamic
baseline and bottlenecks
2. Insights of Client workflow and experience
with software – Help us to predict the client
use cases
14. Agile Testing Alliance Global Testing Retreat 2017
Case Study Contd..
3. Feature usage analytics and predict - Helps in
planning the next releases
15. Agile Testing Alliance Global Testing Retreat 2017
Key Takeaways
Shift from KYC to LYC
Certified and Release software is not
Definition of Done.
Learn and Generate Future requirements
Get Ready for future with Statistical Analysis
of ML
We have Maintenance team who works on fixing the customer reported issues and part of earnings are spent on maintenance activity including support.
How do we reduce the maintenance overhead and increase the client deligment and this is the motivation for writing a paper…
Testing in early phase is recommended but not enough, Continuous feedback from customer is import and let us see how automatically can generate the feedback
Generate a User stories for your agile team instead of waiting for customer give surprises
Generate a User stories for your agile team instead of waiting for customer give surprises .
Computers can learn on its own without writing logic for all the cases based on data classification
Generate a User stories for your agile team instead of waiting for customer give surprises .Shift KYC to LYC
Flipkart example of camera and lens.
Health information of patient stored in hospital today is not same as next 2 years and we need to constatntly monitor their growth
The Peadiatrics and orthoeadics are different as Peadatric is less images and orthopeadic is more images so need optimization
Supervised Learning is all about Classification. Are the defects based on Business rules, test depth or coverage
Doctors or banking domain employees of company spend more time with customer than software
Number of clicks to be reduced. In healthcare industry. Abnormal results click has been reduced
Either educate a client or change the user expierence or is that valueable to customer