Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Flavio Clésio, Movile
Eiti Kimura, Movile
PREVENTING REVENUE LEAKAGE AND
MONITORING DISTRIBUTED SYSTEMS
WITH MACHINE LEARN...
#EUai10
ABOUT US
2
Flávio Clésio
• Core Machine Learning at Movile
• MSc. in Production Engineering (Machine Learning in C...
#EUai10
ABOUT US
3
• IT Coordinator and Software Architect at Movile
• Msc. in Electrical Engineering
• Apache Cassandra M...
Movile is the company behind several
apps that makes the life easier
WE MAKE LIFE BETTER
THROUGH OUR APPS
#EUai10
Agenda
5
● The Movile's Platform Case
● Practical Machine Learning
Model Training
● Key Takeaways and Results
#EUai10
SBS
Subscription and
Billing Platform
6
#EUai10 7
THE INPUT
#EUai10 8
THE
PROCESSING
#EUai10 9
THE OUTPUT
#EUai10 10
#EUai10
Main Problem: Monitoring
11
How can we check if platform is fully
functional based on data analysis only?
Tip: wha...
#EUai10
The Data Volumetry
12
● 236 Millions + of billing requests attempt a day
● 4 main mobile carriers drive the operat...
#EUai10
Stating the problem
13
Sample of data (predicting the number of success)
featureslabel/target
# success carrier_we...
#EUai10
The Modeling Lifecycle
14
Training Data
Testing Data
Feature
Extraction
Train
Score
Model
Evaluation
Dataset
#EUai10 15
#EUai10
Training notebook available
16
github.com/fclesio/watcher-ai-samples
#EUai10
Evaluating Model Results
17
Machine Learning Tested Model Accuracy RMSE
Lasso with SGD Model 35% 0.32
Ridge Regres...
#EUai10
Watcher-ai Introduction
18
Hi I'm Watcher-ai!
It is nice to see you here
Applied Machine Learning
to solving probl...
#EUai10 19
Watcher-ai Training
#EUai10 20
Watcher-ai using models
#EUai10 21
Watcher-ai request predictions
#EUai10 22
Watcher-ai notification
#EUai10 23
Watcher-ai Architecture
Lessons Learned
Empirical observations about this kind of problem
#EUai10
• Regularization doesn't fit so well with our low
dimensional data
• Linear Methods are good for extrapolation but...
#EUai10
• Time Series with thresholds didn't work in the past
because we have several exogenous factors that make the
regu...
#EUai10
Why we changed from RDD to Dataframe?
27
RDD
(2011)
DataFrame
(2013)
distributed collection
of JVM objects
functio...
#EUai10
Why we changed from RDD to Dataframe?
• A good way to perform Grid-Search in our models
• Simpler and cleaner code...
Final Results
#EUai10 30
Avoid to Lose more than
U$ 3 Million Dollars preventing leakage
Saved more than 500
working hours
Recovery Time...
#EUai10
• Able to prevent revenue loss
• The main monitoring system
• Successful case of applied Machine Learning
• Simple...
github.com/fclesio/watcher-ai-samples
http://bit.ly/2gE4sTO
http://bit.ly/2gDEr7m
THANK YOU!
eitikimura eiti.kimura@movile...
Próxima SlideShare
Cargando en…5
×

Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Learning Eiti Kimura and Flavio Clésio

747 visualizaciones

Publicado el

Have you imagined a simple machine learning solution able to prevent revenue leakage and monitor your distributed application? To answer this question, we offer a practical and a simple machine learning solution to create an intelligent monitoring application based on simple data analysis using Apache Spark MLlib. Our application uses linear regression models to make predictions and check if the platform is experiencing any operational problems that can impact in revenue losses. The application monitor distributed systems and provides notifications stating the problem detected, that way users can operate quickly to avoid serious problems which directly impact the company’s revenue and reduce the time for action. We will present an architecture for not only a monitoring system, but also an active actor for our outages recoveries. At the end of the presentation you will have access to our training program source code and you will be able to adapt and implement in your company. This solution already helped to prevent about US$3mi in losses last year.

Publicado en: Datos y análisis
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Learning Eiti Kimura and Flavio Clésio

  1. 1. Flavio Clésio, Movile Eiti Kimura, Movile PREVENTING REVENUE LEAKAGE AND MONITORING DISTRIBUTED SYSTEMS WITH MACHINE LEARNING #EUai10
  2. 2. #EUai10 ABOUT US 2 Flávio Clésio • Core Machine Learning at Movile • MSc. in Production Engineering (Machine Learning in Credit Derivatives/NPL) • Specialist in Database Engineering and Business Intelligence • Blogger at Mineração de Dados (Data Mining) - http://mineracaodedados.wordpress.com • Strata Hadoop World Singapore Speaker (2016) flavioclesio
  3. 3. #EUai10 ABOUT US 3 • IT Coordinator and Software Architect at Movile • Msc. in Electrical Engineering • Apache Cassandra MVP (2014/2015 and 2015/2016) • Apache Cassandra Contributor (2015) • Cassandra Summit Speaker (2014 and 2015) • Strata Hadoop World Singapore Speaker (2016) Eiti Kimura eitikimura
  4. 4. Movile is the company behind several apps that makes the life easier WE MAKE LIFE BETTER THROUGH OUR APPS
  5. 5. #EUai10 Agenda 5 ● The Movile's Platform Case ● Practical Machine Learning Model Training ● Key Takeaways and Results
  6. 6. #EUai10 SBS Subscription and Billing Platform 6
  7. 7. #EUai10 7 THE INPUT
  8. 8. #EUai10 8 THE PROCESSING
  9. 9. #EUai10 9 THE OUTPUT
  10. 10. #EUai10 10
  11. 11. #EUai10 Main Problem: Monitoring 11 How can we check if platform is fully functional based on data analysis only? Tip: what if we ask help to an intelligent system?
  12. 12. #EUai10 The Data Volumetry 12 ● 236 Millions + of billing requests attempt a day ● 4 main mobile carriers drive the operational work
  13. 13. #EUai10 Stating the problem 13 Sample of data (predicting the number of success) featureslabel/target # success carrier_weight hour week response_time #no_credit #errors # attempts 61.083, [4.0, 17h, 3.0, 1259.0, 24.751.650, 2.193.67, 26.314.551] SUPERVISED LEARNING Linear Regression
  14. 14. #EUai10 The Modeling Lifecycle 14 Training Data Testing Data Feature Extraction Train Score Model Evaluation Dataset
  15. 15. #EUai10 15
  16. 16. #EUai10 Training notebook available 16 github.com/fclesio/watcher-ai-samples
  17. 17. #EUai10 Evaluating Model Results 17 Machine Learning Tested Model Accuracy RMSE Lasso with SGD Model 35% 0.32 Ridge Regression with SGD Model 87.5% 0.13 Elastic Net with SGD Model 35% 0.32 Decision Tree Model 93.4% 0.05
  18. 18. #EUai10 Watcher-ai Introduction 18 Hi I'm Watcher-ai! It is nice to see you here Applied Machine Learning to solving problems
  19. 19. #EUai10 19 Watcher-ai Training
  20. 20. #EUai10 20 Watcher-ai using models
  21. 21. #EUai10 21 Watcher-ai request predictions
  22. 22. #EUai10 22 Watcher-ai notification
  23. 23. #EUai10 23 Watcher-ai Architecture
  24. 24. Lessons Learned Empirical observations about this kind of problem
  25. 25. #EUai10 • Regularization doesn't fit so well with our low dimensional data • Linear Methods are good for extrapolation but Decision Trees are more suitable for interpolation problems Regularization and Linear Methods 25
  26. 26. #EUai10 • Time Series with thresholds didn't work in the past because we have several exogenous factors that make the regular algorithms behaving badly. • We avoid (totally removed) fixed thresholds based on standard deviations The Timeseries Thing 26
  27. 27. #EUai10 Why we changed from RDD to Dataframe? 27 RDD (2011) DataFrame (2013) distributed collection of JVM objects functional operators like (map, filter, etc) Distributed collection of Row objects Expression-base operations and UDF Logical plans and optimizer Fast/efficient internal representation
  28. 28. #EUai10 Why we changed from RDD to Dataframe? • A good way to perform Grid-Search in our models • Simpler and cleaner code, better to debug 28
  29. 29. Final Results
  30. 30. #EUai10 30 Avoid to Lose more than U$ 3 Million Dollars preventing leakage Saved more than 500 working hours Recovery Time drops from 6 hours to 1 hour
  31. 31. #EUai10 • Able to prevent revenue loss • The main monitoring system • Successful case of applied Machine Learning • Simple solution with Apache Spark Our Goals 31
  32. 32. github.com/fclesio/watcher-ai-samples http://bit.ly/2gE4sTO http://bit.ly/2gDEr7m THANK YOU! eitikimura eiti.kimura@movile.com flavio.clesio@movile.comflavioclesio

×