7.
Jeff Dean, “An Overview of Google's Work on AutoML and Future Directions” , ICML 2019
https://slideslive.com/38917182/an-overview-of-googles-work-on-automl-and-future-directions
8.
Automated Hyperparameter Optimization
Hyperopt, Optuna, SMAC3, scikit-optimize, …
Jeff Dean, “An Overview of Google's Work on AutoML and Future Directions” , ICML 2019
https://slideslive.com/38917182/an-overview-of-googles-work-on-automl-and-future-directions
9. Jeff Dean, “An Overview of Google's Work on AutoML and Future Directions” , ICML 2019
https://slideslive.com/38917182/an-overview-of-googles-work-on-automl-and-future-directions
HPO + Automated Feature Engineering
featuretools, tsfresh, boruta, …
10.
Automated Algorithm(Model) Selection
Auto-sklearn, TPOT, H2O, auto_ml, MLBox, …
Jeff Dean, “An Overview of Google's Work on AutoML and Future Directions” , ICML 2019
https://slideslive.com/38917182/an-overview-of-googles-work-on-automl-and-future-directions
14. Eric Brochu, Vlad M. Cora, and Nando de Freitas. A tutorial on Bayesian optimization of expensive cost functions,
with application to active user modeling and hierarchical reinforcement learning. 2010. arXiv:1012.2599.
15. Eric Brochu, Vlad M. Cora, and Nando de Freitas. A tutorial on Bayesian optimization of expensive cost functions,
with application to active user modeling and hierarchical reinforcement learning. 2010. arXiv:1012.2599.
:
:
16. Eric Brochu, Vlad M. Cora, and Nando de Freitas. A tutorial on Bayesian optimization of expensive cost functions,
with application to active user modeling and hierarchical reinforcement learning. 2010. arXiv:1012.2599.
17.
Eric Brochu, Vlad M. Cora, and Nando de Freitas. A tutorial on Bayesian optimization of expensive cost functions,
with application to active user modeling and hierarchical reinforcement learning. 2010. arXiv:1012.2599.
18.
Jamieson, K. G. and Talwalkar, A. S.: Non-stochastic Best Arm Identification
and Hyperparameter Optimization, in AIS-TATS (2016).
10 epochs
trial #1
trial #2
trial #3
trial #4
trial #5
trial #6
trial #7
trial #8
trial #9
30 epochs 90 epochs
Liam Li, Kevin Jamieson, Afshin Rostamizadeh, Ekaterina Gonina,
Moritz Hardt, Benjamin Recht, and Ameet Talwalkar.
Massively parallel hyperparameter tuning. arXiv preprint arXiv:1810.05934, 2018.
29. 1. Feature Preprocessing Operators.
StandardScaler, RobustScaler,
MinMaxScaler, MaxAbsScaler,
RandomizedPCA, Binarizer, and
PolynomialFeatures.
2. Feature Selection Operators:
VarianceThreshold, SelectKBest,
SelectPercentile, SelectFwe, and
Recursive Feature Elimination (RFE).
AutoML feature preprocessing
M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter.
Efficient and robust automated machine learning.
In Neural Information Processing Systems (NIPS), 2015
R. S. Olson and J. H. Moore. Tpot: A tree-based pipeline optimization
tool for automating machine learning.
In Workshop on Automatic Machine Learning, 2016
TPOT Auto-sklearn
30. 1. Feature Preprocessing Operators.
StandardScaler, RobustScaler,
MinMaxScaler, MaxAbsScaler,
RandomizedPCA, Binarizer, and
PolynomialFeatures.
2. Feature Selection Operators:
VarianceThreshold, SelectKBest,
SelectPercentile, SelectFwe, and
Recursive Feature Elimination (RFE).
AutoML feature preprocessing
M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter.
Efficient and robust automated machine learning.
In Neural Information Processing Systems (NIPS), 2015
R. S. Olson and J. H. Moore. Tpot: A tree-based pipeline optimization
tool for automating machine learning.
In Workshop on Automatic Machine Learning, 2016
TPOT Auto-sklearn
42. • AutoML 2
• ML
• ML
•
AutoML as a CASH Problem
Combined Algorithm Selection and Hyperparameter optimization
M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter.
Efficient and robust automated machine learning. In Neural Information Processing Systems (NIPS), 2015