Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
•
0 recomendaciones•356 vistas
This presentation introduces Tune and Fugue, frameworks for intuitive and scalable hyperparameter optimization (HPO). Tune supports both non-iterative and iterative HPO problems. For non-iterative problems, Tune supports grid search, random search, and Bayesian optimization. For iterative problems, Tune generalizes algorithms like Hyperband and Asynchronous Successive Halving. Tune allows tuning models both locally and in a distributed manner without code changes. The presentation demonstrates Tune's capabilities through examples tuning Scikit-Learn and Keras models. The goal of Tune and Fugue is to make HPO development easy, testable, and scalable.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
Similar a Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue (20)
Más de Databricks
Más de Databricks (20)
Último
Último (20)
Intuitive & Scalable Hyperparameter Tuning with Apache Spark + Fugue
- 1. Intuitive & Scalable HPO With Spark+Fugue Han Wang
- 3. pip install tune https://github.com/fugue-project/tune pip install fugue https://github.com/fugue-project/fugue
- 4. Introduction
- 5. Questions ● Is parameter tuning a machine learning problem? ● Are there common ways to tune both classical models and deep learning models? ● Why is it so hard to do distributed parameter tuning?
- 6. Tuning Problems In General General Parameter Tuning Hyperparameter Tuning (for Machine Learning) Some Classical Models Deep Learning Models Some Classical Models Non-Iterative Problems Iterative Problems
- 7. Distributed Parameter Tuning ● Not everything can be parallelized ● The tuning logic is always complex and tedious ● Popular tuning frameworks are not distributed environment friendly ● Spark is not suitable for iterative tuning problems
- 8. Distributed Parameter Tuning Tune SQL Validation
- 9. Our Goals ● For non-iterative problems: ○ Unify grid and random search, make other plugable ● For iterative problems: ○ Generalize SOTA algos such as Hyperband and ASHA ● For both ○ Tune both locally and distributedly without code change ○ Make tuning development iterable and testable ○ Minimize moving parts ○ Minimize interfaces
- 11. Grid Search a: Grid(0,1) b: Grid(“a”, “b”) c: 3.14 a:0, b:”a”, c:3.14 a:0, b:”b”, c:3.14 a:1, b:”a”, c:3.14 a:1, b:”b”, c:3.14 Search Space Candidates Pros: determinism, even coverage, interpretable Cons: complexity can increase exponentially
- 12. Random Search a: Rand(0,1) b: Choice(“a”,“b”) c: 3.14 a:0.12, b:”a”, c:3.14 a:0.66, b:”a”, c:3.14 a:0.32, b:”b”, c:3.14 a:0.94, b:”a”, c:3.14 Search Space Candidates Pros: complexity and distribution are controlled, good for continuous variables Cons: by luck, not deterministic, large number of samples are normally needed
- 13. Bayesian Optimization objective: a^2 a: Rand(-1,1) -0.66 -> 0.76 -> -0.18 -> 0.75 -> 0.90 -> 0.07 -> 0.00 -> 0.41 -> 0.12 -> 0.66 Search Space Candidates Pros: less compute to guess the optimal parameters Cons: sequential operations may require more time
- 14. Hybrid Search Space Distributed Hybrid Search Model 1 Model 2 Grid Random Bayesian
- 15. Live Demo Space Concept & Scikit-Learn Tuning
- 17. Challenges ● Realtime asynchronous communication ● The overhead for checkpointing iterations can be significant ● Single iterative problem can’t be parallelized ● A lot of boilerplate code
- 18. Successive Halving (SHA) Rung 1 Rung 2 Rung 3 Rung 4
- 19. Fully Customized Successive Halving 8, [(4,6), (2,2), (6,1)]
- 20. Hyperband
- 21. Asynchronous Successive Halving (ASHA)
- 22. Live Demo Keras Model Tuning
- 23. Summary Space Monitoring Dataset Distributed Execution Abstraction Non-Iterative Random, Grid, BO Iterative SHA, HB, ASHA, PBT ... Specialization Scikit-Learn Specialization Keras, TF, PyTorch
- 24. Let’s Collaborate! ● Create specialized higher level APIs for major tuning cases so users can do tuning with minimal code and without learning distributed systems ● Enable advanced users to create fully customized, platform agnostic and scale agnostic tuning pipelines with tune’s lower level APIs
- 25. pip install tune https://github.com/fugue-project/tune pip install fugue https://github.com/fugue-project/fugue
- 26. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.