We provide a recap of the MLflow R interface which was announced at Spark+AI Summit Europe and discuss recent developments. The session includes a live demo showcasing the intersection of big data (Spark) and deep learning (via TensorFlow) and how the end-to-end lifecycle from prototyping to deployment can be managed by MLflow.

1. Kevin Kuo @kevinykuo, RStudio
Streamlining AI
Prototyping and
Deployment with R and
MLflow
#UnifiedAnalytics #SparkAISummit

2. Daily specials
- Quick update on the R ecosystems for AI stuff
- Recap of MLflow
- Demo
- Discussion + Q&A
2#UnifiedAnalytics #SparkAISummit

3. Sparklyr Update
- Arrow integration to massively speed up UDFs
- XGBoost
- TFRecord read/write
- SparkNLP on the way
https://spark.rstudio.com/
3#UnifiedAnalytics #SparkAISummit

4. TensorFlow Update
4#UnifiedAnalytics #SparkAISummit

5. TensorFlow Update
5#UnifiedAnalytics #SparkAISummit

6. TensorFlow Update
He looks skeptical, as if you
were nothing
get it right.
It drives me crazy, I can do
this
reproachful look no longer
endure.
His name is Olaf.
6#UnifiedAnalytics #SparkAISummit

7. TensorFlow Update
- library(keras) defaults to tf.keras
- TensorFlow Probability for probabilistic modeling
- Eager execution
- Preparing for TF2.0 drop
https://tensorflow.rstudio.com/
https://blogs.rstudio.com/tensorflow/
7#UnifiedAnalytics #SparkAISummit

8. Quick recap of MLflow
Open source platform for
- Experiment instrumentation (Tracking)
- Reproducible runs (Projects)
- Model deployment (Models)
8#UnifiedAnalytics #SparkAISummit

9. Tracking
Keeping track of stuff
mlflow_log_param("num_hidden_units", 64)
mlflow_log_artifact("training_history.png")
mlflow_log_metric("accuracy", metrics$acc)
9#UnifiedAnalytics #SparkAISummit

10. Projects
Packaging up (reproducible) building blocks
mlflow_run("data-prep.R")
10#UnifiedAnalytics #SparkAISummit

11. Models
Deployment
flavors:
keras:
version: 2.2.2
data: model.h5
python_function:
loader_module: mlflow.keras
data: model.h5
env: conda_env.yaml
utc_time_created: 19-04-25T01:00:21.21.72
11#UnifiedAnalytics #SparkAISummit
mlflow_rfunc_serve(
"keras_model",
run_uuid = training_run_id
)

12. Demo!
12#UnifiedAnalytics #SparkAISummit

13. Roadmap
How are package dependencies handled for R
projects?
Conda? Packrat?
What if your packages depend on Java/Python
libraries?
13#UnifiedAnalytics #SparkAISummit

14. Quick excursion on
dependency management
14#UnifiedAnalytics #SparkAISummit

15. Renv is in
15#UnifiedAnalytics #SparkAISummit
renv::init()
renv::restore()

16. Renv is in
16#UnifiedAnalytics #SparkAISummit
Conda support in
progress for reticulated
packages

17. What about...
What about stuff with
Java/rJava
dependencies?!?!
17#UnifiedAnalytics #SparkAISummit

18. Betting on new tech
Why MLflow and not
something else?
18#UnifiedAnalytics #SparkAISummit

19. Roadmap
Better integration with deployment tech
- MLeap (https://github.com/rstudio/mleap)
- H2O
- TensorFlow Serving
- Arbitrary R models (plumber + docker)
19#UnifiedAnalytics #SparkAISummit

20. Resources
- https://mlflow.org/
- https://github.com/mlflow/mlflow
- https://tensorflow.rstudio.com/
- https://spark.rstudio.com/
- https://community.rstudio.com/
- Demo Repo:
https://github.com/kevinykuo/sais2019-mlflow
20#UnifiedAnalytics #SparkAISummit

21. DON’T FORGET TO RATE
AND REVIEW THE SESSIONS
SEARCH SPARK + AI SUMMIT