Apache Spark’s machine learning library provides a simple, elegant, yet powerful framework for creating scalable machine learning pipelines. It provides out of the box components for feature extraction and transformation, as well as various machine learning algorithms. However, in recent years specialized systems (such as TensorFlow, Caffe, PyTorch and Apache MXNet) have been dominant in the domain of AI and deep learning, as they allow greater performance and flexibility for training complex models. While there are a few deep learning frameworks that are Spark specific, in most cases these frameworks are separate from Spark and the ease of integration and feature set exposed varies considerably. This session will explore the role of Spark within the AI landscape, the current state of deep learning on top of Spark and the most recent developments in the Spark project to better integrate Spark with the deep learning ecosystem.

2. About
@MLnick on Twitter & Github
Principal Engineer, IBM
CODAIT - Center for Open-Source Data
& AI Technologies
Machine Learning & AI
Apache Spark committer & PMC
Author of Machine Learning with Spark
Various conferences & meetups

3. Center for Open Source Data
and AI Technologies
CODAIT
codait.org
DBG / Oct 4, 2018 / © 2018 IBM Corporation
CODAIT aims to make AI solutions
dramatically easier to create, deploy,
and manage in the enterprise
Relaunch of the Spark Technology
Center (STC) to reflect expanded
mission
Improving Enterprise AI Lifecycle in Open Source
Gather
Data
Analyze
Data
Machine
Learning
Deep
Learning
Deploy
Model
Maintain
Model
Python
Data Science
Stack
Fabric for
Deep Learning
(FfDL)
Mleap +
PFA
Scikit-LearnPandas
Apache
Spark
Apache
Spark
Jupyter
Model
Asset
eXchange
Keras +
Tensorflow

4. The Machine Learning
Workflow

5. Applying Machine Learning: Perception

6. In reality the workflow spans teams …

7. … and tools

8. Spark provides a unified platform

9. Machine Learning
Pipelines

10. What is a “model”?

11. Pipelines in Spark ML

12. Example – Text Classifier Pipeline

13. Example – Text Classifier PipelineModel

14. Spark ML Components
14
Source: http://spark.apache.org/docs/latest

15. Deep Learning
DBG / June 6, 2018 / © 2018 IBM Corporation

16. Deep Learning Overview
• Original theory from 1940s; computer models
originated around 1960s; fell out of favor in
1980s/90s
• Recent resurgence due to
• Bigger (and better) data; standard datasets (e.g.
ImageNet)
• Better hardware (GPUs)
• Improvements to algorithms, architectures and
optimization
• Leading to new state-of-the-art results in
computer vision (images and video);
speech/text; language translation and more
Source: Wikipedia

17. Modern Neural Networks
• Deep (multi-layer) networks
• Computer vision
• Convolution neural networks (CNNs)
• Image classification, object detection, segmentation
• Sequences and time-series
• Recurrent neural networks (RNNs)
• Machine translation, text generation
• Embeddings
• Text, categorical features
• Deep learning frameworks
• Flexibility, computation graphs, auto-differentiation,
GPUs
Source: Stanford CS231n

18. Deep Learning Frameworks
* Logos trademarks of their respective projects

19. Computation Graphs
Source: Google AI Blog
*MnasNet Network
*Inception V3

20. DL Frameworks on Spark

21. Major Frameworks
21
• Deeplearning4J
• BigDL
• Deep Learning Pipelines
• TensorFlowOnSpark
• Microsoft Machine Learning on Spark
(MMLSpark)

22. Deeplearning4J
22
• Distributed GPU support for all major deep
learning architectures
• CPU / Distributed CPU / Single GPU options exist
• Supports Convolutional Nets, LSTMs / RNNs,
Feedforward Nets, Word2Vec, custom layers
• Supported by startup Skymind.io
• Backed by its own linear algebra library –
ND4J
• APIs in Scala, Java, Python
• Newer Scala API, Keras-like
• Keras import / export for Python API
• Production serving is through proprietary
layer
• DataVec for ETL

23. BigDL
23
• Distributed CPU with Intel MKL
• No GPU support
• Most DL models – CNN, RNN
• Backed by Intel
• Natively integrated with Spark
• Scala, Python API
• Support for Spark ML pipelines
• Uses private internal Spark components for
distributed training
• Load Keras, Caffe, Torch models
• New Keras-style API

24. Deep Learning Pipelines
24
• Created by Databricks
• Focus on scoring models (TensorFlow / Keras) and
basic transfer learning
• No support for training the DL model
• Focus on image data & use cases
• Natively integrated with Spark
• Scala, Python API
• Support for Spark ML pipelines
• Support for scoring models as a SQL UDF
• Largely dormant currently

25. TensorFlowOnSpark
25
• Created by Yahoo
• Scale out TF on Spark clusters
• Use Spark executors to launch TF processes
• Supports distributed training through TF parameter
servers
• RDMA / Infiniband improvement to TF to speed up
distributed training
• Good support for TensorBoard
• Good integration with Spark
• But only Python API
• Some support for Spark ML pipelines
• Relatively inactive recently

26. MMLSpark
26
• Created by Microsoft
• Supports training using CNTK including distributed
• Image, text data
• Good integration with Spark
• Scala, Python, R API
• Support for Spark ML pipelines
• Varied deployment options
• Relatively active, seems quite well supported

27. Other Frameworks
27
• H20 AI / DeepWater
• Apache MXNet Spark integration
• TensorFrames
• CaffeOnSpark
• scalable-deep-learning on Github
• MLlib – MLPClassifier only
• Sparknet (abandoned)

28. Integration Challenges
28
• Moving data from Spark to DL framework (and
back)
• Serialization overhead – especially Python
• Managing DL computation graphs from Spark
executors means fault tolerance is difficult to
achieve
• GPU awareness
• Optimize and standardize data exchange -
SPARK-24579
• Apache Arrow
• Barrier Execution Mode - SPARK-24374
• Accelerator-aware scheduling - SPARK-
24615

29. 29
Thank you!
codait.org
twitter.com/MLnick
github.com/MLnick
developer.ibm.com
FfDL
Sign up for IBM Cloud and try Watson Studio!
https://ibm.biz/BdYhXz
https://datascience.ibm.com/
MAX

30. Brought to you by community.ibm.com/icpfordata Catch the replay at ibmaicommunity.bemyapp.com