Deep learning is a fast growing subset of machine learning. There is an emerging trend to conduct deep learning in the same cluster along with existing data processing pipelines to support feature engineering and traditional machine learning. As the leading framework for Distributed ML, we believe that the addition of deep learning to the super-popular Spark framework is important, because it allows Spark developers to perform a range of data analysis tasks within a single framework that helps avoid the complexity inherent in using multiple frameworks and libraries. As one of the early and top contributors to Apache Spark, Intel is thrilled to share with the community a big deal contribution to open source Spark…”BigDL” -… A distributed deep Learning framework organically built on Big Data (Apache Spark) platform. It combines the benefits of “high performance computing” and “Big Data” architecture for rich deep learning support. With BigDL on Spark, customers can eliminate large volume of unnecessary dataset transfer between separate systems, eliminate separate HW clusters and move towards a CPU cluster, reduce system complexity and the latency for end-to-end learning. Ultimately, customers can achieve better scale, higher resource utilization, ease of use/development, and better TCO. Feature parity with Caffe and Torch, significant performance boost when combined with Intel’s Math Kernel Library (MKL), scale-out, fault tolerance, elasticity and dynamic resource sharing are some of the prominent features of BigDL. BigDL open source project will be launched at the 2017 Spark Summit East and this keynote will help spotlight this new contribution and benefits to the Spark developer community and encourage their wide contribution and collaboration. We will also showcase some real world applications of Big DL from early customers’ adoption.

1. WITH INTEL AND APACHE SPARK

2. AIISTRANSFORMINGINDUSTRIES

3. BIG DATA
ANALYTICS
LOW LEVEL
SOFTWARE
PRIMITIVES
Intel® Math Kernel Library
for Deep Neural Networks
(Intel® MKL-DNN)
Intel® Deep Learning SDK
(Training& Deployment)
WORKFLOW
TOOLS/SDK
DEEP LEARNING
FRAMEWORK
OPTIMIZATIONS
INTEL®
ARCHITECTURE (IA)
BigDL
AIONIA:UNLEASHINGTHENEXTWAVE
LAKE
CREST

4. Easeof
Use
Lower
TCO
High
Performance
AINEEDS
Efficient
Scalability

5. BigDLAnsweringtheNeeds
FEATURE PARITY WITH CAFFE* and TORCH*
DRIVES SPARK/HADOOP AS THE UNIFIED
DATA ANALYTICS PLATFORM, GIVING EASY
CUSTOMER AND DEVELOPER EXPERIENCE
AND LOWER TCO
DEEP LEARNING ON BIG DATA PLATFORM,
ENABLING EFFICIENT SCALE-OUT
HIGH PERFORMANCE AT SINGLE-NODE
Github: github.com/intel-analytics/BigDL
*Other names and brands may be claimed as the property of others

6. CUSTOMERUSECASESFORBIGDL
MANUFACTURING
DEFECT RECOGNITION
FINANCIAL SERVICES
FRAUD DETECTION

7. JUST-IN-TIME
DATA PLATFORM
AUTOMATED
MANAGEMENT
INTERACTIVE
EXPLORATION
JOBS &
WORKFLOWSBigDL
BigDLIn
Product-
ionize
Model
Training&
Prediction
Feature
extraction
Data
IngestClusters

8. Github: github.com/intel-analytics/BigDL Additional info: software.intel.com/AI

9. LegalDisclaimers
• Intel technologies’ features and benefits depend on system configuration and may require
enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or
retailer.
• No computer system can be absolutely secure.
• Tests document performance of components on a particular test, in specific systems.
Differences in hardware, software, or configuration will affect actual performance. Consult
other sources of information to evaluate performance as you consider your purchase. For
more complete information about performance and benchmark results, visit
http://www.intel.com/performance.
Intel, the Intel logo, Xeon, Xeon phi, Lake Crest, etc. are trademarks of Intel Corporation in the
U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© 2017 Intel Corporation