Presensation on why Machine Learning/ Deep Learning is a hot topic and what role Tensorflow plays in getting the most out of your models.

1. Deep Learning Lecture Series
IBM Executive Briefing Center
Austin,TX
Session: Introduction to Tensorflow
Presenter: Clarisse Taaffe-Hedglin
clarisse@us.ibm.com
Executive HPC/HPDA Architect
IBM Systems WW Client Centers

2. © Copyright IBM Corporation 2016
Session Abstract and Outline
Abstract:
This presentation recaps the reasons why deep learning is so popular today and explores the tools
Data Scientists use. We’ll introduce the more commonly used frameworks and dive into
Tensorflow. We’ll compare and contrast frameworks and key factors that drive framework
selection.
Outline:
ØUnderstand what frameworks are and how they are used by data scientists
Ø Have more in depth understanding of the Tensorflow framework
Ø See Tensorflow in action

3. To Begin
https://xkcd.com/license.html

4. 3
Why Deep Learning Now?
→ Data
→ Compute
→ Technique

5. Exploding Data Sources
ImageNet 10,000,000 labeled images
depicting 10,000+ object categories
CIFAR-10 (RBG)
https://quickdraw.withgoogle.com/data
Learned filter for AlexNet, Krizhevsky et al. 2012
MNIST 0-9
300,000 Labeled images
Over 1000 datasets at:
https://www.kaggle.com/datasets

6. Data & Compute Drive Training & Inference
Training
•Data intensive:
historical data sets
•Compute intensive:
100% accelerated
•Develop a model for use
on the edge as inference
Inference
•Enables the computer
to act in real time
•Low Power
•Out at the edge

7. Technique Increasing in Complexity
Artificial Neural Networks are evolving
Perceptron
GoogLeNet
Recurrent Neural Network

8. 7
Frameworks Address Technique
Frameworks enable developers to build, implement and maintain
machine learning systems, generate new projects and create new
impactful systems (Models).
Analytics tools and AI frameworks implemented by data science
engineers are often driven by researcher and data scientist preferences

9. Models Deployed Across All Industries
Automotive and
Transportation
Security and Public
Safety
Consumer Web,
Mobile, Retail
Medicine and Biology Broadcast, Media and
Entertainment
• Autonomous driving:
• Pedestrian detection
• Accident avoidance
Auto, trucking, heavy
equipment, Tier 1
suppliers
• Video Surveillance
• Image analysis
• Facial recognition and
detection
Local and national
police, public and
private safety/ security
• Image tagging
• Speech recognition
• Natural language
• Sentiment analysis
Hyperscale web
companies, large
retail
• Drug discovery
• Diagnostic assistance
• Cancer cell detection
Pharmaceutical, Medical
equipment, Diagnostic
labs
• Captioning
• Search
• Recommendations
• Real time translation
Consumer facing
companies with large
streaming of existing
media, or real time
content

10. 9
Using a Range of Data Science Software
Tool
%
change
2017
% usage
2016
% usage
Microsoft, CNTK 294% 3.4% 0.9%
Tensorflow 195% 20.2% 6.8%
Microsoft Power BI 84% 10.2% 5.6%
Alteryx 76% 5.3% 3.0%
SQL on Hadoop tools 42% 10.3% 7.3%
Microsoft other tools 40% 2.2% 1.6%
Anaconda 37% 21.8% 16.0%
Caffe 32% 3.1% 2.3%
Orange 30% 4.0% 3.1%
DL4J 30% 2.2% 1.7%
Other Deep Learning Tools 30% 4.8% 3.7%
Microsoft Azure ML 26% 6.4% 5.1%
Source: http://www.kdnuggets.com/2017/05/poll-analytics-data-science-machine-learning-software-leaders.html
Deep Learning tools used by 32% of all
respondents (18% in 2016, 9% in 2015)

11. © Copyright IBM Corporation 2016
10
Python Libraries for Data Scientists
Keras works with Theano,
Tensorflow, CNTK, and MXNet. R
users can use Keras without having
to switch over to Python. Sharing
models and weights is easier if all
frameworks unify around same API.
Tensorflow and Caffe most
popular + wrappers and
tools for ease of use, rapid
prototyping, reduced/ no
coding or specialized
networks or uses
Source: github and google

12. TensorFlow Overview
Framework developed by Google (Google Brain Team)
Created for machine learning & deep neural networks research
For numerical computation using data flow graphs
Tensorflow is opensource since Nov 2015, released under the Apache 2.0
https://github.com/tensorflow/tensorflow
Very strong developer/user community: 36,790+ forks, 1,100 Contributors
Written in C++, CUDA, some Python; Python and Matlab interfaces

13. 12
TensorFlow Framework

14. 13
TensorFlow Constructs
Model Development
Learning model described by data flow graphs:
Nodes: represent mathematical operations (a.k.a. ops)
• General purpose
• Neural Net
Edges: represent data in N-D Arrays (Tensors)
Backward graph and update are added automatically to
graph
Inference
Execute forward path on graph
• TensorFlow Core is lowest level API for complete programming control
• Higher level APIs available (e.g. skflow as part of Scikit Learn API)
• Higher level abstractions for common patterns, structures and functionality

15. © Copyright IBM Corporation 2016
TensorFlow Use Cases and Model Zoo
Wide range of models: Vision, NLP, Text, speech
Used in production by many companies starting with Google (over
2000 projects in 2016), AirBnB, eBay, SAP, Qualcomm, Intel, Uber,
Twitter, General Motors, etc
With growing library of best in class models

16. © Copyright IBM Corporation 2016
Running Tensorflow
15
Build
Run
Ref: https://www.tensorflow.org/get_started/get_started
Optimizer to minimize loss
Loss (true vs predicted)
Training data and labels
Setup libraries
Repeat training
Initialize and start session to run
Define parameters to train
Hidden layer
Output prediction accuracy

17. © Copyright IBM Corporation 2016
Navigating www.tensorflow.org and https://github.com/tensorflow/
16
- Install tips
- Getting started and Quickstarts
- API documentation by version
- Programmer’s Guide
- Tutorials
- Community

18. © Copyright IBM Corporation 2016
Why TensorFlow
Large Community and Google!
Cross platform support of heterogeneous devices for rapid deployment
Efficient computations with GPUs

19. © Copyright IBM Corporation 2016
Comparing Frameworks
18
Many framework options and
many evaluation criteria – very
dynamic!
Driven by Modeling capabilities:
Interfaces available
Ease of use
Algorithms supported
Ecosystem and support
Model deployment
Cross platform
Metrics can also play a role:
Speed of execution
Resources required (CPU and Memory capacity)
GPU support and performance
Distributed systems capability
http://www.kdnuggets.com/2017/03/getting-started-deep-learning.html
Data Scientist Challenges (Source:NVIDIA)

20. 19
Framework Scalability and Flexibility
Scalability-oriented Flexibility-oriented
▶ Use-cases in mind
▶ New algorithm research
▶ R&D projects for AI products
▶ Problem type
▶ Various specific applications
▶ 10+ k training samples
▶ 1 node with multiple GPUs
▶ Possible bottleneck
▶ Trial-and-error in prototyping
▶ Debugging, profiling & refactoring
▶ (wait time during compilation)
▶ Use-cases in mind
▶ Image/speech recognition system
▶ Fast DL as a service in cloud
▶ Problem type
▶ A few general applications
▶ 10+ million training samples
▶ 10+ nodes cluster w/ fast network
▶ Possible bottleneck
▶ Tuning of well-known algorithms
▶ Distributed computation for
model/data-parallel training
Source: Preferred Networks presentation,
2017 OpenPOWER Developer Congress

21. © Copyright IBM Corporation 2016
Scalability: Distributed Deep Learning
Problem: Deep learning applications have been
limited by scalability to date. Run your job inside only a
single system, where the number of installed GPUs is
the ceiling on your performance scaling.
Even with the newest Deep Learning frameworks,
performance across servers hasn’t delivered because
the best fabric interfaces aren’t available by default.
Solution: DDL solves these challenges. It
efficiently expands Deep Learning to the
cluster, leverages leading interconnects, &
delivers world-class perf scaling.
Uncap your scaling, and let IBM AI expertise + DDL
bring the right work to the right place for optimal
performance.
Scaling results using Caffe to train a ResNet-50 model using the ImageNet-1K data set on
64 Power Systems servers that have a total of 256 NVIDIA P100 GPU accelerators in them.

22. © Copyright IBM Corporation 2016
Flexibility: Comparison of Domain Specific Language type
21
DSL type Example Pros. Cons.
Text DSL
Caffe (prototext), CNTK (NDL)
f: {
“A”: “Variable”,
“B”: “Variable”,
“C”: [“B”, “*”, “A”],
“ret”: [“C”, “+”, 1]
}
• Human-readable definition
• Non-programmer can easily
edit the network
• Users must study the format
• Format might have to be
extended for new algorithms
Internal DSL
Symbolic
Tensorflow, Theano
A = Variable(‘A’)
B = Variable(‘B’)
C = B * A
D = C + Constant(1)
• Static analysis at compile
• Optimization before training
• Easy to parallelize
• Users must study special
syntax
• May need more efforts to
implement new algorithms
Imperative
Torch, Chainer
a = np.ones(10)
b = np.ones(10) * 2
c = b * a
d = c + 1
• Less efforts to learn syntax
• Easy debugging and profiling
• Suitable for new algorithms
with complex logic
• Hard to optimize in advance
• Less efficient in memory
allocation and parallelization
Source: Preferred Networks presentation,
2017 OpenPOWER Developer Congress

23. © Copyright IBM Corporation 2016
Graph Build Scheme Differences
22
Caffe, Tensorflow
Run
Define
Network
definition
Computational
graph
Gradient
function
Parameters
Computational
graph
Gradient
function
Parameters
Training
data
Update
Loss & gradient
Auto differentiation
Define-by-Run
Model
definition
Computationa
l graph
Gradient
function
Parameters
Training
data
Update
Dynamic change
Conditions
Chainer, Pytorch
Source: Preferred Networks presentation,
2017 OpenPOWER Developer Congress

24. © Copyright IBM Corporation 2016
based on:
https://github.com/googledatalab/notebooks/blob/master/samples/TensorFlow/Machine%20Learning%20with%20Financial%20Data.ipynb
23
Demo: Machine Learning with Financial Data
https://developer.ibm.com/linuxonpower/deep-learning-powerai

25. © Copyright IBM Corporation 2016
In Summary
24
https://xkcd.com/license.html

26. © Copyright IBM Corporation 2016
| 25
ccenter@us.ibm.com
Engage
with us
today !

27. © Copyright IBM Corporation 2016
• https://www.nvidia.com/en-us/deep-learning-ai/solutions/
• https://www.tensorflow.org/
• http://download.tensorflow.org/paper/whitepaper2015.pdf
• https://www.youtube.com/watch?v=oZikw5k_2FM&feature=youtu.be&t=61
• http://colah.github.io/posts/2014-07-Conv-Nets-Modular/
• https://github.com/rasbt/pattern_classification, https://github.com/rasbt/python-
machine-learning-book
• https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0
• http://www.cs.toronto.edu/~hinton/digits.html
• https://www.wired.com/2015/11/google-open-sourcing-tensorflow-shows-ais-
future-is-data-not-code/
• https://www.udacity.com/course/deep-learning--ud730
• http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-
tensorflow/
• http://emnlp2014.org/papers/pdf/EMNLP2014194.pdf
• https://github.com/zer0n/deepframeworks
• https://www.slideshare.net/IndrajitPoddar/fast-scalable-easy-machine-learning-
with-openpower-gpus-and-docker
• https://aitopics.org/
Acknowledgements and References