This talk was presented in Startup Master Class 2017 - http://aaiitkblr.org/smc/ 2017 @ Christ College Bangalore. Hosted by IIT Kanpur Alumni Association and co-presented by IIT KGP Alumni Association, IITACB, PanIIT, IIMA and IIMB alumni. My co-presenter was Biswa Gourav Singh. And contributor was Navin Manaswi. http://dataconomy.com/2017/04/history-neural-networks/ - timeline for neural networks

1. AI and Deep Learning
Biswa Gourav Singh
Deep Learning Practice
AMD
MS Clemson University, 2010
Dr. Subrat Panda
Principal Architect AI and DS
Capillary Technologies
IIT Kharagpur 2002
IIT Kharagpur 2009 - PhD

2. Brief Introduction about Subrat
● BTech ( 2002) , PhD (2009) – CSE, IIT Kharagpur
● Synopsys (EDA), IBM (CPU), NVIDIA (GPU), Taro (Full Stack Engineer), Capillary
(Principal Architect - AI)
● Applying AI to Retail
● Co-Founded IDLI (for social good) with Prof. Amit Sethi (IIT Bombay), Jacob
Minz (Synopsys) and Biswa Gourav Singh (AMD)
● https://www.facebook.com/groups/idliai/
● Linked In - https://www.linkedin.com/in/subratpanda/
● Facebook - https://www.facebook.com/subratpanda
● Twitter - @subratpanda

3. Brief Introduction about Biswa
● BTech ( 2005) , MS (2010) – Clemson University
● Synopsys (EDA), IBM (CPU), ARM, Qualcomm, HP, AMD
● Deep Learning Performance
● Co-Founded IDLI (for social good) with Prof. Amit Sethi (IIT Bombay), Jacob
Minz (Synopsys) and Subrat Panda (Capillary)
● https://www.facebook.com/groups/idliai/
● Linked In - https://www.linkedin.com/in/biswagsingh/
● Facebook - https://www.facebook.com/biswa.singh

4. Agenda
1. Deep Learning: Overview - 5 minutes - Subrat
2. Deep Learning and Industries - 10 minutes - Subrat
3. How to do Deep Learning - 10 minutes - Subrat
4. Tensor-Flow, a DL framework - 20 minutes - Biswa
5. CNN Introduction - 10 minutes - Biswa
6. Transfer Learning - 15 minutes - Subrat
7. Building Deep Learning Application - 10 minutes - Subrat
a. Building a Product in the age of APIs - Subrat
8. Challenges of Deep Learning - 5 minutes - Subrat / Biswa ( Performance )
9. Q and A - 10 minutes

5. The AI landscape
Deep Learning: An Overview
Courtesy: NVIDIA

6. Machine Learning – http://techleer.com
Deep Learning: An Overview

7. The Neuron
• The neuron is the basic information processing unit of a
NN. It consists of:
1 A set of synapses or connecting links, each link
characterized by a weight: W1
, W2
, …, Wm
2 An adder function (linear combiner) which computes the
weighted sum of the inputs:
2 Activation function (squashing function) for limiting
the amplitude of the output of the neuron.

8. The Neuron
Input
signal
Synaptic
weights
Summing
function
Bias
b
Activation
function
Local
Field
v
Output
y
x1
x2
xm
w2
wm
w1

9. Is a single perceptron or layer enough ?
- No
- Perceptrons learn linearly separable functions
- Cannot learn a XOR function
- Minsky & Papert (1969) offered solution to XOR problem
by combining perceptron unit responses using a second layer
of Units. Piecewise linear classification using an MLP with
threshold (perceptron) unit

10. MLP - Multi-Layer Perceptron
• No connections within a layer
• No direct connections between input and output layers
• Fully connected between layers
• Often more than 3 layers
• Number of output units need not equal number of input units
• Number of hidden units per layer can be more or less than
input or output units

11. 3 Layer Neural Network

12. What do each layer do ?
1st layer draws linear
boundaries
2nd layer combines
the boundaries
3rd layer can
generate arbitrarily
complex boundaries

13. Forward Activity - Backward Error

14. Back Propagation
Forward pass phase: computes ‘functional signal’, feed forward
propagation of input pattern signals through network.
Backward pass phase: computes ‘error signal’, propagates
the error backwards through network starting at output units
(where the error is the difference between actual and desired
output values)

15. Deep Learning and Industries

16. HORIZONTAL APPLICATIONS
Deep Learning

17. MARKETING APPLICATIONS
Predicting
Lifetime Value
(LTV)
Wallet share
estimation
Churn
Customer
segmentation
Product mix Cross selling
Recommendatio
n algorithms
Up-selling
Channel
optimization
Discount
targeting
Reactivation
likelihood
Adwords
optimization and
ad buying

18. RISK APPLICATIONS
Credit risk
Treasury or
currency risk
Fraud
detection
Accounts
Payable
Recovery
Anti-money
laundering

19. HUMAN RESOURCES APPLICATIONS
Resume
screening
Employee churn
Training
recommendatio
n
Talent
management

20. MORE HORIZONTAL APPLICATIONS…
Sales
• Lead prioritization
Customer support
• Call routing
• Call center message optimization
• Call center volume forecasting
Logistics
• Demand forecasting

21. VERTICAL APPLICATIONS

22. HEALTHCARE APPLICATIONS
Claims review
prioritization
Medicare/medicai
d fraud
Medical resources
allocation
Alerting and
diagnostics from
real-time patient
data
Prescription
compliance
Physician attrition
Survival analysis
Medication
(dosage)
effectiveness
Readmission risk

23. RETAIL APPLICATIONS
Price
optimization
Location of new
stores
Product layout in
stores
Merchandizing
Inventory
Management
(how many
units)
Shrinkage
analytics
Warranty
Analytics
Market Basket
Analysis
Cannibalization
Analysis
Next Best Offer
Analysis
In store traffic
patterns

24. TRAVEL APPLICATIONS
Aircraft
scheduling
Seat/gate
management
Air crew
scheduling
Dynamic
pricing
Customer
complaint
resolution
Maintenance
optimization
Tourism
forecasting

25. Life Sciences
Identifying
biomarkers
Drug/chemical
discovery
Analyzing study
results
Identifying negative
responses
Diagnostic test
development
Diagnostic targeting
Predicting drug
demand
Prescription
adherence
Putative safety
signals
Social media
marketing
Image analysis
Clinical trial design
COGS optimization
Insurance
Claims prediction
Claims handling
Price sensitivity
Investments
Agent & branch
performance
DM, product mix
Hospitality
Dynamic pricing
Promos/upgrades/
offers
Table
management &
reservations
Workforce
management
Manufacturing
Failure analysis
Quality
management
Inventory
management
Warranty/pricing
Direct
Marketing
Response rates
Segmentations for
mailings
Reactivation
likelihood
RFM
Discount targeting
Phone marketing
Email Marketing
Construction
Contractor
performance
Design issue
prediction
Agriculture
Yield management
Automation
Mall Operators
Tenant capacity to
pay
Tenant selection
Education
Automated essay
scoring
Dynamic courses
Utilities
Optimize
Distribution
Network
Predict
Commodity
Requirements

26. How to do Deep Learning
- Pick up a framework
- Pick up an algorithm to use
- Understand its hyper-parameters
- Get data labelled
- Make sure you understand data well (Visualization Tools etc)
- Is your data enough ? - Very domain specific question.
- Define what accuracy you need for your algorithm ?
- Choose the platform / architecture etc ?

27. Tools

28. Building a Deep Learning Application
- Make sure that you actually need Deep Learning - don’t just jump into it.
- You need to have DATA (Labelled) - curated.
- GIGO is the rule on using data.
- Choose the right algorithm/network/framework for your Use case.
- Find out if you can use Transfer Learning - if yes go ahead try it out.
- If you don’t have data - Prepare a data capture strategy (buy it. Collect it
etc) - and use API based service of known providers if you can use it.
- Do not reinvent the wheel of trying to build a network from scratch.
- Play with hyper-parameters of known models/framework etc.
- Some cases an ensemble is good enough for the problem.
- Look at Kaggle, Arxiv and other data science competition platforms

29. TensorFlow, a DL framework
1) A deep learning library open sourced by google
2) Created for task with heavy numeric computation
3) Based on dataflow graph

30. Tensorflow code

31. Tensorflow code: Getting the output
sess.run(fetches, feeds)
Fetches: List of graph
nodes.
To fetch the the output of
the operation execute the
graph in run().
Feeds:
Feed data to the graph

32. So What we covered so far?
We first built a graph using variables and placeholders
We then deployed the graph onto a session, which is the execution
environment
Next we will see how to train the model

33. Handwritten Digit classification: MNIST

34. Matrix Notation

35. Error Metric

36. Training

37. DEMO
RELU and
DROPOUT

38. Dropout
A regularization technique works great in Deep Neural network
Randomly Drops certain % of nodes in training to reduce overfitting
An efficient way of doing model averaging as we randomly drop nodes for
each training example

39. Convolutional Neural Network (CNN)
Problem With Fully connected network on images:
Losing shape information when we are flattening
the image into a single array
Visual cortex system of brain doesn't work like
fully connected network.
Huble and Wiesel experiment on cat reveals
1) Local connection
2) Hierarchical layer
3) Spatial Invariance : Any size, rotation, shift

40. CNN to rescue

41. Convolution layer

42. Conv Net on MNIST

43. Pooling
The idea of pooling in convolutional neural networks is to do two things:
● Reduce the number of parameters in your network (pooling is also
called “down-sampling” for this reason)
● To make feature detection more robust by making it more impervious
to scale and orientation changes
Max Pooling
Average Pooling

44. Feature extraction

45. DEMO

46. Acknowledgement
Materials have been taken from
1) Martin Gorner Tensorflow Tutorial
2) Stanford CS224d Tensorflow Tutorial
3) Stanford CS231n course material

47. Transfer Learning

48. Transfer Learning - http://ruder.io/transfer-learning/
Transfer Learning

49. Transfer Learning - http://ruder.io/transfer-learning/
Transfer Learning

50. Drivers of ML Success
Transfer Learning

51. Transfer Learning Demo using TF
http://ruder.io/transfer-learning/

52. RNNs

53. LSTMs

54. Combining CNN and RNN - Image Captioning

55. Challenges of Deep Learning
- Explainability - How do you learn what you learn ?
- Debugging - What has gone wrong ?
- Hyper-parameter optimization
- Why is this not converging ?
- Is my learning rate good ?
- Do I need to change the number of layers ?
- Am I resilient to bad or erroneous data ?
- Deep domain knowledge is sometimes required to make sense of what
the data speaks about.

56. Building a AI product in the age of APIs
- Google
- Amazon
- Microsoft
- Other small players.
- Vision, Speech, Video Analytics etc
- Build your own solutions taking inputs from those APIs - enhanced
inferencing

57. Q and A
Thank you!!
Subrat Panda - https://www.linkedin.com/in/subratpanda/
Biswa G Singh - https://www.linkedin.com/in/biswagsingh/
Consider Joining IDLI - https://www.facebook.com/groups/idliai/ - Indian
Deep Learning Initiative
Navin Manaswi - https://www.linkedin.com/in/navin-manaswi-1a708b8/
"Deep Learning by python using TensorFlow" By Navin - Book to be
launched in Jan end

58. Acknowledgements
- Team at Capillary Technologies
- SMC organizing team
Navin Manaswi
Deep Learning-AI Architect and Partner
at Symphony AI Incubator
IIT Kanpur 2004

59. References
- www.tensorflow.org
- https://www.slideshare.net/LuMa921/deep-learning-a-visual-introduction
- http://ruder.io/transfer-learning/
- Many more PPTs available on the internet from Universities and Good
Samaritans