Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Neural Networks with Google TensorFlow

22.791 visualizaciones

Publicado el

This slides explains how Convolution Neural Networks can be coded using Google TensorFlow.
Video available at :

Publicado en: Software
  • Inicia sesión para ver los comentarios

Neural Networks with Google TensorFlow

  1. 1. Neural Networks with Google TensorFlow Darshan Patel Northeastern University
  2. 2. • Overview: 1) Computer Vision Tasks 2) Convolution Neural Network (CNNs) Architecture 3) CNNs using Google TensorFlow 4) Google TensorBoard Neural Networks with Google TensorFlow
  3. 3. Computer Vision Tasks
  4. 4. Source :
  5. 5. Source :
  6. 6. Source :
  7. 7. Convolution Neural Networks (CNNs/ConvNets)
  8. 8. • Mathematical Definition: A function derived from two given functions by integration that expresses how the shape of one is modified by the other What is Convolution?
  9. 9. Neural Networks
  10. 10. Neural Networks - Forward Pass
  11. 11. Neural Networks - Back Propagation Source :
  12. 12. • ConvNet architectures make the explicit assumption that the inputs are images, which allows us to encode certain properties into the architecture. • This assumption makes the forward function more efficient to implement and vastly reduces the amount of parameters in the network. How CNN/ConvNets is different?
  13. 13. Cont. How CNN/ConvNets is different? Source :
  14. 14. LeNet-5 1990 Yann LeCun Director of AI Research at Facebook Handwritten Digits Classification
  15. 15. LeNet-5 Architecture
  16. 16. AlexNet Architecture - ImageNet 2012
  17. 17. LeNet-5 Architecture
  18. 18. Layers in ConvNets 1. Convolution Layer 2. ReLU (Activation) Layer 3. Pooling Layer 4. Fully Connected Layer
  19. 19. Source :
  20. 20. Source :
  21. 21. Source :
  22. 22. Source :
  23. 23. Source :
  24. 24. Source :
  25. 25. Source : Convolution Layer
  26. 26. Source : Convolution Layer
  27. 27. Source : Convolution Layer
  28. 28. Source : Activation Layer
  29. 29. Source :
  30. 30. Source : Pooling Layer
  31. 31. Source : Pooling Layer
  32. 32. Fully Connected Layer
  33. 33. • A ConvNet architecture is a list of Layers that transform the image volume into an output volume (e.g. holding the class scores) • There are a few distinct types of Layers (e.g. CONV/FC/RELU/POOL are by far the most popular) • Each Layer accepts an input 3D volume and transforms it to an output 3D volume through a differentiable function • Each Layer may or may not have parameters (e.g. CONV/FC do, RELU/POOL don't) ConvNets Summary
  34. 34. • Second generation Machine Learning system, followed by DistBelief • TensorFlow grew out of a project at Google, called Google Brain, aimed at applying various kinds of neural network machine learning to products and services across the company. • An open source software library for numerical computation using data flow graphs • Used in following projects at Google 1. DeepDream 2. RankBrain 3. Smart Reply And many more.. Google TensorFlow
  35. 35. Data Flow Graph • Data flow graphs describe mathematical computation with a directed graph of nodes & edges • Nodes in the graph represent mathematical operations. • Edges represent the multidimensional data arrays (tensors) communicated between them. • Edges describe the input/output relationships between nodes. • The flow of tensors through the graph is where TensorFlow gets its name.
  36. 36. Google TensorFlow Basic Elements • Tensor • Variable • Operation • Session • Placeholder • TensorBoard
  37. 37. • TensorFlow programs use a tensor data structure to represent all data • Think of a TensorFlow tensor as an n-dimensional array or list In the following example, c, d and e are symbolic Tensor Objects, where as result is a numpy array Tensor
  38. 38. 1. Constant Value Tensors 2. Sequences 3. Random Tensors Tensor Types
  39. 39. Constant Value Tensors
  40. 40. Sequence Tensors
  41. 41. Random Tensors
  42. 42. • In-memory buffers containing tensors • Initial value defines the type and shape of the variable. • They must be explicitly initialized and can be saved to disk during and after training. Variable
  43. 43. • An Operation is a node in a TensorFlow Graph • Takes zero or more Tensor objects as input, and produces zero or more Tensor objects as output. • Example: c = tf.matmul(a, b) Creates an Operation of type "MatMul" that takes tensors a and b as input, and produces c as output. Operation
  44. 44. • A class for running TensorFlow operations • InteractiveSession is a TensorFlow Session for use in interactive contexts, such as a shell and Ipython notebook. Session & Interactive Session
  45. 45. • A value that we'll input when we ask TensorFlow to run a computation. Placeholder
  46. 46. TensorBoard : Visual Learning
  47. 47. MNIST Dataset
  48. 48. MNIST Dataset
  49. 49. LeNet-5 Architecture
  50. 50. Load MNIST Data
  51. 51. Load MNIST Data Start a session
  52. 52. Placeholders Dynamic Size
  53. 53. Placeholders Weight/Filter & Bias
  54. 54. Convolution and Pooling Stride of one Max Pooling over 2x2 blocks Stride of two
  55. 55. First Convolution Layer including ReLU It will consist of convolution, followed by max pooling Filter/Patch Dimension Number of Input Channels Number of Output Channel Number of Output Channel
  56. 56. Second Convolution Layer including ReLU It will consist of convolution, followed by max pooling
  57. 57. Fully Connected Layer • Reshape the tensor from the pooling layer into a batch of vectors • Multiply by a weight matrix, add a bias, and apply a ReLU
  58. 58. Dropout • To reduce over fitting, we will apply dropout before the readout layer. • Dropout is an extremely effective, simple and recently introduced regularization technique by Srivastava et al. in “Dropout: A Simple Way to Prevent Neural Networks from Overfitting” that complements the other methods (L1, L2, maxnorm). Source :
  59. 59. Dropout • We create a placeholder for the probability that a neuron's output is kept during dropout. • This allows us to turn dropout on during training, and turn it off during testing. • While training, dropout is implemented by only keeping a neuron active with some probability pp (a hyperparameter)
  60. 60. Readout Layer • Finally, we add a softmax layer, just like for the one layer softmax regression.
  61. 61. Train and Evaluate the Model Initialize All Variables Training Accuracy Testing Optimizer Loss Function
  62. 62. • TensorBoard operates by reading TensorFlow events files, which contain summary data that you can generate when running TensorFlow. TensorBoard
  63. 63. • TensorBoard operates by reading TensorFlow events files, which contain summary data that you can generate when running TensorFlow. • First, create the TensorFlow graph that we'd like to collect summary data from, and decide which nodes should be annotated with summary operation. • For example, • For MNIST digits CNNs, we'd like to record how the learning rate varies over time, and how the objective function is changing • We’d like to record distribution of gradients or weights TensorBoard
  64. 64. TensorBoard Graph Representation Graph Representation Histogram Summary
  65. 65. TensorBoard
  66. 66. TensorBoard