Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Deep learning - Conceptual understanding and applications

56.325 visualizaciones

Publicado el

딥러닝 Deep Learning에 관해서 팀에 공유했던 발표자료입니다.

Publicado en: Datos y análisis
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • This is real take it serious, who will believe that a herb can cure herpes, i navel believe that this will work i have spend a lot when getting drugs from the hospital to keep me healthy, what i was waiting for is death because i was broke, one day i hard about this great man who is well know of HIV and cancer cure, i decided to email him, unknowingly to me that this will be the end of the herpes in my body, he prepare the herb for me, and give me instruction on how to take it, at the end of the one month, he told me to go to the hospital for a check up, and i went, surprisingly after the test the doctor confirm me negative, i thought it was a joke, i went to other hospital was also negative, then i took my friend who was also herpes positive to the Dr Agumagu, after the treatment she was also confirm negative . He also have the herb to cure cancer. please i want every one with this virus to be free, that is why am dropping his email address, agumaguspelltemple@outlook.com or agumaguspelltemple@gmail.com do email him he is a great man. the government is also interested in this DR, thank you for saving my life, and I promise I will always testify for your good work call his number +233200116937.
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Deep learning - Conceptual understanding and applications

  1. 1. Deep Learning Conceptual Understanding & Application Variants ben.hur@daumkakao.com The aim of this slide is at conceptual understanding of deep learning, rather than implementing it for practical applications. That is, this is more for paper readers, less for system developers. Note that, some variables are not uniquely defined nor consistently used for convenience. Also, variable/constant and vector/matrix are not clearly distinct. More importantly, some descriptions may be incorrect due to lack of my ability.
  2. 2. Deep Learning (DL) is, in concise, a deep and wide neural network. That is half-true.
  3. 3. Deep Learning is difficult because of lack of understanding about — artificial neural network (ANN) — unfamiliar applications.
  4. 4. DL — ANN — Signal transmission between neurons — Graphical model (Belief network) — Linear regression / logistic regression — Weight, bias & activation — (Feed-forward) Multi-layer perceptron (MLP) — Optimization — Back-propagation — (Stochastic) Gradient-decent
  5. 5. DL — Applications — Speech recognition (Audio) — Natural language processing (Text) — Information retrieval (Text) — Image recognition (Vision) — & yours (+)
  6. 6. Artificial Neural Network (ANN)
  7. 7. ANN is a nested ensemble of [logistic] regressions.
  8. 8. Perceptron & Message Passing
  9. 9. http://en.wikipedia.org/wiki/Neuron
  10. 10. 5 5 y = x Message passing
  11. 11. y = Σ x y = Σ wx y = Σ wx + b Linear regression
  12. 12. y = sigm(Σwx + b) Logistic regression Σwx?Σwx
  13. 13. Activation function — Linear g(a) = a — Sigmoid g(a) = sigm(a) = 1 / (1 + exp(-a)) — Tanh g(a) = tanh(a) = (exp(a) - exp(-a)) / (exp(a) + exp(-a)) — Rectified linear g(a) = reclin(a) = max(0, a) — Step — Gaussian — Softmax, usually for the output layer
  14. 14. Bias controls activation. y = ax b y = ax + b
  15. 15. Multi-Layer Perceptron (MLP)
  16. 16. Linearly non-separable by a single perceptron
  17. 17. Input layer Hidden layer Output layer http://info.usherbrooke.ca/hlarochelle/ neural_networks/content.html Interconnecting weight Bias Activation function
  18. 18. http://info.usherbrooke.ca/hlarochelle/ neural_networks/content.html
  19. 19. Back-Propagation & Optimization
  20. 20. {x, y} h=a(wx+b) y'=a2(w2h+b2) l = y - y'w2 & b2w & b w2 R & b2 RwR & bR y' Back-propagation {x, y} —> feed-forward <— back-propagation
  21. 21. Reinforced through epoch ANN is a reinforcement learning.
  22. 22. Local minimum Plateaus http://info.usherbrooke.ca/hlarochelle/ neural_networks/content.html
  23. 23. Stochastic Gradient Descent The slide excludes details. See reference tutorial for that.
  24. 24. ANN is a good approximator of any functions, if well-designed and trained.
  25. 25. Deep Learning
  26. 26. DL (DNN) is a deep and wide neural network. many (2+) hidden layers many input/hidden nodes
  27. 27. DEEP WIDE Large matrix Many matrix Image from googling
  28. 28. Brain image from Simon Thorpe Other from googling
  29. 29. Many large connection matrices (W) — Computational burden (GPU) — Insufficient labeled data (Big Data) — Under-fitting (Optimization) — Over-fitting (Regularization)
  30. 30. Unsupervised makes DL success.
  31. 31. Unsupervised pre-training — Initialize W matrices in an unsupervised manner — Restricted Boltzmann Machine (RBM) — Auto-encoder — Pre-train in a layer-wise (stacking) fashion — Fine-tune in a supervised manner
  32. 32. Unsupervised representation learning — Construct a feature space — DBM / DBN — Auto-encoder — Sparse coding — Feed new features to a shallow ANN as input cf, PCA + regression
  33. 33. visible hidden Restricted Boltzmann Machine (RBM)
  34. 34. Boltzmann Machine Example
  35. 35. visible hidden
  36. 36. Auto-encoder x x' W WT encoder decoder
  37. 37. if k <= n (single & linear), then AE becomes PCA. otherwise, AE can be arbitrary (over-complete). Hidden & Output layers plays the role of semantic feature space in general - compresses & latent feature (under-complete) - expanded & primitive feature (over-complete)
  38. 38. Denoising auto-encoder x x' nois(x)
  39. 39. Contractive auto-encoder Regularized auto-encoder Same as Ridge (or similar to Lasso) Regression
  40. 40. Hinton and Salakhutdinov, Science, 2006 Deep auto-encoder
  41. 41. (k > n, over-complete) Sparse coding
  42. 42. X Y
  43. 43. RBM3 RBM2 RBM1 Stacking Training in the order of RBM1, RBM2, RBM3, …
  44. 44. Representation Learning Artificial Neural Network+ - Sparse coding - Auto-encoder - DBN / DBM
  45. 45. Stochastic Dropout Training for preventing over-fitting
  46. 46. http://info.usherbrooke.ca/hlarochelle/ neural_networks/content.html
  47. 47. Deep Architecture
  48. 48. Deep architecture is an ensemble of shallow networks.
  49. 49. Key Deep Architectures — Deep Neural Network (DNN) — Deep Belief Network (DBN) — Deep Boltzmann Machine (DBM) — Recurrent Neural Network (RNN) — Convolution Neural Network (CNN) — Multi-modal/multi-tasking — Deep Stacking Network (DSN)
  50. 50. Applications
  51. 51. Applications — Speech recognition — Natural language processing — Image recognition — Information retrieval Application Characteristic - Non-numeric data - Large dimension (curse of dimensionality)
  52. 52. Natural language processing — Word embedding — Language modeling — Machine translation — Part-of-speech (POS) tagging — Named entity recognition — Sentiment analysis — Paraphrase detection — …
  53. 53. Deep-structured semantic modeling (DSSM) BM25(Q, D) vs Corr(sim(Q, D), CTR)
  54. 54. DBN & DBM
  55. 55. Recurrent Neural Network (RNN)
  56. 56. Recurrent Neural Network (RNN)
  57. 57. http://nlp.stanford.edu/courses/NAACL2013/ NAACL2013-Socher-Manning-DeepLearning.pdf Recursive Neural Network
  58. 58. Convolution Neural Network (CNN)
  59. 59. C-DSSM
  60. 60. CBOW & Skip-gram (Word2Vec)
  61. 61. Deep Stacking Network (DSN)
  62. 62. Tensor Deep Stacking Network (TDSN)
  63. 63. Multi-modal / Mulit-tasking
  64. 64. Google Picture to Text
  65. 65. Content-based Spotify Recommendation
  66. 66. In various applications, Deep Learning provides either — better performance than state-of-the-art — similar performance with less labor
  67. 67. Remarks
  68. 68. Deep Personalization with Rich Profile in a Vector Representation - Versatile / flexible input - Dimension reduction - Same feature space ==> Similarity computation
  69. 69. RNN for CF http://erikbern.com/?p=589
  70. 70. Hard work — Application-specific configuration — Network structure — Objective and loss function — Parameter optimization
  71. 71. Yann LeCun —> Facebook Geoffrey Hinton —> Google Andrew Ng —> Baidu Joshua Benjio —> U of Montreal
  72. 72. References Tutorial — http://info.usherbrooke.ca/hlarochelle/neural_networks/content.html — http://deeplearning.stanford.edu/tutorial Paper — Deep learning: Method and Applications (Deng & Yu, 2013) — Representation learning: A review and new perspective (Benjio et al., 2014) — Learning deep architecture for AI (Benjio, 2009) Slide & Video — http://www.cs.toronto.edu/~fleet/courses/cifarSchool09/slidesBengio.pdf — http://nlp.stanford.edu/courses/NAACL2013 Book — Deep learning (Benjio et al. 2015) http://www.iro.umontreal.ca/~bengioy/dlbook/
  73. 73. Open Sources (from wikipedia) — Torch (Lua) https://github.com/torch/torch7 — Theano (Python) https://github.com/Theano/Theano/ — Deeplearning4j (word2vec for Java) https://github.com/SkymindIO/deeplearning4j — ND4J (Java) http://nd4j.org https://github.com/SkymindIO/nd4j — DeepLearn Toolbox (matlab) https://github.com/rasmusbergpalm/DeepLearnToolbox/ graphs/contributors — convnetjs (javascript) https://github.com/karpathy/convnetjs — Gensim (word2vec for Python) https://github.com/piskvorky/gensim — Caffe (image) http://caffe.berkeleyvision.org — More+++
  74. 74. FBI WARNING Some reference citations are missed. Many of them come from the first tutorial and the first paper, and others from googling. All rights of those images captured, albeit not explicitly cited, are reserved to original authors. Free to share, but cautious of that.
  75. 75. Left to Blank

×