SlideShare una empresa de Scribd logo
1 de 43
Handong Global University
Deep Learning:
인공지능/기계학습의 새로운 트랜드
In-Jung Kim
Handong Global University
2014. 11. 7.
Handong Global University
Agenda
 Introduction to Deep Learning
 Deep Learning Algorithms
 Successful Application of Deep Learning
 Q&A
Handong Global University
Machine Learning
 Learn from data
 Data-driven approach ( Knowledge-based approach)
Trainable
Framework
Parameters Train
Training
Samples
(deep) neural networks,
Bayesian classifier,
HMM, MRF, CRF, SVM,
Etc.
Input
result
Handong Global University
Knowledge-base vs. Data-driven
 Knowledge-based approaches
 Intuitive
 Dependent on designer’s knowledge
 Difficult to justify or improve
 Data-driven approaches
 Learn from data
 Requires training data
 Given training data, easy to (re)build
 Difficult to understand trained model
 Given sufficient training samples,
 Data-driven approach > knowledge-based approach
Handong Global University
Neural Networks
 An artificial neural network is a mathematical
model inspired by biological neural networks.
 Intelligence comes from their connection weights
 Connection weights are decided by learning or adaptation
x1
x2
xn
…
o1
om
…
o
Handong Global University
Neural Networks
 Neural networks is a mathematical model to learn
mappings
 Mapping from a vector to another vector (or a scalar value)
Examples)
 Pattern  class (classification)
 Independent variables  dependent variables (regression)
 Information  decision
 History  future
x1
x2
xn
… o1
om
…
input
vector
output
vector
Handong Global University
Neural Networks
 Neural networks can learn probability distribution from
training samples
Examples)
 Approximate joint prob. P(X,Y), or conditional prob. P(X|Y)
 Likelihood P(x|), a posteriori probability P(|x)
 Classification, sampling, restoration
x1
x2
xn
…
o1
om
…
training
Samples from f(X)
{ X1, X2, … }
Handong Global University
Deep Neural Networks
 A deep neural network (DNN) is a neural network with
multiple levels of nonlinear operations.
Layer 1
Input
Layer 2
Output
…
Handong Global University
Network Depth and Decision Region
[Lipman87]
Half Plane
Bounded
by
Hyperplane
Convex
(open or
closed)
Regions
Arbitrary
(Complexity
Limited By
# Nodes)
Handong Global University
Why Deep Networks?
 Efficient in modeling of complex functions
 Representation of some functions needs sufficiently many
layers.
 Stepwise abstraction to learn high-level feature
 Large capacity
 DNN can learn very well from a huge volume of samples
 Integrated learning
 DNN integrates feature extractor and classifier in a single
network
Handong Global University
Stepwise Abstraction
 Abstraction from low level representation to high
level representation.
 Similar to human perception process
Layer 1
Input
Layer 2
Output
…
[Lee12]
Handong Global University
Integrated Learning
 Deep networks optimize both feature extractor and
classifier in a unified framework.
 Conventional system
 Deep neural network
Classifier
Feature
Extractor
DNN
input
input
output
output
Handong Global University
Challenges with Deep Networks
 Hard to optimize
 Back-propagation algorithm does not work well for deep
fully connected networks starting from random weights
 New training algorithms
 A large number of parameters
 A huge volume of training samples is now available.
 Techniques to improve generalization ability
Ex) sparse coding, virtual sample generation, dropout
 Requires heavy computation
 GPU-based massive parallel processing
 H/W implementation (SoC, FPGA)
Handong Global University
The Back-Propagation Algorithm
 Gradient descent algorithm to minimize error E.
Layer 1 (X1)
Input layer (X0)
W1
Layer 2 (X2)
W2
Output layer (XN)
WN
feature
errorsignal
…
netj
𝜕𝐸
𝜕𝑤 𝑖𝑗
=
𝜕𝐸
𝜕𝑛𝑒𝑡 𝑗
𝜕𝑛𝑒𝑡 𝑗
𝜕𝑤 𝑖𝑗
=𝛿𝑗 𝑥𝑖
𝛿𝑗 =
𝜕𝐸
𝜕𝑛𝑒𝑡𝑗
𝛿𝑖 = 𝑓′
(𝑛𝑒𝑡𝑖)
𝑗
𝑤𝑖𝑗 𝛿𝑗
Handong Global University
BP on Deep Network
 BP does not work on deep networks
 Error signals from many nodes are blended together.
 become dim and vague on bottom layers
“Diminishing gradient problem”
 Error signal
at a non-output node i
𝛿𝑖 = 𝑓′(𝑛𝑒𝑡𝑖)
𝑗
𝑤𝑖𝑗 𝛿𝑗
j
wij
i
Handong Global University
Agenda
 Introduction to Deep Learning
 Deep Learning Algorithms
 Successful Application of Deep Learning
 Q&A
Handong Global University
Breakthroughs in Deep Learning
 Conventional back-propagation algorithm does not
work well for deep fully-connected networks starting
from random weights.
 Layer-wise unsupervised pre-training algorithm
Ex) DBN[Hinton2006], stacked auto-encoders[Bengio2006]
 First, place the weights near a local optimal position by
unsupervised learning algorithm
 Then, conventional supervised learning algorithms work fine
 Network structure to prevent diminishing gradient
problem
Ex) Convolutional Neural Networks [Fukushima1980][LeCun1998]
Handong Global University
Layer-wise Unsupervised Pre-training
 Based on generative neural networks
 Training procedure
1. Pre-train each layer to reproduce the input by unsupervised
learning algorithm
2. Fine-tune the whole network by a supervised learning
algorithm
Ex) wake-sleep[Hinton2003], back-propagation
Handong Global University
Generative Neural Networks
 Neural networks with forward–backward connections
Layer 1 (X1)
Input layer (X0)
W1
Layer 2 (X2)
W2
Output layer (XN)
WN
…
Layer 1 (X1)
Input layer (X0)
Layer 2 (X2)
Output layer (XN)
W1
W2
WN
…
WN
W1
W2
Forward-backward networkFeed forward network
Forward connection
for “encoding”
Backward connection
for “decoding”
Handong Global University
Layer-wise Unsupervised Pre-training
 Starting from bottom layer, train each layer to
reproduce the input
Input  encoding  hidden  decoding  reprod. of input
Layer 1 (X1)
Input layer (X0)
Layer 2 (X2)
Output layer (XN)
W1
W2
WN
…
WN
W1
W2
Forward-backward network
Forward propagation
for encoding
Backward propagation
for decoding
1st phase
Handong Global University
Layer-wise Unsupervised Pre-training
 Starting from bottom layer, train each layer to
reproduce the input
Input  encoding  hidden  decoding  reprod. of input
Layer 1 (X1)
Input layer (X0)
Layer 2 (X2)
Output layer (XN)
W1
W2
WN
…
WN
W1
W2
Forward-backward network
Forward propagation
for encoding
Backward propagation
for decoding
2nd phase
Handong Global University
Convolutional Neural Networks
 Neocognitron [Fukushima80]
 Designed to imitate visual processing of human/animals
 Suggested basic concept and network structure of CNN
 LeNet [LeCun98]
 Simplified node and network structure
 Gradient-based learning
 Many improvements and extensions
 [Simard2003], [Ciresan2011]
 Convolutional DBN [Lee2009]
 Siamese network [Chopra2005]
 Locally connected network [Taigman2014]
 Fast training algorithm using FFT [Mathieu2014]
Handong Global University
Convolutional Neural Networks
 Composed of many heterogeneous layers
 Convolution layer – feature extraction
 Max-pooling layer - feature abstraction
 Fully-connected layers - classification
Handong Global University
Convolution Layers
 Odd-numbered layers in
low/middle-level of CNN
 Nodes on each layer are grouped
into 2D planes (or feature maps)
 Each plane is connected to one or
more input planes
 Each node computes weighted sum
of input nodes in a small region
 All nodes on a plane share weight
set
 Extract feature by convolution
operation
Handong Global University
Max-Pooling Layers
 Even-numbered layers in
low/middle-level of CNN
 Nodes on each layer are grouped
into planes
 Each plane is connected to only
one input plane
 Each node chooses maximum
among input nodes in a small
region
 Abstract features
 Reduces feature dimension
 Ignores positional variation of feature
elements
Handong Global University
Fully-connected Layers
 Top 2~3 layers of CNN
 1D structure
 Each node is fully connected to
all input nodes
 Each node computes weighted
sum of all input nodes
 Classify input pattern with high-
level features extracted by previous
layers
Handong Global University
Gradient-based Learning [LeCun98]
 Trains the whole network to minimize a single error
function E.
 At layer n
 At layer n-1
Layer 1 (X1)
Input layer (X0)
W1
Layer 2 (X2)
W2
Output layer (XN)
WN
…
Handong Global University
Why CNN Works Well?
 Network structure effectively guides learning from 2D
images preventing the diminishing gradient problem
 Sparse connection
 Parameter tying
 Good at catching 2D structures
 Training of convolution masks is effective to learn feature
extraction
 Good at handing shape variation
 Abstraction in phases
 Max pooling
 Directly train the network to minimize classification error.
Handong Global University
Agenda
 Introduction to Deep Learning
 Deep Learning Algorithms
 Successful Application of Deep Learning
 Q&A
Handong Global University
Numeral Digit Recognition (MNIST DB)
 Support Vector Machines
 Neural Nets
 Convolutional Neural Networks (Deep Networks)
Handong Global University
Chinese Character Recognition (CASIA DB)
 ICDAR 2013 Competition Result [Yin, et.al 2013]
CNN
Handong Global University
Object Image Recognition
 ImageNet Large Scale Visual Recognition Challenge
2013 (ILSVRC2013, http://www.image-net.org)
 1300 object categories
 Training set: 1,281,167 images
 Validation set: 50,000 images
 Test set: 100,000 images
Handong Global University
Examples of ILSVRC2013 Images
Handong Global University
ILSVRC2013 Results
All high rankers are based on CNNs
Handong Global University
Deep Learning in Face Recognition
 Face recognition flow
1. Detection
2. Alignment (pre-processing)
3. Representation (feature extraction)  CNN
 Robust to variation in lighting, expression, …
 Alternatives: LBP + PCA/FDA
4. Verification / Classification  Siamese network
 Alternatives: Euclidian distance, dot product, 2 distance, SVM,
…
Handong Global University
DeepFace [Taigman2014]
 Facebook AI group, Tel Aviv Univ.
 Y. Taigman, et.al.
 Achieved 97.25% on LFW dataset.
cf. Conventional best performance: 96.33% [Cao 2013]
 Face recognition prodedure
 2D and 3D alignment
 CNN-based representation
 Verification by weighted 2 distance and Siamese network
 A huge volume of training data
 SFC dataset (4,000 identity * 1,000 samples)
Handong Global University
DeepFace [Taigman2014]
 Feature extraction by CNN
 Train a CNN-based face recognizer
 Represent the input face image by the output of (N-1)th
layer
Handong Global University
Deep Learning in Speech Recognition
 Hybrid system (HMM + DNN)
Handong Global University
Deep Learning in Speech Recognition
 Deep Neural Networks for Acoustic Modeling in
Speech Recognition [Hinton2012]
Deep learning
Handong Global University
Deep Learning in Speech Recognition
 CNN for speech recognition [Ossama13]
 Apply CNN on 2D vector (frame, frequency bands)
Handong Global University
Hangul Recognition
 Challenges in Hangul Recognition
 A multitude of similar characters
 Missing one small stroke often result in misclassification
Ex) 에-애–얘, 괟-괱-괠-팰, 흥-홍-훙-흉
 Excessive cursiveness
Handong Global University
Deep Learning in Hangul Recognition
Methods SERI95a PE92
Kim&Kim01
Structural
matching
86.3% 82.2%
Kang&Kim04
Structural
matching
90.3% 87.7%
Jang&Kim02
Structural
matching +
Post-processing
93.4% N/A
Kim&Liu11 MQDF 93.71% 85.99%
Kim CNN 95.96% 92.92%
Error reduction rates 35.71% 42.44%
[Kim2014]
Handong Global University
Q&A

Más contenido relacionado

La actualidad más candente

Deep learning in Computer Vision
Deep learning in Computer VisionDeep learning in Computer Vision
Deep learning in Computer VisionDavid Dao
 
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.GeeksLab Odessa
 
Convolutional Neural Network
Convolutional Neural NetworkConvolutional Neural Network
Convolutional Neural NetworkJunho Cho
 
Scene classification using Convolutional Neural Networks - Jayani Withanawasam
Scene classification using Convolutional Neural Networks - Jayani WithanawasamScene classification using Convolutional Neural Networks - Jayani Withanawasam
Scene classification using Convolutional Neural Networks - Jayani WithanawasamWithTheBest
 
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...GeeksLab Odessa
 
TypeScript and Deep Learning
TypeScript and Deep LearningTypeScript and Deep Learning
TypeScript and Deep LearningOswald Campesato
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networksSi Haem
 
Deep Learning Tutorial
Deep Learning Tutorial Deep Learning Tutorial
Deep Learning Tutorial Ligeng Zhu
 
From neural networks to deep learning
From neural networks to deep learningFrom neural networks to deep learning
From neural networks to deep learningViet-Trung TRAN
 
Deep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT ConferenceDeep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT ConferenceBill Liu
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network Yan Xu
 
Transfer Learning: An overview
Transfer Learning: An overviewTransfer Learning: An overview
Transfer Learning: An overviewjins0618
 
Deep Learning
Deep LearningDeep Learning
Deep LearningJun Wang
 
A Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its ApplicationA Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its ApplicationXiaohu ZHU
 
Convolutional neural networks deepa
Convolutional neural networks deepaConvolutional neural networks deepa
Convolutional neural networks deepadeepa4466
 
101: Convolutional Neural Networks
101: Convolutional Neural Networks 101: Convolutional Neural Networks
101: Convolutional Neural Networks Mad Scientists
 
Machine Learning and Deep Learning with R
Machine Learning and Deep Learning with RMachine Learning and Deep Learning with R
Machine Learning and Deep Learning with RPoo Kuan Hoong
 
Transfer Learning and Fine-tuning Deep Neural Networks
 Transfer Learning and Fine-tuning Deep Neural Networks Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural NetworksPyData
 

La actualidad más candente (20)

Deep learning in Computer Vision
Deep learning in Computer VisionDeep learning in Computer Vision
Deep learning in Computer Vision
 
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.
 
Convolutional Neural Network
Convolutional Neural NetworkConvolutional Neural Network
Convolutional Neural Network
 
Scene classification using Convolutional Neural Networks - Jayani Withanawasam
Scene classification using Convolutional Neural Networks - Jayani WithanawasamScene classification using Convolutional Neural Networks - Jayani Withanawasam
Scene classification using Convolutional Neural Networks - Jayani Withanawasam
 
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
 
TypeScript and Deep Learning
TypeScript and Deep LearningTypeScript and Deep Learning
TypeScript and Deep Learning
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
 
Deep Learning Tutorial
Deep Learning Tutorial Deep Learning Tutorial
Deep Learning Tutorial
 
From neural networks to deep learning
From neural networks to deep learningFrom neural networks to deep learning
From neural networks to deep learning
 
AlexNet
AlexNetAlexNet
AlexNet
 
Deep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT ConferenceDeep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT Conference
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
 
Transfer Learning: An overview
Transfer Learning: An overviewTransfer Learning: An overview
Transfer Learning: An overview
 
Deep Learning
Deep LearningDeep Learning
Deep Learning
 
A Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its ApplicationA Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its Application
 
Convolutional neural networks deepa
Convolutional neural networks deepaConvolutional neural networks deepa
Convolutional neural networks deepa
 
101: Convolutional Neural Networks
101: Convolutional Neural Networks 101: Convolutional Neural Networks
101: Convolutional Neural Networks
 
Machine Learning and Deep Learning with R
Machine Learning and Deep Learning with RMachine Learning and Deep Learning with R
Machine Learning and Deep Learning with R
 
Transfer Learning and Fine-tuning Deep Neural Networks
 Transfer Learning and Fine-tuning Deep Neural Networks Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks
 

Destacado

농업 빅데이터를 활용한 병해충 발생 예측 모형
농업 빅데이터를 활용한 병해충 발생 예측 모형농업 빅데이터를 활용한 병해충 발생 예측 모형
농업 빅데이터를 활용한 병해충 발생 예측 모형datasciencekorea
 
Structures of Twitter Crowds and Conversations Six distinct types of crowds t...
Structures of Twitter Crowds and Conversations Six distinct types of crowds t...Structures of Twitter Crowds and Conversations Six distinct types of crowds t...
Structures of Twitter Crowds and Conversations Six distinct types of crowds t...datasciencekorea
 
Studying Social Selection vs Social Influence in Virtual Financial Communities
Studying Social Selection vs Social Influence in Virtual Financial CommunitiesStudying Social Selection vs Social Influence in Virtual Financial Communities
Studying Social Selection vs Social Influence in Virtual Financial Communitiesdatasciencekorea
 
Analyzing Big Data to Discover Honest Signals of Innovation
Analyzing Big Data to Discover Honest Signals of InnovationAnalyzing Big Data to Discover Honest Signals of Innovation
Analyzing Big Data to Discover Honest Signals of Innovationdatasciencekorea
 
온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호
온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호
온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호datasciencekorea
 
빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원
빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원
빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원datasciencekorea
 
데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석
데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석
데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석datasciencekorea
 
A Unified Music Recommender System Using Listening Habits and Semantics of Tags
A Unified Music Recommender System Using Listening Habits and Semantics of TagsA Unified Music Recommender System Using Listening Habits and Semantics of Tags
A Unified Music Recommender System Using Listening Habits and Semantics of Tagsdatasciencekorea
 
Data-driven biomedical science: implications for human disease and public health
Data-driven biomedical science: implications for human disease and public healthData-driven biomedical science: implications for human disease and public health
Data-driven biomedical science: implications for human disease and public healthdatasciencekorea
 
데이터시장의 트렌드와 예측 - 이영환
데이터시장의 트렌드와 예측 - 이영환 데이터시장의 트렌드와 예측 - 이영환
데이터시장의 트렌드와 예측 - 이영환 datasciencekorea
 
R의 이해와 활용_데이터사이언스학회
R의 이해와 활용_데이터사이언스학회R의 이해와 활용_데이터사이언스학회
R의 이해와 활용_데이터사이언스학회datasciencekorea
 
소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점
소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점
소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점datasciencekorea
 
2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선
2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선
2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선datasciencekorea
 
텍스톰을 이용한 SNA 분석 -전채남
텍스톰을 이용한 SNA 분석 -전채남텍스톰을 이용한 SNA 분석 -전채남
텍스톰을 이용한 SNA 분석 -전채남datasciencekorea
 
Data Centric Art, Science, and Humanities
Data Centric Art, Science, and HumanitiesData Centric Art, Science, and Humanities
Data Centric Art, Science, and Humanitiesdatasciencekorea
 
국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603
국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603
국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603datasciencekorea
 
도시의 마음, 그 발현 - Emergent Mind of City
도시의 마음, 그 발현 - Emergent Mind of City도시의 마음, 그 발현 - Emergent Mind of City
도시의 마음, 그 발현 - Emergent Mind of Citydatasciencekorea
 
International Collaboration Networks in the Emerging (Big) Data Science
International Collaboration Networks in the Emerging (Big) Data ScienceInternational Collaboration Networks in the Emerging (Big) Data Science
International Collaboration Networks in the Emerging (Big) Data Sciencedatasciencekorea
 
DATA CENTRIC EDUCATION & LEARNING
 DATA CENTRIC EDUCATION & LEARNING DATA CENTRIC EDUCATION & LEARNING
DATA CENTRIC EDUCATION & LEARNINGdatasciencekorea
 
소셜미디어 분석방법론과 사례
소셜미디어 분석방법론과 사례소셜미디어 분석방법론과 사례
소셜미디어 분석방법론과 사례datasciencekorea
 

Destacado (20)

농업 빅데이터를 활용한 병해충 발생 예측 모형
농업 빅데이터를 활용한 병해충 발생 예측 모형농업 빅데이터를 활용한 병해충 발생 예측 모형
농업 빅데이터를 활용한 병해충 발생 예측 모형
 
Structures of Twitter Crowds and Conversations Six distinct types of crowds t...
Structures of Twitter Crowds and Conversations Six distinct types of crowds t...Structures of Twitter Crowds and Conversations Six distinct types of crowds t...
Structures of Twitter Crowds and Conversations Six distinct types of crowds t...
 
Studying Social Selection vs Social Influence in Virtual Financial Communities
Studying Social Selection vs Social Influence in Virtual Financial CommunitiesStudying Social Selection vs Social Influence in Virtual Financial Communities
Studying Social Selection vs Social Influence in Virtual Financial Communities
 
Analyzing Big Data to Discover Honest Signals of Innovation
Analyzing Big Data to Discover Honest Signals of InnovationAnalyzing Big Data to Discover Honest Signals of Innovation
Analyzing Big Data to Discover Honest Signals of Innovation
 
온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호
온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호
온라인 데이터 분석을 통한 선거예측- 김찬우, 조인호
 
빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원
빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원
빅데이터 기술을 활용한 뉴스 큐레이션 서비스 - 온병원
 
데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석
데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석
데이터사이언스학회 5월 세미나 데이터저널리즘과 트위터네트워크 분석
 
A Unified Music Recommender System Using Listening Habits and Semantics of Tags
A Unified Music Recommender System Using Listening Habits and Semantics of TagsA Unified Music Recommender System Using Listening Habits and Semantics of Tags
A Unified Music Recommender System Using Listening Habits and Semantics of Tags
 
Data-driven biomedical science: implications for human disease and public health
Data-driven biomedical science: implications for human disease and public healthData-driven biomedical science: implications for human disease and public health
Data-driven biomedical science: implications for human disease and public health
 
데이터시장의 트렌드와 예측 - 이영환
데이터시장의 트렌드와 예측 - 이영환 데이터시장의 트렌드와 예측 - 이영환
데이터시장의 트렌드와 예측 - 이영환
 
R의 이해와 활용_데이터사이언스학회
R의 이해와 활용_데이터사이언스학회R의 이해와 활용_데이터사이언스학회
R의 이해와 활용_데이터사이언스학회
 
소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점
소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점
소셜 텍스트 빅 테이터를 통해 분석한 화장품 유통구조 시사점
 
2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선
2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선
2015-4 혁신기술로서의 빅데이터 국내 기술수용 초기 특성연구- 김정선
 
텍스톰을 이용한 SNA 분석 -전채남
텍스톰을 이용한 SNA 분석 -전채남텍스톰을 이용한 SNA 분석 -전채남
텍스톰을 이용한 SNA 분석 -전채남
 
Data Centric Art, Science, and Humanities
Data Centric Art, Science, and HumanitiesData Centric Art, Science, and Humanities
Data Centric Art, Science, and Humanities
 
국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603
국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603
국가의 신성장 동력으로서 공간정보의 가치와 활용 2016-0603
 
도시의 마음, 그 발현 - Emergent Mind of City
도시의 마음, 그 발현 - Emergent Mind of City도시의 마음, 그 발현 - Emergent Mind of City
도시의 마음, 그 발현 - Emergent Mind of City
 
International Collaboration Networks in the Emerging (Big) Data Science
International Collaboration Networks in the Emerging (Big) Data ScienceInternational Collaboration Networks in the Emerging (Big) Data Science
International Collaboration Networks in the Emerging (Big) Data Science
 
DATA CENTRIC EDUCATION & LEARNING
 DATA CENTRIC EDUCATION & LEARNING DATA CENTRIC EDUCATION & LEARNING
DATA CENTRIC EDUCATION & LEARNING
 
소셜미디어 분석방법론과 사례
소셜미디어 분석방법론과 사례소셜미디어 분석방법론과 사례
소셜미디어 분석방법론과 사례
 

Similar a Deep Learning - 인공지능 기계학습의 새로운 트랜드 :김인중

deeplearning
deeplearningdeeplearning
deeplearninghuda2018
 
Advance deep learning
Advance deep learningAdvance deep learning
Advance deep learningaliaKhan71
 
DEF CON 24 - Clarence Chio - machine duping 101
DEF CON 24 - Clarence Chio - machine duping 101DEF CON 24 - Clarence Chio - machine duping 101
DEF CON 24 - Clarence Chio - machine duping 101Felipe Prado
 
AI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems   - Sunil ChomalAI & ML in Defence Systems   - Sunil Chomal
AI & ML in Defence Systems - Sunil ChomalSunil Chomal
 
A Survey of Deep Learning Algorithms for Malware Detection
A Survey of Deep Learning Algorithms for Malware DetectionA Survey of Deep Learning Algorithms for Malware Detection
A Survey of Deep Learning Algorithms for Malware DetectionIJCSIS Research Publications
 
Recent advances of AI for medical imaging : Engineering perspectives
Recent advances of AI for medical imaging : Engineering perspectivesRecent advances of AI for medical imaging : Engineering perspectives
Recent advances of AI for medical imaging : Engineering perspectivesNamkug Kim
 
RunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkRunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkPutra Wanda
 
UNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptxUNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptxNoorUlHaq47
 
Resnet.pptx
Resnet.pptxResnet.pptx
Resnet.pptxYanhuaSi
 
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...csandit
 
Multimedia data mining using deep learning
Multimedia data mining using deep learningMultimedia data mining using deep learning
Multimedia data mining using deep learningPeter Wlodarczak
 
Attention correlated appearance and motion feature followed temporal learning...
Attention correlated appearance and motion feature followed temporal learning...Attention correlated appearance and motion feature followed temporal learning...
Attention correlated appearance and motion feature followed temporal learning...IJECEIAES
 
Deep learning for image super resolution
Deep learning for image super resolutionDeep learning for image super resolution
Deep learning for image super resolutionPrudhvi Raj
 
Deep learning for image super resolution
Deep learning for image super resolutionDeep learning for image super resolution
Deep learning for image super resolutionPrudhvi Raj
 
Quantum transfer learning for image classification
Quantum transfer learning for image classificationQuantum transfer learning for image classification
Quantum transfer learning for image classificationTELKOMNIKA JOURNAL
 
Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr TeterwakLearn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr TeterwakPyData
 
intro-to-cnn-April_2020.pptx
intro-to-cnn-April_2020.pptxintro-to-cnn-April_2020.pptx
intro-to-cnn-April_2020.pptxssuser3aa461
 
Resnet.pdf
Resnet.pdfResnet.pdf
Resnet.pdfYanhuaSi
 
Dp2 ppt by_bikramjit_chowdhury_final
Dp2 ppt by_bikramjit_chowdhury_finalDp2 ppt by_bikramjit_chowdhury_final
Dp2 ppt by_bikramjit_chowdhury_finalBikramjit Chowdhury
 

Similar a Deep Learning - 인공지능 기계학습의 새로운 트랜드 :김인중 (20)

deeplearning
deeplearningdeeplearning
deeplearning
 
Advance deep learning
Advance deep learningAdvance deep learning
Advance deep learning
 
AI and Deep Learning
AI and Deep Learning AI and Deep Learning
AI and Deep Learning
 
DEF CON 24 - Clarence Chio - machine duping 101
DEF CON 24 - Clarence Chio - machine duping 101DEF CON 24 - Clarence Chio - machine duping 101
DEF CON 24 - Clarence Chio - machine duping 101
 
AI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems   - Sunil ChomalAI & ML in Defence Systems   - Sunil Chomal
AI & ML in Defence Systems - Sunil Chomal
 
A Survey of Deep Learning Algorithms for Malware Detection
A Survey of Deep Learning Algorithms for Malware DetectionA Survey of Deep Learning Algorithms for Malware Detection
A Survey of Deep Learning Algorithms for Malware Detection
 
Recent advances of AI for medical imaging : Engineering perspectives
Recent advances of AI for medical imaging : Engineering perspectivesRecent advances of AI for medical imaging : Engineering perspectives
Recent advances of AI for medical imaging : Engineering perspectives
 
RunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkRunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural Network
 
UNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptxUNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptx
 
Resnet.pptx
Resnet.pptxResnet.pptx
Resnet.pptx
 
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...
 
Multimedia data mining using deep learning
Multimedia data mining using deep learningMultimedia data mining using deep learning
Multimedia data mining using deep learning
 
Attention correlated appearance and motion feature followed temporal learning...
Attention correlated appearance and motion feature followed temporal learning...Attention correlated appearance and motion feature followed temporal learning...
Attention correlated appearance and motion feature followed temporal learning...
 
Deep learning for image super resolution
Deep learning for image super resolutionDeep learning for image super resolution
Deep learning for image super resolution
 
Deep learning for image super resolution
Deep learning for image super resolutionDeep learning for image super resolution
Deep learning for image super resolution
 
Quantum transfer learning for image classification
Quantum transfer learning for image classificationQuantum transfer learning for image classification
Quantum transfer learning for image classification
 
Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr TeterwakLearn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
 
intro-to-cnn-April_2020.pptx
intro-to-cnn-April_2020.pptxintro-to-cnn-April_2020.pptx
intro-to-cnn-April_2020.pptx
 
Resnet.pdf
Resnet.pdfResnet.pdf
Resnet.pdf
 
Dp2 ppt by_bikramjit_chowdhury_final
Dp2 ppt by_bikramjit_chowdhury_finalDp2 ppt by_bikramjit_chowdhury_final
Dp2 ppt by_bikramjit_chowdhury_final
 

Más de datasciencekorea

스마트 시티의 빅데이터 분석론 - 최준영
스마트 시티의 빅데이터 분석론 - 최준영스마트 시티의 빅데이터 분석론 - 최준영
스마트 시티의 빅데이터 분석론 - 최준영datasciencekorea
 
데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁
데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁
데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁datasciencekorea
 
Bayesian Network 을 활용한 예측 분석
Bayesian Network 을 활용한 예측 분석Bayesian Network 을 활용한 예측 분석
Bayesian Network 을 활용한 예측 분석datasciencekorea
 
데이터 시각화의 글로벌 동향 20140819 - 고영혁
데이터 시각화의 글로벌 동향   20140819 - 고영혁데이터 시각화의 글로벌 동향   20140819 - 고영혁
데이터 시각화의 글로벌 동향 20140819 - 고영혁datasciencekorea
 
온라인 물가지수 분석을 위한 빅데이터 융합분석 방법
온라인 물가지수 분석을 위한 빅데이터 융합분석 방법온라인 물가지수 분석을 위한 빅데이터 융합분석 방법
온라인 물가지수 분석을 위한 빅데이터 융합분석 방법datasciencekorea
 
Use of Big Data Technology in the area of Video Analytics
Use of Big Data Technology in the area of Video AnalyticsUse of Big Data Technology in the area of Video Analytics
Use of Big Data Technology in the area of Video Analyticsdatasciencekorea
 
빅 데이터 비즈니스 모델
빅 데이터 비즈니스 모델빅 데이터 비즈니스 모델
빅 데이터 비즈니스 모델datasciencekorea
 

Más de datasciencekorea (7)

스마트 시티의 빅데이터 분석론 - 최준영
스마트 시티의 빅데이터 분석론 - 최준영스마트 시티의 빅데이터 분석론 - 최준영
스마트 시티의 빅데이터 분석론 - 최준영
 
데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁
데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁
데이터에 포함된 동적 패턴의 탐색과 해석을 위한 협업적 탐험 플랫폼 -최진혁
 
Bayesian Network 을 활용한 예측 분석
Bayesian Network 을 활용한 예측 분석Bayesian Network 을 활용한 예측 분석
Bayesian Network 을 활용한 예측 분석
 
데이터 시각화의 글로벌 동향 20140819 - 고영혁
데이터 시각화의 글로벌 동향   20140819 - 고영혁데이터 시각화의 글로벌 동향   20140819 - 고영혁
데이터 시각화의 글로벌 동향 20140819 - 고영혁
 
온라인 물가지수 분석을 위한 빅데이터 융합분석 방법
온라인 물가지수 분석을 위한 빅데이터 융합분석 방법온라인 물가지수 분석을 위한 빅데이터 융합분석 방법
온라인 물가지수 분석을 위한 빅데이터 융합분석 방법
 
Use of Big Data Technology in the area of Video Analytics
Use of Big Data Technology in the area of Video AnalyticsUse of Big Data Technology in the area of Video Analytics
Use of Big Data Technology in the area of Video Analytics
 
빅 데이터 비즈니스 모델
빅 데이터 비즈니스 모델빅 데이터 비즈니스 모델
빅 데이터 비즈니스 모델
 

Último

Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsNeo4j
 
Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1bengalurutug
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptxFurkanTasci3
 
TCFPro24 Building Real-Time Generative AI Pipelines
TCFPro24 Building Real-Time Generative AI PipelinesTCFPro24 Building Real-Time Generative AI Pipelines
TCFPro24 Building Real-Time Generative AI PipelinesTimothy Spann
 
Understanding the Impact of video length on student performance
Understanding the Impact of video length on student performanceUnderstanding the Impact of video length on student performance
Understanding the Impact of video length on student performancePrithaVashisht1
 
2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-ProfitsTimothy Spann
 
Báo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân MarketingBáo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân MarketingMarketingTrips
 
Using DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data WarehouseUsing DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data WarehouseThinkInnovation
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j
 
Data Collection from Social Media Platforms
Data Collection from Social Media PlatformsData Collection from Social Media Platforms
Data Collection from Social Media PlatformsMahmoud Yasser
 
Unleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMUnleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMMarco Wobben
 
Stochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxStochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxjkmrshll88
 
The market for cross-border mortgages in Europe
The market for cross-border mortgages in EuropeThe market for cross-border mortgages in Europe
The market for cross-border mortgages in Europe321k
 
Data Analytics Fundamentals: data analytics types.potx
Data Analytics Fundamentals: data analytics types.potxData Analytics Fundamentals: data analytics types.potx
Data Analytics Fundamentals: data analytics types.potxEmmanuel Dauda
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptxFurkanTasci3
 
How to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentHow to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentAggregage
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
Brain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptxBrain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptxShammiRai3
 

Último (20)

Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge Graphs
 
Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
 
TCFPro24 Building Real-Time Generative AI Pipelines
TCFPro24 Building Real-Time Generative AI PipelinesTCFPro24 Building Real-Time Generative AI Pipelines
TCFPro24 Building Real-Time Generative AI Pipelines
 
Understanding the Impact of video length on student performance
Understanding the Impact of video length on student performanceUnderstanding the Impact of video length on student performance
Understanding the Impact of video length on student performance
 
2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits
 
Target_Company_Data_breach_2013_110million
Target_Company_Data_breach_2013_110millionTarget_Company_Data_breach_2013_110million
Target_Company_Data_breach_2013_110million
 
Báo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân MarketingBáo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân Marketing
 
Using DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data WarehouseUsing DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data Warehouse
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
 
Data Collection from Social Media Platforms
Data Collection from Social Media PlatformsData Collection from Social Media Platforms
Data Collection from Social Media Platforms
 
Unleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMUnleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IM
 
Stochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxStochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptx
 
The market for cross-border mortgages in Europe
The market for cross-border mortgages in EuropeThe market for cross-border mortgages in Europe
The market for cross-border mortgages in Europe
 
Data Analytics Fundamentals: data analytics types.potx
Data Analytics Fundamentals: data analytics types.potxData Analytics Fundamentals: data analytics types.potx
Data Analytics Fundamentals: data analytics types.potx
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
 
How to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentHow to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product Development
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
Brain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptxBrain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptx
 

Deep Learning - 인공지능 기계학습의 새로운 트랜드 :김인중

  • 1. Handong Global University Deep Learning: 인공지능/기계학습의 새로운 트랜드 In-Jung Kim Handong Global University 2014. 11. 7.
  • 2. Handong Global University Agenda  Introduction to Deep Learning  Deep Learning Algorithms  Successful Application of Deep Learning  Q&A
  • 3. Handong Global University Machine Learning  Learn from data  Data-driven approach ( Knowledge-based approach) Trainable Framework Parameters Train Training Samples (deep) neural networks, Bayesian classifier, HMM, MRF, CRF, SVM, Etc. Input result
  • 4. Handong Global University Knowledge-base vs. Data-driven  Knowledge-based approaches  Intuitive  Dependent on designer’s knowledge  Difficult to justify or improve  Data-driven approaches  Learn from data  Requires training data  Given training data, easy to (re)build  Difficult to understand trained model  Given sufficient training samples,  Data-driven approach > knowledge-based approach
  • 5. Handong Global University Neural Networks  An artificial neural network is a mathematical model inspired by biological neural networks.  Intelligence comes from their connection weights  Connection weights are decided by learning or adaptation x1 x2 xn … o1 om … o
  • 6. Handong Global University Neural Networks  Neural networks is a mathematical model to learn mappings  Mapping from a vector to another vector (or a scalar value) Examples)  Pattern  class (classification)  Independent variables  dependent variables (regression)  Information  decision  History  future x1 x2 xn … o1 om … input vector output vector
  • 7. Handong Global University Neural Networks  Neural networks can learn probability distribution from training samples Examples)  Approximate joint prob. P(X,Y), or conditional prob. P(X|Y)  Likelihood P(x|), a posteriori probability P(|x)  Classification, sampling, restoration x1 x2 xn … o1 om … training Samples from f(X) { X1, X2, … }
  • 8. Handong Global University Deep Neural Networks  A deep neural network (DNN) is a neural network with multiple levels of nonlinear operations. Layer 1 Input Layer 2 Output …
  • 9. Handong Global University Network Depth and Decision Region [Lipman87] Half Plane Bounded by Hyperplane Convex (open or closed) Regions Arbitrary (Complexity Limited By # Nodes)
  • 10. Handong Global University Why Deep Networks?  Efficient in modeling of complex functions  Representation of some functions needs sufficiently many layers.  Stepwise abstraction to learn high-level feature  Large capacity  DNN can learn very well from a huge volume of samples  Integrated learning  DNN integrates feature extractor and classifier in a single network
  • 11. Handong Global University Stepwise Abstraction  Abstraction from low level representation to high level representation.  Similar to human perception process Layer 1 Input Layer 2 Output … [Lee12]
  • 12. Handong Global University Integrated Learning  Deep networks optimize both feature extractor and classifier in a unified framework.  Conventional system  Deep neural network Classifier Feature Extractor DNN input input output output
  • 13. Handong Global University Challenges with Deep Networks  Hard to optimize  Back-propagation algorithm does not work well for deep fully connected networks starting from random weights  New training algorithms  A large number of parameters  A huge volume of training samples is now available.  Techniques to improve generalization ability Ex) sparse coding, virtual sample generation, dropout  Requires heavy computation  GPU-based massive parallel processing  H/W implementation (SoC, FPGA)
  • 14. Handong Global University The Back-Propagation Algorithm  Gradient descent algorithm to minimize error E. Layer 1 (X1) Input layer (X0) W1 Layer 2 (X2) W2 Output layer (XN) WN feature errorsignal … netj 𝜕𝐸 𝜕𝑤 𝑖𝑗 = 𝜕𝐸 𝜕𝑛𝑒𝑡 𝑗 𝜕𝑛𝑒𝑡 𝑗 𝜕𝑤 𝑖𝑗 =𝛿𝑗 𝑥𝑖 𝛿𝑗 = 𝜕𝐸 𝜕𝑛𝑒𝑡𝑗 𝛿𝑖 = 𝑓′ (𝑛𝑒𝑡𝑖) 𝑗 𝑤𝑖𝑗 𝛿𝑗
  • 15. Handong Global University BP on Deep Network  BP does not work on deep networks  Error signals from many nodes are blended together.  become dim and vague on bottom layers “Diminishing gradient problem”  Error signal at a non-output node i 𝛿𝑖 = 𝑓′(𝑛𝑒𝑡𝑖) 𝑗 𝑤𝑖𝑗 𝛿𝑗 j wij i
  • 16. Handong Global University Agenda  Introduction to Deep Learning  Deep Learning Algorithms  Successful Application of Deep Learning  Q&A
  • 17. Handong Global University Breakthroughs in Deep Learning  Conventional back-propagation algorithm does not work well for deep fully-connected networks starting from random weights.  Layer-wise unsupervised pre-training algorithm Ex) DBN[Hinton2006], stacked auto-encoders[Bengio2006]  First, place the weights near a local optimal position by unsupervised learning algorithm  Then, conventional supervised learning algorithms work fine  Network structure to prevent diminishing gradient problem Ex) Convolutional Neural Networks [Fukushima1980][LeCun1998]
  • 18. Handong Global University Layer-wise Unsupervised Pre-training  Based on generative neural networks  Training procedure 1. Pre-train each layer to reproduce the input by unsupervised learning algorithm 2. Fine-tune the whole network by a supervised learning algorithm Ex) wake-sleep[Hinton2003], back-propagation
  • 19. Handong Global University Generative Neural Networks  Neural networks with forward–backward connections Layer 1 (X1) Input layer (X0) W1 Layer 2 (X2) W2 Output layer (XN) WN … Layer 1 (X1) Input layer (X0) Layer 2 (X2) Output layer (XN) W1 W2 WN … WN W1 W2 Forward-backward networkFeed forward network Forward connection for “encoding” Backward connection for “decoding”
  • 20. Handong Global University Layer-wise Unsupervised Pre-training  Starting from bottom layer, train each layer to reproduce the input Input  encoding  hidden  decoding  reprod. of input Layer 1 (X1) Input layer (X0) Layer 2 (X2) Output layer (XN) W1 W2 WN … WN W1 W2 Forward-backward network Forward propagation for encoding Backward propagation for decoding 1st phase
  • 21. Handong Global University Layer-wise Unsupervised Pre-training  Starting from bottom layer, train each layer to reproduce the input Input  encoding  hidden  decoding  reprod. of input Layer 1 (X1) Input layer (X0) Layer 2 (X2) Output layer (XN) W1 W2 WN … WN W1 W2 Forward-backward network Forward propagation for encoding Backward propagation for decoding 2nd phase
  • 22. Handong Global University Convolutional Neural Networks  Neocognitron [Fukushima80]  Designed to imitate visual processing of human/animals  Suggested basic concept and network structure of CNN  LeNet [LeCun98]  Simplified node and network structure  Gradient-based learning  Many improvements and extensions  [Simard2003], [Ciresan2011]  Convolutional DBN [Lee2009]  Siamese network [Chopra2005]  Locally connected network [Taigman2014]  Fast training algorithm using FFT [Mathieu2014]
  • 23. Handong Global University Convolutional Neural Networks  Composed of many heterogeneous layers  Convolution layer – feature extraction  Max-pooling layer - feature abstraction  Fully-connected layers - classification
  • 24. Handong Global University Convolution Layers  Odd-numbered layers in low/middle-level of CNN  Nodes on each layer are grouped into 2D planes (or feature maps)  Each plane is connected to one or more input planes  Each node computes weighted sum of input nodes in a small region  All nodes on a plane share weight set  Extract feature by convolution operation
  • 25. Handong Global University Max-Pooling Layers  Even-numbered layers in low/middle-level of CNN  Nodes on each layer are grouped into planes  Each plane is connected to only one input plane  Each node chooses maximum among input nodes in a small region  Abstract features  Reduces feature dimension  Ignores positional variation of feature elements
  • 26. Handong Global University Fully-connected Layers  Top 2~3 layers of CNN  1D structure  Each node is fully connected to all input nodes  Each node computes weighted sum of all input nodes  Classify input pattern with high- level features extracted by previous layers
  • 27. Handong Global University Gradient-based Learning [LeCun98]  Trains the whole network to minimize a single error function E.  At layer n  At layer n-1 Layer 1 (X1) Input layer (X0) W1 Layer 2 (X2) W2 Output layer (XN) WN …
  • 28. Handong Global University Why CNN Works Well?  Network structure effectively guides learning from 2D images preventing the diminishing gradient problem  Sparse connection  Parameter tying  Good at catching 2D structures  Training of convolution masks is effective to learn feature extraction  Good at handing shape variation  Abstraction in phases  Max pooling  Directly train the network to minimize classification error.
  • 29. Handong Global University Agenda  Introduction to Deep Learning  Deep Learning Algorithms  Successful Application of Deep Learning  Q&A
  • 30. Handong Global University Numeral Digit Recognition (MNIST DB)  Support Vector Machines  Neural Nets  Convolutional Neural Networks (Deep Networks)
  • 31. Handong Global University Chinese Character Recognition (CASIA DB)  ICDAR 2013 Competition Result [Yin, et.al 2013] CNN
  • 32. Handong Global University Object Image Recognition  ImageNet Large Scale Visual Recognition Challenge 2013 (ILSVRC2013, http://www.image-net.org)  1300 object categories  Training set: 1,281,167 images  Validation set: 50,000 images  Test set: 100,000 images
  • 33. Handong Global University Examples of ILSVRC2013 Images
  • 34. Handong Global University ILSVRC2013 Results All high rankers are based on CNNs
  • 35. Handong Global University Deep Learning in Face Recognition  Face recognition flow 1. Detection 2. Alignment (pre-processing) 3. Representation (feature extraction)  CNN  Robust to variation in lighting, expression, …  Alternatives: LBP + PCA/FDA 4. Verification / Classification  Siamese network  Alternatives: Euclidian distance, dot product, 2 distance, SVM, …
  • 36. Handong Global University DeepFace [Taigman2014]  Facebook AI group, Tel Aviv Univ.  Y. Taigman, et.al.  Achieved 97.25% on LFW dataset. cf. Conventional best performance: 96.33% [Cao 2013]  Face recognition prodedure  2D and 3D alignment  CNN-based representation  Verification by weighted 2 distance and Siamese network  A huge volume of training data  SFC dataset (4,000 identity * 1,000 samples)
  • 37. Handong Global University DeepFace [Taigman2014]  Feature extraction by CNN  Train a CNN-based face recognizer  Represent the input face image by the output of (N-1)th layer
  • 38. Handong Global University Deep Learning in Speech Recognition  Hybrid system (HMM + DNN)
  • 39. Handong Global University Deep Learning in Speech Recognition  Deep Neural Networks for Acoustic Modeling in Speech Recognition [Hinton2012] Deep learning
  • 40. Handong Global University Deep Learning in Speech Recognition  CNN for speech recognition [Ossama13]  Apply CNN on 2D vector (frame, frequency bands)
  • 41. Handong Global University Hangul Recognition  Challenges in Hangul Recognition  A multitude of similar characters  Missing one small stroke often result in misclassification Ex) 에-애–얘, 괟-괱-괠-팰, 흥-홍-훙-흉  Excessive cursiveness
  • 42. Handong Global University Deep Learning in Hangul Recognition Methods SERI95a PE92 Kim&Kim01 Structural matching 86.3% 82.2% Kang&Kim04 Structural matching 90.3% 87.7% Jang&Kim02 Structural matching + Post-processing 93.4% N/A Kim&Liu11 MQDF 93.71% 85.99% Kim CNN 95.96% 92.92% Error reduction rates 35.71% 42.44% [Kim2014]