PR100: SeedNet: Automatic Seed Generation with Deep Reinforcement Learning for Robust Interactive Segmentation
1. PR-100:
SeedNet: Automatic Seed Generation with Deep
Reinforcement Learning for Robust Interactive Segmentation
CVPR2018
Gwangmo Song, Heesoo Myeong, Kyoung Mu Lee
인공지능연구원
이광희
2. 2
논문 선정의 이유..
Chen, Tao, et al. "PhotoSketch: Internet image montage." SIGGRAPH Asia (2009).
3. 3
논문 선정의 이유..
스케치
배경
오브젝트
사진 선택
이미지 조합
사용될 이미지 생성
팔레트
최종 결과물
텍스트소나무
+
전체 스타일 변환 (팔레트)
브러시
부분 수정 및 조정 (브러시)
이미지 생성 모델
스타일 변환 모델
검색
5. 5
Related Works : Interactive Segmentation
Deep extreme cut: From extreme points to object segmentation. CVPR2018
Grabcut: Interactive foreground extraction using iterated graph cuts. Siggraph2003
Methods:
Grabcut
Random walk
Geodesic
Deep extremecut
.
.
.
Seed types:
Rectangle
Scribble
Contour
Extreme point
.
.
6. 6
Classification, image captioning, video tracking, face
hallucination, …
Related Works : RL in Computer Vision
Active Object Localization with Deep Reinforcement Learning. ICCV2015
Distort-and-Recover: Color enhancement using deep reinforcement learning. CVPR2018
7. 7
An automatic seed generation technique with deep RL to solve the interactive segmentation
problem
Robust and consistent object extraction with less human effort
User first select two points- foreground & background
A sequence of artificial user input is automatically generated
Markov Decision Process(MDP) / Deep Q-Network(DQN)
Motivation
8. 8
Introduction of a MDP formulation for the interactive segmentation
task
The novel reward function design: Intersection Over Union(IOU)
score
Why deep RL?
• Cannot define globally optimal seed at some stage of interactive segmentation
Contributions
9. 9
Automatic Seed Generation System
Markov Decision Process(MDP)
- State: input image + segmented mask by new seeds
- Action: 800 actions, label(fg/bg), position of the seed in the 2D grid(20x20)
- Reward:
Segmentation method: Random Walk(RW) segmentation
Binary Mask
- Compute reward signal
- An observation of the next iteration
Termination: 10 seed points
DQN architecture
SF: Strong Foreground
SB: Strong Background
WF: Weak Foreground
WB: Weak Background
10. 10
Experiments
MSRA10K saliency dataset
Training: 9000 images, Test: 1000 images, Total: 10000 images
Image size: about 400x300 pixels
Training/testing Input size: 84x84
Segmentation
• Training: 84x84(for accelerate), seed point size(3 pixels)
• Testing: original size, seed point size(13 pixels)
Termination: 10 times (average number of seeds until saturation:
5.39)