SlideShare a Scribd company logo
1 of 31
Download to read offline
RVOS: End-to-End Recurrent Network
for Video Object Segmentation
Carles Ventura Míriam Bellver Andreu Girbau Amaia Salvador Ferran Marqués Xavi Giró
Video Object Segmentation
One-shot video object
segmentation
Zero-shot video
object segmentation
VS
Motivation
● End-to-End Trainable model
○ YouTube-VOS include 3,471 training videos
○ No dependency on other pre-trained networks like optical flow
● Recurrent Network
○ Extend RSIS (Recurrent Semantic Instance Segmentation) to Video Object Segmentation
● Spatial and Temporal Recurrence
○ Study if spatio-temporal recurrence outperforms spatial and temporal recurrence
● One-shot and zero-shot video object segmentation
○ RSIS is able to discover the object instances in an image
○ No published results for zero-shot multiple object video object segmentation
● Fast method
○ No need of fine-tuning at inference (no online learning)
Related Work
● Sequence-to-Sequence (S2S) [1]
○ Drawbacks
■ Each instance is trained and segmented independently
■ Designed only for one-shot video object segmentation
[1] N. Xu et al, YouTube-VOS: Sequence-to-Sequence Video Object Segmentation. ECCV 2018
Related Work
● ConvGRU [2]
○ Drawbacks
■ Each instance is trained and segmented independently
■ Optical flow depends on a network trained for another task: model is not end-to-end trainable
[2] P. Tokmakov et al., Learning video object segmentation with visual memory. ICCV 2017
Related Work
● RSIS [3]
○
○ Drawbacks
■ Model is designed for image object segmentation
[3] A. Salvador et al., Recurrent Neural Networks for Semantic Instance Segmentation. arXiv
Proposed model
● RVOS: Recurrent Video Object Segmentation
○
Proposed model
● RVOS: Recurrent Video Object Segmentation
○
Proposed model
● RVOS: Recurrent Video Object Segmentation
○
Experiments
● Two different benchmarks
○ YouTube-VOS
■ 3,471 training videos
■ 474 testing videos (validation set)
○ DAVIS-2017
■ 90 training videos (train+val)
■ 30 testing videos (test-dev set)
● Two different tasks
○ One-shot video object segmentation
○ Zero-shot video object segmentation
● Evaluation measures
○ Region similarity J
○ Contour accuracy F
Experiments: One-shot VOS on YouTube
● Ablation study
Experiments: One-shot VOS on YouTube
● Comparison with SoA techniques
Experiments: One-shot VOS on YouTube
● Performance VS Number of Instances
Experiments: One-shot VOS on YouTube
Experiments: One-shot VOS on YouTube
● Qualitative results
Experiments: One-shot VOS on YouTube
● Qualitative results
Experiments: One-shot VOS on YouTube
● Qualitative results
Experiments: One-shot VOS on YouTube
● Qualitative results
Experiments: Zero-shot VOS on YouTube
● Problem related with annotated objects
○ Missing object annotations
Experiments: Zero-shot VOS on YouTube
● Ablation study
● First results reported for zero-shot VOS on YouTube-VOS
Experiments: Zero-shot VOS on YouTube
Experiments: Zero-shot VOS on YouTube
● Qualitative results
Experiments: Zero-shot VOS on YouTube
● Qualitative results
Experiments: Zero-shot VOS on YouTube
● Qualitative results
Experiments: One-shot VOS on DAVIS-2017
● We take advantage of the model already trained on YouTube-VOS
○ Apply directly the pre-trained model
○ Finetune on DAVIS-2017 the pre-trained model
● S2S model (SoA also trained on YouTube-VOS)
○ Results on DAVIS-2016 (single object, foreground-background video object segmentation)
○ No results on DAVIS-2017 (multiple object)
Experiments: One-shot VOS on DAVIS-2017
● Comparison with SoA techniques
Experiments: One-shot VOS on DAVIS-2017
● Qualitative results
Experiments: Zero-shot VOS on DAVIS-2017
● First results reported for zero-shot VOS on DAVIS-2017
● Results for zero-shot VOS on YouTube for unseen categories were also low
Experiments: Zero-shot VOS on DAVIS-2017
● Qualitative results
Conclusions
● Fully end-to-end trainable model for video object segmentation
● Designed for multiple object video object segmentation
● Designed for one-shot and zero-shot video object segmentation
● Spatio-temporal recurrence outperforms spatial and temporal recurrence
● One-shot video object segmentation:
○ YouTube-VOS: Comparable results to SoA techniques (S2S)
○ DAVIS-2017:
■ Outperform other SoA techniques that do not use online learning
■ Comparable results to some SoA techniques that use online learning
● Zero-shot video object segmentation:
○ No results reported both on YouTube-VOS and DAVIS-2017
Thank you for your attention
Carles Ventura Royo
cventuraroy@uoc.edu
https://imatge-upc.github.io/rvos/

More Related Content

Similar to RVOS: End-to-End Recurrent Network for Video Object Segmentation (CVPR 2019)

[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...
[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...
[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...
Sunghyun Park
 

Similar to RVOS: End-to-End Recurrent Network for Video Object Segmentation (CVPR 2019) (15)

Building Your Own Native App Visual Testing Module with Appium by Wim Selles
Building Your Own Native App Visual Testing Module with Appium by Wim SellesBuilding Your Own Native App Visual Testing Module with Appium by Wim Selles
Building Your Own Native App Visual Testing Module with Appium by Wim Selles
 
Building Your Own Native App Visual Testing Module with Appium
Building Your Own Native App  Visual Testing Module with AppiumBuilding Your Own Native App  Visual Testing Module with Appium
Building Your Own Native App Visual Testing Module with Appium
 
Google | Infinite Nature Zero Whitepaper
Google | Infinite Nature Zero WhitepaperGoogle | Infinite Nature Zero Whitepaper
Google | Infinite Nature Zero Whitepaper
 
Building Your Own Native App Visual Testing Module with Appium
Building Your Own Native App Visual Testing Module with AppiumBuilding Your Own Native App Visual Testing Module with Appium
Building Your Own Native App Visual Testing Module with Appium
 
"Revisiting self supervised visual representation learning" Paper Review
"Revisiting self supervised visual representation learning" Paper Review"Revisiting self supervised visual representation learning" Paper Review
"Revisiting self supervised visual representation learning" Paper Review
 
[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...
[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...
[AAAI 2021] Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Di...
 
Rahul Sukthankar at AI Frontiers: Large-Scale Video Understanding: YouTube an...
Rahul Sukthankar at AI Frontiers: Large-Scale Video Understanding: YouTube an...Rahul Sukthankar at AI Frontiers: Large-Scale Video Understanding: YouTube an...
Rahul Sukthankar at AI Frontiers: Large-Scale Video Understanding: YouTube an...
 
Meetup TestingUy 2017: Automating the Viewer: a cross-functional team approach
Meetup TestingUy 2017: Automating the Viewer: a cross-functional team approachMeetup TestingUy 2017: Automating the Viewer: a cross-functional team approach
Meetup TestingUy 2017: Automating the Viewer: a cross-functional team approach
 
Meetup TestingUy 2017 - Automating the Viewer: a cross-functional team approach
Meetup TestingUy 2017 - Automating the Viewer: a cross-functional team approachMeetup TestingUy 2017 - Automating the Viewer: a cross-functional team approach
Meetup TestingUy 2017 - Automating the Viewer: a cross-functional team approach
 
Visual Search for Musical Performances and Endoscopic Videos
Visual Search for Musical Performances and Endoscopic VideosVisual Search for Musical Performances and Endoscopic Videos
Visual Search for Musical Performances and Endoscopic Videos
 
Building turn-key recommendations for 5% of internet video
Building turn-key recommendations for 5% of internet videoBuilding turn-key recommendations for 5% of internet video
Building turn-key recommendations for 5% of internet video
 
Video+Language: From Classification to Description
Video+Language: From Classification to DescriptionVideo+Language: From Classification to Description
Video+Language: From Classification to Description
 
Video assessment recipes
Video assessment recipesVideo assessment recipes
Video assessment recipes
 
Video + Language: Where Does Domain Knowledge Fit in?
Video + Language: Where Does Domain Knowledge Fit in?Video + Language: Where Does Domain Knowledge Fit in?
Video + Language: Where Does Domain Knowledge Fit in?
 
Video + Language: Where Does Domain Knowledge Fit in?
Video + Language: Where Does Domain Knowledge Fit in?Video + Language: Where Does Domain Knowledge Fit in?
Video + Language: Where Does Domain Knowledge Fit in?
 

More from Universitat Politècnica de Catalunya

More from Universitat Politècnica de Catalunya (20)

Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
 
Deep Generative Learning for All
Deep Generative Learning for AllDeep Generative Learning for All
Deep Generative Learning for All
 
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
 
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoTowards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
 
The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021
 
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
 
Open challenges in sign language translation and production
Open challenges in sign language translation and productionOpen challenges in sign language translation and production
Open challenges in sign language translation and production
 
Discovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in MinecraftDiscovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in Minecraft
 
Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...
 
Intepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural NetworksIntepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural Networks
 
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
 
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
 
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
 
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
 
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
 
Curriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object SegmentationCurriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object Segmentation
 
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
 
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
 

Recently uploaded

➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
amitlee9823
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
amitlee9823
 
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
amitlee9823
 
CHEAP Call Girls in Rabindra Nagar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Rabindra Nagar  (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Rabindra Nagar  (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Rabindra Nagar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...
amitlee9823
 
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
amitlee9823
 
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
amitlee9823
 
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
amitlee9823
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
amitlee9823
 
➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men 🔝Thrissur🔝 Escor...
➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men  🔝Thrissur🔝   Escor...➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men  🔝Thrissur🔝   Escor...
➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men 🔝Thrissur🔝 Escor...
amitlee9823
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
amitlee9823
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
amitlee9823
 
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
karishmasinghjnh
 

Recently uploaded (20)

Anomaly detection and data imputation within time series
Anomaly detection and data imputation within time seriesAnomaly detection and data imputation within time series
Anomaly detection and data imputation within time series
 
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
 
Capstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics ProgramCapstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics Program
 
CHEAP Call Girls in Rabindra Nagar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Rabindra Nagar  (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Rabindra Nagar  (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Rabindra Nagar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
 
Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...
 
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
 
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
 
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men  🔝Bangalore🔝   Esc...
➥🔝 7737669865 🔝▻ Bangalore Call-girls in Women Seeking Men 🔝Bangalore🔝 Esc...
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
 
➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men 🔝Thrissur🔝 Escor...
➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men  🔝Thrissur🔝   Escor...➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men  🔝Thrissur🔝   Escor...
➥🔝 7737669865 🔝▻ Thrissur Call-girls in Women Seeking Men 🔝Thrissur🔝 Escor...
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
 
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
 

RVOS: End-to-End Recurrent Network for Video Object Segmentation (CVPR 2019)

  • 1. RVOS: End-to-End Recurrent Network for Video Object Segmentation Carles Ventura Míriam Bellver Andreu Girbau Amaia Salvador Ferran Marqués Xavi Giró
  • 2. Video Object Segmentation One-shot video object segmentation Zero-shot video object segmentation VS
  • 3. Motivation ● End-to-End Trainable model ○ YouTube-VOS include 3,471 training videos ○ No dependency on other pre-trained networks like optical flow ● Recurrent Network ○ Extend RSIS (Recurrent Semantic Instance Segmentation) to Video Object Segmentation ● Spatial and Temporal Recurrence ○ Study if spatio-temporal recurrence outperforms spatial and temporal recurrence ● One-shot and zero-shot video object segmentation ○ RSIS is able to discover the object instances in an image ○ No published results for zero-shot multiple object video object segmentation ● Fast method ○ No need of fine-tuning at inference (no online learning)
  • 4. Related Work ● Sequence-to-Sequence (S2S) [1] ○ Drawbacks ■ Each instance is trained and segmented independently ■ Designed only for one-shot video object segmentation [1] N. Xu et al, YouTube-VOS: Sequence-to-Sequence Video Object Segmentation. ECCV 2018
  • 5. Related Work ● ConvGRU [2] ○ Drawbacks ■ Each instance is trained and segmented independently ■ Optical flow depends on a network trained for another task: model is not end-to-end trainable [2] P. Tokmakov et al., Learning video object segmentation with visual memory. ICCV 2017
  • 6. Related Work ● RSIS [3] ○ ○ Drawbacks ■ Model is designed for image object segmentation [3] A. Salvador et al., Recurrent Neural Networks for Semantic Instance Segmentation. arXiv
  • 7. Proposed model ● RVOS: Recurrent Video Object Segmentation ○
  • 8. Proposed model ● RVOS: Recurrent Video Object Segmentation ○
  • 9. Proposed model ● RVOS: Recurrent Video Object Segmentation ○
  • 10. Experiments ● Two different benchmarks ○ YouTube-VOS ■ 3,471 training videos ■ 474 testing videos (validation set) ○ DAVIS-2017 ■ 90 training videos (train+val) ■ 30 testing videos (test-dev set) ● Two different tasks ○ One-shot video object segmentation ○ Zero-shot video object segmentation ● Evaluation measures ○ Region similarity J ○ Contour accuracy F
  • 11. Experiments: One-shot VOS on YouTube ● Ablation study
  • 12. Experiments: One-shot VOS on YouTube ● Comparison with SoA techniques
  • 13. Experiments: One-shot VOS on YouTube ● Performance VS Number of Instances
  • 15. Experiments: One-shot VOS on YouTube ● Qualitative results
  • 16. Experiments: One-shot VOS on YouTube ● Qualitative results
  • 17. Experiments: One-shot VOS on YouTube ● Qualitative results
  • 18. Experiments: One-shot VOS on YouTube ● Qualitative results
  • 19. Experiments: Zero-shot VOS on YouTube ● Problem related with annotated objects ○ Missing object annotations
  • 20. Experiments: Zero-shot VOS on YouTube ● Ablation study ● First results reported for zero-shot VOS on YouTube-VOS
  • 22. Experiments: Zero-shot VOS on YouTube ● Qualitative results
  • 23. Experiments: Zero-shot VOS on YouTube ● Qualitative results
  • 24. Experiments: Zero-shot VOS on YouTube ● Qualitative results
  • 25. Experiments: One-shot VOS on DAVIS-2017 ● We take advantage of the model already trained on YouTube-VOS ○ Apply directly the pre-trained model ○ Finetune on DAVIS-2017 the pre-trained model ● S2S model (SoA also trained on YouTube-VOS) ○ Results on DAVIS-2016 (single object, foreground-background video object segmentation) ○ No results on DAVIS-2017 (multiple object)
  • 26. Experiments: One-shot VOS on DAVIS-2017 ● Comparison with SoA techniques
  • 27. Experiments: One-shot VOS on DAVIS-2017 ● Qualitative results
  • 28. Experiments: Zero-shot VOS on DAVIS-2017 ● First results reported for zero-shot VOS on DAVIS-2017 ● Results for zero-shot VOS on YouTube for unseen categories were also low
  • 29. Experiments: Zero-shot VOS on DAVIS-2017 ● Qualitative results
  • 30. Conclusions ● Fully end-to-end trainable model for video object segmentation ● Designed for multiple object video object segmentation ● Designed for one-shot and zero-shot video object segmentation ● Spatio-temporal recurrence outperforms spatial and temporal recurrence ● One-shot video object segmentation: ○ YouTube-VOS: Comparable results to SoA techniques (S2S) ○ DAVIS-2017: ■ Outperform other SoA techniques that do not use online learning ■ Comparable results to some SoA techniques that use online learning ● Zero-shot video object segmentation: ○ No results reported both on YouTube-VOS and DAVIS-2017
  • 31. Thank you for your attention Carles Ventura Royo cventuraroy@uoc.edu https://imatge-upc.github.io/rvos/