4. What’s ICLR?
● International Conferene on Learning Representations. Called ‘I CLeaR’.
● 3 days conference, held at the beginning of May.
● ICLR2017 is the 5th-ICLR, the 1st-ICLR was held in 2013
● Website : http://www.iclr.cc/doku.php
○ Facebook page : https://www.facebook.com/iclr.cc/?fref=ts
○ Twitter account (This year) : https://twitter.com/iclr2017?lang=en
● http://www.kdnuggets.com/2016/02/iclr-deep-learning-scientific-publishing-ex
periment.html
5. Main features of ICLR (1)
● Focus on Deep Learning and its Application
○ Most of the accepted papers concern neural networks.
Variational Auto-Encoder
(ICLR2014)
ADAM optimizer
(ICLR2015)
6. Main features of ICLR (2)
● Open Review System https://openreview.net/
○ Single blind submission
○ Everyone can see all of the reviews and rebuttals.
○ Everyone can “join” the reviewing process.
7. Other features
● 2 tracks : Conference track and Workshop track
● Single oral track
● Lots of IT giants sponsors
https://blogs.sap.com/2017/05/22/dive-deep-into-deep-learning-
sap-at-the-international-conference-on-learning-representations
-iclr/
15. Oral sessions:
● Single oral track
● Two sessions on each day : morning and afternoon
○ 1 invited talk and 2~3 oral selected presentations in each session.
● Live streamining: https://www.facebook.com/iclr.cc/?fref=ts
16. Poster presentations
● Two poster sessions : morning and afternoon
● Conference track
○ ~40 posters at each session
○ 10~20 audiences around each poster
● Workshop track
○ ~20 posters
○ ~5 audiences
● (last year, poster)
○ ~5 audiences
(Taken by Okumura-san)
17. After hours parties
Individually hosted by Google, Facebook,
Salesforce ...etc.
○ Networking
○ Discussing with researchers and
engineers
■ until late at night!
18. Major & popular topics at ICLR2017
● Generalization ability of Neural Networks
○ Rethinking generalization https://arxiv.org/abs/1611.03530
○ Large batch training coverges to sharp minima https://arxiv.org/abs/1609.04836
● Generative Adversarial Networks
○ Towards principled methods of training GANs https://openreview.net/pdf?id=Hk4_qw5xe
○ Energy based GANs https://arxiv.org/abs/1702.01691
○ Two-sample tests by classifier https://openreview.net/forum?id=SJkXfE5xx¬eId=SJkXfE5xx
● Deep Reinforcement Learning
○ Reinforcement Learning with Unsupervised Auxiliary Tasks https://openreview.net/pdf?id=SJ6yPD5xg
○ (Not RL) Learning to act by predicting future https://arxiv.org/abs/1611.01779
● Neural Programmer
○ Making Neural Programming Architectures Generalize via Recursion
https://openreview.net/forum?id=BkbY4psgg¬eId=BkbY4psgg
19. Generative Adversarial Networks (GANs)
● Originally proposed by Ian Goodfellow et al. (2014)
● Quite a lot of researchers have been conducting works on GANs
○ GAN Zoo https://github.com/hindupuravinash/the-gan-zoo
https://github.com/hindupuravinash/the-gan-zoo
(GANs)
20. GANs frameworks
● Discriminator D(x) : trained to
discriminate between real
(dataset) examples and
generated examples by the
Generator G(z)
● Generator G(z) : trained to fool
the Discriminator D(x).
(GANs)
21. What is good about GANs?
● We don’t need explicit expression of the denstiy for
the generative models pG
(x)
○ Only requires a stochastic generative
process : x ~ pG
(x)
● The training process can be incorporated into
semi-supervised learning https://arxiv.org/abs/1606.03498
○ Achieved the state of the art performance, especially on a few
labeled semi-supervised dataset.
https://arxiv.org/pdf/1702.08896.pdf
(GANs)
22. Important GANs works (submitted to ICLR2017)
● b-GAN (uehara et al. https://arxiv.org/pdf/1610.02920.pdf )
○ The discriminator of GANs is the density ratio estimator rD
(x) of
r(x)=q(x) / pG
(x)
○ Directly learn q(x) / pG
(x) by minimizing Bregman Divergence between
rD
(x) and q(x) / pG
(x).
● Implicit Generative Models (Mohamed et al. https://arxiv.org/abs/1610.03483 )
○ Likelihood-free estimation through the GANs algorithm
● Deep and Hireachical Implicit models (Tran et al. https://arxiv.org/pdf/1702.08896.pdf ) (※Not
subimitted to ICLR)
○ Likelihood-free variational inference (LFVI) through the GANs
algorithm
■ it only requires that we can sample from qVI
(x, z) and pModel
(x, z)
(My selection of)
