8. Related Works: Summary of Existing Methods
• Not deep (Fixed feature space)
– [Gong, 2012] GFK
– [Baktashmotlagh, 2013] UDICA
– [Sun, 2016] CORAL
– Many
• Deep (Neural based, Flexible feature space)
– Feature Adaptation
– Classifier Adaptation (<- This Work)
• [Long, 2016] RTN (NIPS2016)
• [Sener, 2016] knn-Ad (NIPS2016)
• [Saito, 2017] ATDA (ICML2017)
8
9. Related Works: Feature Adaptation
Mathematical Foundation
[Ganin, 2016] “Domain-Adversarial Training of Neural Networks”
Visualization
[Ben-David, 2010] “A theory of learning from different domains”
ドメイン間の距離ソース損失
理想的なhを使うと
きの損失の差
9
17. 17
Feature Adaptation Classifier Adaptation?
Name Ref. Minimize By Means Enc unshared? Y/N By Means
DDC [Tzeng, 2014] E|P(Zs) – P(Zt)| MMD N N /
DDA [Long, 2015] E|P(Zs) – P(Zt)| MK-MMD N N /
DANN [Ganin, 2014] E|P(Zs)/P(Zt)| Adversarial N N /
CORAL [Sun, 2016] E|P(Zs) – P(Zt)| 2nd order moment N N /
VFAE [Louizos, 2016] E|P(Zs) – P(Zt)| MMD + Graphical N N /
AdaBN [Li, 2017] E|P(Zs) – P(Zt)| Domain-wise BN N N /
CMD [Zellinger, 2017] E|P(Zs) – P(Zt)| k-th order moment N N /
UPLDA [Bousmalis, 2016] E|P(Xs)/P(Xt)| GAN / N /
DSN [Bousmalis, 2016]
E|P(Zs) – P(Zt)|
or E|P(Zs)/P(Zt)| MMD or Adversarial Y N /
ADDA [Tzeng, 2017] E|P(Zs)/P(Zt)| Adversarial Y N /
KNN-Ad [Sener, 2016] ?? ?? ?? Y ??
RTN [Long, 2016] E|P(Zs) – P(Zt)| MMD N Y Residual Classifier
ATDA [Saito, 2017] / / N Y Tri-Training
DRCN [Ghifary, 2016] Implicit Reconstruction N N /
CoGAN [Liu, 2016] Implicit GAN Y N /
GTA [Swami, 2017] Implicit Conditional GAN N N /
- Feature
- Shared Enc
- Feature
- Unshared Enc
- Classifier
- Feature
- Implicit
Category
18. 18
Feature Adaptation Classifier Adaptation?
Name Ref. Minimize By Means Enc unshared? Y/N By Means
DDC [Tzeng, 2014] E|P(Zs) – P(Zt)| MMD N N /
DDA [Long, 2015] E|P(Zs) – P(Zt)| MK-MMD N N /
DANN [Ganin, 2014] E|P(Zs)/P(Zt)| Adversarial N N /
CORAL [Sun, 2016] E|P(Zs) – P(Zt)| 2nd order moment N N /
VFAE [Louizos, 2016] E|P(Zs) – P(Zt)| MMD + Graphical N N /
AdaBN [Li, 2017] E|P(Zs) – P(Zt)| Domain-wise BN N N /
CMD [Zellinger, 2017] E|P(Zs) – P(Zt)| k-th order moment N N /
UPLDA [Bousmalis, 2016] E|P(Xs)/P(Xt)| GAN / N /
DSN [Bousmalis, 2016]
E|P(Zs) – P(Zt)|
or E|P(Zs)/P(Zt)| MMD or Adversarial Y N /
ADDA [Tzeng, 2017] E|P(Zs)/P(Zt)| Adversarial Y N /
KNN-Ad [Sener, 2016] ?? ?? ?? Y ??
RTN [Long, 2016] E|P(Zs) – P(Zt)| MMD N Y Residual Classifier
ATDA [Saito, 2017] / / N Y Tri-Training
DRCN [Ghifary, 2016] Implicit Reconstruction N N /
CoGAN [Liu, 2016] Implicit GAN Y N /
GTA [Swami, 2017] Implicit Conditional GAN N N /
<- Proposal