Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

論文紹介: Mojitalk: generating emotional responses at scale

1.968 visualizaciones

Publicado el

ACL2018 読み会の発表資料です

Publicado en: Ingeniería
  • Sé el primero en comentar

論文紹介: Mojitalk: generating emotional responses at scale

  1. 1. MOJITALK: Generating Emotional Responses at Scale ACL2018
  2. 2. : (B4, ) : , , : NLP 1 , CV : , 3D , NMT, HCI
  3. 3. 🐠 MOJITALK: Generating Emotional Responses at Scale : ▶︎ ▶︎ ( ) ▶︎ #kawaii_paper_name
  4. 4. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 2.Conclusion 3.Related Work 4.Dataset 5.Model 6.Experimental Results 7.Conclusion & Future Work
  5. 5. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 🐶 2.Conclusion 3.Related Work 4.Dataset 5.Model 6.Experimental Results 7.Conclusion & Future Work
  6. 6. … Abstract 🐥
  7. 7. Introduction 🐶 : ▶︎ (emotional) : ▶︎ … [Pang et al., 2002; Maas et al., 2011; Socher et al., 2013]
  8. 8. Introduction 🐶 + ▶︎ [Go et al., 2016], [Li et al., 2017b] ▶︎ [Pang et al., 2002; Maas et al., 2011; Socher et al., 2013] Twitter
  9. 9. Introduction 🐶 Original tweet( ) Response( )
  10. 10. Introduction 🐶 ▶︎ IR [Huang et al., 2017] ▶︎ CVAE VAE [Zhao et al., 2017] ▶︎ sentence-to-emoji classifier[Feibo et al., 2017]
  11. 11. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 2.Conclusion 🐱 3.Related Work 4.Dataset 5.Model 6.Experimental Results 7.Conclusion & Future Work
  12. 12. Conclusion 🐱
  13. 13. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 2.Conclusion 3.Related Work 🐭 4.Dataset 5.Model 6.Experimental Results 7.Conclusion & Future Work
  14. 14. Related Work 🐭 Twitter Emoji2vec [Eisner et al., 2016] ▶︎
  15. 15. Related Work 🐭 Twitter Emoji2vec [Eisner et al., 2016] ▶︎
  16. 16. Related Work 🐭 Twitter DeepMoji [Felbo et al., 2017] ▶︎ bi-LSTM https://deepmoji.mit.edu/
  17. 17. Related Work 🐭 ConditionalVAE [Sohn et al., 2015] ▶︎ ( )
  18. 18. Related Work 🐭 [Li et al., 2016][Li et al., 2017] ▶︎ 
 Action , State , Policy[pi, qi] pRL(pi+1 |pi, qi)
  19. 19. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 2.Conclusion 3.Related Work 4.Dataset 🐹 5.Model 6.Experimental Results 7.Conclusion & Future Work
  20. 20. Dataset 🐹 ▶︎ SNS ▶︎ 64 DeepMoji
  21. 21. Dataset 🐹 ▶︎ 2017 8 12~14 ▶︎ ▶︎ URL, , ▶︎
  22. 22. Dataset 🐹 ▶︎ ▶︎
  23. 23. Dataset 🐹 ▶︎ ▶︎ yess yes
  24. 24. Dataset 🐹
  25. 25. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 2.Conclusion 3.Related Work 4.Dataset 5.Model 🐰 6.Experimental Results 7.Conclusion & Future Work
  26. 26. Model 🐰 seq2seq + Attention [Luong et al., 2015] ▶︎ ▶︎ ▶︎ embedding->Dense
  27. 27. Model 🐰 CVAE [Sohn et al., 2015] p(x|c) = ∫ p(x|z, c)p(z|c)dz [vo; ve]
  28. 28. Model 🐰 CVAE p(x|c) = ∫ p(x|z, c)p(z|c)dz ▶︎ cf)VAE BoW loss
  29. 29. Model 🐰 CVAE recog/prior network Reparameterization trick[Kingma and Welling, 2013] recog net z decoder Prior net
  30. 30. Model 🐰 CVAE CVAE Attention ( ) ▶︎ VAE Decoder [Bowman et al., 2015] KL loss construction loss ▶︎ KL annealing early stopping [Bowman et al., 2015] bag-of-words loss [Zhao et al., 2017]
  31. 31. Model 🐰 Reinforced CVAE ▶︎ CVAE policy gradient( ) policy gradient bi-GRU [Felbo et al., 2017] ▶︎ policy training x c CVAE x’ x’ R x emoji clf
  32. 32. Model 🐰 Reinforced CVAE REINFORCE ▶︎ ▶︎ Reinforce hybrid objective α ∈ [0,1]
  33. 33. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 2.Conclusion 3.Related Work 4.Dataset 5.Model 6.Experimental Results 🦊 7.Conclusion & Future Work
  34. 34. Experimental Results 🦊 CVAE seq2seq ▶︎ Perplexity ( ) : www.slideshare.net/hoxo_m/perplexity ▶︎ top5 ▶︎ SotA
  35. 35. Experimental Results 🦊 ▶︎ seq2seq ▶︎ CVAE ▶︎ 1-gram, 2-gram, 3-gram ▶︎ Reinforced CVAE (?)
  36. 36. Experimental Results 🦊 ▶︎ top5 CVAE Reinforced CVAE
  37. 37. Experimental Results 🦊 ▶︎ Amazon Mechanical Turk 100 5 ▶︎
  38. 38. Experimental Results 🦊 ▶︎ Turing Test ▶︎ 18% 27% ▶︎ (inter-rater reliability)
  39. 39. Experimental Results 🦊 ▶︎ seq2seq I’m ( ) ▶︎ CVAE ▶︎ Reinforced CVAE
  40. 40. MOJITALK: Generating Emotional Responses at Scale 1.Introduction 2.Conclusion 3.Related Work 4.Dataset 5.Model 6.Experimental Results 7.Conclusion & Future Work 🐮
  41. 41. Conclusion & Future Work 🐮
  42. 42. Thank you for listening 👻

×