Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

「知識」のDeep Learning

2015/06/04にPFIセミナーで発表した、知識表現に対する表現学習に関するスライドです

  • Sé el primero en comentar

「知識」のDeep Learning

  1. 1. Deep Learning 1 Preferred Infrastructure (@unnonouno) 2015/06/04 PFI
  2. 2. !  1 !  !  NLP 12014-
  3. 3. NLP 1YANS !  YANS 19 !  140 !  !  !  YANS 13 !  !  "
  4. 4. Knowledge Representation
  5. 5. 1. 3. 4. 2.
  6. 6. 1. 3. 4. 2.
  7. 7. 1. 3. 4. 2. RNN
  8. 8. Recurrent Neural Network Language Model (RNNLM) [Mikolov+10] !  RNN !  !  !  LSTM
  9. 9. Long Short-Term Memory (LSTM) !  !  1 ! 
  10. 10. RNN1LSTM !  ! 
  11. 11. !  !  argmaxy P(y|x) = argmaxy P(x|y) P(y)
  12. 12. 1. 3. 4. 2.
  13. 13. [ 15] !  1 !  -‐‑‒
  14. 14. -‐‑‒ [ 15] !  Pydata Tokyo
  15. 15. 1. 3. 4. 2.
  16. 16. -‐‑‒ !  !  ! 
  17. 17. Knowledge Representation
  18. 18. !  EMNLP2014 Bordes Weston 1 part 2 [Bordes&Weston14]
  19. 19. !  [ 15]
  20. 20. Q: Deep Learning Representation Learning -‐‑‒ A: Deep
  21. 21. 推論を導けるような知識の表現、およびその方法 を開発する人工知能研究の領域である。
  22. 22. !  !  !  1
  23. 23. 1 !  2 t t 3 !  t t !  RDF (x, r, y) x yr
  24. 24. 1Knowledge Base
  25. 25. !  New York NY
  26. 26. !  !  t t !  t t !  t t
  27. 27. -‐‑‒ !  -‐‑‒ NLP
  28. 28. !  {(xi, ri, yi)}: !  x, y: t t !  r: t t !  x, y !  r
  29. 29. !  !  1 argmax ∑i f(xi, ri, yi)
  30. 30. Distance model (Structured Embedding) [Bordes +11] !  e !  Rleft, Rright !  !  f f(x, r, y) = || Rleft(r) e(x) – Rright(r) e(y) ||1
  31. 31. TransE model [Brodes+13] !  r r !  f(x, r, y) = || e(x) + r – e(y) ||2 2
  32. 32. TransE model x y r
  33. 33. TransE model 1 TransE !  TransM: !  TransH:
  34. 34. TransM model [Fan+14] !  r !  wr r x, y f(x, r, y) = wr|| e(x) + r – e(y) ||2 2
  35. 35. TransH model [Wang+14] !  TransE
  36. 36. Bilinear model !  r !  f(x, r, y) = e(x)T Wr e(y)
  37. 37. Neural Tensor Network (NTN) [Socher+13] f(x, r, y) = ur tanh(e(x)Wre(y) + V1 r e(x) + V2 r e(y) + br) !  r ! 
  38. 38. [Yang+15] !  2 !  Bilinear
  39. 39. Q: -‐‑‒ A: [Nickel+11]
  40. 40. Link prediction !  t t 1 !  QA !  -‐‑‒ !  t t (e1, r, e2) (e1, r, ? )
  41. 41. TransH TransE [Bordes&Weston14]
  42. 42. !  ! 
  43. 43. [Weston+13] !  x y r !  [Bordes&Weston14]
  44. 44. [Weston+13] !  !  -‐‑‒ [Bordes&Weston14]
  45. 45. [Bordes&Weston14]
  46. 46. Link prediction 1QA !  Link prediction ! 
  47. 47. QA [Bordes+14] !  !  q t f(q) g(t) !  f(q), g(t) q, t
  48. 48. Memory networks: [Weston+15] !  I: I(x) !  G: mi = G(mi, I(x), m) !  O: o = O(I(x), m) !  R: r = R(o)
  49. 49. I: O: R: G:
  50. 50. !  !  !  ! 
  51. 51. !  Deep Learning -‐‑‒
  52. 52. -‐‑‒ !  !  -‐‑‒ !  -‐‑‒ !  -‐‑‒ !  !  factoid !  -‐‑‒ ! 
  53. 53. RNN [Peng&Yao15] !  RNN ! 
  54. 54. !  !  !  !  !  !  !  ! 
  55. 55. 1/4 !  [Mikolov+10] T. Mikolov, M. Karafiat, L. Burget, J. H. Cernocky, S. Khudanpur. Recurrent neural network based language model. Interspeech 2010. !  [ 15] . . 2015. !  [ 15] . NLP Introduction based on Project Next NLP. PyData.Tokyo Meetup #5, 2015. !  [Bordes&Weston14] A. Bordes, J. Weston. Embedding Methods for Natural Language Processing. EMNLP2014 tutorial. !  [ 15] . . JSAI 2015 .
  56. 56. 2/4 !  [Bordes+11] A. Bordes, J. Weston, R. Collobert, Y. Bengio. Learning structured embeddings of knowledge bases. AAAI2011. !  [Bordes+13] A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O. Yakhnenko. Translating Embeddings for Modeling Multi-relational Data. NIPS 2013. !  [Fan+14] M. Fan, Q. Shou, E. Chang, T. F. Zheng. Transition-based Knowledge Graph Embedding with Relational Mapping Properties. PACLIC 2014. !  [Wang+14] Z. Wang, J. Zhang, J. Feng, Z. Chen. Knowledge Graph Embedding by Translating on Hyperplanes. AAAI 2014.
  57. 57. 3/4 !  [Socher+13] R. Socher, D. Chen, C. D. Manning, A. Y. Ng. Reasoning With Neural Tensor Networks for Knowledge Base Completion. NIPS 2013. !  [Yang+15] B. Yang, W. Yih, X. He, J. Gao, L. Deng. Embedding Entities and Relations for Learning and Inferenece in Knowledge Bases. ICLR 2015. !  [Nickel+11] M. Nickel, V. Tresp, H. P. Kriegel. A Three-Way Model for Collective Learning on Multi-Relational Data. ICML 2011.
  58. 58. 4/4 !  [Bordes+14] A. Bordes, J. Weston, N. Usunier. Open Question Answering with Weakly Supervised Embedding Models. ECML/PKDD 2014. !  [Weston+15] J. Weston, S. Chopra, A. Bordes. Memory Networks. ICLR 2015. !  [Peng&Yao15] B. Peng, K. Yao. Recurrent Neural Networks with External Memory for Language Understanding. arXiv:1506.00195, 2015.

×