Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio

Eche un vistazo a continuación

1 de 96 Anuncio

ACL 2018 Recap

Descargar para leer sin conexión

ACL 2018에 다녀오신 NAVER 개발자 분들께서 그 내용을 공유해 주십니다.

1. Overview - Lucy Park
2. Tutorials - Xiaodong Gu
3. Main conference
a. Semantic parsing - Soonmin Bae
b. Dialogue - Kyungduk Kim
c. Machine translation - Zae Myung Kim
d. Summarization - Hye-Jin Min
4. Workshops - Minjoon Seo

ACL 2018에 다녀오신 NAVER 개발자 분들께서 그 내용을 공유해 주십니다.

1. Overview - Lucy Park
2. Tutorials - Xiaodong Gu
3. Main conference
a. Semantic parsing - Soonmin Bae
b. Dialogue - Kyungduk Kim
c. Machine translation - Zae Myung Kim
d. Summarization - Hye-Jin Min
4. Workshops - Minjoon Seo

Anuncio
Anuncio

Más Contenido Relacionado

Presentaciones para usted (20)

Similares a ACL 2018 Recap (20)

Anuncio

Más de NAVER Engineering (20)

Más reciente (20)

Anuncio

ACL 2018 Recap

  1. 1. July 15-20, 2018 @Melbourne, Australia Recap on 2018-08-13 Naver Techtalks
  2. 2. 1. Overview - Lucy Park 2. Tutorials - Xiaodong Gu 3. Main conference a. Semantic parsing - Soonmin Bae b. Dialogue - Kyungduk Kim c. Machine translation - Zae Myung Kim d. Summarization - Hye-Jin Min 4. Workshops - Minjoon Seo
  3. 3. Overview Lucy Park
  4. 4. ACL (Association for Computational Linguistics) - EMNLP, NAACL, COLING 등과 더불어 NLP 관련 최고 학회 - 이들끼리는 workshop이나 tutorial도 joint-call을 함 - ACL since 1962 (56th annual meeting) - EACL since 1982 - NAACL since 2000 - AACL (Asia-Pacific) since 2018! (학회는 biannual, 2020년에 시작)
  5. 5. 1551 papers submitted [1]: https://acl2018.org/2018/03/12/reviewing-statistics/ (408 accepted) 4 topics to be covered today!
  6. 6. Schedule Workshops Main conference + Booth Tutorials (메인 세션은 조만간 비디오 공개 예정)
  7. 7. Tutorials Morning: ● 100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask ● Neural Approaches to Conversational AI ● Variational Inference and Deep Generative Models ● Connecting Language and Vision to Actions Afternoon: ● Beyond Multiword Expressions: Processing Idioms and Metaphors ● Neural Semantic Parsing ● Deep Reinforcement Learning for NLP ● Multi-lingual Entity Discovery and Linking To be covered by Xiadong
  8. 8. Thursday: ● BioNLP 2018 (BioNLP) ● Cognitive Aspects of Computational Language Learning and Processing (CogCL) ● Deep Learning Approaches for Low Resource Natural Language Processing (DeepLo) ● Multilingual Surface Realization: Shared Task and Beyond (MSR) ● The 5th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA) ● Third Workshop on Computational Approaches to Linguistic Code-Switching (CALCS) ● Workshop on Machine Reading for Question Answering (MRQA) ● Workshop on Relevance of Linguistic Structure in Neural Architectures for NLP (RELNLP) Friday: ● 1st Workshop on Economics and Natural Language Processing (ECONLP) ● 3rd Workshop on Representation Learning for NLP (RepL4NLP) ● First Workshop on Computational Modeling of Human Multimodal Language (MML_Challenge) ● Sixth International Workshop on Natural Language Processing for Social Media (SocialNLP) ● The 2nd Workshop on Neural Machine Translation and Generation (NMT) ● The Seventh Named Entities Workshop (NEWS) ● Workshop for NLP Open Source Software (NLPOSS) Workshops 7 out of 15 recurring workshops
  9. 9. Thursday: ● BioNLP 2018 (BioNLP): Room 207, MCEC ● Cognitive Aspects of Computational Language Learning and Processing (CogCL) ● Deep Learning Approaches for Low Resource Natural Language Processing (DeepLo) ● Multilingual Surface Realization: Shared Task and Beyond (MSR) ● The 5th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA) ● Third Workshop on Computational Approaches to Linguistic Code-Switching (CALCS) ● Workshop on Machine Reading for Question Answering (MRQA) ● Workshop on Relevance of Linguistic Structure in Neural Architectures for NLP (RELNLP) Friday: ● 1st Workshop on Economics and Natural Language Processing (ECONLP) ● 3rd Workshop on Representation Learning for NLP (RepL4NLP) ● First Workshop on Computational Modeling of Human Multimodal Language (MML_Challenge) ● Sixth International Workshop on Natural Language Processing for Social Media (SocialNLP) ● The 2nd Workshop on Neural Machine Translation and Generation (NMT) ● The Seventh Named Entities Workshop (NEWS) ● Workshop for NLP Open Source Software (NLPOSS) Workshops To be covered by Minjoon
  10. 10. Booth
  11. 11. Best papers Short: ● Know What You Don’t Know: Unanswerable Questions for SQuAD. Pranav Rajpurkar, Robin Jia and Percy Liang ● ‘Lighter’ Can Still Be Dark: Modeling Comparative Color Descriptions. Olivia Winn and Smaranda Muresan Long: ● Finding syntax in human encephalography with beam search. John Hale, Chris Dyer, Adhiguna Kuncoro and Jonathan Brennan. ● Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information. Sudha Rao and Hal Daumé III. ● Let’s do it “again”: A First Computational Approach to Detecting Adverbial Presupposition Triggers. Andre Cianflone,* Yulan Feng,* Jad Kabbara* and Jackie Chi Kit Cheung. (* equal contribution) 새로운 태스크 + 새로운 데이터
  12. 12. Know what you don’t know: Unanswerable Questions for SQuAD 배경: MRC (machine reading comprehension) 문제: Unanswerable questions in MRC 제안: SQuAD 2.0 - 50,000 unanswerable questions written adversarially by crowdworkers - a model with 86% F1 on SQuAD 1.1 achieves only 66% F1 on SQuAD 2.0
  13. 13. Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information 문제: Make a machine generate questions - 기존: Reading comprehension - 제안: Find missing information 데이터: 77k Unix-related posts from StackExchange 방법: 1. p와 유사한 10개의 질문에서 q 후보 추출 2. 각 q에 적합한 a 도출 3. p가 a로 개선된 정도(EVPI)로 q 랭킹 p q a
  14. 14. Best demo: uroman ● Romanization for 290 langs ● Unicode 기반이지만 직접 설계한 1088개의 룰 추가 ● https://www.isi.edu/~ulf/uroman.html
  15. 15. Lifetime Achievement Award * https://aclweb.org/aclwiki/ACL_Lifetime_Achievement_Award_Recipients “Algorithms like LSTM and RNN may work in practice. But do they work in theory?” ● Mark Steedman (U. Edinburgh) ● Known for work on CCG ○ Combinatory Categorial Grammar ○ Phrase structure grammar의 일종
  16. 16. Tutorials Xiaodong Gu
  17. 17. Variational Inference and Deep Generative Models Let X and Z be random variables, where X is our observed data and Z is a latent variable. A generative model is any model that defines their joint distributions: p(x, z) Our goal: estimate the posterior over latent variables p(z|x) which is often intractable! Our solution: approximate p(z|x) with q(z;θ) which is computable! L = log p(x) >= - KL(q(z;θ)||p(z|x)) + log p(x) ELBO: VI turns inference into optimization
  18. 18. Variational Autoencoder Generative Network Gradient Inference Network Gradient Solution - VAE Again requires approximation by sampling Generative NetworkInference Network
  19. 19. Deep Reinforcement Learning for NLP - RL introduction - Fundamentals and Overview - Deep Reinforcement Learning for Dialogue - Challenges
  20. 20. Fundamentals and Overview Why use RL in NLP? 1. Learning to search and reason 2. Instead of minimizing the surrogate loss (e.g., XE, hinge loss), optimize the end metric (e.g., BLEU, ROUGE) directly. 3. Select the right (unlabeled) data. 4. Back-propagate the reward to update the model.
  21. 21. Deep Reinforcement Learning for Dialogue Sequence-to-Sequence Model for Dialogue Problem1: Repetitive responses Shut up! -> You shut up! -> You shut up!-> …... Problem2: Short-sighted conversation decisions How old are you? -> I’m 16 -> 16? -> I don’t know what you are talking about -> you don’t know what you’re saying -> I don’t know ……->...... Reinforcement Learning State: message encoding Action: response decoding Rewards: - Ease of answering r(response)=-log(dull utt | resp) - Information Flow r2 = - log sigmoid(cos(s1 ,s2 )) - Meaningfulness r = log p(resp|message) + log p (message|resp)
  22. 22. Challenges NLP problems that presents new challenges to RL • An unbounded action space defined by natural language • Dealing with combinatorial actions and external knowledges • Learning reward functions for NLG RL problems that are particularly relevant to NLP • Sample complexity • Model-based vs. model free RL • Acquiring rewards
  23. 23. Main conference (1) - Semantic parsing Soonmin Bae
  24. 24. Intro Semantic Parsing : mapping natural language utterances to machine interpretable representations (e.g., executable queries or logical forms) Semantic Role Labeling : shallow semantic parsing, assigning labels to words (e.g., predicate-argument structure or “who does what to who, when, and where”) Semantic Parsing @ ACL 2018 3 Talk Sessions for Semantic Parsing (2A, 6A, 7A) : 12 papers 2 Poster Sessions for Semantics (1B, 3C) : 15 + 14 papers ⇒ half SP/SRL, half embeddings 1 Tutorial Session for Neural Semantic Parsing 2 Talk Sessions for Semantics (5A, 8A) : 8 papers ⇒ mostly embeddings, but a few SRL 2 Talk Sessions for Parsing (3F, 4F) : 8 papers ⇒ grammars/syntax + dependency parsing 1 Poster Session for Morphology, Tagging, Parsing (3F)
  25. 25. Improving Text-to-SQL Evaluation Methodology [1/3] - Intro https://arxiv.org/abs/1806.09029 “How to evaluate Text-to-SQL” must be improved. - DBs are not realistic - question-based split vs query-based split Template-based, Slot-filling Baseline must fail but works for question-based split. 1B
  26. 26. Improving Text-to-SQL Evaluation Methodology [2/3] - DB Issues Issue 1: Human-written questions require more complex queries than automatically generated data (e.g., joins, nesting) Issue 2: Training and test sets should be splitted based on the SQL query instead of question. 1B
  27. 27. Improving Text-to-SQL Evaluation Methodology [3/3] - Template-based, Slot-filling Baseline must fail, but works Oracle Entities : To provide accurate anonymization, we annotated query variables using a combination of automatic and manual processing. DB from NLP communities DB from DB communities 1B
  28. 28. Coarse-to-Fine Decoding for Neural Semantic Parsing [1/2] - Honourable Mention for Best Paper https://arxiv.org/abs/1805.04793 Structure-aware neural architecture 1. Generate a rough sketch of input utterance’s meaning, where low-level info is glossed over 2. Experimental results on four datasets 2A
  29. 29. Coarse-to-Fine Decoding for Neural Semantic Parsing [2/2] - Honourable Mention for Best Paper 2A
  30. 30. Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling https://arxiv.org/abs/1805.04787 End-to-end approach for jointly predicting all predicates, arguments spans, and the relations b/w them 5A
  31. 31. Large-Scale QA-SRL Parsing [1/3] - Honourable Mention for Best Paper https://arxiv.org/abs/1805.05377 Contributions 1. Scalable crowdsourcing QA-SRL 2. New models for QA-SRL parser : span detection → question generation Challenges 1. Scale up QA-SRL data annotation 2. Train a QA-SRL Parser 3. Improve Recall Results - 133,479 verbs from 64,018 sentences - across 3 domains (Wikipedia, Wikinews, Science) - 265,140 question-answer pairs in 9 days 7A
  32. 32. Large-Scale QA-SRL Parsing [2/3] - Honourable Mention for Best Paper Crowdsourcing pipeline 1. Generation 1.1. Write QA-SRL questions for each verb ⇒ Auto-suggest complete questions reduces # strokes 1.2. Highlight answers in the sentence 2. Validation 2.1. Answer each question or mark it invalid Model 7A
  33. 33. Large-Scale QA-SRL Parsing [3/3] 7A
  34. 34. Main conference (2) - Dialogue Kyungduk Kim
  35. 35. Intro
  36. 36. Neural Approaches - Neural Response Generation - Dialog Policy Optimization - Personalized Dialog System Discourse Analysis Dialog System @ ACL 2018 Dialog System Oral presentation: 12 long, 4 short papers Poster presentation: 16 papers (Sigdial 2018 is co-located) Question Answering Oral presentation: 8 long, 4 short papers Poster presentation: 11+ papers Intro
  37. 37. Intro ● 결론 ○ Task-Oriented 대화 시스템에서 neural approaches를 사용하기 위한 방법론이 활발히 연구 중 ○ Amazon Alexa와 같은 산업체에서 발표한, (Clova에도) 실제 적용가능 할만한 연구도 눈에 띔 ○ 대화 시스템 평가에 대한 방법론은 아직도 정리 중 ■ Perplexity등을 사용하기도 하나, mTurk등을 통해 사람이 직접 성능을 확인 ■ ConvAI2 등의 shared task 개최에서도 mTurk을 통한 evaluation이 진행됨
  38. 38. Exemplar Encoder-Decoder for Neural Conversation Generation Neural Conversation Generation using example contexts and their responses
  39. 39. Exemplar Encoder-Decoder for Neural Conversation Generation
  40. 40. Efficient Large-Scale Neural Domain Classification with Personalized Attention ● Proposes finding proper domains without explicit invocation name of domain ● Handling 40,000+ 3rd party skills.
  41. 41. Efficient Large-Scale Neural Domain Classification with Personalized Attention
  42. 42. Efficient Large-Scale Neural Domain Classification with Personalized Attention Evaluation Week: 100K+ utterances from 600K+ users across 1,000+ Alexa skills Generated from utterances with explicit invocation patterns. e.g.) “Ask TAXI to get me a ride” → (get me a ride, TAXI) MTurk: 20K+ human-paraphrased utterances of randomly sampled commands from test set of Weak data e.g.) “get me a ride” → I need a ride, can I get a taxi, order a car for me, … Evaluate performance with natural language
  43. 43. Efficient Large-Scale Neural Domain Classification with Personalized Attention Baseline model 1-bit model (user-enabled domain bitmap) Attention model (user-enabled domain attention)
  44. 44. Improving Slot Filling in Spoken Language Understanding with Joint Pointer and Attention
  45. 45. Improving Slot Filling in Spoken Language Understanding with Joint Pointer and Attention
  46. 46. Improving Slot Filling in Spoken Language Understanding with Joint Pointer and Attention
  47. 47. Main conference (3) - Machine translation Zae Myung Kim
  48. 48. Intro ● Machine translation ○ Translating a sequence of tokens in lang. A into that of lang. B ○ All MT papers @ ACL 2018 focused on neural MT (NMT) as opposed to statistical MT (SMT) ● 11 long, 8 shorts, 1 demo, 1 NMT workshop (16 papers) ○ Main conf. papers (see next page) ○ Papers @ NMT workshop ○ Talks @ NMT workshop ● Trends in MT @ ACL 2018 ○ Less exploration on DL architectures than last year, but more NLP-oriented ○ Linguistic structure, document-level translation, data augmentation, efficient computation, domain adaptation, handling inadequate resources, and analysis of models
  49. 49. List of MT related papers (main conf.) 1. Unsupervised Neural Machine Translation with Weight Sharing. Yang et al. [link] 2. Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. Kudo. [link] 3. Forest-Based Neural Machine Translation. Ma et al. [link] 4. Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings. Kuang et al. [link] 5. Context-Aware Neural Machine Translation Learns Anaphora Resolution. Voita et al. [link] 6. The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation. Chen et al. [link] 7. Towards Robust Neural Machine Translation. Cheng et al. [link] 8. Document Context Neural Machine Translation with Memory Networks. Maruf and Haffari. [link] 9. A Stochastic Decoder for Neural Machine Translation. Schulz et al. [link] 10. Accelerating Neural Transformer via an Average Attention Network. Zhang et al. [link] 11. Are BLEU and Meaning Representation in Opposition?. Cífka and Bojar. [link] 12. A Simple and Effective Approach to Coverage-Aware Neural Machine Translation. Li et al. [link] 13. Compositional Representation of Morphologically-Rich Input for Neural Machine Translation. Ataman and Federico. [link] 14. Extreme Adaptation for Personalized Neural Machine Translation. Michel and Neubig. [link] 15. Learning from Chunk-based Feedback in Neural Machine Translation. Petrushkov et al. [link] 16. Sparse and Constrained Attention for Neural Machine Translation. Malaviya et al. [link] 17. Bag-of-Words as Target for Neural Machine Translation. Ma et al. [link] 18. Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation. Shu and Nakayama. [link] 19. Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation. Zaremoodi et al. [link] 20. Marian: Fast Neural Machine Translation in C++. Junczys-Dowmunt et al. [link]
  50. 50. Document-level translation Context-Aware Neural Machine Translation Learns Anaphora Resolution. Voita et al. [link] ● OpenSubtitles 2018; En-Ru; takes the previous subtitle as “context” ● Modifies Transformer’s encoder part to incorporate the context ● Model attends to the context when translating ambiguous pronouns
  51. 51. Data augmentation Towards Robust Neural Machine Translation. Cheng et al. [link] ● Small perturbations in an input can dramatically deteriorate its translation results ● Perturb input by 1) randomly replacing words, or 2) adding Gaussian noise to word embedding ● Maintain the consistency of behaviors through the NMT model for the source sentence x and its perturbed counterpart x’ through adversarial learning for the perturbation-invariant encoder
  52. 52. Domain adaptation Extreme Adaptation for Personalized Neural Machine Translation. Michel and Neubig. [link]
  53. 53. Main conference (4) - Summarization Hye-Jin Min
  54. 54. Overview of Text Summarization ● TASK ○ Extractive summarization / Abstractive Summarization ● DATA ○ CNN/Daily Mail data sets / English giga word corpus ● ISSUES ○ OOV (Out-Of-Vocabulary) / Repetition ● Techniques ○ Seq2seq model + α
  55. 55. 3 selected papers Model TASK DATA Issues Techniques Unified Model Abstractive Summarization CNN/Daily Mail data sets advantages from extractor and abstractor Unified model & Inconsistency mechanism SWAP-NET Extractive summarization CNN/Daily Mail data sets the interaction of key words and salient sentences 2-level Pointer Network Re 3 Sum Abstractive Summarization English giga word corpus Repetition Short length IR-approach + seq2seq
  56. 56. Unified Model ● Combining sentence-level and word-level attentions to take advantage of both extractive and abstractive summarization approaches
  57. 57. Unified Model ● Combining sentence-level and word-level attentions to take advantage of both extractive and abstractive summarization approaches
  58. 58. SWAP-NET ● Sentences and Words from Alternating Pointer Network ● 2 level pointer network ○ The sentence-level ○ The word-level ● Switch mechanism
  59. 59. Retrieve, Rerank and Rewrite (Re3Sum) ● Soft Template Based Neural Summarization ○ to improve the readability and stability of seq2seq summarization systems ● fuse the popular IR-based and seq2seq-based summarization systems
  60. 60. Retrieve, Rerank and Rewrite (Re3Sum) ● Jointly Rerank and Rewrite ○ Bi-directional RNN encoder to read input x and template r ○ soft template r resembles the actual summary y∗ as much as possible
  61. 61. References ● Unified Model: A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss ○ https://arxiv.org/abs/1805.06266 ○ https://hsuwanting.github.io/unified_summ/ ● SWAP-NET: Extractive Summarization with SWAP-NET: Sentences and Words from Alternating Pointer Networks ○ http://aclweb.org/anthology/P18-1014 ○ https://github.com/aishj10/swap-net/tree/master/swap-net-summer ● Re 3 Sum: Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization ○ http://aclweb.org/anthology/P18-1015 ○ http://www4.comp.polyu.edu.hk/~cszqcao/
  62. 62. 학생들끼리 AI 워크샵 주최하기 Minjoon Seo
  63. 63. MRC vs MRQA?
  64. 64. 1. 제안
  65. 65. 워크샵 주최? 우린 학생인데?
  66. 66. 2. Pivoting
  67. 67. MRQA Interpretability?
  68. 68. QA에 대해 높아진 관심도
  69. 69. WMT 급으로 만들어 보자!
  70. 70. MRQA Interpretability → MRQA
  71. 71. 3. 사람 모으기
  72. 72. Steering & Speakers
  73. 73. Program Committee: 35
  74. 74. 4. 스폰서 구하기
  75. 75. 5. 제안서 제출
  76. 76. 10월 22일 제출, 11월 17일 결과
  77. 77. 6. 논문제출 받기, 결정하기
  78. 78. Encouraging cross-submission
  79. 79. ~25 제출, ~50% 합격
  80. 80. 6. 갑작스런 상황들
  81. 81. 바로 전주에도 스피커가 못온다고...
  82. 82. 그리고 ACL의 아주 느린 답변
  83. 83. 7. 주최
  84. 84. 8. 내년에도?
  85. 85. 새로운 사람들? 빠질 사람들?
  86. 86. Challenge 추가?
  87. 87. Thank you!

×