We introduce a new method for unsupervised knowledge-based word sense disambiguation (WSD) based on a resource that links two types of sense-aware lexical networks: one is induced from a corpus using distributional semantics, the other is manually constructed. The combination of two networks reduces the sparsity of sense representations used for WSD. We evaluate these enriched representations within two lexical sample sense disambiguation benchmarks. Our results indicate that (1) features extracted from the corpus-based resource help to significantly outperform a model based solely on the lexical resource; (2) our method achieves results comparable or better to four state-of-the-art unsupervised knowledge-based WSD systems including three hybrid systems that also rely on text corpora. In contrast to these hybrid methods, our approach does not require access to web search engines, texts mapped to a sense inventory, or machine translation systems. See the full paper at: http://www.aclweb.org/anthology/W/W17/W17-1909.pdf Panchenko A., Faralli S., Ponzetto S. P., and Biemann C. (2017): Using Linked Disambiguated Distributional Networks for Word Sense Disambiguation. In Proceedings of the Workshop on Sense, Concept and Entity Representations and their Applications (SENSE) co-located with the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL'2017). Valencia, Spain. Association for Computational Linguistics

1. Universität Hamburg, – Name of Presenter: Titel of Talk/Slideset (Version: 27.6.2014) – Slide 1
Using Linked Disambiguated Distributional
Networks for Word Sense Disambiguation
Chris Biemann
biemann@informatik.uni-hamburg.de
Alexander Panchenko
panchenko@informatik.uni-hamburg.de
Stefano Faralli
stefano@informatik.uni-mannheim.de
Simone Paolo Ponzetto
simone@informatik.uni-mannheim.de
Dmitry Ustalov
dmitry.ustalov@urfu.ru
Presented by:

2. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 2
Contribution
An unsupervised knowledge-based approach to WSD based on
the Hybrid Aligned Resource (HAR) by Faralli et al. (2016):
• Learning sparse distributional sense representations from text;
• Linking them to the language resource (LR);
• Expanding sense representations of the LR.
Combines distributional and knowledge-based sense
representations.
Faralli S., Panchenko A., Biemann C., and Ponzetto S.P. (2016). Linked disambiguated distributional semantic networks.
In International Semantic Web Conference (ISWC’2016), pages 56–64, Kobe, Japan. Springer.

3. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 3
Contribution
An unsupervised knowledge-based approach to WSD based on
the Hybrid Aligned Resource (HAR) by Faralli et al. (2016):
• Learning sparse distributional sense representations from text;
• Linking them to the language resource (LR);
• Expanding sense representations of the LR.
Combines distributional and knowledge-based sense
representations.
The method requires no linking of texts to a sense inventory and
thus can be applied to large text collections.
Faralli S., Panchenko A., Biemann C., and Ponzetto S.P. (2016). Linked disambiguated distributional semantic networks.
In International Semantic Web Conference (ISWC’2016), pages 56–64, Kobe, Japan. Springer.

4. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 4
Linked Disambiguated Distributional Networks for WSD
Distributional corpus-derived information
- Hybrid Aligned Resource (HAR) by Faralli et al. (2016)
- Distributional sense representations linked to a lexical resource (WordNet, ...)
- Sample entries of the HAR for the words “mouse” and “keyboard”.

5. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 5
Linked Disambiguated Distributional Networks for WSD
- Hybrid Aligned Resource (HAR) by Faralli et al. (2016)
- Distributional sense representations linked to a lexical resource (WordNet, ...)
- Sample entries of the HAR for the words “mouse” and “keyboard”.
Information from the knowledge base

6. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 6
Construction of the Hybrid Aligned Resource (HAR):
1. Building a Distributional Thesaurus (DT).
2. Word Sense Induction.
3. Labeling Word Senses with Hypernyms.
4. Disambiguation of Related Terms and Hypernyms.
5. Retrieval of Context Clues.
Linked Disambiguated Distributional Networks for WSD

7. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 7
Construction of the Hybrid Aligned Resource (HAR):
1. Building a Distributional Thesaurus (DT).
2. Word Sense Induction.
3. Labeling Word Senses with Hypernyms.
4. Disambiguation of Related Terms and Hypernyms.
5. Retrieval of Context Clues.
HAR Datasets used in our experiment (Faralli et al., 2016):
− news:
• a 100 million sentence news corpus
• average polysemy of 2.3
− wiki:
• a 35 million sentence Wikipedia corpus
• average polysemy of 1.8
Linked Disambiguated Distributional Networks for WSD

8. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 8
Using the Hybrid Aligned Resource in Word Sense Disambiguation:
− WordNet: this baseline model relies solely on the WordNet:
• Synonyms
• Glosses
• Target synset + synsets directly connected to it
− WordNet + Related: augments the WordNet-based representation
with related terms from the corpus-induced word senses.
− WordNet + Related + Context: all features of the previous model
plus context clues obtained by aggregating features of the sense
cluster words.
Linked Disambiguated Distributional Networks for WSD

9. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 9
Linked Disambiguated Distributional Networks for WSD
- The third sense of the word “disk” in the WordNet:
- The initial WordNet-based sense representation vs
- The enriched via linking to HAR sense representation
- Enriched with related words from the HAR

10. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 10
Evaluation: Research Questions
RQ 1:
Does the enriched sense representation improve WSD performance
compared to the original WordNet-based representations?

11. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 11
Evaluation: Research Questions
RQ 1:
Does the linked sense representation improve WSD performance
compared to the original WordNet-based sense representation?
RQ 2:
What is the quality of our approach compared to the SOTA
unsupervised knowledge-based WSD systems?

12. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 12
Evaluation: Dataset and Evaluation Metrics
SemEval-2007 Task 16 “Evaluation of wide-coverage knowledge
resources” (Cuadros and Rigau, 2007):
- specifically designed for evaluating the impact of lexical
resources on the WSD performance

13. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 13
Evaluation: Dataset and Evaluation Metrics
SemEval-2007 Task 16 “Evaluation of wide-coverage knowledge
resources” (Cuadros and Rigau, 2007):
- specifically designed for evaluating the impact of lexical
resources on the WSD performance
- the task dataset is based on the WordNet-labeled sentences from:
- Senseval-3 (Mihalcea et al., 2004)
- SemEval-2007 Task 17 (Pradhan et al., 2007)

14. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 14
RQ1: Results
Does the linked sense representation improve WSD performance
compared to the original WordNet-based sense representation?

15. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 15
RQ1: Dataset and Evaluation Metrics
Does the linked sense representation improve WSD performance
compared to the original WordNet-based sense representation?

16. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 16
RQ2: Baselines
What is the quality of our approach compared to the SOTA
unsupervised knowledge-based WSD systems?
The state of the art unsupervised knowledge-based methods:
- WN+XWN (Cuadros and Rigau, 2007)
- WordNet + eXtend WordNet (parsing WordNet glosses)
- KnowNet (Cuadros and Rigau, 2008)
- based on snippets retrieved with a web search engine
- BabelNet (Navigli and Ponzetto, 2012)
- Wikipedia articles + WordNet synsets
- NASARI (Camacho-Collados et al., 2015):
- vector representations of senses based on Wikipedia and WordNet
- lexical or sense-based feature spaces
- The links between WordNet and Wikipedia are retrieved from BabelNet

17. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 17
RQ2: Results
What is the quality of our approach compared to the SOTA
unsupervised knowledge-based WSD systems?

18. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 18
RQ2: Results
What is the quality of our approach compared to the SOTA
unsupervised knowledge-based WSD systems?

19. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 19
Conclusions
− We presented a novel approach to knowledge-based WSD:
• Learning sparse distributional sense representations from text;
• Linking them to the language resource (LR);
• Expanding sense representations of the LR.
− Possibility to use large corpora: not limited to Wikipedia-linked
texts as in BabelNet, NASARI.

20. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 20
Conclusions
− We presented a novel approach to knowledge-based WSD:
• Learning sparse distributional sense representations from text;
• Linking them to the language resource (LR);
• Expanding sense representations of the LR.
− A possibility to use large corpora: the method is not limited to
Wikipedia-linked texts as in BabelNet, NASARI.
− RQ1: Distributional sense representations let us substantially
outperform the model based solely on the lexical resource.
− RQ2: Comparable performance to the state-of-the-art hybrid
approaches leveraging corpus-based features.

21. Universität Hamburg, – Name of Presenter: Titel of Talk/Slideset (Version: 27.6.2014) – Slide 21
We acknowledge the support of:

22. Universität Hamburg – Panchenko et al.: Using Linked Disambiguated Distributional Networks for WSD (04.04.2017) – Slide 22
References
[1] Chris Biemann and Martin Riedl. 2013. Text: Now in 2D! A Framework for Lexical Expansion with
Contextual Similarity. Journal of Language Modelling, 1(1):55–95
[2] Chris Biemann. 2006. Chinese whispers - an efficient graph clustering algorithm and its application to
natural language processing problems. In Proceedings of TextGraphs: the First Workshop on Graph
Based Methods for Natural Language Processing, pages 73–80, New York City. Association for
Computational Linguistics.
[3] Jose Camacho-Collados, Mohammad Taher Pilehvar, ´ and Roberto Navigli. 2015a. Nasari: a novel
approach to a semantically-aware representation of items. In Proceedings of the 2015 Conference of
the North American Chapter of the Association for Computational Linguistics: Human Language
Technologies, pages 567–577, Denver, Colorado. Association for Computational Linguistics.
[4] Montse Cuadros and German Rigau. 2007. Semeval- 2007 task 16: Evaluation of wide coverage
knowledge resources. In Proceedings of the Fourth International Workshop on Semantic Evaluations
(SemEval-2007), pages 81–86, Prague, Czech Republic. Association for Computational Linguistics.
[5] Montse Cuadros and German Rigau. 2008. KnowNet: Building a large net of knowledge from the web.
In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008),
pages 161–168, Manchester, UK, August. Coling 2008 Organizing Committee.
[6] Stefano Faralli, Alexander Panchenko, Chris Biemann, and Simone P. Ponzetto. 2016. Linked
disambiguated distributional semantic networks. In International Semantic Web Conference
(ISWC’2016), pages 56–64, Kobe, Japan. Springer.
[7] Roberto Navigli and Simone Paolo Ponzetto. 2012. Babelnet: The automatic construction, evaluation
and application of a wide-coverage multilingual semantic network. Artificial Intelligence, 193:217– 250.