Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

TREC 2016: Looking Forward Panel

478 visualizaciones

Publicado el

Opening statement at the "Looking forward" panel at the 25 years of TREC celebration event, Nov 15th, 2016.

Webcast to appear within a week:

Publicado en: Ciencias
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

TREC 2016: Looking Forward Panel

  1. 1. Looking Forward Arjen P. de Vries Gaithersburg MD, November 15th, 2016
  2. 2. Q: “TREC Anniversary”
  3. 3. Top Result:  50 years of Star Trek (Article on the Verge about Facebook Like buttons)
  4. 4. Science Fiction  Defining a TREC task or a track is like time-travel in Back to the Future Note to the audience: that is just 74 characters You could even add the hashtag #TREC #TRECCelebrations and my Twitter handle @arjenpdevries
  5. 5. Better Search – “Deep Personalization”  “Even more broadly than trying to get people the right content based on their context, we as a community need to be thinking about how to support people through the entire search experience.” Jaime Teevan on “Slow Search”  Search as a dialogue My first journal paper: De Vries, Van der Veer and Blanken: Let’s talk about it: dialogues with multimedia databases (1998)
  6. 6. Moving Forward  Elements of the “Slow Search movement” at TREC today: - Sessions - Tasks - Dynamic domains - Total recall - Complex Answer Retrieval (new!)
  7. 7. Missing from TREC!  Access to rich personal data including email, browsing history, documents read and contents of the user’s home directory…
  8. 8. Trade log data! IR-809: (2011) Feild, H., Allan, J. and Glatt, J., "CrowdLogging: Distributed, private, and anonymous search logging," Proceedings of the International Conference on Research and Development in Information Retrieval (SIGIR'11), pp. 375-384. [View bibtex] We describe an approach for distributed search log collection, storage, and mining, with the dual goals of preserving privacy and making the mined information broadly available. [..] The approach works with any search behavior artifact that can be extracted from a search log, including queries, query reformulations, and query- click pairs.
  9. 9. Open challenges  How to select the part of your log data you are willing to trade?  How to estimate the value of this log data?  And a social challenge, not so much scientific: How to get people to participate?
  10. 10. Branding
  11. 11. Branding (NL)
  12. 12. The TREC Brand  A community that creates reusable test collections
  13. 13. Extra Slides
  14. 14. Reproducibility vs. Representativeness  Increasing representativeness of a TREC task should not come at the cost of sacrificing reproducibility (104 characters ) Samar, T., Bellogín, A. & de Vries, A.P. Inf Retrieval J (2016) 19: 230. doi: 10.1007/s10791-015-9276-9
  15. 15. Baltimore
  16. 16. Baltimore  Title query of TREC topic 478 for the information need “Who is the mayor of Baltimore”  “The honest conclusion of this year’s evaluation should be that we underestimated the problem of handling Web data. Surprising is the performance of the title-only queries doing better than queries including description or even narrative. It seems that the web-track topics are really different from the previous TREC topics in the ad- hoc task, for which we never weighted title terms different from description or narrative.” (Quote from the CWI TREC-9 paper)