Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Beyond search 
queries 
Jano Suchal 
searchd.co
Search 
as seen by developers 
{ 
"query": { 
"query_string": { 
"query": "elasticsearch book" 
} 
} 
} 
return response.h...
Search 
as experienced by users 
query: elasticsarch 
Typo in query. 
No results. 
query: elasticsearch Too many hits. 
No...
Measuring 
search quality
Cpt. Obvious: 
“Hits, clicks and order 
do matter.”
Accurately interpreting clickthrough 
data as implicit feedback 
Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembroo...
Accurately interpreting clickthrough 
data as implicit feedback
Search quality metrics 
● Mean Average Precision @ N 
○ probability of target result being in top N items 
● Mean Reciproc...
Search KPIs 
● CTR trend 
● # of queries w/o results or clicks 
● # of searches per session 
● Search engine latency
Search quality 
optimization
Optimizing search engines using 
clickthrough data 
Thorsten Joachims. Optimizing search engines using clickthrough data. ...
Optimizing search engines using 
clickthrough data
Query chains: learning to rank from 
implicit feedback 
Filip Radlinski and Thorsten 
Joachims. Query chains: learning 
to...
Fighting Search Engine Amnesia: 
Reranking Repeated Results 
In this paper, we observed that the same results are often sh...
searchd.co 
Search Analytics
searchd.co dashboard
A/B testing
A/B testing lists
A/B testing 
A B
A/B testing with interleaving 
A B
Interleaving & scoring 
● Balanced 
● Team Draft 
● Probabilistic 
● Binary preference 
● Linear rank difference 
● Invers...
A/B testing with interleaving
A/B testing with interleaving
A/B testing with interleaving 
● Pros 
○ Lower risk of loosing conversions 
● Cons 
○ Harder to interpret 
○ Harder to imp...
searchd.co 
Search Analytics 
● Identify and fix key search problems 
● KPIs for site search 
● Actionable tips for search...
Bad search experience is a lost 
opportunity. Let's fix it. 
searchd.co 
Search Analytics 
www.searchd.co 
info@searchd.co
Beyond search queries
Beyond search queries
Beyond search queries
Beyond search queries
Próxima SlideShare
Cargando en…5
×

Beyond search queries

976 visualizaciones

Publicado el

Slides from Banalytics talk

Publicado en: Tecnología
  • Have you ever used the help of ⇒ www.HelpWriting.net ⇐? They can help you with any type of writing - from personal statement to research paper. Due to this service you'll save your time and get an essay without plagiarism.
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Beyond search queries

  1. 1. Beyond search queries Jano Suchal searchd.co
  2. 2. Search as seen by developers { "query": { "query_string": { "query": "elasticsearch book" } } } return response.hits.hits
  3. 3. Search as experienced by users query: elasticsarch Typo in query. No results. query: elasticsearch Too many hits. Not relevant. query: elasticsearch book Click! Success! Or?
  4. 4. Measuring search quality
  5. 5. Cpt. Obvious: “Hits, clicks and order do matter.”
  6. 6. Accurately interpreting clickthrough data as implicit feedback Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in Information retrieval, SIGIR ’05, pages 154–161, New York, NY, USA, 2005. ACM.
  7. 7. Accurately interpreting clickthrough data as implicit feedback
  8. 8. Search quality metrics ● Mean Average Precision @ N ○ probability of target result being in top N items ● Mean Reciprocal Rank ○ 1 / rank of target result ● Normalized Discounted Cumulative Gain ● Expected Reciprocal Rank
  9. 9. Search KPIs ● CTR trend ● # of queries w/o results or clicks ● # of searches per session ● Search engine latency
  10. 10. Search quality optimization
  11. 11. Optimizing search engines using clickthrough data Thorsten Joachims. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’02, pages 133–142, New York, NY, USA, 2002. ACM.
  12. 12. Optimizing search engines using clickthrough data
  13. 13. Query chains: learning to rank from implicit feedback Filip Radlinski and Thorsten Joachims. Query chains: learning to rank from implicit feedback. In KDD ’05: Proceeding of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pages 239–248, New York, NY, USA, 2005. ACM.
  14. 14. Fighting Search Engine Amnesia: Reranking Repeated Results In this paper, we observed that the same results are often shown to users multiple times during search sessions. We showed that there are a number of effects at play, which can be leveraged to improve information retrieval performance. In particular, previously skipped results are much less likely to be clicked, and previously clicked results may or may not be re-clicked depending on other factors of the session. Milad Shokouhi, Ryen W. White, Paul Bennett, and Filip Radlinski. Fighting search engine amnesia: reranking repeated results. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’13, pages 273–282, New York, NY, USA, 2013. ACM.
  15. 15. searchd.co Search Analytics
  16. 16. searchd.co dashboard
  17. 17. A/B testing
  18. 18. A/B testing lists
  19. 19. A/B testing A B
  20. 20. A/B testing with interleaving A B
  21. 21. Interleaving & scoring ● Balanced ● Team Draft ● Probabilistic ● Binary preference ● Linear rank difference ● Inverse rank difference
  22. 22. A/B testing with interleaving
  23. 23. A/B testing with interleaving
  24. 24. A/B testing with interleaving ● Pros ○ Lower risk of loosing conversions ● Cons ○ Harder to interpret ○ Harder to implement
  25. 25. searchd.co Search Analytics ● Identify and fix key search problems ● KPIs for site search ● Actionable tips for search tuning ● Safe A/B testing ● Easy setup ● In Beta, sending out invites
  26. 26. Bad search experience is a lost opportunity. Let's fix it. searchd.co Search Analytics www.searchd.co info@searchd.co

×