Soumettre la recherche
Mettre en ligne
Food portfolio
•
0 j'aime
•
133 vues
SaskiaLelieveld
Suivre
Art & Photos
Signaler
Partager
Signaler
Partager
1 sur 32
Télécharger maintenant
Télécharger pour lire hors ligne
Recommandé
Pantry items
Pantry
Pantry
GraceDeWald
Oledo fête de la nativité
Oledo fête de la nativité
vadrouilleur
Zakelijk Oostland september 2011
Zakelijk Oostland september 2011
alexvanholland
Zakelijk Oostland Juni 2011 Gezamenlijke uitgave van Nieuwsblad De Heraut en Telstar Uitgeverij, waarin de bedrijven en ondernemers centraal staan.
Zakelijk Oostland Juni 2011
Zakelijk Oostland Juni 2011
alexvanholland
Gabriella Kazai, Emine Yilmaz, Nick Craswell, and S.M.M. Tahaghoghi. 2013. User intent and assessor disagreement in web search evaluation. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management (CIKM '13). ACM, New York, NY, USA, 699-708. DOI=10.1145/2505515.2505716 http://doi.acm.org/10.1145/2505515.2505716 Preference based methods for collecting relevance data for information retrieval (IR) evaluation have been shown to lead to better inter-assessor agreement than the traditional method of judging individual documents. However, little is known as to why preference judging reduces assessor disagreement and whether better agreement among assessors also means better agreement with user satisfaction, as signalled by user clicks. In this paper, we examine the relationship between assessor disagreement and various click based measures, such as click preference strength and user intent similarity, for judgments collected from editorial judges and crowd workers using single absolute, pairwise absolute and pairwise preference based judging methods. We find that trained judges are significantly more likely to agree with each other and with users than crowd workers, but inter-assessor agreement does not mean agreement with users. Switching to a pairwise judging mode improves crowdsourcing quality close to that of trained judges. We also find a relationship between intent similarity and assessor-user agreement, where the nature of the relationship changes across judging modes. Overall, our findings suggest that the awareness of different possible intents, enabled by pairwise judging, is a key reason of the improved agreement, and a crucial requirement when crowdsourcing relevance data.
Kazai cikm2013-intent
Kazai cikm2013-intent
Gabriella Kazai
Το Πείραμα στην τάξη μας!!
Το Πείραμα στην τάξη μας!!
antonis11
зөвлөгөө 112
зөвлөгөө 112
nsaruul75
F* Alarm - Alarm Clock for Android
F* Alarm - Alarm Clock for Android
F* Alarm - Alarm Clock for Android
Huy Nguyen Quang
Recommandé
Pantry items
Pantry
Pantry
GraceDeWald
Oledo fête de la nativité
Oledo fête de la nativité
vadrouilleur
Zakelijk Oostland september 2011
Zakelijk Oostland september 2011
alexvanholland
Zakelijk Oostland Juni 2011 Gezamenlijke uitgave van Nieuwsblad De Heraut en Telstar Uitgeverij, waarin de bedrijven en ondernemers centraal staan.
Zakelijk Oostland Juni 2011
Zakelijk Oostland Juni 2011
alexvanholland
Gabriella Kazai, Emine Yilmaz, Nick Craswell, and S.M.M. Tahaghoghi. 2013. User intent and assessor disagreement in web search evaluation. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management (CIKM '13). ACM, New York, NY, USA, 699-708. DOI=10.1145/2505515.2505716 http://doi.acm.org/10.1145/2505515.2505716 Preference based methods for collecting relevance data for information retrieval (IR) evaluation have been shown to lead to better inter-assessor agreement than the traditional method of judging individual documents. However, little is known as to why preference judging reduces assessor disagreement and whether better agreement among assessors also means better agreement with user satisfaction, as signalled by user clicks. In this paper, we examine the relationship between assessor disagreement and various click based measures, such as click preference strength and user intent similarity, for judgments collected from editorial judges and crowd workers using single absolute, pairwise absolute and pairwise preference based judging methods. We find that trained judges are significantly more likely to agree with each other and with users than crowd workers, but inter-assessor agreement does not mean agreement with users. Switching to a pairwise judging mode improves crowdsourcing quality close to that of trained judges. We also find a relationship between intent similarity and assessor-user agreement, where the nature of the relationship changes across judging modes. Overall, our findings suggest that the awareness of different possible intents, enabled by pairwise judging, is a key reason of the improved agreement, and a crucial requirement when crowdsourcing relevance data.
Kazai cikm2013-intent
Kazai cikm2013-intent
Gabriella Kazai
Το Πείραμα στην τάξη μας!!
Το Πείραμα στην τάξη μας!!
antonis11
зөвлөгөө 112
зөвлөгөө 112
nsaruul75
F* Alarm - Alarm Clock for Android
F* Alarm - Alarm Clock for Android
F* Alarm - Alarm Clock for Android
Huy Nguyen Quang
Portfolio Saskia Lelieveld
Portfolio Saskia Lelieveld
SaskiaLelieveld
Amberes 8590
Amberes 8590
SaskiaLelieveld
Amberes 8552
Amberes 8552
SaskiaLelieveld
Amberes 8507
Amberes 8507
SaskiaLelieveld
Amberes 8469
Amberes 8469
SaskiaLelieveld
Amberes 8414
Amberes 8414
SaskiaLelieveld
Amberes 8286
Amberes 8286
SaskiaLelieveld
Amberes 8188
Amberes 8188
SaskiaLelieveld
Contenu connexe
Plus de SaskiaLelieveld
Portfolio Saskia Lelieveld
Portfolio Saskia Lelieveld
SaskiaLelieveld
Amberes 8590
Amberes 8590
SaskiaLelieveld
Amberes 8552
Amberes 8552
SaskiaLelieveld
Amberes 8507
Amberes 8507
SaskiaLelieveld
Amberes 8469
Amberes 8469
SaskiaLelieveld
Amberes 8414
Amberes 8414
SaskiaLelieveld
Amberes 8286
Amberes 8286
SaskiaLelieveld
Amberes 8188
Amberes 8188
SaskiaLelieveld
Plus de SaskiaLelieveld
(8)
Portfolio Saskia Lelieveld
Portfolio Saskia Lelieveld
Amberes 8590
Amberes 8590
Amberes 8552
Amberes 8552
Amberes 8507
Amberes 8507
Amberes 8469
Amberes 8469
Amberes 8414
Amberes 8414
Amberes 8286
Amberes 8286
Amberes 8188
Amberes 8188
Food portfolio
1.
2.
the village coffee
& music
3.
4.
5.
6.
7.
8.
9.
10.
Amberes
11.
12.
13.
14.
15.
16.
garden
17.
18.
19.
20.
21.
gastmaal en de
tafel
22.
23.
24.
25.
food
Télécharger maintenant
