2. Ask for true opinion?
• Will you buy Samsung Galaxy S3 when it
comes out? (Yes/No)
• Will you vote in the next presidential election?
– (definitely/probably/probably not/definitely not)
• Have you had more than 20 sexual partners
over the past year (Yes/No)
3. What is BTS?
• Survey scoring method that provides truth-
telling incentives for respondents answering
multiple-choice questions
• Respondents to supply not only their own
answers, but also percentage estimates of
others’ answers.
• The formula then assigns high scores to
answers that are surprisingly common
A Bayesian Truth Serum for Subjective Data by Drazen Prelec
Science 15 October 2004: Vol. 306 no. 5695 pp. 462-466
4. BTS simplified
• “The premise behind this approach is the
following. If people truly hold a particular
belief, they are more likely to think that others
agree or have had similar experiences.”
• you are your best estimator
– or your estimation reveals you
– posterior probability
You
your estimation the unknown world
(distribution of different opinions)
8. The Information score: measures
surprisingly common
ex. log(0.15/0.05)
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
9. prediction score measures prediction
accuracy
equals zero for
a perfect prediction
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
10. Conclusion First
• The best strategy for the respondent is to tell
the truth
Your preference “wins” to the extent that it
is more popular than collectively estimated
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
11. The intuitive argument for m=2
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
12. and I happen to like Red
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
13. This is my best estimate of the Red
share (e.g., 50%)
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
14. Bayesian reasoning implies that someone who
likes White will estimate a smaller share for Red
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
15. The average predicted share for Red will fall
somewhere between these two estimates
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
16. Hence, if I like Red I should believe that
the share for Red will be underestimated
or ‘surprisingly popular’
My prediction of the
average Red share
estimate
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
17. The argument holds even if I know that my
preferences are unusual
reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
18. Application?
• Honest signals subjective preferences
– BTS draws more truth opinions from the users
– reality mining captures the objective ground truths
• Are there relations between these two?
– I feel stressful when multiple people around me
– I feel depressed when I am alone
• A improvement on psychological-social probe
– developing an opinion probe on funf-framework
– capture preferences and context at the same time
Notas del editor
The algorithm assigns more points to responses to answers that are "surprisingly common", that is, answers that are more common that collectively predicted. For example, let's say you are being asked about which political candidate you support. A candidate who is chosen (in the first question) by 10% of the respondents, but only predicted as being chosen (the second question) by 5% of the respondents is a surprisingly common answer. This technique gets more true opinions because it is believed that people systematically believe that their own views are unique, and hence will underestimate the degree to which other people will predict their own true views.
example:X_bar = 0.15Y_bar = 0.05 information score = log(3)The Information score measures whether ananswer is surprisingly common
example:X_bar = 0.15Y_bar = 0.05 information score = log(3)The Information score measures whether ananswer is surprisingly common