Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

How Non-native English Speakers Respond to English Surveys

We like to think that everyone answering our surveys is perfectly fluent in English but let's be realistic. About 10% of Americans have difficulty reading/writing in English because it is not their native language. And when we apply standard techniques to identify which survey takers provide good data and which are simply giving random answers, we are often making the mistake of applying measurements that all require high level language skills.

  • Sé el primero en comentar

  • Sé el primero en recomendar esto

How Non-native English Speakers Respond to English Surveys

  1. 1. Please tweet! @PeanutLabsMedia @LoveStats Please tweet! @LoveStats @PeanutLabsMediaPlease tweet! @LoveStats @PeanutLabsMedia I. Can’t. Understand. You: How non-native English speakers respond to English surveys Presented by Annie Pettit Chief Research Officer at Peanut Labs DIY Sample and Polling
  2. 2. Please tweet! @PeanutLabsMedia @LoveStats 20% of US residents don’t speak English at home 9% of US residents don’t speak English well
  3. 3. Please tweet! @PeanutLabsMedia @LoveStats Survey •12 minutes •Driving, shopping, electronics, eating out, working
  4. 4. Please tweet! @PeanutLabsMedia @LoveStats N=1212 N=228 N=204 Method
  5. 5. Please tweet! @PeanutLabsMedia @LoveStats DATA QUALITY
  6. 6. Please tweet! @PeanutLabsMedia @LoveStats Native Speakers Are Click Happy % of respondents choosing many answer options
  7. 7. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Are Click Averse % of respondents choosing minimal answer options
  8. 8. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Choose Red Herrings Sounds familiar? Looks familiar?
  9. 9. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Don’t Follow Instructions Failure rate on “Choose 3 items”
  10. 10. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Choose None of Above % choosing many None of Above answers Somewhat Good Very Good Extremely Good
  11. 11. Please tweet! @PeanutLabsMedia @LoveStats ESL Provide Shorter Verbatims % providing shortest responses Blank % in Q1 % in Q2 % in Q3 % in Q4 Answer Shortest Answers Longest Answers
  12. 12. Please tweet! @PeanutLabsMedia @LoveStats ESL are really fast or really slow % in Slowest to Fastest Deciles of Responses Slowest Decile Fastest Decile
  13. 13. Please tweet! @PeanutLabsMedia @LoveStats ESL Responders Make More Errors 0 1 2 3 4 5 6 7 8 9 10 11 Number of errors made
  14. 14. Please tweet! @PeanutLabsMedia @LoveStats RESULT OF THESE DATA QUALITY MEASURES
  15. 15. Please tweet! @PeanutLabsMedia @LoveStats Higher % of ESL People Cut Are their opinions less important? Is it fair to cut them? Somewhat good English Good English Excellent English
  16. 16. Please tweet! @PeanutLabsMedia @LoveStats Honest ESL People Get Cut Any final comments on this survey? •great and interesting survey •Have nothing for u guy, sorry im dumb lol lmao XD •i 'm very active •I enjoyed this survey •it was a good survey •it was very great to fill all ur surveys •very nice survey, not enough points •Well nothing to share now but their are something friendly way of living now. What 3 things do you do with your phone? •1..To call someone 2..To know what is going on around the world. text my friends •coupons, texts, facebook •i phone, awesome, can download games and I used iPhone and mobile •internet and check emails and talk and text •Looking information on the internet. Call the doctors. •Play games, Watch Youtube, and Watch read, Talk Text Listen to music •text. chatting with friends and talking on the phone
  17. 17. Please tweet! @PeanutLabsMedia @LoveStats Most Language Discriminatory Variables Beta Sig Under Click V1 -.283 .000 Contradict V1 -.150 .002 Can’t Rank -.137 .006 Lies -.151 .011 Under Click V2 -.132 .030 Eliminate based on language
  18. 18. Please tweet! @PeanutLabsMedia @LoveStats Eliminate based on quality Least Language Discriminatory Variables Beta Sig. Over Click V1 .004 .944 Over Click V2 -.005 .927 Straightlining V1 -.005 .926 Over Click V3 .005 .923 Don’t Follow Instructions -.008 .903 Verbatim V1 .011 .838 Invalid Phone Number .015 .756 Contradict V2 -.019 .704 quality quality
  19. 19. Please tweet! @PeanutLabsMedia @LoveStats CONCLUSIONS
  20. 20. Please tweet! @PeanutLabsMedia @LoveStats Recommendations •Choose less language discriminative data quality questions • Overclicking • Straightlining • Follow instructions • Verbatim quality • Contradictions • Red herrings • None of the above
  21. 21. Please tweet! @PeanutLabsMedia @LoveStats Recommendations • Avoid cultural references, slang, norms • Consider forcing minimum responses • Keep questions short • Create white space Have you seen any advertising for this brand? Please consider TV ads, billboards, internet ads, and other ads.
  22. 22. Please tweet! @PeanutLabsMedia @LoveStats Thank you! Annie Pettit Chief Research Officer Jonathan Cheriff Director of Sales & Marketing Find PeanutLabs on LinkedIn Facebook Twitter YouTube DIY SURVEY PROGRAMMING DIY POLLINGDIY SAMPLE