Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

RepliCHI: Graduate Student Perspectives

  • Sé el primero en comentar

RepliCHI: Graduate Student Perspectives

  1. 1. RepliCan’t<br />Graduate Student Perspectives<br />Michael Bernstein<br />MIT Computer Science and Artificial Intelligence Laboratory<br />msbernst@mit.edu | @msbernst<br />mit human-computer interaction<br />
  2. 2. Speaking for every graduate student in SIGCHI, I can say one thing:<br />
  3. 3. I can’t speak for every<br />graduate student<br />in SIGCHI.<br />
  4. 4. My Unassailable, Extremely ScientificData Collection Protocol:<br />Survey of CHI student volunteers, CHI-students ACM listserv, and snowballed recruitment through Facebook and Twitter.<br />N=93 responses<br />
  5. 5. Have you ever replicateda study or system?<br />17% yes, 83% no<br />
  6. 6. Do you ever plan to replicatea study or system?<br />38% “Hell yes”, 62% “Hell no”<br />
  7. 7. Set Yourself Apart<br />“<br />The point of research is to come up with exciting ideas that solve problems. Not copy others’ work.<br />”<br />
  8. 8. Set Yourself Apart<br />“<br />”<br />I'm more creative than that.<br />
  9. 9. There’s No Reward<br />“<br />New studies confirming old studies have no chance of publication.<br />”<br />
  10. 10. There’s No Reward<br />Reviewers […] didn't feel replication was necessary even though the original study was specific to a single company.<br />“<br />”<br />
  11. 11. There’s No Reward<br />I very frequently see reviewers criticize submissions for presenting results that are “not novel” or “have already been shown”.<br />“<br />”<br />
  12. 12. There’s No Reward<br />I very frequently see reviewers criticize submissions for presenting results that are “not novel” or “have already been shown”.<br />“<br />”<br />
  13. 13. Responding to Incentives<br />Open access and replication. <br />A true scientist’s ideals, but see:The grad student must conform. <br />“<br />”<br />
  14. 14. Haikus<br />studies should break ground replication wastes our timelet's find new problems <br />“<br />”<br />
  15. 15. Haikus<br />think analyzing<br />CMC is tough? try it <br />reproducibly!<br />“<br />”<br />
  16. 16. Haikus<br />repeat to be sure<br />we stand on giants’ shoulders<br />but do so on faith<br />“<br />”<br />
  17. 17. 83% have not<br />62% will not<br />(But we’ll need to replicate the study to be sure.)<br />
  18. 18.
  19. 19. Why?<br />“<br />repeat to be sure<br />we stand on giant's shoulders<br />but do so on faith<br />”<br />
  20. 20. Why?<br />“<br />Replication is a critical component of scientific research, and it should be encouraged and rewarded. The lack of it is detrimental to the scientific soundness of our discipline. <br />”<br />
  21. 21. Why?<br />“<br />we lack the time for<br />replication of studies<br />just review strictly<br />”<br />
  22. 22. Why?<br />“<br />think analyzing<br />CMC is tough? try it <br />reproducibly!<br />”<br />
  23. 23. Why?<br />“<br />It seems like the best outcome for a replication, rather than success, is actually a refutation of the original study.<br />”<br />
  24. 24. Why?<br />“<br />Because its's a waste of time: HCI studies are so small, I know they surely WON'T replicate, so why bother!<br />”<br />
  25. 25. Why?<br />“<br />CHI is too cutting edge for things like replication, or good science, or careful analysis, or the humility to accept that other topics besides Fitts' Law deserve dozens of nearly-identical studies.<br />”<br />
  26. 26.
  27. 27. Why?<br />“<br />I do not intend on taking the risk of replicating some of my favorite works unless I see evidence that the CHI community supports such a thing.<br />”<br />
  28. 28. There’s No Reward<br />Case A. Confirmation of the earlier results (very boring)Case B. Conflict with earlier results (unpublishable problem) <br />“<br />”<br />

×