Fiona O'Donovan, Waterford Institute of Technology, Psychology Matters Day. However, in recent years, concerns have been steadily growing about how valid or ‘true’ some of these claims may be. Criticisms about scientific rigour are causing turmoil within the field. In 2015, over 270 social psychologists published the Reproducibility Project. 100 psychology experiments were repeated to see if they could generate the same result. Almost two-thirds of the studies failed to replicate. In this talk, I will discuss the hallmarks of good science and how misaligned incentive systems can derail scientific progress. While fraud is rare, selective reporting, cherry picking data and poor analytic procedures are not uncommon. A positive outcome of this crisis has been a cultural shift in psychological science. More sophisticated techniques and a move towards openness and transparency have changed the landscape of psychological research and we are in the midst of what some are calling a scientific revolution.

1. How Credible is
Psychology?
PSI Psych Matters Day
2018 WIT
Fiona O’ Donovan

2. Life hack:
Power Posing
Image licensed under CC BY 2.0, via Wikimedia Commons

3. But is it really that simple?

5. How science should work
Establish topic
Review literature
Form hypothesis
Test with experiment
Analyze results
Hypothesis confirmed Hypothesis disconfirmed
Report
Replication

6. How it is to
work in
science
Insecure employment
Limited resources
“Publish or perish”
Journals want novel results
File drawer problem
P - hacking

7. Lies, damned
lies, and
statistics

9. Cultural Revolution in Psychological Science
• Move towards open science and data sharing
• e.g. Open Science Framework, Centre for Open Science, Pre prints, MOOCs, Berkeley
Initiative for Transparency in Social Sciences
• Renewed interest in scientific vigour
• e.g. Metapsychology journal, Meta-Research Center at Tilburg University, Society for
Improvement of Psychological Science, more sophisticated analyses
• Replications are valued
• Including adversarial collaborations
• Accountability
• e.g. statcheck.io, GRIM (Granularity-Related Inconsistency of Mean) technique,
Pubpeer
• Reforming the publishing model
• e.g. Registered reports

10. Registered Reports

12. Conclusion
Failure is moving
psychological science forward
As psychologists we should be
aware of our own biases
Science is self correcting
Science is hard!

13. Thank you
fionaodonovan2@gmail.com
@Fiona_od