Abstract:
Reproducibility and replicability are at heart of science, yet increasing evidence from recent years suggests that many of the findings in psychological science are irreproducible and non-replicable in what some termed as a “replication/reproducibility crisis” and a new “credibility revolution” calling for significant changes in the way we do science. Is there a crisis? Should this affect our research agenda? If so – how?
I will present an ongoing mass replication effort at the University of Hong Kong (HKU), with ~45 completed and ~25 ongoing pre-registered replications in guided thesis work and six undergraduate one-semester courses at HKU, outlined in http://mgto.org/pre-registered-replications/ .
Undergraduate students analyzed impactful Judgment and Decision Making literature classics and attempted to reproduce methods and materials to conduct effect-size calculations and power analyses, design experiments, and adopt latest tools and templates in preregistering replications and extensions on the Open Science Framework. We ran the experiments on well-powered online Prolific and MTurk samples (power = 0.95-0.99; N = 400-1500). Students wrote full submission-ready manuscripts of their findings with comprehensive supplementary materials. In the spirit of full transparency and open-science, all procedures, decisions, materials, datasets, and analyses/code are in the process of being shared on the Open Science Framework. See sample projects here: https://tinyurl.com/hkureplicationexamples
I will briefly present the process, findings, main insights, and my own take-aways, and will review open resources we developed such as course materials, collaborative guides, and tools that can be used by all in their research labs.
Bio:
Gilad is an assistant professor with the psychology department at the University of Hong Kong. Gilad’s work is in the areas of social-cognitive psychology and judgment and decision-making. Research focuses on the topics of biases and heuristics, especially regarding agency and action. In recent years, with the emergence of the credibility revolution (so-called “replication/reproducibility crisis”), Gilad has taken a special interest in the movement for advancing psychological science by giving workshops and seminars and guiding researchers and students about open-science, statistical tools and simulations using open-source software (R), meta-analyses, pre-registrations, replications, and reproducibility. Open workshop
materials: http://mgto.org/workshops/; Open teaching materials:
http://mgto.org/teaching-courses/
More details on Gilad can be found at http://giladfeldman.org