Unfounded
causal claims from the internet, the fact that randomized control trials (RCTs)
cannot address many critical issues, and reports that scientific studies fail
replication attempts suggest reconsidering how students learn to evaluate
causal claims. Traditionally, students learn RCTs are at the top of the research methods
hierarchy, and that they cannot infer causation from associations (e.g.,
correlations). Both traditions are debatable. Students need to
learn how to evaluate causal claims they encounter in daily life as well as
claims supported by scientific evidence. Students will become better critical
thinkers from learning a definition of evidence that applies inside and outside
the psychology laboratory and how to use anecdotes, associations, and RCTs in
defense of causal claims. They must learn to question all evidence and seek
patterns of supporting evidence for causal claims.
References
[1]
Abelson, R. P. (1995). Statistics as Principled Argument. Lawrence Erlbaum Associates.
[2]
American Psychological Association (2013). APA Guidelines for the Undergraduate Psychology Major: Version 2.0. Author. https://tinyurl.com/2c5ybmcs
[3]
Aron, A., Coups, E. J., & Aron, E. N. (2011). Statistics for the Behavioral and Social Sciences: A Brief Course (5th ed.). Pearson.
[4]
Barrett, T. (2015, February 27). Inhofe Brings Snowball on Senate Floor as Evidence Globe Is Not Warming. CNN. https://www.cnn.com/2015/02/26/politics/james-inhofe-snowball-climate-change/index.html
[5]
Biddlestone, M., Flavio Azevedo, F., & van der Linden, S. (2022). Climate of Conspiracy: A Meta-Analysis of the Consequences of Belief in Conspiracy Theories about Climate Change. Current Opinion in Psychology, 46, Article ID: 101390. https://doi.org/10.1016/j.copsyc.2022.101390
[6]
Blanco, F. (2017). Positive and Negative Implications of the Causal Illusion. Consciousness and Cognition, 50, 56-68. https://doi.org/10.1016/j.concog.2016.08.012
[7]
Boudry, M., Vlerick, M., & McKay, R. (2015). Can Evolution Get Us off the Hook? Evaluating the Ecological Defense of Human Rationality. Consciousness and Cognition, 33, 524-535. https://doi.org/10.1016/j.concog.2014.08.025
[8]
Campbell, D. T., & Stanley, J. C. (1966). Experimental and Quasi-Experimental Designs for Research. Rand McNally.
[9]
Čavojová, V., Šrol, J., & Jurkovič, M. (2020). Why Should We Try to Think like Scientists? Scientific Reasoning and Susceptibility to Epistemically Suspect Beliefs and Cognitive Biases. Applied Cognitive Psychology, 34, 85-95. https://doi.org/10.1002/acp.3595
[10]
Chance, P. (1999). Thorndike’s Puzzle Boxes and the Origins of the Experimental Analysis of Behavior. Journal of the Experimental Analysis of Behavior, 72, 433-440. https://doi.org/10.1901/jeab.1999.72-433
[11]
Darwin, C. (1964). On the Origin of Species by Charles Darwin: A Facsimile of the First Edition with an Introduction by Ernst Mayr. Harvard University Press.
[12]
Drake, R. E., Latimer, E. A., Leff, H. S., McHugo, G. J., & Burns, B. J. (2004). What Is Evidence? Child and Adolescent Psychiatric Clinics of North America, 13, 717-728. https://doi.org/10.1016/j.chc.2004.05.005
[13]
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021, December 7). Investigating the Replicability of Preclinical Cancer Biology. eLife, 10, e71601. https://doi.org/10.7554/eLife.71601.sa2
[14]
Fiedler, K., & Schwarz, N. (2015). Questionable Research Practices Revisited. Social Psychological and Personality Science, 7, 45-52. https://doi.org/10.1177/1948550615612150
[15]
File Drawer Problem (2007). In N. J. Salkind (Ed.), Encyclopedia of Measurement and Statistics (Vol. 1, pp. 353-354). SAGE.
[16]
Flora, D. B. (2020). Thinking about Effect Sizes: From the Replication Crisis to a Cumulative Psychological Science. Canadian Psychology, 61, 318-330. https://doi.org/10.1037/cap0000218
[17]
Goldberg, M. H. (2019). How Often Does Random Assignment Fail? Estimates and Recommendations. Journal of Environmental Psychology, 66, Article ID: 101351. https://doi.org/10.1016/j.jenvp.2019.101351
[18]
Hatfield, J., Faunce, G. J., & Soames Job, R. F. (2006). Avoiding Confusion Surrounding the Phrase “Correlation Does Not Imply Causation”. Teaching of Psychology, 33, 49-51.
[19]
Intergovernmental Panel on Climate Change (2022). Summary for Policymakers. https://tinyurl.com/fneejxkm
[20]
Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLOS Medicine, 2, e124. https://doi.org/10.1371/journal.pmed.0020124
[21]
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling. Psychological Science, 23, 524-532. https://doi.org/10.1177/0956797611430953
[22]
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
[23]
Kenny, D. A. (2019). Enhancing Validity in Psychological Research. American Psychologist, 74, 1018-1028. https://doi.org/10.1037/amp0000531
[24]
Lewandowsky, S. (2021). Climate Change Disinformation and How to Combat It. Annual Review of Public Health, 42, 1-21. https://doi.org/10.1146/annurev-publhealth-090419-102409
[25]
Lilienfeld, S. O., Ammirati, R., & Landfield, K. (2009). Giving Debiasing Away. Perspectives on Psychological Science, 4, 390-398. https://doi.org/10.1111/j.1745-6924.2009.01144.x
[26]
Lindsay, D. S. (2015). Replication in Psychological Science. Psychological Science, 26, 1827-1832. https://doi.org/10.1177/0956797615616374
[27]
Loftus, E. M. (2019). Eyewitness Testimony. Applied Cognitive Psychology, 33, 498-503. https://doi.org/10.1002/acp.3542
[28]
Marek, S., Tervo-Clemmens, B., Calabro, F. J., Montez, D. F., Kay, B. P., Hatoum, A. S., Donohue, M. R., Foran, W., Miller, R. L., Hendrickson, T. J., Malone, S. M., Kandala, S., Feczko, E., Miranda-Dominguez, O., Graham, A. M., Earl, E. A., Perrone, A J., Cordova, M., Doyle, O., & Dosenbach, N. U. F. (2022). Reproducible Brain-Wide Association Studies Require Thousands of Individuals. Nature, 603, 654-660. https://doi.org/10.1038/s41586-022-04492-9
[29]
Martín, M., & Valiña, M. D. (2023). Heuristics, Biases and the Psychology of Reasoning: State of the Art. Psychology, 14, 264-294. https://doi.org/10.4236/psych.2023.142016
[30]
Marx, W., Haunschild, R., Thor, A., & Bornmann, L. (2017). Which Early Works Are Cited Most Frequently in Climate Change Research Literature? A Bibliometric Approach Based on Reference Publication Year Spectroscopy. Scientometrics, 110, 335-353. https://doi.org/10.1007/s11192-016-2177-x
[31]
Matute, H., Blanco, F., Yarritu, I., Díaz-Lago, M., Vadillo, M. A., & Barberia, I. (2015). Illusions of Causality: How They Bias Our Everyday Thinking and How They Could Be Reduced. Frontiers in Psychology, 6, Article No. 888. https://doi.org/10.3389/fpsyg.2015.00888
[32]
Matute, H., Yarritu, I., & Vadillo, M. A. (2011). Illusions of Causality at the Heart of Pseudoscience. British Journal of Psychology, 102, 392-405. https://doi.org/10.1348/000712610X532210
[33]
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is Psychology Suffering from a Replication Crisis? What Does “Failure to Replicate” Really Mean? American Psychologist, 70, 487-498. https://doi.org/10.1037/a0039400
[34]
Mealey, L. (2000). Sex Differences: Developmental and Evolutionary Strategies. Academic Press.
[35]
Meltzoff, J., & Cooper, H. (2018). Critical Thinking about Research: Psychology and Related Fields (2nd ed.). American Psychological Association. https://doi.org/10.1037/0000052-000
[36]
Moravec, P. L., Kim, A., & Dennis, A. L. (2018). Appealing to Sense and Sensibility: System 1 and System 2 Interventions for Fake News on Social Media. Information Systems Research, 31, 987-1006. https://doi.org/10.2139/ssrn.3269902
[37]
Morling, B. (2018). Research Methods in Psychology (3rd ed.). Norton.
[38]
Morling, B. (2021). Research Methods in Psychology (4th ed.). Norton.
[39]
National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and Replicability in Science. The National Academies Press.
[40]
Open Science Collaboration (2015). Estimating the Reproducibility of Psychological Science. Science, 349, aac4716. https://doi.org/10.1126/science.aac4716
[41]
Pearl, J. (2018). Challenging the Hegemony of Randomized Controlled Trials: A Commentary on Deaton and Cartwright. Social Science & Medicine, 210, 60-62. https://doi.org/10.1016/j.socscimed.2018.04.024
[42]
Reiss, J. (2009). Causation in the Social Sciences: Evidence, Inference, and Purpose. Philosophy of the Social Sciences, 39, 20-40. https://doi.org/10.1177/0048393108328150
[43]
Richmond, A. S, Boysen, G. A., Hudson, D. L., Gurung, R. A. R., Naufel, K. Z., Neufeld, G., Landrum, R. E., Dunn, D. S., & Beers, M. (2021). The Introductory Psychology Census: A National Study. Scholarship of Teaching and Learning in Psychology, 7, 163-180. https://doi.org/10.1037/stl0000277
[44]
Sawilowsky, S. S. (2004). Teaching Random Assignment: Do You Believe It Works? Journal of Modern Applied Statistical Methods, 3, 221-226. http://digitalcommons.wayne.edu/coe_tbf/16 https://doi.org/10.22237/jmasm/1083370980
[45]
Schmidt, F. L., & Oh, I.-S. (2016). The Crisis of Confidence in Research Findings in Psychology: Is Lack of Replication the Real Problem? Or Is It Something Else? Archives of Scientific Psychology, 4, 32-37. https://doi.org/10.1037/arc0000029
[46]
Schoolov, K. (2021, October 1). Why It’s Not Possible for the Covid Vaccines to Contain a Magnetic Tracking Chip That Connects to 5G. CNBC. https://www.cnbc.com/2021/10/01/why-the-covid-vaccines-dont-contain-a-magnetic-5g-tracking-chip.html
[47]
Schwarcz, J. (2021). Can Vaccines Make Our Body Magnetic? McGill Office for Science and Society. https://tinyurl.com/2evdeavy
[48]
Sharpe, D., & Poets, S. (2020). Meta-Analysis as a Response to the Replication Crisis. Canadian Psychology, 61, 377-387. https://doi.org/10.1037/cap0000215
[49]
Stapleton, P. (2019). Avoiding Cognitive Biases: Promoting Good Decision Making in Research Methods Courses. Teaching in Higher Education, 24, 578-586. https://doi.org/10.1080/13562517.2018.1557137
[50]
Taccone, F. S., Hites, M., & Dauby, M. (2022). From Hydroxychloroquine to Ivermectin: How Unproven “Cures” Can Go Viral. Clinical Microbiology and Infection, 28, 472-474. https://doi.org/10.1016/j.cmi.2022.01.008
[51]
Thorndike, E. L. (1911). Animal Intelligence. Hafner.
[52]
Tryon, W. W. (2016). Replication Is about Effect Size: Comment on Maxwell, Lau, and Howard (2015). American Psychologist, 71, 236-237. https://doi.org/10.1037/a0040191
[53]
Wagner, P. A. (2022). Tools for Teaching and Role-Modeling Critical Thinking. Psychology, 13, 1335-1341. https://doi.org/10.4236/psych.2022.138086
[54]
Wielkiewicz, R. M. (2016). Sustainability and Psychology (2nd ed.). Main Event Press. https://www.amazon.com/dp/B012LJ9ACQ/
[55]
Wielkiewicz, R. M. (2022). A Quick Review of Statistical Thinking (4th ed.). Main Event Press. https://www.amazon.com/Quick-Review-Statistical-Thinking-ebook/dp/B09YBYXD4L/
[56]
Wilkinson, L., & the Task Force on Statistical Inference (TFSI) (1999). Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist, 54, 594-604. https://doi.org/10.1037/0003-066X.54.8.594
[57]
Worrall, J. (2010). Evidence: Philosophy of Science Meets Medicine. Journal of Evaluation in Clinical Practice, 16, 356-362. https://doi.org/10.1111/j.1365-2753.2010.01400.x
