For those wondering, “How do I critique a study” or “Where’s the research gap”?
A fav way for me is to find a ‘tennis match’ in a journal (can be riveting). Use keywords like “reply to [author]” or “comment on [author]” in the same journal as the one you’d like to critique.
Exhibit A:
Recent retractions of high-profile scientific studies highlight the challenges of conducting good, reliable research.
A study using the scientific method, aims to improve the credibility of scientific investigation. This is hard work. For example, Protzko et al. (2024) [now Retracted].
Pic: Tennis match in progress, c. 1925, possibly Rockhampton, State Library of Queensland
The paper focused on replication of psychological studies and reported a replication rate of 86%. However, letters-to-the-editors by other psychologist-scientists pointed out that replicability was not a pre-registered outcome; it was not one of the original hypotheses.
So…? It’s not clear whether they added a hypothesis After they had analysed the data (p-hacking).
“ … inference relies on a post hoc comparison of a tenuous metric of replicability to past research that relied on incommensurable metrics and sampling frames” (Bak-Coleman & Devezer, 2023).
Bak-Coleman, J. B., & Devezer, B. (2023, November 20). Causal claims about scientific rigor require rigorous causal evidence. https://doi.org/10.31234/osf.io/5u3kj
Protzko, J., Krosnick, J., Nelson, L. et al. RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable. Nat Hum Behav 8, 311–319 (2024). https://doi.org/10.1038/s41562-023-01749-9
Some of the other ‘players’ (who in turn, link to others):
https://statmodeling.stat.columbia.edu/2023/11/21/more-on-the-so-called-rigor-enhancing-practices-in-quantitative-psychology-research/
https://statmodeling.stat.columbia.edu/2023/11/21/of-course-its-preregistered-just-give-me-a-sec/
Light & Life~ Charmayne
Today's affiliate:"The Australian Firefighters' Calendar, established in 1993 to support the Children's Hospital Foundation, providing funds for research into childhood burns. Now in its 31st year…"
Buy me a coffee
コメント