Treffer: Ten particularly frequent and consequential questionable research practices in quantitative research: Bias mechanisms, preventive strategies, and a simulation-based framework.

Ten particularly frequent and consequential questionable research practices in quantitative research: Bias mechanisms, preventive strategies, and a simulation-based framework.

Kyriazos TA; Department of Psychology, University of Western Macedonia School of Social Sciences and Humanities, Florina, Greece. th.Kyriazos@gmail.com., Poga M; Independent Researcher, Athens, Greece.

Behavior research methods [Behav Res Methods] 2026 Jan 23; Vol. 58 (2), pp. 46. Date of Electronic Publication: 2026 Jan 23.

Journal Article

English

Publisher: Springer Country of Publication: United States NLM ID: 101244316 Publication Model: Electronic Cited Medium: Internet ISSN: 1554-3528 (Electronic) Linking ISSN: 1554351X NLM ISO Abbreviation: Behav Res Methods Subsets: MEDLINE

Publication: 2010- : New York : Springer
Original Publication: Austin, Tex. : Psychonomic Society, c2005-

Research Design*/standards , Computer Simulation*, Humans ; Bias

Aczel, B., Szaszi, B., Nilsonne, G., van den Akker, O. R., Albers, C. J., van Assen, M. A. L. M., Bastiaansen, J. A., Benjamin, D., Boehm, U., Botvinik-Nezer, R., Bringmann, L. F., Busch, N. A., Caruyer, E., Cataldo, A. M., Cowan, N., Delios, A., van Dongen, N. N., Donkin, C., van Doorn, J. B., ..., & Wagenmakers, E.-J. (2021). Consensus-based guidance for conducting and reporting multi-analyst studies. eLife, 10, e72185. https://doi.org/10.7554/eLife.72185.
Altman, D. G., Moher, D., & Schulz, K. F. (2017). Harms of outcome switching in reports of randomised trials: CONSORT perspective. BMJ, j396. https://doi.org/10.1136/bmj.j396.
Andrade, C. (2021). HARKing, Cherry-Picking, P-Hacking, Fishing Expeditions, and Data Dredging and Mining as Questionable Research Practices. The Journal of Clinical Psychiatry, 82(1). https://doi.org/10.4088/jcp.20f13804.
Bakker, M., Veldkamp, C. L. S., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020). Ensuring the quality and specificity of preregistrations. PLoS Biology, 18(12), e3000937. https://doi.org/10.1371/journal.pbio.3000937. (PMID: 10.1371/journal.pbio.3000937332963587725296)
Baruch, Y. (2023). Harking can be good for science: Why, when, and how c/should we Hypothesizing after results are known or proposing research questions after results are known. Human Resource Management Journal, 34(4), 865–878. https://doi.org/10.1111/1748-8583.12534. (PMID: 10.1111/1748-8583.12534)
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B: Statistical Methodology, 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x. (PMID: 10.1111/j.2517-6161.1995.tb02031.x)
Berggren, C. (2024). Developing surveys on questionable research practices: Addressing precision, social desirability, and coverage issues. Journal of Empirical Research on Integrity. https://doi.org/10.1007/s10805-024-09565-0. (PMID: 10.1007/s10805-024-09565-0)
Bertram, M. G., Sundin, J., Roche, D. G., Sánchez-Tójar, A., Thoré, E. S. J., & Brodin, T. (2023). Open science. Current Biology, 33, R792–R797. https://doi.org/10.1016/j.cub.2023.05.036. (PMID: 10.1016/j.cub.2023.05.03637552940)
Bishop, D. V. M., & Thompson, P. (2023). Evaluating What Works. https://doi.org/10.1201/9781003453079.
Bradshaw, T. J., Huemann, Z., Hu, J., & Rahmim, A. (2023). A Guide to Cross-Validation for Artificial Intelligence in Medical Imaging. Radiology: Artificial Intelligence, 5(4). https://doi.org/10.1148/ryai.220232.
Braga-Neto, U. M., & Dougherty, E. R. (2004). Is cross-validation valid for small-sample microarray classification? Bioinformatics, 20(3), 374–380. https://doi.org/10.1093/bioinformatics/btg419. (PMID: 10.1093/bioinformatics/btg41914960464)
Bridges, A. J. (2022). Hypothesizing After Results Are Known: HARKing Avoiding Questionable Research Practices in Applied Psychology (pp. 175–190). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-04968-2_8. (PMID: 10.1007/978-3-031-04968-2_8)
Brooker, R., & Allum, N. (2024). Investigating the links between questionable research practices, scientific norms and organisational culture. Research Integrity and Peer Review, 9, Article 12. https://doi.org/10.1186/s41073-024-00151-x. (PMID: 10.1186/s41073-024-00151-x3939701311472529)
Cantone, G. G., & Tomaselli, V. (2023). Theory and methods of the multiverse: An application for panel-based models. Quality and Quantity, 58(2), 1447–1480. https://doi.org/10.1007/s11135-023-01698-5. (PMID: 10.1007/s11135-023-01698-5)
Cartwright, N. (2011). Predicting ‘It will work for us’: (Way) beyond statistics. Causality in the Sciences (pp. 750–768). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199574131.003.0035. (PMID: 10.1093/acprof:oso/9780199574131.003.0035)
Chambers, C. D., & Tzavella, L. (2021). The past, present and future of Registered Reports. Nature Human Behaviour, 6(1), 29–42. https://doi.org/10.1038/s41562-021-01193-7. (PMID: 10.1038/s41562-021-01193-734782730)
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. JAMA, 291(20), 2457–2465. https://doi.org/10.1001/jama.291.20.2457. (PMID: 10.1001/jama.291.20.245715161896)
Clark, C. J., & Tetlock, P. E. (2023). Adversarial collaboration: The next science reform. Ideological and political bias in psychology: Nature, scope, and solutions (pp. 905–927). Springer International Publishing. https://doi.org/10.1007/978-3-031-29148-7_32. (PMID: 10.1007/978-3-031-29148-7_32)
Colavizza, G., Cadwallader, L., LaFlamme, M., Dozot, G., Lecorney, S., & Hrynaszkiewicz, I. (2024). An analysis of the effects of sharing research data, code, and preprints on citations. PLoS One. https://doi.org/10.1371/journal.pone.0311493. (PMID: 10.1371/journal.pone.03114933947584911524460)
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD). Circulation, 131(2), 211–219. https://doi.org/10.1161/circulationaha.114.014508. (PMID: 10.1161/circulationaha.114.014508255615164297220)
Cumming, G. (2013). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. Routledge. https://doi.org/10.4324/9780203807002.
Ding, J. (2022). Open science. https://doi.org/10.31219/osf.io/m8qdc.
Dreber, A., & Johannesson, M. (2025). The Credibility Gap. https://doi.org/10.4324/9781003569954.
Dwan, K., Gamble, C., Williamson, P. R., & Kirkham, J. J. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias-An updated review. PLoS One, 8(7), e66844. https://doi.org/10.1371/journal.pone.0066844. (PMID: 10.1371/journal.pone.0066844238617493702538)
Dwan, K., Altman, D. G., Clarke, M., Gamble, C., Higgins, J. P. T., Sterne, J. A. C., Williamson, P. R., & Kirkham, J. J. (2014). Evidence for the selective reporting of analyses and discrepancies in clinical trials: A systematic review of cohort studies of clinical trials. PLoS Medicine, 11(6), Article e1001666. https://doi.org/10.1371/journal.pmed.1001666. (PMID: 10.1371/journal.pmed.1001666249597194068996)
Elliott, G., Kudrin, N., & Wüthrich, K. (2022). Detecting p-hacking. Econometrica, 90(2), 887–906. https://doi.org/10.3982/ecta18583. (PMID: 10.3982/ecta18583)
Elson, M., Huff, M., & Utz, S. (2020). Metascience on peer review: Testing the effects of a study’s originality and statistical significance in a field experiment. Advances in Methods and Practices in Psychological Science, 3(1), 53–65. https://doi.org/10.1177/2515245919895419. (PMID: 10.1177/2515245919895419)
Fanelli, D. (2020). Pressures to Publish: What Effects Do We See? Gaming the Metrics (pp. 111–122). MIT Press. https://doi.org/10.7551/mitpress/11087.003.0011. (PMID: 10.7551/mitpress/11087.003.0011)
Fanelli, D., Costas, R., & Ioannidis, J. P. A. (2017). Meta-assessment of bias in science. Proceedings of the National Academy of Sciences of the United States of America, 114(14), 3714–3719. https://doi.org/10.1073/pnas.1618569114. (PMID: 10.1073/pnas.1618569114283209375389310)
Fidler, F., & Cumming, G. (2018). Effect sizes and confidence intervals. The reviewer’s guide to quantitative methods in the social sciences (pp. 72–85). Routledge. https://doi.org/10.4324/9781315755649-6. (PMID: 10.4324/9781315755649-6)
Frank, M. C., Braginsky, M., Cachia, J., Coles, N., Hardwicke, T. E., Hawkins, R. D., Mathur, M. B., & Williams, R. (2025). Experimentology, MIT Press. https://doi.org/10.7551/mitpress/14810.001.0001.
Fraser, H., Parker, T., Nakagawa, S., Barnett, A., & Fidler, F. (2018). Questionable research practices in ecology and evolution. PLoS One, 13(7), e0200303. https://doi.org/10.1371/journal.pone.0200303. (PMID: 10.1371/journal.pone.0200303300112896047784)
Freuli, F., Held, L., & Heyard, R. (2023). Replication success under questionable research practices-a simulation study. Statistical Science. https://doi.org/10.1214/23-STS904. (PMID: 10.1214/23-STS904)
Friese, M., & Frankenbach, J. (2020). P-hacking and publication bias interact to distort meta-analytic effect size estimates. Psychological Methods, 25(4), 456–471. https://doi.org/10.1037/met0000246. (PMID: 10.1037/met000024631789538)
García-Pérez, M. A. (2023). Use and misuse of corrections for multiple testing. Methods in Psychology, 8, Article 100120. https://doi.org/10.1016/j.metip.2023.100120. (PMID: 10.1016/j.metip.2023.100120)
Gebre, S., Cattrysse, D., Alemayehu, E., & Van Orshoven, J. (2021). Multi-criteria decision making methods to address rural land allocation problems: A systematic review. International Soil and Water Conservation Research. https://doi.org/10.1016/j.iswcr.2021.04.005. (PMID: 10.1016/j.iswcr.2021.04.005)
Gelman, A., & Loken, E. (2014). The statistical crisis in science. American Scientist, 102(6), 460. https://doi.org/10.1511/2014.111.460. (PMID: 10.1511/2014.111.460)
Götz, M., Sarma, A., & O’Boyle, E. H. (2024). The multiverse of universes: A tutorial to plan, execute and interpret multiverses analyses using the R package multiverse. International Journal of Psychology, 59(6), 1003–1014. https://doi.org/10.1002/ijop.13229 . Portico. (PMID: 10.1002/ijop.1322939030767)
Gou, J. (2023). On dependence assumption in p-value based multiple test procedures. Journal of Biopharmaceutical Statistics, 33(5), 596–610. https://doi.org/10.1080/10543406.2022.2162066. (PMID: 10.1080/10543406.2022.216206636607042)
Häckl, S., Koch, A., & Lasch, F. (2023). Type I error rate inflation in mixed models for repeated measures. Pharmaceutical Statistics. https://doi.org/10.1002/pst.2328. (PMID: 10.1002/pst.232837519010)
Hall, B. D., Liu, Y., Jansen, Y., Dragicevic, P., Chevalier, F., & Kay, M. (2022). A survey of tasks and visualizations in multiverse analysis reports. Computer Graphics Forum, 41(1), 402–426. https://doi.org/10.1111/cgf.14443. (PMID: 10.1111/cgf.14443)
Hastie, T., Tibshirani, R., & Wainwright, M. (2015). Statistical learning with sparsity. Chapman and Hall/CRC. https://doi.org/10.1201/b18401.
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biology, 13(3), e1002106. https://doi.org/10.1371/journal.pbio.1002106. (PMID: 10.1371/journal.pbio.1002106257683234359000)
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124. (PMID: 10.1371/journal.pmed.0020124160607221182327)
Jamieson, K. H., Kearney, B., & Mazza, A.-M. (Eds.). (2024). Realizing the promise and minimizing the perils of AI for science and the scientific community. https://doi.org/10.9783/9781512827491.
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953. (PMID: 10.1177/095679761143095322508865)
Kampman, J. M., Sperna Weiland, N. H., Hollmann, M. W., Repping, S., & Hermanides, J. (2021). High incidence of outcome switching observed in follow-up publications of randomized controlled trials: Meta-research study. Journal of Clinical Epidemiology, 137, 236–240. https://doi.org/10.1016/j.jclinepi.2021.05.003. (PMID: 10.1016/j.jclinepi.2021.05.00334004339)
Kerr, N. L. (1998). HARking: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4. (PMID: 10.1207/s15327957pspr0203_415647155)
Klebel, T. (2025). The academic impact of open science: A scoping review. Royal Society Open Science, 12, Article 241248. https://doi.org/10.1098/rsos.241248. (PMID: 10.1098/rsos.2412484004666311879623)
Klein, R. A., Vianello, M., Hasselman, F., et al. (2018). Many labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225. (PMID: 10.1177/2515245918810225)
Lakens, D. (2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355–362. https://doi.org/10.1177/1948550617697177 . (Original work published 2017). (PMID: 10.1177/1948550617697177287366005502906)
Lakens, D. (2020). Pandemic researchers–recruit your own best critics. Nature, 581(7807), 121–122. https://doi.org/10.1038/d41586-020-01392-8. (PMID: 10.1038/d41586-020-01392-832393923)
Lakens, D. (2021). The practical alternative to the p value is the correctly used p value. Perspectives on Psychological Science, 16(3), 639–648. https://doi.org/10.1177/1745691620958012. (PMID: 10.1177/1745691620958012335601748114329)
Lakens, D. (2025). Concerns about theorizing, relevance, generalizability, and methodology across two crises in social psychology. International Review Of Social Psychology, 38(1), Article 6. https://doi.org/10.5334/irsp.1038. (PMID: 10.5334/irsp.10384094859312372686)
Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259–269. https://doi.org/10.1177/2515245918770963. (PMID: 10.1177/2515245918770963)
Liberatore, M., & Nydick, R. (2008). The analytic hierarchy process in medical and health care decision making: A literature review. European Journal of Operational Research, 189, 194–207. https://doi.org/10.1016/j.ejor.2007.05.001. (PMID: 10.1016/j.ejor.2007.05.001)
Luan, J. J., Mai, W., & Chow, S. C. (2021). Two One-Sided Tests Versus Confidence Interval Approach. Methodologies in Biosimilar Product Development (pp. 97–113). Chapman and Hall/CRC. https://doi.org/10.1201/9781003205807-5. (PMID: 10.1201/9781003205807-5)
Marcot, B. G., & Hanea, A. M. (2020). What is an optimal value of k in k-fold cross-validation in discrete Bayesian network analysis? Computational Statistics, 36(3), 2009–2031. https://doi.org/10.1007/s00180-020-00999-9. (PMID: 10.1007/s00180-020-00999-9)
Miller, G., & Spiegel, E. (2025). Guidelines for Research Data Integrity (GRDI). Scientific Data, 12(1). https://doi.org/10.1038/s41597-024-04312-x.
Morse, J. M. (2009). “Cherry picking”: Writing from thin data. Qualitative Health Research, 20(1), 3–3. https://doi.org/10.1177/1049732309354285. (PMID: 10.1177/1049732309354285)
Munafò, M., Nosek, B., Bishop, D., et al. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, Article 0021. https://doi.org/10.1038/s41562-016-0021. (PMID: 10.1038/s41562-016-0021339542587610724)
Murphy, K. R., & Aguinis, H. (2017). Harking: How badly can cherry-picking and question trolling produce bias in published results? Journal of Business and Psychology, 34(1), 1–17. https://doi.org/10.1007/s10869-017-9524-7. (PMID: 10.1007/s10869-017-9524-7)
Naeem, M., Jamal, T., Diaz-Martinez, J., Butt, S. A., Montesano, N., Tariq, M. I., ... & De-La-Hoz-Valdiris, E. (2021, November). Trends and future perspective challenges in big data. In Advances in intelligent data analysis and applications: Proceeding of the sixth euro-China conference on intelligent data analysis and applications, 15-18 October 2019, Arad, Romania (pp. 309–325). Singapore: Springer Singapore. https://doi.org/10.1007/978-981-16-5036-9_30.
Nagy, T., Hergert, J., Elsherif, M. M., Wallrich, L., Schmidt, K., Waltzer, T., Payne, J. W., Gjoneska, B., Seetahul, Y., Wang, Y. A., Scharfenberg, D., Tyson, G., Yang, Y.-F., Skvortsova, A., Alarie, S., Graves, K., Sotola, L. K., Moreau, D., & Rubínová, E. (2025). Bestiary of questionable research practices in psychology. Advances in Methods and Practices in Psychological Science, 8(3), 1–36. https://doi.org/10.1177/25152459251348431. (PMID: 10.1177/25152459251348431)
Nanjundeswaraswamy, T. S., & Divakar, S. (2021). Determination of sample size and sampling methods in applied research. Proceedings on Engineering Sciences, 3(1), 25–32. https://doi.org/10.24874/pes03.01.003. (PMID: 10.24874/pes03.01.003)
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences of the United States of America, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114. (PMID: 10.1073/pnas.1708274114295310915856500)
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73(1), 719–748. https://doi.org/10.1146/annurev-psych-020821-114157. (PMID: 10.1146/annurev-psych-020821-11415734665669)
Olsson-Collentine, A., van Aert, R. C. M., Bakker, M., & Wicherts, J. (2025). Meta-analyzing the multiverse: A peek under the hood of selective reporting. Psychological Methods, 30(3), 441–461. https://doi.org/10.1037/met0000559. (PMID: 10.1037/met000055937166859)
Pawel, S., Kook, L., & Reeve, K. (2024). Pitfalls and potentials in simulation studies: Questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method. Biometrical Journal, 66(1), e202200091. https://doi.org/10.1002/bimj.202200091. (PMID: 10.1002/bimj.202200091)
Rakow, T. (2022). Adversarial collaboration. Avoiding questionable research practices in applied psychology (pp. 359–377). Springer International Publishing. https://doi.org/10.1007/978-3-031-04968-2_16. (PMID: 10.1007/978-3-031-04968-2_16)
Riesthuis, P. (2024). Simulation-based power analyses for the smallest effect size of interest: A confidence-interval approach for minimum-effect and equivalence testing. Advances in Methods and Practices in Psychological Science, 7(2), Article 25152459241240722. https://doi.org/10.1177/25152459241240722. (PMID: 10.1177/25152459241240722)
Riesthuis, P., Mangiulli, I., Broers, N., & Otgaar, H. (2022). Expert opinions on the smallest effect size of interest in false memory research. Applied Cognitive Psychology, 36(1), 203–215. https://doi.org/10.1002/acp.3911. (PMID: 10.1002/acp.3911)
Riesthuis, P., Rassin, E., Bücken, C., Booker, A., Chin, J. M., Goldfarb, D., ..., & Otgaar, H. (2025). Through the lens of legal professionals: Examining the smallest effect size of interest for eyewitness memory research. The International Journal of Evidence & Proof. https://doi.org/10.1177/13657127251357630.
Rizopoulos, D. (2018). Max Kuhn and Kjell Johnson. Applied Predictive Modeling. New York, Springer. Biometrics, 74(1), 383–383. https://doi.org/10.1111/biom.12855. (PMID: 10.1111/biom.12855)
Roozeboom, M. A. (2007). A modified Bonferroni procedure for multiple tests. University of Northern Colorado.
Rubin, M. (2017). When does HARKing hurt? Identifying when different types of undisclosed post hoc hypothesizing harm scientific progress. Review Of General Psychology, 21(4), 308–320. https://doi.org/10.1037/gpr0000128. (PMID: 10.1037/gpr0000128)
Rubin, M. (2022). The costs of HARKing. The British Journal for the Philosophy of Science, 73(2), 535–560. https://doi.org/10.1093/bjps/axz050. (PMID: 10.1093/bjps/axz050)
Sagarin, B. J., Ambler, J. K., & Lee, E. M. (2014). An ethical approach to peeking at data. Perspectives on Psychological Science, 9(3), 293–304. https://doi.org/10.1177/1745691614528214. (PMID: 10.1177/174569161452821426173265)
Saini, P., Loke, Y. K., Gamble, C., Altman, D. G., Williamson, P. R., & Kirkham, J. J. (2014). Selective reporting bias of harm outcomes within studies: Findings from a cohort of systematic reviews. BMJ, 349(nov21 3), g6501–g6501. https://doi.org/10.1136/bmj.g6501. (PMID: 10.1136/bmj.g6501254164994240443)
Schönbrodt, F. D., & Wagenmakers, E. J. (2018). Bayes factor design analysis: Planning for compelling evidence. Psychon Bull Rev, 25, 128–142. https://doi.org/10.3758/s13423-017-1230-y. (PMID: 10.3758/s13423-017-1230-y28251595)
Shaffer, J. P. (1986). Modified sequentially rejective multiple test procedures. Journal Of The American Statistical Association, 81(395), 826–831. https://doi.org/10.1080/01621459.1986.10478341. (PMID: 10.1080/01621459.1986.10478341)
Short, C., Breznau, N., Bruntsch, M., Burkhardt, M., Busch, N., Cesnaite, E., Frank, M., Gießing, C., Krähmer, D., Kristanto, D., Lonsdorf, T. B., Neuendorf, C., Nguyen, H. H. V., Rausch, M., Schmalz, X., Schneck, A., Tabakci, C., & Hildebrandt, A. (2025). Multi-curious: A Multi-Disciplinary Guide to Multiverse Analysis. https://doi.org/10.31222/osf.io/4yzeh_v1.
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., & Aust, F. (2025). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science., 1(3), 337–356. (PMID: 10.1177/2515245917747646)
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2012). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632. (PMID: 10.1177/0956797611417632)
Sivarajah, U., Kamal, M. M., Irani, Z., & Weerakkody, V. (2017). Critical analysis of big data challenges and analytical methods. Journal of Business Research, 70, 263–286. https://doi.org/10.1016/j.jbusres.2016.08.001. (PMID: 10.1016/j.jbusres.2016.08.001)
Skubera, M., Korbmacher, M., Evans, T. R., Azevedo, F., & Pennington, C. R. (2025). International initiatives to enhance awareness and uptake of open research in psychology: A systematic mapping review. Royal Society Open Science, 12(241726). https://doi.org/10.1098/rsos.241726.
Spitzer, L., & Mueller, S. (2023). Registered report: Survey on attitudes and experiences regarding preregistration in psychological research. PLoS One, 18(3), e0281086. https://doi.org/10.1371/journal.pone.0281086. (PMID: 10.1371/journal.pone.02810863692866410019715)
Stallard, N., Todd, S., Ryan, E. G., & Gates, S. (2020). Comparison of Bayesian and frequentist group-sequential clinical trial designs. Bmc Medical Research Methodology, 20(1), 1–14. https://doi.org/10.1186/s12874-019-0892-8. (PMID: 10.1186/s12874-019-0892-8)
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712. https://doi.org/10.1177/1745691616658637. (PMID: 10.1177/174569161665863727694465)
Stefan, A. M., & Schönbrodt, F. D. (2023). Big little lies: A compendium and simulation of p-hacking strategies. Royal Society Open Science, 10(2), Article 220346. https://doi.org/10.1098/rsos.220346. (PMID: 10.1098/rsos.220346367789549905987)
Stoudt, S., Vásquez, V. N., & Martinez, C. C. (2021). Principles for data analysis workflows. PLoS Computational Biology. https://doi.org/10.1371/journal.pcbi.1008770. (PMID: 10.1371/journal.pcbi.1008770337352087971542)
Tincani, M., Travers, J., et al. (2025). Questionable and improved research practices in single-case experimental design: Initial investigation and findings. Perspectives on Behavior Science, 48, 447–473. https://doi.org/10.1007/s40614-025-00441-9. (PMID: 10.1007/s40614-025-00441-94052058212162424)
TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. (2024). BMJ, q902. https://doi.org/10.1136/bmj.q902.
van den Akker, O. R., Bakker, M., van Assen, M. A. L. M., Pennington, C. R., Verweij, L., Elsherif, M. M., Claesen, A., Gaillard, S. D. M., Yeung, S. K., Frankenberger, J.-L., Krautter, K., Cockcroft, J. P., Kreuer, K. S., Evans, T. R., Heppel, F. M., Schoch, S. F., Korbmacher, M., Yamada, Y., Albayrak-Aydemir, N., ..., & Wicherts, J. M. (2024a). The potential of preregistration in psychology: Assessing preregistration producibility and preregistration-study consistency. Psychological Methods. https://doi.org/10.1037/met0000687 . Advance online publication.
van den Akker, O. R., van Assen, M. A. L. M., Bakker, M., Elsherif, M. M., Wong, T. K., & Wicherts, J. M. (2024b). Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology. Behavior Research Methods, 56, 5424–5433. https://doi.org/10.3758/s13428-023-02277-0. (PMID: 10.3758/s13428-023-02277-037950113)
van de Schoot, R., Depaoli, S., King, R., Kramer, B., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J., & Yau, C. (2021). Bayesian statistics and modelling. Nature Reviews Methods Primers. https://doi.org/10.1038/s43586-020-00001-2. (PMID: 10.1038/s43586-020-00001-2)
Vendé, B. (2025). A decade of empirical research on research integrity: What have we (not) looked at? Research Integrity Journal, 10, 5.
Wagenmakers, E. J., Sarafoglou, A., & Aczel, B. (2022). One statistical analysis must not rule them all. Nature, 605(7910), 423–425. https://doi.org/10.1038/d41586-022-01332-8. (PMID: 10.1038/d41586-022-01332-835581494)
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832. https://doi.org/10.3389/fpsyg.2016.01832. (PMID: 10.3389/fpsyg.2016.01832279330125122713)
Williams, S., Carson, R., & Tóth, K. (2023). Moving beyond P values in The Journal of Physiology: A primer on the value of effect sizes and confidence intervals. The Journal of Physiology. https://doi.org/10.1113/JP285575.

Keywords: Analytical flexibility; Inferential bias; Questionable research practices (QRPs); Reproducibility; Research integrity; Research workflow

Date Created: 20260123 Date Completed: 20260124 Latest Revision: 20260123

20260124

10.3758/s13428-025-02920-y

41577935

MEDLINE