• Integration of non-randomized...

Treffer: Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions.

Title:
Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions.
Authors:
Mei F; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China., Yao M; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China., Wang Y; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China., Huan J; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China., Ma Y; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China., Li G; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.; Center for Clinical Epidemiology and Methodology, Guangdong Second Provincial General Hospital, Guangzhou, China.; Biostatistics Unit, Research Institute at St. Joseph's Healthcare Hamilton, Hamilton, Ontario, Canada., Zou K; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China., Li L; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China. liling@wchscu.cn.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China. liling@wchscu.cn.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China. liling@wchscu.cn., Sun X; Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-Based Medicine Center, Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China. sunxin@wchscu.cn.; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, China. sunxin@wchscu.cn.; Sichuan Center of Technology Innovation for Real World Data, Chengdu, China. sunxin@wchscu.cn.; Department of Epidemiology and Biostatistics, School of Public Health and West China Fourth Hospital, Sichuan University, Chengdu, China. sunxin@wchscu.cn.
Source:
BMC medicine [BMC Med] 2024 Dec 02; Vol. 22 (1), pp. 571. Date of Electronic Publication: 2024 Dec 02.
Publication Type:
Journal Article; Meta-Analysis; Research Support, Non-U.S. Gov't
Language:
English
Journal Info:
Publisher: BioMed Central Country of Publication: England NLM ID: 101190723 Publication Model: Electronic Cited Medium: Internet ISSN: 1741-7015 (Electronic) Linking ISSN: 17417015 NLM ISO Abbreviation: BMC Med Subsets: MEDLINE
Imprint Name(s):
Original Publication: [London] : BioMed Central, 2003-
MeSH Terms:
Randomized Controlled Trials as Topic*/methods , Randomized Controlled Trials as Topic*/standards , Epidemiologic Studies*, Humans ; Non-Randomized Controlled Trials as Topic/methods ; Meta-Analysis as Topic ; Systematic Reviews as Topic/methods
References:
JAMA. 2023 Apr 25;329(16):1376-1385. (PMID: 37097356)
Stat Med. 2017 Apr 15;36(8):1210-1226. (PMID: 28083901)
BMC Med. 2022 May 11;20(1):174. (PMID: 35538478)
BMJ. 2021 Sep 15;374:n1864. (PMID: 34526355)
Eur Heart J. 2012 Aug;33(15):1893-901. (PMID: 22711757)
Am J Epidemiol. 2007 Nov 15;166(10):1203-9. (PMID: 17712019)
J Clin Epidemiol. 2022 Dec;152:314-315. (PMID: 36441078)
PLoS Med. 2011 May;8(5):e1001026. (PMID: 21559325)
Res Synth Methods. 2013 Mar;4(1):12-25. (PMID: 26053536)
Nat Med. 2023 Oct;29(10):2383-2386. (PMID: 37794251)
BMC Med Res Methodol. 2016 Mar 22;16:35. (PMID: 27004721)
J Clin Med. 2023 Feb 20;12(4):. (PMID: 36836227)
J Clin Epidemiol. 2020 Feb;118:18-28. (PMID: 31698062)
BMJ. 2014 Jan 24;348:f7450. (PMID: 24464277)
PLoS Med. 2016 May 24;13(5):e1002028. (PMID: 27218655)
BMC Med Res Methodol. 2024 Apr 27;24(1):99. (PMID: 38678213)
Cochrane Database Syst Rev. 2014 Apr 29;(4):MR000034. (PMID: 24782322)
J Clin Epidemiol. 2023 Jun;158:44-52. (PMID: 36822441)
BMJ. 2014 Jan 08;348:f7668. (PMID: 24401468)
J Clin Epidemiol. 2022 Feb;142:200-208. (PMID: 34800676)
J Clin Epidemiol. 2018 Jun;98:33-40. (PMID: 29452221)
J Clin Epidemiol. 2024 Jun;170:111356. (PMID: 38604271)
Res Synth Methods. 2023 Sep;14(5):689-706. (PMID: 37309821)
J Clin Epidemiol. 2011 Nov;64(11):1178-86. (PMID: 21636246)
Res Synth Methods. 2013 Mar;4(1):26-35. (PMID: 26053537)
BMJ. 2019 Aug 28;366:l4898. (PMID: 31462531)
BMC Med. 2022 Oct 24;20(1):355. (PMID: 36274131)
BMC Med Res Methodol. 2024 Sep 27;24(1):219. (PMID: 39333867)
Health Technol Assess. 2000;4(34):1-154. (PMID: 11134917)
Lancet. 2017 Jul 22;390(10092):415-423. (PMID: 28215660)
BMC Med. 2021 Dec 6;19(1):307. (PMID: 34865623)
J Clin Epidemiol. 2014 Jun;67(6):645-53. (PMID: 24725644)
Res Synth Methods. 2015 Mar;6(1):45-62. (PMID: 26035469)
BMJ Open. 2023 Jul 26;13(7):e073232. (PMID: 37495391)
J Clin Epidemiol. 2022 Dec;152:300-306. (PMID: 36245131)
J Evid Based Med. 2024 Sep;17(3):550-558. (PMID: 39107946)
N Engl J Med. 2016 Dec 8;375(23):2293-2297. (PMID: 27959688)
Grant Information:
72204173 National Natural Science Foundation of China; 82274368 National Natural Science Foundation of China; 82225049 National Science Fund for Distinguished Young Scholars; ZYGD23004 1.3.5 project for disciplines of excellence, West China Hospital, Sichuan University; 2024zd023 special fund for traditional Chinese medicine of Sichuan Provincial Administration of Traditional Chinese Medicine
Contributed Indexing:
Keywords: Meta-analysis; Meta-epidemiology; Non-randomized studies of interventions; Randomized controlled trials; Systematic review
Entry Date(s):
Date Created: 20241202 Date Completed: 20241203 Latest Revision: 20241211
Update Code:
20250114
PubMed Central ID:
PMC11613474
DOI:
10.1186/s12916-024-03778-1
PMID:
39623370
Database:
MEDLINE

Weitere Informationen

Backgrounds: Syntheses of non-randomized studies of interventions (NRSIs) and randomized controlled trials (RCTs) are increasingly used in decision-making. This study aimed to summarize when NRSIs are included in evidence syntheses of RCTs, with a particular focus on the methodological issues associated with combining NRSIs and RCTs.
Methods: We searched PubMed to identify clinical systematic reviews published between 9 December 2017 and 9 December 2022, randomly sampling reviews in a 1:1 ratio of Core and non-Core clinical journals. We included systematic reviews with RCTs and NRSIs for the same clinical question. Clinical scenarios for considering the inclusion of NRSIs in eligible studies were classified. We extracted the methodological characteristics of the included studies, assessed the concordance of estimates between RCTs and NRSIs, calculated the ratio of the relative effect estimate from NRSIs to that from RCTs, and evaluated the impact on the estimates of pooled estimates when NRSIs are included.
Results: Two hundred twenty systematic reviews were included in the analysis. The clinical scenarios for including NRSIs were grouped into four main justifications: adverse outcomes (n = 140, 63.6%), long-term outcomes (n = 36, 16.4%), the applicability of RCT results to broader populations (n = 11, 5.0%), and other (n = 33, 15.0%). When conducting a meta-analysis, none of these reviews assessed the compatibility of the different types of evidence prior, 203 (92.3%) combined estimates from RCTs and NRSIs in the same meta-analysis. Of the 203 studies, 169 (76.8%) used crude estimates of NRSIs, and 28 (13.8%) combined RCTs and multiple types of NRSIs. Seventy-seven studies (35.5%) showed "qualitative disagree" between estimates from RCTs and NRSIs, and 101 studies (46.5%) found "important difference". The integration of NRSIs changed the qualitative direction of estimates from RCTs in 72 out of 200 studies (36.0%).
Conclusions: Systematic reviews typically include NRSIs in the context of assessing adverse or long-term outcomes. The inclusion of NRSIs in a meta-analysis of RCTs has a substantial impact on effect estimates, but discrepancies between RCTs and NRSIs are often ignored. Our proposed recommendations will help researchers to consider carefully when and how to synthesis evidence from RCTs and NRSIs.
(© 2024. The Author(s).)

Declarations. Ethics approval and consent to participate: Not applicable. Consent for publication: Not applicable. Competing interests: The authors declare no competing interests.