Very large work published on September 6, 2024 in JAMA Network Open with the title: Agreement Between Mega-Trials and Smaller Trial. A Systematic Review and Meta-Research Analysis. A preprint was posted on medRxiv in May. Very large article (21 pages PDF format) withc 10 authors and affiliations from 6 countries (USA, Switzerland, Germany, Holland, Austria, Iran) and the participation of JP Ioannidis (METRICS, Stanford).
The message is that mega-trials, defined here as involving more than 10,000 patients, should be carried out more often, excluding vaccine trials. Good message, but not so easy to implement for various reasons. Beware of sweeping generalizations.
“Conclusions and relevance” from the article abstract:
Hang in there to read the conclusion (DeepL translation reviewed) : In this meta-research, meta-analyses of smaller trials showed results broadly comparable to those of mega-trials, but smaller trials published before the mega-trials showed more favorable results than mega-trials. tests. These results suggest that mega-trials should be performed more often given the relatively small number of mega-trials found, their low significance rate, and the fact that small trials published before mega-trials report more beneficial results than larger ones. mega-trials and subsequent small trials.
An impressive systematic review
It takes a lot of time to read this work, very documented, with a protocol, many appendices, a data sharing declaration, etc. Quality meta-science. WELL DONE.
The research strategy was complex because to make a comparison, similar judgment criteria were needed: We searched for mega-trials in ClinicalTrials.gov (last updated January 2023) and then performed PubMed searches (until June 2023) to identify the most recent meta-analyses that included the results of these mega-trials for the primary outcome of the mega-trial and for all-cause mortality.
Unsurprisingly, out of 82 mega-trials included, 64 were in the cardiovascular field, 17 in the nutritional field, and one in pharmacology. So conclusions generalizable to these areas only or to others? In cardiovascular, there are a lot of mega-trials… so a lot of money and a search for a small effect???
Here is the ‘Results’ paragraph of the summary (ROR pour odds ratio) : Of 120 mega-trials identified, 41 showed a significant result for the primary outcome and 22 showed a significant result for all-cause mortality. In 35 comparisons of primary outcomes (including 85 point estimates from 69 unique mega-trials and 272 point estimates from smaller trials) and 26 comparisons of all-cause mortality (including 70 point estimates from 65 unique mega-trials and 267 point estimates from smaller trials), no difference existed between the outcomes of the mega-trials and smaller trials for primary outcome (ROR, 1.00; 95% CI, 0.97-1.04) nor for all-cause mortality (ROR, 1.00; 95% CI, 0.97-1.04). For the primary outcomes, smaller trials published before the mega-trials had more favorable results than the mega-trials (ROR, 1.05; 95% CI, 1.01-1.10) and subsequent smaller trials published after the mega-trials (ROR, 1.10; 95% CI, 1.04-1.18).
PS: I thank Pierre Rimbaud
