Reverse fragility index shows high fragility in Cochrane meta-analyses with P values between 0.05 and 0.20

Abstract

OBJECTIVE: To quantify fragility of statistically nonsignificant meta-analysis results with P values between 0.05 and 0.20 using the reverse fragility index.

STUDY DESIGN AND SETTING: This meta-epidemiological study examined Cochrane systematic reviews published in 2025. We calculated the reverse fragility index (minimum number of outcome event changes needed to convert a nonsignificant result to statistically significant) and reverse fragility quotient. Results represent medians (Q1-Q3) with 95% confidence interval (CI) from 10,000 bootstrap iterations.

RESULTS: We analyzed 280 nonsignificant pooled effect estimates from 81 Cochrane systematic reviews. Meta-analyses included a median of five studies and 1,002 participants with event risks of 11.7% and 12.6% in the intervention and control groups, respectively. The median reverse fragility index was 3 (95% CI, 2-3; Q1-Q3, 2-5), indicating that modifying three events among the primary studies included in a meta-analysis could change the status of nonsignificant results. The median reverse fragility quotient was 2.7 per 1,000 participants (95% CI, 2.2-3.3; Q1-Q3, 1.0-5.8). Meta-analyses with P values 0.05-0.10 were more fragile than those with P values 0.10-0.20. Event changes did not substantially increase heterogeneity.

CONCLUSION: Meta-analyses with P values between 0.05 and 0.20 showed statistical fragility. Reverse fragility metrics provide quantifiable evidence for nonsignificant results that may have a signal of clinical relevance and challenge their misinterpretation as robust null findings.