Evaluating evidential strength of statistically nonsignificant meta-analyses with likelihood ratios

Abstract

OBJECTIVE: To determine the proportion of statistically nonsignificant meta-analyses that provide evidence favoring the null hypothesis of no difference, evidence favoring the alternative hypothesis, or inconclusive findings using likelihood ratios.

STUDY DESIGN AND SETTING: In this meta-epidemiological study, Cochrane systematic reviews published in 2025 were screened. Eligible meta-analyses pooled data from two or more primary studies, reported a pooled effect estimate with a 95% confidence interval including the value of no difference, and had a P value between 0.05 and 0.20. Meta-analyses using Hartung-Knapp/Sidik-Jonkman were excluded. In the primary analysis, likelihood ratios compared the null hypothesis against the observed pooled effect. In the secondary analysis, likelihood ratios were calculated against prespecified small, medium, and large effect thresholds; meta-analyses reporting weighted mean differences were excluded as no agreed-upon thresholds exist for these outcomes.

RESULTS: A total of 936 meta-analyses from 172 Cochrane reviews were eligible. In the primary analysis, 693 (74%) provided weak evidence favoring the alternative hypothesis and 243 (26%) were inconclusive; none favored the null hypothesis. All meta-analyses with P values between 0.05 and 0.10 showed weak evidence favoring the alternative, while those with P values between 0.11 and 0.20 were split between weak evidence favoring the alternative (55%) and inconclusive (45%) findings. In the secondary analysis of 732 meta-analyses, the proportion supporting the null hypothesis increased from 20% for small to 55% for medium to 75% for large effect thresholds, while inconclusive findings declined correspondingly.

CONCLUSION: Most nonsignificant meta-analyses provide weak evidence favoring the observed treatment effect rather than supporting no difference, but evidence increasingly favors no difference as prespecified effect thresholds increase in magnitude. Likelihood ratios enable guideline developers and clinicians to distinguish genuine evidence of no difference from inconclusive findings and support more informed clinical recommendations and better-targeted research prioritization.