Your new experience awaits. Try the new design now and help us make it even better

ORIGINAL RESEARCH article

Front. Res. Metr. Anal.

Sec. Research Assessment

Volume 10 - 2025 | doi: 10.3389/frma.2025.1679842

A pilot study investigating the relationship between journal impact factor and methodological quality of real-world observational studies

Provisionally accepted
  • Bridge Medical Consulting Ltd, Richmond, United Kingdom

The final, formatted version of the article will be published soon.

Introduction: The primary objective of this study was to investigate the association between journal impact factor (IF) and study quality in real-world observational studies. The secondary objective was to explore whether the association changes as a function of different study factors (study design, funding type and geographic location). Methods: Study quality was assessed using the Newcastle-Ottawa Scale (NOS). IFs were obtained from journal websites. The association between journal IF and NOS score was evaluated firstly using Spearman’s correlation coefficient, and secondly using one-way analysis of variance (ANOVA). Results: We selected 457 studies published in 208 journals across 11 consecutive systematic literature reviews (SLRs) conducted at our organization over the last 5 years. Most studies were cross-sectional and from North America or Europe. Mean (SD) NOS score was 6.6 (1.03) and mean (SD) IF was 5.2 (4.5). Overall, there was a weak positive correlation between NOS score and IF (Spearman’s coefficient (ρ) = 0.23 [95% CI: 0.13 – 0.31]; p<0.001). There was no correlation between NOS score and IF for prospective cohort studies (ρ = 0.07 [95% CI: -0.12 – 0.25]) and industry-funded studies (ρ = 0.06 [95% CI: -0.09 – 0.21]). Based on ANOVA, the effect size, eta squared (η²), was 0.04 (95% CI: 0.01 – 0.08), indicating a small effect. Discussion: While there is some correlation between journal quality and study quality, our findings indicate that high-quality research can be found in journals with lower IF, and assessing study quality requires careful review of study design, methodology, analysis, interpretation, and significance of the findings. Notably, in industry-funded studies, no correlation was found between methodological quality and IF.

Keywords: Impact Factor, Study quality, Newcastle-Ottawa scale, Critical appraisal, Observational study

Received: 05 Aug 2025; Accepted: 03 Oct 2025.

Copyright: © 2025 Gupta, Kaur and Malik. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Mansi Malik, mansimalik@bridgemedical.org

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.