ORIGINAL RESEARCH article
Front. Res. Metr. Anal.
Sec. Research Assessment
Volume 10 - 2025 | doi: 10.3389/frma.2025.1679842
A pilot study investigating the relationship between journal impact factor and methodological quality of real-world observational studies
Provisionally accepted- Bridge Medical Consulting Ltd, Richmond, United Kingdom
Select one of your emails
You have multiple emails registered with Frontiers:
Notify me on publication
Please enter your email address:
If you already have an account, please login
You don't have a Frontiers account ? You can register here
Introduction: The primary objective of this study was to investigate the association between journal impact factor (IF) and study quality in real-world observational studies. The secondary objective was to explore whether the association changes as a function of different study factors (study design, funding type and geographic location). Methods: Study quality was assessed using the Newcastle-Ottawa Scale (NOS). IFs were obtained from journal websites. The association between journal IF and NOS score was evaluated firstly using Spearman’s correlation coefficient, and secondly using one-way analysis of variance (ANOVA). Results: We selected 457 studies published in 208 journals across 11 consecutive systematic literature reviews (SLRs) conducted at our organization over the last 5 years. Most studies were cross-sectional and from North America or Europe. Mean (SD) NOS score was 6.6 (1.03) and mean (SD) IF was 5.2 (4.5). Overall, there was a weak positive correlation between NOS score and IF (Spearman’s coefficient (ρ) = 0.23 [95% CI: 0.13 – 0.31]; p<0.001). There was no correlation between NOS score and IF for prospective cohort studies (ρ = 0.07 [95% CI: -0.12 – 0.25]) and industry-funded studies (ρ = 0.06 [95% CI: -0.09 – 0.21]). Based on ANOVA, the effect size, eta squared (η²), was 0.04 (95% CI: 0.01 – 0.08), indicating a small effect. Discussion: While there is some correlation between journal quality and study quality, our findings indicate that high-quality research can be found in journals with lower IF, and assessing study quality requires careful review of study design, methodology, analysis, interpretation, and significance of the findings. Notably, in industry-funded studies, no correlation was found between methodological quality and IF.
Keywords: Impact Factor, Study quality, Newcastle-Ottawa scale, Critical appraisal, Observational study
Received: 05 Aug 2025; Accepted: 03 Oct 2025.
Copyright: © 2025 Gupta, Kaur and Malik. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Mansi Malik, mansimalik@bridgemedical.org
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.