Following initial deduplication 6,653 potential eligible titles were returned by searches, 519 (7.8%) of which were included following title and abstract screening. Full-text screening identified 202 unique eligible studies (Figure 1). Three (1.5%) of these reports were theses with the remaining 198 (98.5%) consisting of articles in academic journals (Table 1). Excluding 2016, the frequency of eligible publication increased year-on-year, with a monthly rate of 4.9 publications averaged over January-October 2022 compared to 0.6 between January-December 2014 (Figure 2). Thirty-five different countries hosted the healthcare context under study, with the United States (n = 56, 27.7%), United Kingdom (n = 29, 14.4%), Canada (n = 16, 8.0%), Australia (n = 16, 7.9%) and Germany (n = 11, 5.4%) the most frequent countries studied. Six studies (3.0%) were based in countries categorized by the United Nations as having a medium or low human development index (31). Of the 172 studies focused on a single clinical specialty, primary care (n = 48, 27.9%) and psychiatry (n = 16, 9.3%) were the most common of 27 distinct clinical specialties.

Seventy eligible reports (34.7%) applied at least one of 50 distinct TMFs in the main text (Table 2), 7 (14.0%) of these were new TMFs developed within the eligible article itself. Theory application was increasingly prevalent as studies focused closer toward real-world use, with studies of hypothetical, simulated or active clinical use cases applying TMFs in 26.9%, 34.8% and 42.3% of studies respectively. There was no significant difference between the frequency of TMF application before and after the start of 2021, the median year of publication (Chi squared test, p = 0.17). Twelve (17.1%) of the 70 reports drawing on a TMF applied more than one [maximum 5 (82)]. Of the 87 instances that a TMF was applied it originated from the fields of technology acceptance (n = 36, 41.4%), practice theory (n = 21, 24.1%), organizational theory (n = 19, 21.8%) or usability (n = 11, 12.6%) according to Liberati's taxonomy (30). Similarly, under Nilsen's taxonomy of TMFs the purpose of each TMF applied could be classified as determinant framework (n = 49, 56.3%), process model (n = 18, 20.7%), classic theory (n = 10, 11.5%), evaluation framework (n = 9, 10.3%) or implementation theory (n = 1, 1.1%) (29).

The Technology Acceptance Model was the most frequent choice when a TMF was applied (n = 9, 12.9%), but 40 (80.0%) of the TMFs were only applied once across all eligible reports. Across the 87 instances of reports explicitly applying a TMF, 4 different modes of application emerged; to inform the study or intervention design (n = 9, 10.3%), to inform data collection (n = 29, 33.3%), to inform data analysis (n = 44, 50.6%) and to relate or disseminate findings to the literature (n = 25, 28.7%). The majority of instances in which a report applied a TMF carried no explanation or justification (n = 51, 58.6%). Five (5.7%) reports made isolated endorsement of the TMF's popularity or quality, e.g., “The sociotechnical approach has been applied widely…” (83). Thirty-one (35.6%) outlined the alignment of the TMF and the present research question, e.g., “our findings are consistent with disruptive innovation theory…” (84). Eleven (12.6%) reports discussed the disadvantages and alternatives that had been considered, e.g., “Because this model does not consider the unique characteristics of the clinical setting… we further adopted qualitative research techniques based on the CFIR [Consolidated Framework for Implementation Research] to further identify barriers and facilitators of the AI-based CDSS [Clinical Decision Support System]” (85).

This study shows that a minority of clinical AI qualitative research applies TMFs, with no suggestion of a change in the relative frequency of TMF application over time. This appears to contrast with research funders and policy makers increasingly valuing more theory-based definitions of evidence and the consistent requirement for TMFs in related reporting guidelines and evaluation criteria (25, 86–88). Underlying this increasing appreciation of the contribution that TMFs can make, is a perception that specific research questions with unique configurations of complexity can draw on prior knowledge through the application of a well-matched theoretical approach (29). It is the great variety of unique research questions that may justify the ever-increasing variety of available TMFs. If considered matching of a specific research question's demands and a specific TMF's value is not taking place however, the ongoing proliferation of TMFs may only serve to further alienate practitioners trying to make sense of the shifting landscape of TMFs (7).

Within this study's relatively narrow eligibility criterion of qualitative clinical AI research, the variety and inconsistency of TMFs applied was striking, with 80% of the 50 TMFs encountered only applied once. This variation in TMF selection was also mirrored by the their varied purpose and mode of application. Across these applications of TMFs, a convincing rationale for their selection was usually absent. This heterogenous TMF selection coupled with little evidence of considered selection, suggests that current TMF application in qualitative clinical AI research usually fails to satisfy established definitions of good practice in implementation research (2, 25). If it is assumed that meeting these definitions of good practice would more effectively support implementation science's goal of bridging know-do-gaps, then it seems likely TMF application is currently under-delivering for efforts to translate clinical AI into practice. The observed heterogeneity in TMF selection is also set to grow, as 15% of the theories applied in eligible articles were novel. This may improve current practice in TMF application if these novel TMFs better serve the needs of research questions in clinical AI implementation. However, only 1 of these 7 novel TMFs has been applied within the other eligible reports of this bibliometric study and so there is a real risk of exacerbating unjustified heterogeneity in TMF usage (45).

To the best of our knowledge, there are no other reviews of TMF application in qualitative implementation research of digital health. Smaller scoping reviews concerning specific disease areas and clinical guideline implementation, and a survey of implementation scientist practices are published, but their findings differ to the present study's in two important regards. Firstly, the heterogeneity of TMF application selection appears to be much greater in the present study, with half of guideline implementation studies applying at least one of the same 2 TMFs (20, 21). The preferences across implementation scientists in general also seem to differ from researchers working on clinical AI implementation as only 2 of the TMFs identified in the present study (Theoretical Domains Framework and Consolidated Framework for Implementation Research) appeared in the 10 most frequently applied TMFs from a survey of an international cohort of 223 implementation scientists (6). These differing preferences may be accounted for by the prominence of TMFs in qualitative clinical AI research from Technology Acceptance disciplines (40.9%), as described by Liberati's taxonomy, which do not have such natural relevance across implementation science as a whole (30). Secondly, the frequency with which any degree of rationale for TMF selection was described in the present study (42%) appears much lower than the 83% observed in guideline implementation research (21). Both of these differences seem to reflect the field of clinical AI and its nascent engagement with formally trained implementation scientists who have more established means of selecting TMFs (6). Taken together, the heterogenous and unjustified selection of TMFs suggests superficial use or misuse of TMFs is common and that clinical AI research is yet to benefit from the full value of TMF-research question alignment experienced by other areas of implementation research (18, 25, 86–89). Given the potential of unjustified heterogeneity to lower the accessibility of implementation research to relevant stakeholders, avoidance of TMF application may be preferable to their superficial use or misuse (6).

There are a number of tools which have been designed, validated and disseminated to reduce the underuse, misuse and superficial use of TMFs demonstrated here and in implementation research generally (2, 90). To aid researchers in the rationalised selection of TMFs, interactive open access libraries and selection tools are available with embedded learning resources (91, 92). Following selection of a TMF, many of the authors of more prominent TMFs develop and maintain toolkits to support the appropriate and effective mobilization of their TMF to varied applications (93, 94). There are also reporting guidelines and quality criteria which support peer reviewers and academic journal editors in identifying quality research and incentivizing researchers to adopt good practices. Apart from occasional exceptions in the present study however, none of these tools were mentioned or used (86, 89, 95, 96). The present study adds to these resources for implementation researchers working in clinical AI by summarizing TMF use to date within the field, with examples of good practice (55, 56, 85). Paradoxically, it seems that the limitation on improving TMF application is not the presence of solutions, but their implementation.

A strength of this study is the eligibility criteria, which facilitated the large number of eligible articles relative to pre-existent bibliometric studies of TMF applications in implementation research (20–22). The study also summarizes TMF applications in clinical AI research, a prominent and growing category of digital health implementation research which had not yet been subject to any similar bibliometric studies. Without clear incentives for authors to report the perceived impact, mode or rationale of TMF application, a lack of information in eligible articles for the present study does not exclude a theoretical foundation. This risk of over-interpreting negative findings is not unique to the present study but is a further limitation to hold in mind (97). A final limitation comes from the eligibility criteria for the present study which focus on qualitative research of clinical AI, to maximise the representation of TMFs among eligible articles at the cost of implementation studies which exclusively use quantitative methods. Whilst this does limit comparability to bibliometric studies of guideline implementation research or other areas, it appears to have succeeded in identifying a greater sample of TMF applications within clinical AI than found by alternative criteria in more established fields of research (20, 21).

Firstly, the ambiguity over the value of ensuring that implementation research that is “theoretically informed”, in a well-characterized and reproducible way, should be minimized through adequately resourced programmes of research. This is not in order to generate more TMFs, but to establish the impact of TMF application under current definitions of good practice. Without it, the challenge laid out in one of the first issues of the journal Implementation Science will continue to limit support from stakeholders influencing the implementation of TMFs: “Until there is empirical evidence that interventions designed using theories are generally superior in impact on behavior choice to interventions not so designed, the choice to use or not use formal theory in implementation research should remain a personal judgment” (19). A negative finding would also prevent future research waste in championing the proliferation and application of TMFs.

Secondly, if TMFs are proven to improve implementation outcomes then scalable impact within clinical AI and elsewhere cannot depend upon the oversight of implementation experts on any more than a small number of high priority implementation endeavors. Therefore, work to improve the accessibility and apparent value of existent TMFs and tools to promote their uptake should be prioritized (2, 91, 92). A focus on training and capacity building across a wider community of researchers and practitioners may also be beneficial (92, 98). Academic journal editors and grant administrators could be influential in endorsing or demanding relevant tools and guidelines, helping to improve the quality, consistency and transparency of theoretically informed clinical AI implementation research. Improved accessibility across existent TMFs would also help to tighten the relationship between frequency of application and efficacy of TMFs, helping to reduce the potentially overwhelming variety of TMFs available. If such a shortlist of popular TMFs emerged, with a clearer rationale and value for application, it could improve the accessibility of TMFs to a greater breadth of the implementation community. This could establish a virtuous cycle of improving frequency and quality of TMF application, mitigating against the researcher-practitioner divide described in implementation science (7).