Target product profiles for digital health technologies including those with artificial intelligence: a systematic review

Abstract

Digital health technologies (DHTs), including those incorporating artificial intelligence (AI), have the potential to improve healthcare access, efficiency, and quality, reducing gaps between healthcare capacity and demand. Despite prioritisation in health policy, the adoption of DHTs remains limited, especially for AI, in part due to complex system requirements. Target product profiles (TPPs) are documents outlining the characteristics necessary for medical technologies to be utilised in practice and offer a way to align DHTs’ research and development with health systems’ needs. This systematic review examines current DHT TPPs’ methodologies, stakeholders, and contents. A total of 14 TPPs were identified, most targeted at low- and middle-income settings and communicable diseases. Only one TPP outlined the requirements for an AI device specifically. In total, 248 different characteristics were reported across the TPPs identified and were consolidated down to 33 key characteristics. Some considerations for DHTs’ successful adoption, such as regulatory requirements or environmental sustainability, were reported inconsistently or not at all. There was little standardisation in TPP development or contents, and limited transparency in reporting. Our findings emphasise the need for guidelines for TPP development, could help inform these, and could be used as a basis to develop future DHT TPPs.

Systematic Review Registration: https://www.researchprotocols.org/2024/1/e50568/authors.

1 Introduction

Digital health technologies (DHTs), including those incorporating artificial intelligence (AI), promise improved access, efficiency, and quality of healthcare, helping meet a growing mismatch between capacity and demand. Consequently, they have attracted significant public (1) and private investment (2), as well as prioritisation in health policy (3–5). The UK provides an example of a country with a strong political mandate to accelerate the adoption of DHTs and AI within the National Health Service (NHS) (1, 3), but where few have been integrated at scale (6). Many innovations fall into a widening implementation “gap” or “chasm” (7, 8) as they fail to meet the complex requirements of the wider UK health system (8–10). This is a problem shared with other countries (11, 12) but particularly pronounced in the UK, where multiple stakeholders are tasked with evaluating, implementing, and monitoring DHTs, including regulators, health technology assessment bodies, and local or national commissioners. These stakeholders' requirements can range from place-based evaluations of diagnostic or clinical utility (13), to cybersecurity (14) and environmental sustainability (15); however, many are poorly understood or defined, particularly for frontier technologies such as AI. This makes product development challenging, resulting in significant waste in research and development (16, 17).

Target product profiles (TPPs) offer a potential solution, providing a mechanism for health systems to “demand signal” to innovators. TPPs outline the desired characteristics of a product aimed at a particular disease or diseases (18). First utilised in the pharmaceutical industry, they have since been adapted by governments (19) and non-governmental organisations (NGOs) (18, 20) to outline the characteristics necessary for products to improve outcomes for patients and healthcare systems (21, 22) and enhance research efficiency. In a UK context, TPPs can fulfil key policy priorities to improve “demand signalling” (23), and facilitate wider digital transformation (3, 4) and innovation in life sciences (24). As a result, TPPs have attracted significant interest from key UK stakeholders, including the Medicines and Healthcare products Regulatory Agency (MHRA) (19), National Institute of Health and Care Excellence (NICE) (25), and Cancer Research UK (CRUK) (26, 27).

The absence of consensus on best practice for TPP development and contents presents a challenge to those seeking to develop them however (22, 28). Added to this, most TPPs to date have focused on in vitro diagnostics or therapeutics aimed at infectious diseases and low- and middle-income countries (LMICs) (22, 28), making their methods and contents potentially less generalisable to the UK context and DHTs, particularly those incorporating AI as or in a medical device.

This review aims to provide an overview of current DHT TPP methods, stakeholders, and contents to support the development of future such TPPs, including those that could be fulfilled by AI technologies.

2 Methods

This study is reported in line with the Preferred Reporting Items for Systematic review and Meta-Analyses (PRISMA) reporting guideline, with a checklist provided in Supplementary File S1. It was conducted in line with a protocol for our wider programme of work previously published (29).

MEDLINE, EMBASE, Web of Science (full collection and preprint), and ACM Digital Library were searched on 23 May 2023 using queries with search terms relevant to TPPs and DHTs such as “quality by design,” “target product profil*,” “QTPP*,” “digital health*,” “(online or web or internet or digital*),” and “(app or apps)” (see Supplementary File S2 for the full search strategy). Searches were developed using terms identified by an information specialist or published in previous systematic reviews (22) or online (30). No date or language limits were used.

A web search was conducted using methods outlined by Godin et al. (31) (see Supplementary File S1). Two researchers (TM and LV) performed Google searches and searched specific websites independently on 22 June 2023 and 8 July 2023, respectively, screening hits and their associated web pages. Records potentially relevant to the review were recorded on a spreadsheet on Microsoft Excel (Version 365; Microsoft Corporation, Redmond, WA, USA).

The online literature review platform Rayyan (Rayyan Systems, Cambridge, MA, USA) (32) was used to conduct this review. Rayyan's duplicate identification function was used to identify duplicates from the database search, which were then reviewed and removed manually from the list of potentially relevant records generated by manual web searches by one researcher (TBM). Two researchers (TBM and HDJH or LV) independently screened all remaining records by title and abstract and then full text against the inclusion and exclusion criteria. Records were included if they contained a TPP outlining minimum and/or desired characteristics for a product for use in healthcare and were for a DHT as defined by the NICE Evidence Standards Framework (ESF) for DHTs (13). References were excluded if they did not contain a TPP, the target product did not affect patient care (e.g., if it described a product or process used in pharmaceutical manufacture), or was not for a DHT as defined by the ESF. Disagreements between reviewers were resolved by discussion and arbitration by the senior author (AKD). The bibliographies of records included after the full-text screening were hand-searched for relevant references.

Two researchers (TM and HDJH) independently extracted information regarding the included TPPs and their development methods. The ESF was used to stratify target products into seven risk categories based on their potential risk to patients or healthcare systems: Tier A: System services; Tier B: Communicating about health and care; Tier B: Health and care diaries; Tier B: promoting good health; Tier C: Inform clinical management; Tier C: Drive clinical management; Tier C: Treat a specific condition; and Tier C: Diagnose a specific condition. The subdivision of Tier C aligns with the software as a medical device classification framework proposed by the International Medical Device Regulators Forum (33). The TPP development stages “scoping,” “drafting,” and “consensus-building” were taken from the study by Cocco et al. (22). Disagreements between reviewers were resolved by discussion and arbitration by the senior author (AD).

One researcher (TM) extracted all characteristics reported in previous TPPs, grouping these into the clusters “unmet clinical need,” “analytical performance,” “clinical validity,” “clinical utility,” “cost,” “environmental impact,” “regulatory requirements,” “human factors,” and “infrastructural requirements” outlined by Cocco et al. (22). Characteristics were deduplicated and consolidated by one researcher (TM), focusing on those relevant to software or in a medical device. All the characteristics originally reported, their clusters, consolidated characteristics, and exclusions were reviewed two other authors (HDJH and AKD). Disagreements were resolved by discussion and arbitration by the senior author (AKD).

Risk of bias assessments were not completed as no formal tools exist to assess TPPs.

3 Results

Figure 1 outlines the results of the search and selection process.

Figure 1

3.1 Target product profile publication details and funding

Seven records met the inclusion criteria (34–40) and are listed in Table 1. Four records were identified from the database search (34, 36, 37, 39) and three from the Internet search (35, 38, 40). The publication year ranged from 2016 to 2023. Six records were journal articles (34, 36–40) and one was a PhD thesis (35). All were open access (34–40). A total of 14 TPPs were reported in the seven records. Falzon et al. included nine TPPs developed during the same study, eight of which met the inclusion criteria (34). Government agencies played a role in funding 12 (85.7%) TPPs. This was through USA (34) or UK (36, 37, 40) foreign aid, although one study (35) received funding from a UK research council. NGOs were involved in funding 10 (71.4%) TPPs (34, 37, 39), and universities funded 1 (7%). One TPP (7%) was funded solely by industry (38).

3.2 Target conditions, settings, and technologies

Of the 14 TPPs, 12 (85.7%) focused on infectious diseases (34, 35, 38, 40). Two targeted primary care, one to read rapid diagnostic tests (39) and the other provided clinical decision support (37), neither of which specified a communicable or non-communicable disease target. An LMIC target setting was explicit or implicit in all included TPPs, although some clinical problems could be seen as priorities for both high- and low-income countries, such as influenza (35) or antimicrobial resistance (40). Only one TPP specified the target product as being an AI device (38), although this was an adaptation of a technology agnostic TPP (41), rather than a de novo AI TPP.

Of the 14 TPPs, 5 (35.7%) outlined products with a NICE ESF Tier A risk classification (34, 40), 2 (14.3%) with a Tier B classification (34), and 7 (50%) with a Tier C classification (34–39). Of those in Tier C, one was classified as “inform clinical management” (34), four as “drive clinical management” (34, 36, 37, 39), and two as “diagnose a condition” (35, 38). No TPPs outlined a therapeutic target product.

3.3 Methods for target product profile development

Table 2 outlines included the TPPs’ development methods and participants. Scoping methods included expert opinion (40), “landscape analysis” (35), literature reviews (35, 36, 39), online questionnaires (34), and feedback gathered from pilots (40). The stakeholders involved in scoping were not clearly defined in 10/12 (71.4%) TPPs. In 4/14 (28.6%) TPPs where scoping stakeholders were defined, these comprised academics (35), NGOs (36, 37, 40), and clinicians (40). The number of participants involved in scoping was not stated in 12/14 (85.7%) TPPs, with two (36) and six (37) participants involved in scoping for the other two.

Of the 14 TPPs, characteristics for 6 (42.8%) were drafted through a meeting (34, 37) with the drafting method not stated in 8 (57.1%). Stakeholders involved in drafting included academics (35, 37, 38), government (37, 38), industry (38), and NGOs (35, 36, 38); however, 10/14 (71.4%) TPPs did not state the stakeholders involved. The number of stakeholders involved in drafting was stated in 4/14 (28.5%) TPPs (35–38) and ranged from two (36) to 39 (37) participants.

The use of a consensus method was reported in 12/14 (85.6%) TPPs (34, 36, 37, 39, 40). Four TPPs (28.5%) stated using either a “Delphi-like process” (36, 37) or “e-Delphi” (39, 40). Maximum Delphi round participant numbers were 28 (37), 40 (39), 43 (40), and 51 (36). Participants included academics (36, 37, 40), clinicians (37, 40), consultants (36), industry (36, 39, 40), government (37, 39, 40), and NGOs (36, 37, 39, 40). The consensus method was an in-person meeting and online iteration (methodology unclear) in 8/14 (57.1%) TPPs (34), although the number of participants and their stakeholder groups were not stated.

3.4 Characteristics reported in target product profiles

The TPPs reported 248 different characteristics (see Supplementary Table S1). Of the 14 TPPs, all reported characteristics in the clusters “unmet clinical need” and “infrastructural requirements,” 13 (92.9%) reported characteristics in the “human factors,” 11 (78.6%) in the clusters “clinical utility,” “costs,” and “regulatory requirements,” 8 (57.1%) in “clinical validity,” and 5 (35.7%) in “analytic performance” (see Table 3). None reported characteristics in “environmental impact.”

The number of characteristics reported in previous TPPs was reduced to 33 after deduplication, consolidation, and exclusion (see Table 3). Supplementary Table S1 outlines the destination of each originally reported characteristic.

4 Discussion

To our knowledge, this study represents the first systematic review of TPPs for DHTs and is of particular relevance given the increasing interest in these technologies’ wider adoption. We adapted established methods to create a robust strategy for the identification and evaluation of TPPs. Those included predominantly focus on LMICs and lack transparency and patient and end-user input in their development. Standardisation of TPP methods, contents, and transparency is strikingly lacking. Despite this, the identified TPPs consistently report a range of characteristics that could form the basis of future TPP development.

We used established, peer-reviewed methods for the identification, categorisation, and evaluation of TPPs for DHTs, increasing our review's comprehensiveness, robustness, and reliability. The terms used to identify records in bibliographic databases were developed in consultation with an information specialist and included combinations of terms to identify TPPs for DHTs as varied as health informatics solutions, electronic health records, software as a medical device, apps, artificial intelligence, and telemedicine. The filters used to identify TPPs have been published previously in peer-reviewed literature (22), while our Internet search strategy used established methods (31) and resulted in the identification of a further three TPPs. NICE's ESF was used to categorise target technologies and was developed through an extensive consensus process (13), while the methods used to categorise TPPs’ development and characteristics have been published previously after peer review (22). TPPs were evaluated by two researchers working independently. This approach is likely to have identified the majority of TPPs for DHTs published up to the search dates, with their methods and contents evaluated in a robust, reliable, and unbiased way.

Every TPP identified by this review focused on LMICs and predominantly communicable diseases. These are findings similar to previous reviews for diagnostic tests (22) and medical technologies in general (including therapeutics) (28). Although unsurprising given TPPs’ prior utilisation and championing by NGOs with a LMIC/global health focus (18, 20, 42), this potentially makes their contents and characteristics less generalisable to high-income country (HIC) contexts.

We found a lack of clear, transparent reporting of TPPs’ development methods and participants, again echoing the findings of previous reviews (22, 28). Recognising TPPs’ noble ambitions to draw funding towards neglected diseases and contexts, and that TPP research is likely to be similarly under-resourced, this lack of transparency makes critical appraisal challenging and undermines DHT TPPs’ reliability and comprehensiveness. Greater transparency is particularly important if TPPs are to be used in HICs, as this would represent a significant opportunity for regulatory or policy capture (43). We therefore echo previous calls for standardisation in TPP methods, contents, and reporting (22, 28) to improve transparency. Although a World Health Organization (WHO) TPP generation process was utilised by a number of included TPPs (37–40), this document is not in the public domain. TPPs developed using this process used Delphi methods, an established and validated consensus process; however, TPP development would benefit from formal guidelines for development and reporting published open access, similar to those published for reporting guidelines (44) and core outcome sets (45).

A key output of this review is a list of characteristics reported in previous DHT TPPs (see Table 3; Supplementary Table S1). Although the number of TPPs we identified was relatively small (n = 14), they reported 248 different characteristics. Their consolidation down to 33 characteristics (Table 3) suggests a reassuring level of consistency in TPPs’ scope and contents, although the reporting of considerations key to the successful use of DHTs was inconsistent. This included key elements, such as target population (4/14, 28.6%) and pathway position (3/14, 21.4%) [key components of an intended use statement (46)], to effects on clinical and service outcomes (8/14, 57.1% and 4/14, 28.6%, respectively), data governance and security (9/14, 64.3%), and regulatory requirements (6/14, 42.9%). Guidelines for DHT TPPs’ development and contents could help address these gaps in future, improving such documents’ comprehensiveness and reliability.

Despite variability in the scope and comprehensiveness of individual TPPs, the input of stakeholders with significant knowledge and expertise in DHT development and implementation to TPP development, such as government agencies (34–37, 40), NGOs (34, 37, 39), and industry (38), means the characteristics they report as a whole are likely to be fairly comprehensive. Using these as the basis for future TPPs’ development may therefore ensure future TPPs have sufficient scope and granularity.

Before doing so, however, it is important to consider if key characteristics may have been omitted from previous DHT TPPs as a whole. Using the UK context as an example, comparison to relevant policy documents, such as those used to guide health technology assessments (13) or AI procurement (47), highlights significant gaps. This includes factors such as DHTs’ environmental sustainability, with digital transformation set to play a key role in fulfilling the NHS’ commitment to net zero by 2045 (48); social value, an essential part of government procurement and commissioning (15); and effects on health inequalities, a persistent UK policy concern and priority (4). These are concerns and considerations shared with other HICs (49–51). TPPs for DHTs that may be met by AI technologies should also address concerns regarding AI's potential for algorithmic bias (52) and performance changes over time (53), taking into account requirements utilised to mitigate these in evaluation and implementation (54–56).

TPPs must reflect the needs of end users to be of utility. Only 3/14 (21.4%) TPPs stated that clinicians were involved in their development (37, 39, 40) and none stated that they involved patients in the development process. Although this would have been challenging given many TPPs’ supranational focus, end-user and patient involvement is essential for the development of future documents, particularly in HIC contexts. In the UK, patient involvement is essential in healthcare research and priority setting (13, 47, 57), a position increasingly adopted in other HICs (58–60). Patient involvement in DHT TPP development is essential not only because DHTs may affect patient care or handle sensitive information, but because these technologies must meet wider public expectations to be sustainably adopted (6). This is particularly relevant as many DHTs are seen as a means to empower patients to better manage their own health (61), meaning patients may be the target product's end user.

As well as significant patient, public, and end-user involvement, the development of future TPPs for specific national contexts would likely benefit from the close involvement of relevant regulators, health technology assessment bodies, and healthcare systems. These stakeholders could seek to develop their own TPPs, a role similar to that taken by WHO and other NGOs in LMIC settings or by the UK's MHRA during the COVID-19 pandemic, when it signalled to industry the UK's demand for such tests as well as the agency's likely product requirements. Alternatively, these stakeholders may wish to contribute to the TPP development processes led by others, such as academia or patient advocacy bodies. This could be by providing legislative requirements, standards, or guidance in general, or recommendations tailored to a specific product or disease area. Given regulators’, health technology assessment bodies’, and healthcare systems’ crucial role in approving, commissioning, and monitoring DHTs, their involvement is likely to be crucial to impart these documents with sufficient accuracy and authority, particularly in HIC settings.

4.1 Limitations

This study has several limitations. First, publicly available TPPs likely represent a fraction of those developed, with many internal to pharmaceutical or medical device companies (21, 28) and therefore not in the public domain. These documents may offer more refined methods, characteristics, or best practice, not captured by this review.

In addition, much of our assessment of TPPs’ details, methods, and characteristics was subjective. Although two researchers performed data extraction and analysis to mitigate this, there remains a residual risk of misinterpretation and bias, particularly as TPPs were often poorly reported. Scoping, drafting, and consensus methods and participants were often hard to identify, with information having to be pieced together from limited information in the manuscript. For example, stakeholder involvement often had to be deciphered from authors’ affiliations. It is possible that a doctor specialising in infectious diseases could be a clinician, academic, member of governmental or non-governmental organisations, or industry consultant, with an affiliation provided to only one or a limited number of these, thus making our judgement of stakeholder involvement less accurate. This further strengthens the argument for increased standardisation and transparency in TPPs’ development and reporting.

Finally, the inclusion of TPPs for DHTs integrated within or working downstream of in-vitro diagnostics (IVDs), such as lateral flow tests, introduces the potential for misinterpretations of characteristics. For example, terms like “sensitivity” and “calibration” may differ in meaning between DHT and IVD contexts. “Sensitivity” may refer to analytical or diagnostic sensitivity, while “calibration” may refer to setting or adjusting the measurement system of a laboratory instrument or assay, or the agreement between the predicted and actual observed outcomes for a prognostic DHT. Confusion could be avoided in future with agreed definitions for these terms included in DHT TPP development guidelines.

5 Conclusions

This review highlights the current state of the art in the development and contents of DHT TPPs, as recorded in the medical and grey literature. It has found significant weaknesses in TPPs’ methods, contents, and reporting, emphasising the need for greater standardisation and transparency. This review could inform best practice or formal reporting guidelines for TPPs. In addition, we report a list of characteristics distilled from existing DHT TPPs that could provide a starting point for the development of similar documents in future, including those incorporating AI.

References

• 1.

  The Artificial Intelligence in Health and Care Award. NHS Transformation Directorate. London: NHS Transformation Directorate (2020). Available at:https://transform.england.nhs.uk/ai-lab/ai-lab-programmes/ai-health-and-care-award/ (Accessed August 22, 2023).

  • Google Scholar
  • View reference in article

• 2.

  KrasnianskyAEvansBZweigMShahPHawksCDell’AquiloGet al2021 Year-end Digital Health Funding: Seismic Shifts Beneath the Surface. San Francisco, CA: Rock Health (2022). We’re Powering the Future of Healthcare. Rock Health is a Seed and Early-stage venture Fund That Supports Startups Building the Next Generation of Technologies Transforming Healthcare. Available at:https://rockhealth.com/insights/2021-year-end-digital-health-funding-seismic-shifts-beneath-the-surface/ (Accessed February 22, 2023).

  • Google Scholar
  • View reference in article

• 3.

  NHS Long Term Plan. NHS Aims to Be a World Leader in Artificial Intelligence and Machine Learning Within 5 Years. London: NHS England (2019). Available at:https://www.longtermplan.nhs.uk/nhs-aims-to-be-a-world-leader-in-artificial-intelligence-and-machine-learning-within-5-years/

  • Google Scholar
  • View reference in article

• 4.

  DarziA. Independent Investigation of the National Health Service in England. London: Department of Health and Social Care (2024). Available at:https://assets.publishing.service.gov.uk/media/66f42ae630536cb92748271f/Lord-Darzi-Independent-Investigation-of-the-National-Health-Service-in-England-Updated-25-September.pdf

  • Google Scholar
  • View reference in article

• 5.

  Office of the Chief Information Officer (OCIO). HHS Artificial Intelligence (AI) Strategy. Hhs.gov. Washington, DC: US Department of Health and Human Services (2021). Available at:https://www.hhs.gov/programs/topic-sites/ai/strategy/index.html (Accessed November 7, 2024).

  • Google Scholar
  • View reference in article

• 6.

  GreenhalghTWhertonJPapoutsiCLynchJHughesGA’CourtCet alBeyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. (2017) 19:e367. 10.2196/jmir.8775

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 7.

  AristidouAJenaRTopolEJ. Bridging the chasm between AI and clinical implementation. Lancet. (2022) 399:620. 10.1016/S0140-6736(22)00235-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 8.

  LaunderM. Babylon seeks to sell UK business. Health Serv J. (2023). Available at: http://hsj.co.uk/primary-care/babylon

  • Google Scholar
  • View reference in article

• 9.

  HernA. Royal Free Breached UK Data law in 1.6m Patient Deal with Google’s DeepMind. London: The Guardian (2017). Available at:http://www.theguardian.com/technology/2017/jul/03/google-deepmind-16m-patient-royal-free-deal-data-protection-act (Accessed March 18, 2022)

  • Google Scholar
  • View reference in article

• 10.

  O’LearyL. How IBM’s Watson Went from the Future of Health Care to Sold Off for Parts. New York, NY: Slate (2022). Available at:https://slate.com/technology/2022/01/ibm-watson-health-failure-artificial-intelligence.html (Accessed August 22, 2023).

  • Google Scholar
  • View reference in article

• 11.

  The Learning Health System Series, National Academy of Medicine. In: MathenyMIsraniSTAhmedMWhicherD, editors. Artificial Intelligence in Health Care. Washington, DC: National Academies Press (2019). 10.17226/27111

  • CrossRef
  • Google Scholar
  • View reference in article

• 12.

  LekadirKQuaglioGGallinC. Artificial Intelligence in Healthcare: Applications, Risks, and Ethical and Societal Impacts. Brussels: European Parliamentary Research Service (2022). Available at:https://www.europarl.europa.eu/RegData/etudes/STUD/2022/729512/EPRS_STU(2022)729512_EN.pdf

  • Google Scholar
  • View reference in article

• 13.

  National Institute for Health and Care Excellence. Evidence Standards Framework for Digital Health Technologies. London: National Institute for Health and Care Excellence (2022). Available at:https://www.nice.org.uk/corporate/ecd7/resources/evidence-standards-framework-for-digital-health-technologies-pdf-1124017457605 (Accessed September 29, 2022).

  • Google Scholar
  • View reference in article

• 14.

  NHS Digital. Clinical Risk Management: Its Application in the Manufacture of Health IT Systems—Specification. Leeds, England: NHS Digital (2018). Report No.: DCB0129.

  • Google Scholar
  • View reference in article

• 15.

  Greener NHS. Applying net Zero and Social Value in the Procurement of NHS Goods and Services. London: Greener NHS (2022). Available at:https://www.england.nhs.uk/greenernhs/publication/applying-net-zero-and-social-value-in-the-procurement-of-nhs-goods-and-services/ (Accessed September 3, 2024).

  • Google Scholar
  • View reference in article

• 16.

  WynantsLVan CalsterBCollinsGSRileyRDHeinzeGSchuitEet alPrediction models for diagnosis and prognosis of COVID-19: systematic review and critical appraisal. Br Med J. (2020) 369:m1328. 10.1136/bmj.m1328

  • CrossRef
  • Google Scholar
  • View reference in article

• 17.

  PriceWN2ndCohenIG. Privacy in the age of medical big data. Nat Med. (2019) 25:37–43. 10.1038/s41591-018-0272-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 18.

  Target product profiles. Available at:https://www.who.int/observatories/global-observatory-on-health-research-and-development/analyses-and-syntheses/target-product-profile/who-target-product-profiles (Accessed November 7, 2024).

  • Google Scholar
  • View reference in article

• 19.

  Medicines and Healthcare products Regulatory Agency. Target Product Profile: Point of Care SARS-CoV-2 Detection Tests. London: Gov.uk. (2020). Available at:https://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/target-product-profile-point-of-care-sars-cov-2-detection-tests (Accessed November 15, 2022).

  • Google Scholar
  • View reference in article

• 20.

  Target Product Profiles. Geneva: FIND (2022). Available at:https://www.finddx.org/tools-and-resources/rd-and-innovation/target-product-profiles/ (Accessed November 7, 2024).

  • Google Scholar
  • View reference in article

• 21.

  BrederCDDuWTyndallA. What’s the regulatory value of a target product profile?Trends Biotechnol. (2017) 35:576–9. 10.1016/j.tibtech.2017.02.011

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 22.

  CoccoPAyaz-ShahAMessengerMPWestRMShinkinsB. Target product profiles for medical tests: a systematic review of current methods. BMC Med. (2020) 18:119. 10.1186/s12916-020-01582-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 23.

  NHS Accelerated Access Collaborative. Horizon Scanning and Demand Signalling. London: NHS Accelerated Access Collaborative (2022). Available at:https://www.england.nhs.uk/aac/what-we-do/demand-signalling/ (Accessed October 18, 2024).

  • Google Scholar
  • View reference in article

• 24.

  HM Government. Life Sciences VisionLondon: HM Government (2021). Available at:https://assets.publishing.service.gov.uk/media/612763b4e90e0705437230c3/life-sciences-vision-2021.pdf

  • Google Scholar
  • View reference in article

• 25.

  Target Product Profiles of Digital Health Technologies. London: National Institute for Health and Care Excellence (2024).

  • Google Scholar
  • View reference in article

• 26.

  CablingMLDawneyJNapierMMarciniak-NuquiZOlumogbaFKesslerLet alAdvancing the Development and use of Diagnostic Target Product Profiles for Cancer. London: Cancer Research UK (2024).

  • Google Scholar
  • View reference in article

• 27.

  LloydJMoorthieS. Developing Target Product Profiles to Support Better Cancer Tests. London: Cancer Research UK—Cancer News (2024). Available at:https://news.cancerresearchuk.org/2024/10/17/target-product-profiles-tpps-for-cancer-diagnostic-tests/ (Accessed October 18, 2024).

  • Google Scholar
  • View reference in article

• 28.

  IbnidrisALiaskosNEldemEGunnAStrefferJGoldMet alFacilitating the use of the target product profile in academic research: a systematic review. J Transl Med. (2024) 22:693. 10.1186/s12967-024-05476-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 29.

  MacdonaldTDinnesJManiatopoulosGTaylor-PhillipsSShinkinsBHoggJet alTarget product profile for a machine learning-automated retinal imaging analysis software for use in English diabetic eye screening: protocol for a mixed methods study. JMIR Res Protoc. (2024) 13:e50568. 10.2196/50568

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 30.

  WuL. Research Guides: Electronic Health Records: EHR Literature. Memphis, TN: The University of Tennessee Health Science Center (2017). Available at:https://libguides.uthsc.edu/c.php?g=657371&p=4615105 (Accessed May 12, 2023).

  • Google Scholar
  • View reference in article

• 31.

  GodinKStapletonJKirkpatrickSIHanningRMLeatherdaleST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev. (2015) 4:138. 10.1186/s13643-015-0125-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 32.

  OuzzaniMHammadyHFedorowiczZElmagarmidA. Rayyan—a web and mobile app for systematic reviews. Syst Rev. (2016) 5:210. 10.1186/s13643-016-0384-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 33.

  IMDRF Software as a Medical Device (SaMD) Working Group. “Software as a Medical Device”: Possible Framework for Risk Categorization and Corresponding Considerations. International Medical Device Regulators Forum (2014). Available at:https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf

  • Google Scholar
  • View reference in article

• 34.

  FalzonDTimimiHKurosinskiPMiglioriGBVan GemertWDenkingerCet alDigital health for the end TB strategy: developing priority products and making them work. Eur Respir J. (2016) 48:29–45. 10.1183/13993003.00424-2016

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 35.

  KeaneCE. Towards a Smartphone Connected Test for Influenza. London: UCL (University College London) (2019). Available at:https://discovery.ucl.ac.uk/id/eprint/10071946/1/Keane_10071946_thesis.pdf

  • Google Scholar
  • View reference in article

• 36.

  KadamRWhiteWBanksNKatzZDittrichSKelly-CirinoC. Target product profile for a mobile app to read rapid diagnostic tests to strengthen infectious disease surveillance. PLoS One. (2020) 15:e0228311. 10.1371/journal.pone.0228311

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 37.

  PelléKGRambaud-AlthausCD’AcremontVMoranGSampathRKatzZet alElectronic clinical decision support algorithms incorporating point-of-care diagnostic tests in low-resource settings: a target product profile. BMJ Glob Health. (2020) 5:e002067. 10.1136/bmjgh-2019-002067

  • CrossRef
  • Google Scholar
  • View reference in article

• 38.

  WardPBroadfieldLADahlbergPLetaGMekonnenZNabatteBet alThe development of an artificial intelligence-based digital pathology for neglected tropical diseases: a platform specific analysis of the World Health Organization diagnostic target product profile for soil-transmitted helminthiasis. Front Trop Dis. (2022) 3. 10.3389/fitd.2022.990304

  • CrossRef
  • Google Scholar
  • View reference in article

• 39.

  WhiteWKadamRMoussyF. Target product profile for readers of rapid diagnostic tests. Bull World Health Organ. (2023) 101:331–40. 10.2471/BLT.23.289728

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 40.

  TobinMFerreyraCPitonJKelly-CirinoCKatzZKadamR. Development of a target product profile for a one health antimicrobial resistance surveillance service. Oxf Open Digit Health. (2023) 1:1–6. 10.1093/oodh/oqac001

  • CrossRef
  • Google Scholar
  • View reference in article

• 41.

  Diagnostic Target Product Profiles for Monitoring and Evaluation of Soil-transmitted Helminth Control Programs.Geneva: World Health Organization (2021).

  • Google Scholar
  • View reference in article

• 42.

  PATH. Geneva: PATH (formerly known as the Program for Appropriate Technology in Health) (1996). Available at: https://www.path.org/ (Accessed November 7, 2024)

  • Google Scholar
  • View reference in article

• 43.

  Organisation for Economic Co-operation and Development. Preventing Policy Capture: Integrity in Public Decision Making. Paris: Organisation for Economic Co-operation and Development (2017). Available at:https://www.oecd-ilibrary.org/governance/preventing-policy-capture_9789264065239-en (Accessed October 24, 2024).

  • Google Scholar
  • View reference in article

• 44.

  How to develop a reporting guideline. Available at:https://www.equator-network.org/toolkits/developing-a-reporting-guideline/ (Accessed October 25, 2024).

  • Google Scholar
  • View reference in article

• 45.

  WilliamsonPRAltmanDGBagleyHBarnesKLBlazebyJMBrookesSTet alThe COMET handbook: version 1.0. Trials. (2017) 18:280. 10.1186/s13063-017-1978-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 46.

  Crafting an Intended Purpose in the Context of Software as a Medical Device (SaMD). London: Gov.uk (2023). Available at:https://www.gov.uk/government/publications/crafting-an-intended-purpose-in-the-context-of-software-as-a-medical-device-samd (Accessed November 28, 2024).

  • Google Scholar
  • View reference in article

• 47.

  JoshiICushnanD. A Buyer’s Guide to AI in Health and Care. London: NHS-X (2020). Available at:https://transform.england.nhs.uk/media/documents/NHSX_A_Buyers_Guide_to_AI_in_Health_and_Care.pdf

  • Google Scholar
  • View reference in article

• 48.

  VäänänenT. Why We’ve put Sustainability into the NHS Digital Design Principles. London: NHS England Digital (2023). Available at:https://digital.nhs.uk/blog/design-matters/2023/why-weve-put-sustainability-into-the-nhs-digital-design-principles (Accessed October 24, 2024).

  • Google Scholar
  • View reference in article

• 49.

  Sustainability for Health Care—Achieving your Sustainability Goals. Chicago, IL: American Hospital Association (2023). Available at:https://www.aha.org/sustainability (Accessed November 7, 2024).

  • Google Scholar
  • View reference in article

• 50.

  WhittyJALittlejohnsP. Social values and health priority setting in Australia: an analysis applied to the context of health technology assessment. Health Policy. (2015) 119:127–36. 10.1016/j.healthpol.2014.09.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 51.

  European Parliamentary Research Service. Addressing Health Inequalities in the European Union Concepts, Action, State of Play. Brussels: European Parliament (2020). Available at:https://www.europarl.europa.eu/RegData/etudes/IDAN/2020/646182/EPRS_IDA(2020)646182_EN.pdf

  • Google Scholar
  • View reference in article

• 52.

  ObermeyerZPowersBVogeliCMullainathanS. Dissecting racial bias in an algorithm used to manage the health of populations. Science. (2019) 366:447–53. 10.1126/science.aax2342

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 53.

  KellyCJKarthikesalingamASuleymanMCorradoGKingD. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. (2019) 17:195. 10.1186/s12916-019-1426-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 54.

  GanapathiSPalmerJAldermanJECalvertMEspinozaCGathJet alTackling bias in AI health datasets through the STANDING together initiative. Nat Med. (2022) 28:2232–3. 10.1038/s41591-022-01987-w

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 55.

  LiuXGlockerBMcCraddenMMGhassemiMDennistonAKOakden-RaynerL. The medical algorithmic audit. Lancet Digit Health. (2022) 4:e384–97. 10.1016/S2589-7500(22)00003-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 56.

  Medicines and Healthcare Products Regulatory Agency. Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles. London: Gov.uk (2023). Available at:https://www.gov.uk/government/publications/predetermined-change-control-plans-for-machine-learning-enabled-medical-devices-guiding-principles (Accessed October 25, 2024).

  • Google Scholar

• 57.

  Public Participation Team. Patient and Public Participation Policy. London: NHS England (2017). Available at:https://www.england.nhs.uk/wp-content/uploads/2017/04/ppp-policy-edit.pdf

  • Google Scholar

• 58.

  Consumer and community engagement. Available at:https://www.nhmrc.gov.au/about-us/consumer-and-community-involvement/consumer-and-community-engagement (Accessed November 28, 2024).

  • Google Scholar

• 59.

  Horizon Europe Strategic Plan (2021–2024). Brussels: European Commission (2021). Available at:https://www.eeas.europa.eu/sites/default/files/horizon_europe_strategic_plan_2021-2024.pdf

  • Google Scholar

• 60.

  The Value of Engagement in Research. The Value of Engagement in Research. Washington, DC: PCORI (2018). Available at:https://www.pcori.org/engagement-research/value-engagement-research (Accessed November 28, 2024).

  • Google Scholar

• 61.

  What is Digital Health Technology and What can it do for me?London: National Institute for Health Research (2022). Available at:https://evidence.nihr.ac.uk/collection/what-is-digital-health-technology/ (Accessed November 7, 2024).

  • Google Scholar