Expanding access to veterinary clinical decision support in resource-limited settings: a scoping review of clinical decision support tools in medicine and antimicrobial stewardship

Abstract

Introduction:

Digital clinical decision support (CDS) tools are of growing importance in supporting healthcare professionals in understanding complex clinical problems and arriving at decisions that improve patient outcomes. CDS tools are also increasingly used to improve antimicrobial stewardship (AMS) practices in healthcare settings. However, far fewer CDS tools are available in lowerand middle-income countries (LMICs) and in animal health settings, where their use in improving diagnostic and treatment decision-making is likely to have the greatest impact. The aim of this study was to evaluate digital CDS tools designed as a direct aid to support diagnosis and/or treatment decisionmaking, by reviewing their scope, functions, methodologies, and quality. Recommendations for the development of veterinary CDS tools in LMICs are then provided.

Methods:

The review considered studies and reports published between January 2017 and October 2023 in the English language in peer-reviewed and gray literature.

Results:

A total of 41 studies and reports detailing CDS tools were included in the final review, with 35 CDS tools designed for human healthcare settings and six tools for animal healthcare settings. Of the tools reviewed, the majority were deployed in high-income countries (80.5%). Support for AMS programs was a feature in 12 (29.3%) of the tools, with 10 tools in human healthcare settings. The capabilities of the CDS tools varied when reviewed against the GUIDES checklist.

Discussion:

We recommend a methodological approach for the development of veterinary CDS tools in LMICs predicated on securing sufficient and sustainable funding. Employing a multidisciplinary development team is an important first step. Developing standalone CDS tools using Bayesian algorithms based on local expert knowledge will provide users with rapid and reliable access to quality guidance on diagnoses and treatments. Such tools are likely to contribute to improved disease management on farms and reduce inappropriate antimicrobial use, thus supporting AMS practices in areas of high need.

1 Introduction

To address the global threat of antimicrobial resistance, antimicrobial stewardship (AMS) programs have been implemented in both the human and animal health sectors (1, 2). AMS refers to a coherent set of actions that promote the responsible use of antimicrobials, including optimal selection, dose, route of administration, duration, and control of antibiotic treatment (3–5). These programs play a pivotal role in slowing the development of antimicrobial resistance by promoting responsible and judicious use of antimicrobial agents and preserving their future effectiveness. However, the success of AMS programs often hinges on the support, education, and behavior of those who make prescribing decisions and administer antimicrobial agents (6, 7). Inappropriate or overuse of antimicrobials persists because a significant portion of antimicrobial prescriptions and treatments are overseen by individuals who may possess a limited understanding of the risks associated with antimicrobial resistance (8, 9). While rapid and accurate diagnostic tools are necessary for effective AMS, treatment decisions are often made in the absence of information on the infectious agent and antimicrobial sensitivity profile. This is especially so in resource-limited settings, where treatment decisions frequently result in inappropriate antimicrobial use (10–12). In response to these challenges, digital clinical decision support (CDS) tools have been designed to assist clinicians in making evidence-based diagnostic and therapeutic decisions to enhance AMS at the point-of-care.

Digital CDS tools are of increasing importance in human medicine. They are used in diverse medical sectors and clinical environments, ranging from hospitals to primary healthcare settings, and increasingly for interdisciplinary care of patients with complex needs. CDS tools integrate patient-specific data with clinical knowledge into a computerized “consultation” process to support clinical decision-making (13). These tools offer a wide variety of functions, including interpreting diagnostic test results, aiding in surgical decision-making, and generating prescriptions. CDS tools can also provide additional features such as integration with electronic health records, warning alerts and pop-up reminders on patient encounters, computerized treatment guidelines, automated data entry, and dynamic interactive programs customized for individual patients (14). CDS tools are also increasingly used to assist clinical decision-making in antimicrobial management by facilitating the selection of appropriate antimicrobials and determining the correct dosage, route of administration, and duration of treatment (15, 16). Over time, CDS tools have demonstrated many benefits, including improving diagnostic accuracy, treatment selection, antimicrobial prescription, reducing antimicrobial usage and hospital admissions, shortening hospital stays, and decreasing mortality rates and healthcare costs (17, 18).

The rapid growth of information technology developments has seen “simple” CDS tools evolve to include advanced mathematical modeling such as multivariate regression, neural networks, decision trees, and probabilistic models, such as Bayesian networks (19, 20). These mathematical models can be applied as CDS tool inference engine algorithms to enable the estimation of the likelihood of a patient suffering from a particular disease (informing diagnostic decisions) (21, 22), identification of the optimal treatment strategy for the patient’s condition (informing treatment decisions) (23, 24), and/or estimation of the probability that the selected treatment will result in a certain outcome (informing prognosis estimations) (25, 26). This rapid technological advancement has seen CDS tools transformed from traditional paper-based formats to digital forms, encompassing computer software, websites, or mobile phone applications. They can function as standalone tools or be integrated into complex IT systems, such as hospital electronic health records (27). For example, it has been reported that approximately 40% of US hospitals have adopted CDS integration with electronic health records, alongside other high-income countries such as Canada, the United Kingdom, Denmark, and Australia (28). As health data become increasingly large and complex, new CDS tools require increasingly powerful and sophisticated methodologies, such as artificial intelligence and machine learning. These advanced technologies have the potential to transform diagnosis and therapy at the point of care.

In contrast to human health, CDS tools are less widespread in veterinary settings, although several CDS tools have been reported in the literature for companion animals (29), cattle (30), pigs (31), poultry (32), horses (33), and aquaculture (34, 35). Veterinary CDS tools appear to be primarily developed for sophisticated livestock farming enterprises or for veterinary hospitals equipped with advanced technologies, two scenarios mostly found in high-income countries. However, arguably the greatest potential for veterinary CDS tools lies in those countries where veterinary infrastructure (human resources and diagnostic laboratories) is limited. Rapid diagnosis of the disease is essential to controlling an outbreak, and while it is ideal that veterinarians are available, this is not always possible, especially in remote locations or where there is an absence of veterinary infrastructure. In these cases, having diagnostic support in the form of a CDS tool that can emulate the way a veterinarian thinks could be an aid in managing disease. For example, a study carried out in Ethiopia demonstrated that the use of a smartphone-based application can be a valuable means to provide disease diagnosis and appropriate treatment recommendations by less experienced animal health professionals, which may lead to increased animal productivity (36). Another CDS tool, Fish-VET has been in operation since 1996 and has shown to be a useful diagnostic aid for veterinarians, students, and others seeking to manage disease in tropical and pond fish (34). Properly designed CDS tools have been shown to be able to be used by other animal health professionals, such as paravet or directly by producers. Thus, CDS tools would appear to be valuable for managing disease in livestock populations in remote areas of low- and middle-income countries (LMICs) with limited access to animal healthcare institutions or veterinary professionals (37, 38).

Several publications have reviewed CDS tools, focusing on their features, outputs, benefits, and limitations (39–41), and others have reported on the underlying technologies and methodologies adopted by CDS tools (28, 42–44). Meanwhile, none have explored digital CDS tools that support AMS, and none have explored CDS tool use in the context of animal health in low- and middle-income countries. Moreover, there is limited information regarding CDS tool development in the early design phase, in the construction of mathematical models, or in the evaluation of factors that contribute to the successful implementation of CDS tools. Therefore, our scoping review aims to address these knowledge gaps. Specifically, we aim to (i) identify the scope, functions, and impact of digital CDS tools on diagnosis, treatment, and AMS, (ii) describe the methodologies and technical designs used in the development of these CDS tools, (iii) assess their quality and effectiveness using the GUIDES checklist (45), and (iv) provide recommendations for the development of veterinary CDS tools aimed at enhancing AMS in LMICs.

2 Methods

This study was designed as a scoping review to investigate the literature and identify knowledge gaps using a systematic search process. This scoping review was conducted and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for Scoping Reviews (PRISMA-ScR) (46).

2.1 Eligibility criteria

With the definition of CDS tool varies depending on its function and feature, digital CDS tools reviewed in this study were defined as tools designed as a direct aid to support diagnosis and or treatment decision-making by providing real-time output based on computerized clinical knowledge (28). The published scientific literature and gray literature considered for inclusion in this review were those that described any form of a digital CDS tool that was (i) currently implemented and used, or being piloted, or in trial, (ii) intended to support diagnosis and/or treatment, (iii) targeted toward human or animal health, and (iv) published in scientific or gray literature between 1 January 2017 and 24 October 2023. Studies were excluded if the CDS tool was not digital (i.e., not computer software, website, and/or mobile application based), if the tool did not fit within the definition of a digital CDS tool previously described, if the study did not provide sufficient information on the features and functions of the tool (including where full text was not accessible), and/or if the article was not published in English.

2.2 Search strategy

PubMed Central was the primary information source for this review and was searched on three occasions in June 2022, January 2023, and October 2023. Google was used as a secondary database to scan for additional studies and gray literature using similar search terms as the primary database. To enhance the electronic database search, Google was also searched for gray literature on CDS tools. Additionally, a “backward snowballing” approach developed by Wohlin (47) was utilized by scanning the reference lists of articles included for extraction and analysis along with gray literature to identify digital CDS tools missed from the primary and secondary search strategies. The search strategy used multiple combinations of keywords relating to the development and implementation of digital CDS tools and CDS systems in human or veterinary health. The final search terms used for PubMed Central were (“digital tool”[Title/Abstract] OR “digital support tool”[Title/Abstract] OR “decision support tool”[Title/Abstract] OR “decision support system”[Title/Abstract]) AND (“health”[Title/Abstract] OR “veterinary”[Title/Abstract]) AND 2017/01/01: 3000/12/31[Date—Publication]. Keyword searches used in Google included “decision support tool” and “decision support system”; additional words were added at the beginning of the keywords, including “digital,” “diagnosis,” “animal,” and “veterinary.” Screening was conducted on the first 100 results of each keyword combination.

2.3 Study selection

Study selection was performed in February 2023 and October 2023. Titles and abstracts retrieved from each search were exported into Microsoft Excel. Titles and abstracts of identified publications were screened against the eligibility criteria by one of the two reviewers (HY and SB), along with a subset of excluded studies. All potentially relevant publications were independently checked by the second reviewer (HY or SB). Full texts were accessed to assess eligibility and determine inclusion by both reviewers (HY and SB). For gray literature, a full-text assessment was conducted to determine the eligibility of the article to be included in the analysis. Disagreements were resolved mutually (See Figure 1).

Figure 1

2.4 Data extraction and analysis

Study characteristics, including CDS type, study design, features or functions of the tool, technical workflow design, and relevant outcomes, were extracted manually into an MS Excel spreadsheet, according to the review’s data collection categories (Table 1). The quality of the CDS tools reviewed in this study was assessed using an adapted version of the GUIDES checklist (45, 49). The checklist consists of four domains that include: (i) CDS context, which refers to the set of conditions for the CDS tool to be potentially successful (ii) CDS content, which refers to the factors that determine the success of output produced by the tool, (iii) CDS system, which refers to the features available on the CDS tool, and (iv) CDS implementation, which refers to the factors influencing how CDS is incorporated into practice setting. Each domain consisted of four success factors, and information on each factor being addressed by each CDS tool is extracted.

3 Results

The primary search resulted in 1,741 articles for the review screening process. On the first screening (title and abstract), 1,263 articles were excluded because they did not meet the definition of a CDS tool specified for this review or were duplicates. On the second screening (full text), 451 articles were excluded, leaving 27 articles within the scope for this review. The Google search added 52 items to the screening process, comprising 25 peer-reviewed publications and 26 gray literature publications in the form of websites containing information about the tool’s scope, functions, and features. Many peer-reviewed publications found during the Google search were not identified in the primary search performed on PubMed Central. We believe this was related to the large number of errors reported on information retrieval in PubMed (50, 51), due to a lack of standardization and formatting of the articles, including title and abstract, inconsistent terminology to describe CDS tools (52), and incorrect indexing (53). The gray literature included in the screening process consisted of commercially available CDS tools (n = 23), government-funded CDS tools (n = 2), not-for-profit development projects (n = 2), a university institution tool (n = 2), and a CDS tool developed from a collaboration between a university and private company (n = 1). Out of 30 gray literature, 8 were included in the analysis, consisting of 7 commercially available CDS tools and 1 CDS tool developed by the university. The snowballing approach added a further five peer-reviewed scientific publications. Publications included from snowballing were not captured on the primary and secondary searches because the title or abstract did not contain the search query keywords (for example, an article used the name of the tool). Of the 57 items from secondary searches, 41 were ineligible for inclusion, leaving 16 items for inclusion. Thus, for the review, a total of 44 CDS tools were included for data extraction and analysis, comprising 36 peer-reviewed publications, 7 commercial CDS tools, and 1 free-access CDS tool developed by the university. The final 44 CDS tools are summarized in Supplementary Table S1.

The majority of CDS tools evaluated were designed for human health (n = 35), in addition to a number of CDS tools for animal health (n = 9) (Table 2). Of the CDS tools for humans, 13 targeted general patients in hospitals (15, 17, 18, 54–57), healthcare facilities (21, 36, 58–60), or primary healthcare (61), while 5 tools targeted pediatric patients (23, 26, 62–64) (Table 2). The remaining CDS tools assisted with decision-making for pregnancy (65), diabetes (66), critically ill patients (67), older adult patients with polypharmacy (68), patients with syncope (69), drug allergies (25), diarrhea (70), neurological disorders (71), urinary tract infections (72), dermatology symptoms (22), obstructive sleep apnea (73), hypertension (74), systemic lupus erythematosus (75), as well as patients requiring joint replacements (76) and oncology surgery (77). The CDS tools for animals were for cattle (36, 80, 81), swine (31, 85), companion animals (29, 82, 83), and a CDS tool for dogs, cats, cattle, horses, sheep, goats, swine, birds, and poultry (84). Eleven digital CDS tools were commercially available, all implemented in high-income countries (15, 18, 22, 29, 31, 58, 59, 62, 82, 83, 85). Of the commercially available CDS tools, six were for humans (15, 18, 22, 58, 59, 62) and five for animals (22, 29, 31, 82, 83). A subscription fee was required for three human CDS tools (15, 22, 58) and four veterinary tools (29, 31, 82, 83). Comprehensive information on each digital CDS tool is available in Supplementary Table S1.

Most CDS tools reviewed were developed or implemented in high-income countries (n = 36, 82%) (Figure 2). The United States of America had the most CDS tools (n = 12, 27%), followed by Germany (n = 5, 11%). Among CDS tools developed or implemented in countries with lower economic status, 7% were from middle-income countries, 9% were from lower-middle-income countries, and 2% were from low-income countries (Figure 2). The majority of the CDS tools reviewed (n = 28) were standalone CDS tools, while 13 CDS tools were integrated into electronic health records in hospital settings. Two CDS tools had both standalone and integrated features (18, 83).

Figure 2

3.1 Digital CDS tool scope and function

The CDS tools reviewed vary in functions, including patient safety, clinical management, cost containment, administrative functions/automation, diagnostics support (including diagnostic imaging, laboratory diagnostics, and pathology), patient decision support, better documentation, and workflow improvement (Table 3). Most of the human and veterinary CDS tools (n = 40) have the capacity to improve patient safety by reducing diagnostic and or treatment errors. Twenty-seven CDS tools generate diagnostic output, with treatment output being generated by 25 tools. Seventeen CDS tools can augment and visualize laboratory tests when providing diagnosis or treatment recommendations. Eighteen CDS tools demonstrated cost-effectiveness by reducing unnecessary treatment (15, 17, 18, 23, 25, 66, 70, 71, 76), duration of stay in hospital (69), laboratory tests (59, 83), and health provider workload (31, 36, 54, 55, 58, 82).

3.2 CDS tool development and methodologies

The teams for the development of the digital CDS tools varied in terms of the skills and numbers of contributors. Thirty CDS tools were developed by a multidisciplinary team consisting of people from medical, pharmacy, and/or veterinary backgrounds collaborating with people from computer engineering, economics, and private companies. Four CDS tools were developed by a team of people of varying medical backgrounds, e.g., intensive care medicine, emergency medicine, pediatric medicine, infectious disease, cardiovascular medicine, college of medicine, and department of pathology (57, 64, 67, 69). One digital CDS tool was developed by a team of medical engineers (21). In terms of funding sources for the tool development, 12 tools reviewed in this study were funded by a research grant (24, 36, 55–57, 61, 62, 67, 73, 76–78), 10 by commercial companies (15, 22, 29, 31, 58, 66, 71, 82, 83), 8 by a development project grant (17, 23, 64, 69, 70, 74, 75, 80), and 4 were funded by more than 1 funding source, such as awards and foundation grants (25), foundation and research grants (65, 79) and collaboration of medical research from two countries (59).

The stage of CDS tool development varied in the reviewed papers. Of the 44 studies reviewed, 32 present outcomes from preliminary work into their CDS tool design, such as a literature review, guidelines review, or expert elicitation process (Table 4). Twenty-five studies detail the development of the CDS tool database layer in the form of a computer hardware system or cloud computing, which is used to house data inputs (Table 4). Thirty-one studies describe the development of an inference engine, which is used to determine if CDS conditions have been met and execute queries. Of all the studies reviewed, 25 describe knowledge-based CDS, in which the inference engines run the built-in logic using if-then rules generated from guidelines or expert elicitation and 15 describe non-knowledge-based CDS, in which the engine applies machine learning of mathematical models such as genetic algorithm and artificial neural networks (86). Thirty-one CDS tools provide information on the user interface of the tool for data input from the patient toward the visualization of the results in the form of graphs and charts for the user. Of the 36 published studies evaluated, 20 detailed the study design during the development and/or implementation of the CDS tools. Seven studies used a retrospective study design (15, 25, 56, 59, 62, 71, 76), four studies conducted a pilot study (26, 36, 63, 72), two conducted a usability evaluation (74, 75), and two conducted a prospective observational cohort study (57, 64). The remaining published studies examined in this scoping review detailed cross-sectional (61), interrupted time series study (18), intervention study (70), longitudinal (55), and randomized control trial study designs (67).

Various mathematical methodologies were utilized for the CDS tools reviewed. Most reported methods included Bayesian theorems (n = 7), with two each for decision tree and rules-based (IF/THEN) methods (Table 5). For example, VetAfrica-Ethiopia, a CDS tool to assist in the diagnosis of cattle diseases in Africa, utilizes Bayesian algorithms to estimate the probabilities of diseases based on various presented clinical signs in Ethiopian cattle (36). Meanwhile, Autokinetics uses a self-built clinical dosing algorithm to recommend antibiotic selection and dosage. If the tool advice is being followed or if plasma data are available, the tool will use Bayesian estimation to approximate the true pharmacokinetic profile and forecasted pharmacokinetic profile (67). A combination of regression models was utilized by LHSpred (78) to calculate the computed tomography severity score (CTSS) for COVID-19 diagnosis. The “Tool to Reduce Inappropriate Medications” (TRIM), designed to prevent polypharmacy in older adults, utilizes a rules-based algorithm to assess the medication’s appropriateness (68). Details of various mathematical methodologies used by the CDS tools identified in this review are presented in Table 5.

3.3 CDS tools quality assessment

Of 16 success factors included in the GUIDES checklist, 14 were used to evaluate the CDS tools included in this study. Two factors, namely, the amount of decision support manageable for the target user and governance of the CDS implementation, were excluded as limited evidence was available to assess the CDS tools against these criteria. Many of the included studies and reports describe features that were evaluated in at least one domain, most commonly the CDS context. For some studies, inferences were made to evaluate the quality of the tool where it was not explicitly stated. While most studies describe the context for the CDS tool, fewer provide sufficient information to assess its content, system design, and implementation. Despite the limited information, CDS tool content quality appeared to improve diagnosis decision-making compared to other diagnosis methods, retrospective data of laboratory confirmation, or without the use of the CDS tool. CDS tools also appeared to improve treatment quality compared to other options, including (i) treatment generated without the use of CDS tools (15, 18, 61, 71), (ii) treatment provided by clinicians (62, 70), and (iii) treatment provided by previous tools (66, 67). Details on the CDS tool quality assessment are provided in Table 6.

3.4 Antimicrobial stewardship

Some CDS tools reviewed in this study are capable of supporting AMS principles (n = 12), including two CDS tools designed for animals (Table 2). AMS CDS tools for humans were implemented in hospitals (15, 17, 18, 54, 62, 70), healthcare facilities (36, 58, 64, 72), and primary healthcare (61). The AMS CDS tools primarily targeted clinicians, physicians, and pharmacists. These tools are designed to reduce medication dosing errors (54) or generate recommendations on antimicrobial dosing and duration based on prescribing guidelines, antibiograms, and pathogen information (15). Four CDS tools demonstrated improvement in AMS practices by reducing unnecessary or inappropriate antimicrobial use (15, 18, 70, 72). For example, one study reported that a CDS tool deployed in two academic hospitals in Canada saved $111,296 in antimicrobial expenditure over the 6 months period of implementation (15). Significant changes in antimicrobial use practices were reported in some studies where the CDS tool was designed to relatively increase or decrease the use of specific antimicrobials through its recommendations (18) and provided recommendations for antimicrobial use that were consistent with prescribing guidelines and recommendations (72).

4 Discussion

Our review of current CDS tools found that while such tools exist in a range of human healthcare settings, there is a dearth of such tools in veterinary settings. Further, while several CDS tools supported AMS practices in human healthcare settings, few supported AMS practices in animal health, an area of particular concern in the control of antimicrobial resistance (10, 11). The disparity in CDS tool availability based on a country’s economic status is stark, with most tools designed for hospital settings in higher-income countries. Very few CDS tools are available in LMICs, and this is where such tools may have the greatest impact. CDS tools designed for animals are of particular interest to countries with limited veterinary services and where decisions on the diagnosis and treatment of unwell animals are often made by untrained personnel (87). Our findings suggest that the field is ripe for the development of CDS tools across all livestock sectors, particularly those that support AMS practices that optimize antimicrobial use.

4.1 CDS tools context

The use of CDS tools in human healthcare settings has undergone rapid evolution and expansion since their first use in the 1980s (28). Technological advances, a desire to mitigate errors in healthcare, and access to funding have driven much of this evolution. High-income countries have been able to make the most of the technology and funding opportunities compared to LMICs (88). The slow progress in the development and adoption of digital CDS tools in LMICs is multifactorial. Factors such as limited funding for health programs, prioritization of resource allocation (89), high costs associated with tool development, subscription fees, and training users (90) are considered blockers to the development of CDS tools in these countries. Additionally, data ownership and protection in high-income countries are significantly better and protected by law (91). In contrast, in LMICs, personal data breach are a major challenge in digital innovation due to infrastructure limitations and insufficient legal protections (92, 93). This situation deters significant private-sector investment in CDS tool development due to the risk of legal repercussions from patient or client data breaches (94). Thus, the development and implementation of CDS tools in LMICs tend to be heavily dependent on support through international grants or agencies (95). Identifying long-term sustainable funding models in countries that could benefit most from using CDS tools should be a key consideration in the early phases of development.

In comparison to the human health sector, the development of CDS tools for animal health has been slow (96), despite the genuine need for such CDS tools in veterinary medicine in LMICs (97). The development of CDS tools in livestock species is situation-specific and no one CDS tool can be applied across species; further, in livestock industries, treating and nursing a sick animal is not always a priority for economic reasons, thus creating barriers to development. This is particularly the case given that investment in this sector is generally focused on equipment, tools, and software products to monitor production parameters (e.g., milk yields and feed conversion ratios), management parameters (e.g., herd structure and feed), and environmental parameters (e.g., temperature and humidity) (98, 99). When CDS tools have been developed for animals, they have focused primarily on livestock species, with the end users being veterinarians (29, 31, 36) or farm workers (31). An example of a highly effective livestock CDS tool we evaluated is VetAfrica-Ethiopia, which is used in Africa to improve herd productivity by supporting better diagnostic decisions where the availability of veterinarians and laboratory capacity is scarce (36).

To design effective tools, extensive interactions are required between biological disciplines (e.g., animal production, health, epidemiology, and microbiology) and computer science, economics, and sociology (37). Fifteen digital CDS tools were developed by multidisciplinary teams (e.g., medical and computer engineering). In line with recommendations for a multidisciplinary approach, as an example, the study conducted to develop the AMS Computerized Decision Support Tool in a Swiss hospital strongly recommended the involvement of a team with expertise both in clinical medicine and IT early in the project (17). Clinicians have an extensive knowledge of clinical workflow that can be translated into the architecture of CDS tools and inform the provision of output that is valuable from the user perspective (100). Nonetheless, with multidisciplinary teams comes the risk of miscommunication and misunderstanding (17). To overcome that, a recently published Delphi study highlighted the additional benefit of having hybrid positions: someone who has knowledge in both medical and IT, understands both languages, and is capable of holding the responsibility of translating the language into the multidisciplinary team (101).

The benefits of CDS tools in clinical practice should be balanced by an understanding of the well-described limitations of these tools. In any situation where disease diagnosis is required, a CDS tool can only suggest plausible diagnoses with an accuracy that reflects the accuracy of the information it receives (35). Like a human expert relying on their clinical judgment, there is always a degree of uncertainty with any outputs from a CDS tool; in some cases, CDS tool even has different sensitivity and specificity for each disease, which must be understood by the user when interpreting those outputs (21, 35, 102). Other challenges limiting the utility of CDS tools in healthcare settings include having tools that require information outside of their workflow, causing alert fatigue due to excessive amount of alerts that are often inconsequential, requiring upskilling and technological aptitude, being limited by poor data quality or “black box” scenarios which lower confidence in outputs, entail high costs, and so on (28). However, the potential positive impacts of CDS tools should outweigh these limitations, many of which could be overcome by proper planning, engagement with stakeholders, and good design. Certainly, in animal health, CDS tools can help unskilled animal careers make informed decisions and take precautionary actions when managing disease outbreaks in the absence of diagnostic tools or veterinary support. In resource-limited settings, even when the CDS tool is unable to make diagnosis, it is still capable of determining if there is a significant risk that the disease is important, thus directing the user and available resources toward only the most important cases.

4.2 Development of digital CDS tool

Referring to the GUIDES checklist on the factors that determine the successful implementation of a CDS tool (45), most factors need to be addressed from the development stage. These start with the basic architectural design of a CDS tool, which is comprised of three layers: (i) a database to capture data, (ii) an inference engine that performs data processing, and (iii) a user interface (103). Nineteen CDS tools reviewed in this study reported database development, indicating a crucial step in the early phase of tool development. The database is often a relational database; this should be designed with an appropriate level of performance, security, availability, manageability, and accessibility. Often, the relational database is programmed with a standardized language, for example, Structural Query Language (SQL), to retrieve, store, and manipulate the database and its contents (104). Database maintenance is an important element of tool operationalization (105). In human healthcare CDS tools, 11 studies reported CDS tools that shared a database with electronic health record systems (17, 23, 26, 54, 58, 61–63, 66, 67, 72). Using an existing, well-maintained, and functional database such as an electronic health record system allowed some of these tools (23, 26, 54, 61, 67) to provide real-time patient alerts and recommendations at the moment the prescribing choice is made, without the need for any intervention (17, 106).

The Bayesian algorithm was one of the most common inference engines (n = 7) used in the CDS tools evaluated, comprised of four naïve Bayesian classifiers and three Bayesian networks. Naïve Bayesian classifiers are relatively simple probabilistic classification algorithms that are computationally efficient (96). Naïve Bayesian classifiers assume that all input attributes are independent of each other and are shown to be optimal in minimizing misclassification in outcomes. Thus, reducing the need for an expert in pathogenesis to establish causal dependencies, as required in traditional Bayesian algorithms (19, 107). Studies have shown the naive Bayesian classifier model performs well when compared to more complex classification models, even if assumptions of independence of input attributes are violated (108–110). For example, the CPT tool was designed with a naïve Bayesian algorithm to support physicians with the treatment of oncologic disease using laryngeal cancer (LC), and the prototype provided correct model predictions in all cases throughout the validation (77). In veterinary settings, there is a desire to have CDS tools capable of aiding in the diagnosis of many diseases. Bayesian classifier models are particularly appealing for this purpose, given their flexibility compared to rules-based models (102), given the complexities involved in diagnosing diseases in animals. These complexities include the similarity of clinical signs among many diseases, variations in how animals within a population express clinical signs, changes in clinical signs as a disease progresses, and the potential for animals to suffer from more than one disease simultaneously.

However, Bayesian algorithms are not without some disadvantages. For example, McKendrick et al. highlight that the logical processing of Bayesian algorithms is not clear to users, as data inputted into the system are processed using a mathematical structure unlike rules-based algorithms (102). Additionally, Bayesian algorithms process all available information each time a user adds new data to the system, and so the new data are analyzed in the context of the previous information. In the case of Bayesian networks, while it can model complex problems where there is a significant degree of uncertainty and causal dependencies are involved, they require intensive computational development, limiting their practicality in a big data setting (19, 55). Despite these limitations, a significant benefit of Bayesian models as CDS tools in animal health is their potential for application in other livestock sectors. Once the investment in designing the algorithm and software has been made, they can be utilized in other sectors, provided conditional probabilities for that sector are available (102). As an example of the use of a Bayesian algorithm, the VetAfrica-Ethiopia CDS tool assists clinical diagnosis of cattle disease in a resource-limited setting and has demonstrated benefits in assisting less experienced animal health professionals despite having low accuracy for some diseases (36).

There are two categories of supervised machine learning models that currently exist: the white-box and black-box models. Bayesian networks, decision trees, and rule-based (IF/THEN) models are categorized as white-box model and characterized by having lower predictive performance compared to black-box model. However, white-box models possess excellent interpretability since the calculation at each point can be explained (55). In contrast to Bayesian, the decision tree is recognized for its user-friendly structure at the point of practice that allows the underlying logic to be seen by the user. Decision trees are closely related to rule-based models, as the hierarchical data structure of nodes from the root to a leaf in the decision tree corresponds to a rule. The advantage of decision trees over a rule-based model is that it is easier to grasp the model globally because of the tree structure compared to a sequential structure (55, 96). However, the training decision tree classifier is quite complex and potentially spiraling out of control due to the number of nodes in some cases, while a small training set also does not work very well. In addition, only one attribute is tested at a time, consuming a lot of time and being computationally expensive for certain domains (111, 112). On the other hand, black-box models, such as artificial neural networks, possess excellent predictive performance despite having minimal to zero interpretability (55). A study to develop an artificial neural network-based CDS tool for COVID-19 can offer high specificity and good sensitivity. The tool, however, developed through a small population in a single center and had the probability of overfitting problems. Thus, obtaining a multicenter dataset will allow external validation to improve the tool (56).

Validation of the CDS tool is a critical step in the design process. In the tools evaluated in this review, approximately half reported that a validation process was undertaken, with two studies describing the validation study in detail (25, 64). Validation in this review refers to a study in which the developer compares the accuracy of the CDS tool with a “gold standard” measure such as a laboratory test result (97). As most of the tools in this review were categorized as novel tool in their respective field, the availability of a so-called gold standard test was limited. Thus, most of the studies reported that they undertook internal validation using retrospective data from hospital records or publicly available data, and some also undertook external validation with experts. None of the studies reporting on CDS tools for animals reported on a validation study. This may be because any validation exercise using laboratory tests as the gold standard is costly and few clinical records are available for livestock. However, CDS tools will show their benefit if the diagnosis output of a particular disease is comparable to the laboratory test gold standard (28). For example, when the sensitivity and specificity between the CDS tool and laboratory standards are not significantly different (56, 78). Approaches to validation of CDS tools in animals are an important area requiring further consideration.

Several limitations were found in the reviewed CDS tools, mainly associated with the missing validation to provide evidence of tool accuracy (23, 54, 60, 65, 74). Fifteen CDS tools provide information on the tool’s accuracy, which corresponds to the sensitivity and specificity of the tool. However, positive predictive value (PPV) and negative predictive value (NPV) are more relevant when people or animals are being screened (113). Unfortunately, there was limited evidence to extract positive PPV–NPV from the study. As most of the tools are still in the development stage, limited data used for the trial and pilot, and the lack of sensitivity, specificity, PPV, NPV, and validation test might affect the accuracy of tool output when facing complex clinical situations during implementation (23, 25, 59). In addition, several tools also mentioned difficulties in capturing data on the output of the CDS tool compared to the subsequent action taken by the user, as well as the outcome of the action, to quantify the impact of the tool (54, 60, 65). There is also the need for user acceptability testing to ensure the user understands the context-specific information of the tool (24, 54). For CDS tools intended to aid diagnosis, the plausibility/accuracy of suggested diagnoses will only be as good as the base knowledge embedded into the inference engine (56, 77) and information inputted by the user (66, 79). CDS tools are not capable of giving a definitive diagnosis because there will always be a level of uncertainty with the outputs. However, these limitations should not detract from their potentially considerable positive impact on disease management. For situations where a veterinarian is not available, CDS tools can help people take informed action much sooner than they may have in the absence of informed guidance from the tool.

4.3 Impact on diagnosis and treatment

In our review, we found evidence that CDS tools can improve the quality of diagnosis of COVID-19 (56, 59), general diseases (79), and respiratory diseases (21). There is great potential for well-designed CDS tools to reduce diagnostic errors, particularly in primary healthcare and resource-limited settings. The CDS tools evaluated demonstrated they could automatically extract laboratory test results (59, 63, 66, 67, 71, 72), including diagnostic imaging (15, 23, 60) and pathology (22, 29) into the CDS tool to improve diagnostic accuracy. In addition to enhanced diagnosis decision-making, four CDS tools demonstrated that users made better treatment decisions, including regarding antimicrobial use, compared to the previous clinical practice tools (15, 18, 62, 67). An additional notable feature of a CDS tool was the generation of a list of treatment options with the prognosis of each option for the user to review before making treatment decisions (71). CDS tools were also shown to be successful in reducing prescribing errors, dosing errors, and contraindications when designed to have safeguards for dosing, duplication of therapies, and drug–drug interactions (61, 71). This is possible because the tools are equipped with features such as automated warnings and drug-event monitoring (114, 115). Among the limited number of CDS tools available for animal health, two CDS tools have a diagnosis function (36, 85), one has a treatment function (31), and one tool has both (29). However, no studies of tools in animal health settings have demonstrated improvement in diagnosis and treatment output.

It has been demonstrated that conventional clinical guidelines for diagnosis and treatment pathways are challenging to put into practice, with low clinical adherence (116, 117). One study suggested that up to 65% of hospital inpatients can be exposed to one or more potentially harmful combinations of medications (118). Meanwhile, CDS tools are able to process complex information and provide patient-specific treatment recommendations, given the probability of the diagnosis, which can inform more appropriate and effective prescription of medications (including the use of antimicrobials) (17, 54, 62, 70, 77). In addition, CDS tools can have features to alert clinicians when a patient has not adhered to a management plan or needs to be followed up, including reminders for testing when particular procedures for the patients need to be applied (119, 120). CDS tools that improve diagnostic accuracy and treatment of disease may be particularly valuable in locations where there is a lack of or limited access to clinical experts (121).

4.4 Impact on antimicrobial stewardship

Of the CDS tools designed to improve AMS practices, most were targeted at human healthcare professionals. The key objective of these tools was to verify prudent drug prescriptions, reduce inappropriate use of antimicrobials, and achieve healthcare-related cost savings (16). For example, Guidance MS, a commercially available tool, reported an improvement in appropriate antimicrobial use, a decrease in costs associated with prescribing antimicrobials, and a decrease in healthcare-associated Clostridium difficile infection rates, without increasing the length of hospital stay or mortality rates (18). A pilot study of a CDS tool developed for Canadian hospitals reported potential antimicrobial cost savings of $111,296 annually, or $403.98 per acute care bed per year, from improving prescribing behaviors, including appropriate dose and duration, and managing patient medication allergies (15). The CDS tools evaluated also reported improved prescribing practices in resource-limited settings (36, 70), demonstrating that these tools can be an effective intervention for AMS. However, other studies reported that the verification of prescriptions was found to be time-consuming and occasionally unpredictable due to human factors (106, 122). Additionally, in human healthcare settings, it is important to tailor AMS interventions to patients who are most at risk for prescription errors to receive priority care (17).

Just two CDS tools identified in this scoping review were designed to improve AMS practices in animals. Given global concerns about antimicrobial use in livestock species, particularly in LMICs (29, 31), this represents a significant opportunity for innovators to develop CDS tools that improve diagnostic and treatment decision-making and, in doing so, improve AMS practices.

4.5 A vision for veterinary CDS tools in low- and middle-income countries

Previous studies have evaluated common technologies and methodologies used to develop and implement CDS tools, but they have predominantly focused on human healthcare settings in high-income countries. Consequently, they often overlook significant global health challenges, such as infectious diseases and antimicrobial resistance, which are prevalent in LMICs. This review aimed to fill this gap by providing additional insights into digital CDS tools and the essential quality factors for their successful development, implementation, and application in human and animal health contexts, particularly in LMICs. Additionally, the review delves into the underlying mathematical methodologies and architectures of various formats and versions of CDS tools, offering further insights into designing future veterinary CDS tools tailored for resource-limited settings.

Development of digital innovations such as information systems and CDS in LMIC is often hindered by limited funding and resources, causing the tool to be unable to perform at the optimal level (64). The reliance on international development projects for development often faces a conflict of interest between stakeholders and a limited duration of project funding, making the tools not adequately developed (17). Furthermore, the user and stakeholders often perceive the innovation as a “free” aid, resulting in a lack of eagerness to maintain sustainability (123). Here, sustainability, means the ability to continue supporting a CDS tool after initial funding and technical assistance have ended (124). Planning for the development of the CDS tool is a crucial first step, including analyzing return on investment and ensuring long-term sustainability so that the CDS tool remains relevant to the sector it is targeted for, but unfortunately, reports of such analyses in the literature are rare (28). Without proper planning, it seems inevitable that a CDS tool will not survive beyond the initial design phase. Factors influencing sustainability include the functionality of the tool, the use of the right technology for the tool, training, and support for end users, leadership and governance, and securing finances beyond the design phase (28, 124). Sustainable business models need to be included in the planning phase of tool development, for example, by adding a subscription fee from the user. CDS tools for animals with a subscription fee have been shown to be more sustainably used when designed to provide real-time recommendations to the user. In addition, CDS tool users would be more willing to pay for the service if the tool can show cost-saving benefits (28). The development phase requires extensive infrastructure and human resources with expert knowledge of the sector and knowledge of coding, programming, computer science, and engineering (90). A multidisciplinary team comprising experienced veterinarians, IT developers, and end-users (e.g., para veterinarians, technicians, and farmers) is necessary to overcome important early challenges in CDS tool development in terms of collaboration and communication of concepts relevant to the development of a tool. It is important that concepts of tool functionality and algorithms are transparent and shared early to assess fitness-for-purpose and feasibility (17). Flexibility in the methodology and approach is essential to allow for the tool to adapt to changing diagnostic and or treatment protocols. In addition, effective validation exercises should occur at each development step to ensure the CDS tool fits with the user’s needs (17).

Another important aspect of CDS tool development is ensuring the successful use of CDS implementation. Making use of standardized guides and protocols, such as the GUIDE checklist, during the development and implementation of the CDS tool will likely improve the probability of designing a CDS tool that is fit-for-purpose. In animal health, CDS tools need to be designed with the end user in mind, especially given the unique and challenging situations where the tool will be needed. For example, a standalone CDS tool as a smartphone app that can be used offline will likely be necessary for users in remote locations or where mobile coverage is limited. Providing easy-to-understand information on how to use the tool and troubleshoot problems without the need for onsite training will likely have a positive impact on adoption, and reduce the funding required for implementation (36, 64).

In the absence of patient data to develop a referral database commonly found in animal healthcare in LMICs, veterinary handbooks and guidelines can be utilized to develop the reference data. However, limitations might occur due to country-specific conditions not covered by the guideline. To overcome this, referral databases can be built based on expert knowledge elicitation to ensure the CDS content provides trustworthy, relevant, and accurate evidence-based information. Transparency of this information and the sources is important to build confidence in the tool. With such limitations, the development of non-knowledge-based inference engines using Bayesian algorithms is recommended. Such algorithms are capable of modeling complex information with causal dependencies such as clinical signs of disease and animal management factors (36, 62, 64, 71). Non-knowledge-based systems have been found to offer significantly reduced healthcare costs and stress on medical practitioners in human healthcare settings (86). In LMICs, access to veterinarians and diagnostic tests is often limited, leading technicians and farmers to make diagnoses based on their judgment alone. This can result in misdiagnosis and treatment errors. Therefore, developing a CDS tool that can guide disease diagnosis with more accuracy than their (untrained) judgment alone and is developed through expert elicitation, will likely be perceived positively by the user. In addition, CDS tools that feature treatment recommendations based on the likely diagnosis should be integrated to encourage users to avoid inappropriate and excessive use of antimicrobials in situations where they may not be needed, thus supporting the AMS program.

Ultimately, the successful implementation of veterinary CDS tools for LMICs is dependent on a number of factors, many of which may not be easily addressed. However, the success of CDS tool implementation will be defined by several factors during development that include (i) proper planning in the initial development of the CDS tool, including a sustainable business model beyond the initial funding, (ii) collaborating with multidisciplinary stakeholders from the start to ensure the development of an effective tool, (iii) designing a low-cost tool that can be used in field situations and with minimal technology requirements, and (iv) being transparent and flexible in its outputs, which will, we think, go a long way toward improving the management of diseases on-farm and ultimately the use of antimicrobials in livestock.

4.6 Study limitations

Methodological limitations in this review include potential publication bias during the screening process and the analysis of the CDS tool scope and functions due to the nature of studies identified in this study. This review only used PubMed Central for the peer-reviewed publications source, and other search engines may have identified additional CDS tools for inclusion. The Google search keywords we used in this study were limited, as many other combinations of keywords were not sufficiently discriminating. Our search terms were deliberately narrowed to focus on publications where a CDS tool was mentioned in the title or abstract. However, it became apparent during our search that not all publications that described the development of a CDS tool used the terms included in our search strategy. For example, one study used the term “Intelligent System for Solving Problems” when describing a CDS tool developed to diagnose cow disease and assist in treatment decisions (80). Without checking for this publication using other methods, it would have been missed from our review. Thus, the lack of standardization in describing CDS tools in publications is a limitation of this study and also explains why a number of tools included were identified from the secondary search of gray literature or from the reference lists of publications. Further, we did not undertake a search of the Apple or Android App Store applications. In addition, we only searched among literature in the English language, thus excluding CDS tools that may have been specific to non-English language countries. It was challenging to properly assess CDS tool design and features, as some studies did not provide sufficient detail or were not sufficiently advanced to evaluate them. Therefore, in some instances, we made inferences about tool features that may be misclassified. For example, some peer-reviewed articles utilized commercially available CDS tools (15, 18, 59, 62) where the features and functions of the CDS tools were not presented. In addition, some commercially available tools require a subscription fee before gaining access to them. Therefore, the data on these tools were extracted from available websites or video demonstrations. In this case, shortcomings in the description of the CDS tool made it difficult to tabulate the profile, scope, and functions of the CDS tool. It is understandable that some commercially available CDS tool developers draw a line to maintain competition and protect intellectual property when sharing information about their products. However, we recommend that information related to the design and overall effectiveness of CDS tools be made available for open publication, that will allow other developers, especially in LMICs, to observe.

5 Conclusion

To support the development of veterinary CDS tools in LMICs, information regarding tool development, the construction of mathematical models, and quality factors determining the successful implementation of the CDS tool is limited. In addition, none of the previous studies addressed CDS tools in the context of animal health and LMICs, in which global health challenges such as infectious disease and antimicrobial resistance remain an issue. This review summarizes current digital CDS tools available in humans and animals to support clinicians by providing easy and rapid access to information required for diagnosis and treatment decisions at the point of care. Factors that determined the quality and effective use of the CD tool were being addressed, including the development phase, structure, methodology, and common factors that can help increase the probability of successful CDS tool implementation. Ensuring sufficient funding for the development and return on investment analysis to ensure long-term sustainability in the planning is a crucial first step. Deploying a multidisciplinary development team will also help design an effective tool. Developers can use the GUIDE checklist to ensure quality factors are addressed during the development stage, to help achieve quality in design, and to facilitate the successful implementation of the tool. Non-knowledge-based CDS tools using Bayesian algorithms and expert opinion-based referral data are recommended in the absence of data sources. In addition, CDS tools available as mobile phone apps were shown to overcome some of the limitations of veterinary professional support often found in LMICs. We believe that CDS tools with the capacity to model complex information such as clinical signs and management factors by assuming all input attributes are independent to generate diagnostic and treatment recommendations are sorely needed in resource-limited settings. The successful development and implementation of veterinary CDS tools in LMICs will likely contribute to improving treatment decisions and minimizing the inappropriate use of antimicrobial agents, with the primary goal of supporting AMS programs.

References

• 1.

  World Health Organization. ANTIMICROBIAL RESISTANCE Global Report on Surveillance. (2014), Available at:https://apps.who.int/iris/bitstream/handle/10665/112642/9789241564748_eng.pdf

  • Google Scholar

• 2.

  OIE—World Organisation for Animal Health. The OIE Strategy on Antimicrobial Resistance and the Prudent Use of Antimicrobials. (2016).

  • Google Scholar

• 3.

  DyarOJHuttnerBSchoutenJPulciniC. What is antimicrobial stewardship?Clin Microbiol Infect. (2017) 23:793–8. doi: 10.1016/j.cmi.2017.08.026

  • CrossRef
  • Google Scholar

• 4.

  ShlaesDMGerdingDNJohnJFCraigWABornsteinDLDuncanRAet al, Society for Healthcare Epidemiology of America and Infectious Diseases Society of America joint committee on the prevention of antimicrobial resistance, Guidelines for the Prevention of Antimicrobial Resistance in Hospitals

  • Google Scholar

• 5.

  NicolauDP. Current challenges in the management of the infected patient. Curr Opin Infect Dis. (2011) 24:S1–S10. doi: 10.1097/01.qco.0000393483.10270.ff

  • CrossRef
  • Google Scholar

• 6.

  CharaniECookeJHolmesA. Antibiotic stewardship programmes—what’s missing?J Antimicrob Chemother. (2010) 65:2275–7. doi: 10.1093/jac/dkq357

  • CrossRef
  • Google Scholar

• 7.

  DaveyPBrownECharaniEFenelonLGouldIMHolmesAet al. Interventions to improve antibiotic prescribing practices for hospital inpatients. (2013)

  • Google Scholar

• 8.

  PulciniCWilliamsFMolinariNDaveyPNathwaniD. Junior doctors’ knowledge and perceptions of antibiotic resistance and prescribing: a survey in France and Scotland. Clin Microbiol Infect. (2011) 17:80–7. doi: 10.1111/j.1469-0691.2010.03179.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9.

  GemedaBAAmenuKMagnussonUDohooIHallenbergGSAlemayehuGet al. Antimicrobial use in extensive smallholder livestock farming Systems in Ethiopia: knowledge, attitudes, and practices of livestock keepers. Front Vet Sci. (2020) 7:55. doi: 10.3389/fvets.2020.00055

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10.

  World Health Organisation. Antimicrobial stewardship programmes in health-care facilities in low- and middle-income countries: a WHO practical toolkit. JAC-Antimicrob Resist. (2019) 1:dlz072. doi: 10.1093/jacamr/dlz072

  • CrossRef
  • Google Scholar

• 11.

  OIE—World Organisation for Animal Health. OIE standards, guidelines and resolution on antimicrobial resistance and the use of antimicrobial agents. 2nd ed. Paris: OIE (2020). 146 p.

  • Google Scholar

• 12.

  Food and Agriculture Organization of the United Nations. The FAO Action Plan on Antimicrobial Resistance 2021–2025. FAO; (2021). Available from:http://www.fao.org/documents/card/en/c/cb5545en

  • Google Scholar

• 13.

  GargAXAdhikariNKJMcDonaldHRosas-ArellanoMPDevereauxPJBeyeneJet al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes. Decis SUPPORT. 16:1223–38.

  • Google Scholar

• 14.

  KabeneSM, (Ed.), Healthcare and the effect of technology: developments, challenges and advancements. IGI Global; (2010). (MoumtzoglouA, (Ed.), Advances in Healthcare Information Systems and Administration)

  • Google Scholar

• 15.

  DoyleDMcDonaldGPrattCRehanZBenteauTPhillipsJet al. Impact of a mobile decision support tool on antimicrobial stewardship indicators in St. John’s, Canada. PLoS One. (2021) 16:e0252407. doi: 10.1371/journal.pone.0252407

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  Carracedo-MartinezEGonzalez-GonzalezCTeixeira-RodriguesAPrego-DominguezJTakkoucheBHerdeiroMTet al. Computerized clinical decision support systems and antibiotic prescribing: a systematic review and meta-analysis. Clin Ther. (2019) 41:552–81. doi: 10.1016/j.clinthera.2019.01.018

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  CathoGCentemeroNSWaldispühl SuterBVernazNPortelaJDa SilvaSet al. How to develop and implement a computerized decision support system integrated for antimicrobial stewardship? Experiences from two Swiss hospital systems. Front Digit Health. (2021) 2:50. doi: 10.3389/fdgth.2020.583390

  • CrossRef
  • Google Scholar

• 18.

  BondSEChubatyAJAdhikariSMiyakisSBoutlisCSYeoWWet al. Outcomes of multisite antimicrobial stewardship programme implementation with a shared clinical decision support system. J Antimicrob Chemother. (2017) 72:2110–8. doi: 10.1093/jac/dkx080

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  KyrimiEDubeKFentonNFahmiANevesMRMarshWet al. Bayesian networks in healthcare: what is preventing their adoption?Artif Intell Med. (2021) 116:102079. doi: 10.1016/j.artmed.2021.102079

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  AdamsSTLevesonSH. Clinical prediction rules. BMJ. (2012) 344:d8312. doi: 10.1136/bmj.d8312

  • CrossRef
  • Google Scholar

• 21.

  JiangYQiuBXuCLiC. The research of clinical decision support system based on three-layer Knowledge Base model. J Healthc Eng. (2017) 2017:e6535286:1–8. doi: 10.1155/2017/6535286

  • CrossRef
  • Google Scholar

• 22.

  VisualDx. Visualdx. Available at: https://www.visualdx.com/who-we-help/individual-clinicians/

• 23.

  Abdel-RahmanSMGillHCarpenterSLGueyePWicklundBBreitkreutzMet al. Design and usability of an electronic health record—integrated, point-of-care, clinical decision support tool for modeling and simulation of Antihemophilic factors. Appl Clin Inform. (2020) 11:253–64. doi: 10.1055/s-0040-1708050

  • CrossRef
  • Google Scholar

• 24.

  FortmannJLutzMSpreckelsenC. System for context-specific visualization of clinical practice guidelines (GuLiNav): concept and software implementation. JMIR Form Res. (2022) 6:e28013. doi: 10.2196/28013

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25.

  ElkhalifaSBhanaRBlagaAJoshiSSvejdaMKasilingamVet al. Development and validation of a Mobile clinical decision support tool for the diagnosis of drug allergy in adults: the drug allergy app. J Allergy Clin Immunol Pract. (2021) 9:4410–4418.e4. doi: 10.1016/j.jaip.2021.07.057

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26.

  ChatzakisIVassilakisKLionisCGermanakisI. Electronic health record with computerized decision support tools for the purposes of a pediatric cardiovascular heart disease screening program in Crete. Comput Methods Prog Biomed. (2018) 159:159–66. doi: 10.1016/j.cmpb.2018.03.009

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  DiasDPaulo Silva CunhaJ. Wearable health devices—vital sign monitoring, systems and technologies. Sensors. (2018) 18:82414. doi: 10.3390/s18082414

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  SuttonRTPincockDBaumgartDCSadowskiDCFedorakRNKroekerKI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. (2020) 3:1–10. doi: 10.1038/s41746-020-0221-y

  • CrossRef
  • Google Scholar

• 29.

  GekkoVet Oy. GekkoVet. Available at: https://www.gekkovet.com/vet

• 30.

  Clinical Mastitis Decision Support Tool|Project|Food Agility CRC. Available at: https://www.foodagility.com/research/clinical-mastitis-decision-support-tool

• 31.

  Data Pig|The ultimate app for Swine Health & Production data. Available at: https://www.datapig.com.au/

• 32.

  GarciaABMadsenALVigreH. A decision support system for the control of Campylobacter in chickens at farm level using data from Denmark. J Agric Sci. (2016) 154:720–31. doi: 10.1017/S0021859616000101

  • CrossRef
  • Google Scholar

• 33.

  Villalba-OreroMRoanes-LozanoE. A prototype of a decision support system for equine cardiovascular diseases diagnosis and management. Mathematics. (2021) 9:2580. doi: 10.3390/math9202580

  • CrossRef
  • Google Scholar

• 34.

  ZeldisDPrescottS. Fish disease diagnosis program—problems and some solutions. Aquac Eng. (2000) 23:3–11. doi: 10.1016/S0144-8609(00)00047-9

  • CrossRef
  • Google Scholar

• 35.

  BarbedoJ. Computer-aided disease diagnosis in aquaculture: current state and perspectives for the future. Innover. (2014) 1:19–32.

  • Google Scholar

• 36.

  BeyeneTJEshetuAAbduAWondimuEBeyiAFTufaTBet al. Assisting differential clinical diagnosis of cattle diseases using smartphone-based technology in low resource settings: a pilot study. BMC Vet Res. (2017) 13:323. doi: 10.1186/s12917-017-1249-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  EzannoPPicaultSBeaunéeGBaillyXMuñozFDubozRet al. Research perspectives on animal health in the era of artificial intelligence. Vet Res. (2021) 52:40. doi: 10.1186/s13567-021-00902-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38.

  GrahamTWTurkJMcDermottJBrownC. Preparing veterinarians for work in resource-poor settings. J Am Vet Med Assoc. (2013) 243:1523–8. doi: 10.2460/javma.243.11.1523

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  OstropoletsAZhangLHripcsakG. A scoping review of clinical decision support tools that generate new knowledge to support decision making in real time. J Am Med Inform Assoc. (2020) 27:1968–76. doi: 10.1093/jamia/ocaa200

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40.

  ShahsavaraniAMAbadiEAMKalkhoranMHJafariSQaranliS. Clinical decision support systems (CDSSs). State Art Rev Literature. (2015) 2:299–308.

  • Google Scholar

• 41.

  BeelerPBatesDHugB. Clinical decision support systems. Swiss Med Wkly. (2014); Available at:https://smw.ch/index.php/smw/article/view/1954

  • Google Scholar

• 42.

  PapadopoulosPSoflanoMChaudyYAdejoWConnollyTM. A systematic review of technologies and standards used in the development of rule-based clinical decision support systems. Health Technol. (2022) 12:713–27. doi: 10.1007/s12553-022-00672-9

  • CrossRef
  • Google Scholar

• 43.

  HakFGuimarãesTSantosM. Towards effective clinical decision support systems: a systematic review. PLoS One. (2022) 17:e0272846. doi: 10.1371/journal.pone.0272846

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  BanningM. A review of clinical decision making: models and current research. J Clin Nurs. (2007) 70621074500075.

  • Google Scholar

• 45.

  GUIDES expert panelvan de VeldeSKunnamoIRoshanovPKortteistoTAertgeertsBet al. The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support. Implement Sci. (2018) 13:86. doi: 10.1186/s13012-018-0772-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46.

  TriccoACLillieEZarinWO’BrienKKColquhounHLevacDet al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. (2018) 169:467–73. doi: 10.7326/M18-0850

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47.

  WohlinC.Guidelines for snowballing in systematic literature studies and a replication in software engineering. Proceedings of the 18th international conference on evaluation and assessment in software engineering. London England United Kingdom: ACM; (2014). 1–10

  • Google Scholar

• 48.

  FantomNSerajuddinU. The World Bank’s classification of countries by income. Washington, DC: World Bank (2016).

  • Google Scholar

• 49.

  BVBA Z. zenjoy.be. Homepage Guides Checklist. Available at: https://www.guidesproject.org/

• 50.

  Salvador-OlivánJAMarco-CuencaGArquero-AvilésR. Errors in search strategies used in systematic reviews and their effects on information retrieval. J Med Libr Assoc. (2019) 107:210–21. doi: 10.5195/jmla.2019.567

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 51.

  LaneD., Capture-mark-recapture as a tool for estimating the number of articles available for systematic reviews in critical care medicine

  • Google Scholar

• 52.

  DelaneyATamásPA. Searching for evidence or approval? A commentary on database search in systematic reviews and alternative information retrieval methodologies

  • Google Scholar

• 53.

  WielandLS. Understanding why evidence from randomised clinical trials may not be retrieved from Medline: Comparison of indexed and non-indexed records. (2012)

  • Google Scholar

• 54.

  HirschJSBrarRForrerCSungCRoycroftRSeelamneniPet al. Design, development, and deployment of an indication- and kidney function-based decision support tool to optimize treatment and reduce medication dosing errors. JAMIA Open. (2021) 4:ooab039. doi: 10.1093/jamiaopen/ooab039

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55.

  VandewieleGDe BackereFLannoyeKVanden BergheMJanssensOVan HoeckeSet al. A decision support system to follow up and diagnose primary headache patients using semantically enriched data. BMC Med Inform Decis Mak. (2018) 18:98. doi: 10.1186/s12911-018-0679-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56.

  ShanbehzadehMNopourRKazemi-ArpanahiH. For detecting COVID-19 disease. J Educ Health Promot. (2022) 11:2. doi: 10.4103/jehp.jehp_387_21

  • CrossRef
  • Google Scholar

• 57.

  FixlerAOliaroBFriedenMGirardoCWinterbottomFAFortLBet al. Alert to action: implementing artificial intelligence–driven clinical decision support tools for Sepsis. Ochsner J. (2023) 23:222–31. doi: 10.31486/toj.22.0098

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58.

  2021|Comorbidities Manager from BMJ Best Practice. Available at:https://bestpractice.bmj.com/info/comorbidities/

• 59.

  McRaeMPDapkinsIPSharifIAndermanJFenyoDSinokrotOet al. Managing COVID-19 with a clinical decision support tool in a community health network: algorithm development and validation. J Med Internet Res. (2020) 22:e22033. doi: 10.2196/22033

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 60.

  Ghoddusi JohariMDabaghmaneshMHZareHSafaeianARAbdollahifardG. Smart diabetic screening and managing software, a novel decision support system. J biomed phys eng. (2018) 8:e870. doi: 10.31661/jbpe.v8i3Sep.870

  • CrossRef
  • Google Scholar

• 61.

  Pons-MesquidaMÀOms-AriasMDiogène-FadiniEFiguerasA. Safer prescription of drugs: impact of the PREFASEG system to aid clinical decision-making in primary care in Catalonia. BMC Med Inform Decis Mak. (2021) 21:349. doi: 10.1186/s12911-021-01710-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 62.

  JarugulaPScottSIvaturiVNoackAMoffettBSBhuttaAet al. Understanding the role of Pharmacometrics-based clinical decision support Systems in Pediatric Patient Management: a case study using Lyv software. J Clin Pharmacol. (2021) 61:61. doi: 10.1002/jcph.1892

  • CrossRef
  • Google Scholar

• 63.

  WulffAHaarbrandtBTuteEMarschollekMBeerbaumPJackT. An interoperable clinical decision-support system for early detection of SIRS in pediatric intensive care using openEHR. Artif Intell Med. (2018) 89:10–23. doi: 10.1016/j.artmed.2018.04.012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 64.

  GarbernSCNelsonEJNasrinSKeitaAMBrintzBJGaineyMet al. External validation of a mobile clinical decision support system for diarrhea etiology prediction in children: a multicenter study in Bangladesh and Mali. eLife. (2022) 11:e72294. doi: 10.7554/eLife.72294

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 65.

  HaddadSMSouzaRTCecattiJGBarreixMTamratTFootittCet al. Building a digital tool for the adoption of the World Health Organization’s antenatal care recommendations: methodological intersection of evidence, clinical logic, and digital technology. J Med Internet Res. (2020) 22:e16355. doi: 10.2196/16355

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66.

  TarumiSTakeuchiWChalkidisGRodriguez-LoyaSKuwataJFlynnMet al. Leveraging artificial intelligence to improve chronic disease care: methods and application to pharmacotherapy decision support for type-2 diabetes mellitus. Methods Inf Med. (2021) 60:e32–43. doi: 10.1055/s-0041-1728757

  • CrossRef
  • Google Scholar

• 67.

  RoggeveenLFGuoTDriessenRHFleurenLMThoralPvan der VoortPHJet al. Right dose, right now: development of AutoKinetics for real time model informed precision antibiotic dosing decision support at the bedside of critically ill patients. Front Pharmacol. (2020) 11:646. doi: 10.3389/fphar.2020.00646

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 68.

  CharpentierPAMeccaMCBrandtCFriedTR. Development of REDCap-based architecture for a clinical decision support tool linked to the electronic health record for assessment of medication appropriateness. JAMIA Open. (2023) 6:e41. doi: 10.1093/jamiaopen/ooad041

  • CrossRef
  • Google Scholar

• 69.

  AminSGuptaVDuGMcMullenCSirrineMWilliamsMVet al. Developing and demonstrating the viability and availability of the multilevel implementation strategy for Syncope optimal care through engagement (MISSION) Syncope app: evidence-based clinical decision support tool. J Med Internet Res. (2021) 23:e25192. doi: 10.2196/25192

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 70.

  HaqueFBallRLKhatunSAhmedMKacheSChistiMJet al. Evaluation of a smartphone decision-support tool for diarrheal disease Management in a Resource-Limited Setting. PLoS Negl Trop Dis. (2017) 11:e0005290. doi: 10.1371/journal.pntd.0005290

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 71.

  BergmannAStangelMWeihMvan HövellPBrauneSKöchlingMet al. Development of registry data to create interactive doctor-patient platforms for personalized patient care, taking the example of the DESTINY system. Front Digit Health. (2021) 3:633427. doi: 10.3389/fdgth.2021.633427

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 72.

  EudaleySTMihmAEHigdonRJeterJChamberlinSM. Development and implementation of a clinical decision support tool for treatment of uncomplicated urinary tract infections in a family medicine resident clinic. J Am Pharm Assoc. (2019) 59:579–85. doi: 10.1016/j.japh.2019.03.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 73.

  Casal-GuisandeMCeide-SandovalLMosteiro-AñónMTorres-DuránMCerqueiro-PequeñoJBouza-RodríguezJBet al. Design of an Intelligent Decision Support System Applied to the diagnosis of obstructive sleep apnea. Diagnostics. (2023) 13:1854. doi: 10.3390/diagnostics13111854

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 74.

  MaiAVoigtKSchübelJGräßerF. A drug recommender system for the treatment of hypertension. BMC Med Inform Decis Mak. (2023) 23:89. doi: 10.1186/s12911-023-02170-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 75.

  Web-based eHealth Clinical Decision Support System as a tool for the treat-to-target management of patients with systemic lupus erythematosus: development and initial usability evaluation. Open Access

  • Google Scholar

• 76.

  ZotovEHillsAFde MelloFLAramPSayersABlomAWet al. JointCalc: a web-based personalised patient decision support tool for joint replacement. Int J Med Inform. (2020) 142:104217. doi: 10.1016/j.ijmedinf.2020.104217

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 77.

  CypkoMAStoehrM. Digital patient models based on Bayesian networks for clinical treatment decision support. Minim Invasive Ther Allied Technol. (2019) 28:105–19. doi: 10.1080/13645706.2019.1584572

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 78.

  BhattacharjeeSSahaBBhattacharyyaPSahaS. LHSPred: a web based application for predicting lung health severity. Biomed Signal Process Control. (2022) 77:103745. doi: 10.1016/j.bspc.2022.103745

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 79.

  ShenYYuanKChenDCollocJYangMLiYet al. An ontology-driven clinical decision support system (IDDAP) for infectious disease diagnosis and antibiotic prescription. Artif Intell Med. (2018) 86:20–32. doi: 10.1016/j.artmed.2018.01.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 80.

  OlzhasSIgorTAisuluI. Intelligent system for solving problems of veterinary medicine on the example of dairy farms. Computer Science & Information Technology (CS & IT) [internet]. AIRCC Publishing Corporation; (2021). 17–34

  • Google Scholar

• 81.

  SicardVAssiéSDorsoLChocteauFPicaultS. A diagnosis support system for veterinary necropsy based on Bayesian networks: Proceedings of the 13th International Conference on Agents and Artificial Intelligence. Online Streaming—Select a Country—SCITEPRESS—Science and Technology Publications; (2021), 645–654

  • Google Scholar

• 82.

  Ask Sofie|The Best Veterinary Search Tool & Information Resource. LifeLearn Inc. Available at: https://www.lifelearn.com/products/sofie/

  • Google Scholar

• 83.

  IDEXX VetConnect PLUS—IDEXX US. Available at:https://www.idexx.com/en/veterinary/software-services/vetconnect-plus/?&utm_source=vetiverse&utm_medium=link_site&utm_campaign=vcplus2023&utm_content=&_gl=1*1r51tns*_ga*MTc3ODI4NjQwLjE2OTEzOTIzMzU.*_ga_1WK05ERXV8*MTY5MTM5MjMzNS4xLjEuMTY5MTM5MjM3MS4yNC4wLjA

  • Google Scholar

• 84.

  Consultant [Internet]. Available from:https://consultant.vet.cornell.edu/

• 85.

  Pig production blog: swine nutrition, pig farming—pig333, pig to pork community. (2023). Available at:https://www.pig333.com/

  • Google Scholar

• 86.

  BernerES. Clinical decision support systems. New York: Springer (2007).

  • Google Scholar

• 87.

  MachilaNWanyanguSWMcDermottJWelburnSCMaudlinIEislerMC. Cattle owners’ perceptions of African bovine trypanosomiasis and its control in Busia and Kwale districts of Kenya. Acta Trop. (2003) 86:25–34. doi: 10.1016/S0001-706X(02)00288-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 88.

  DevarajSSharmaSKFaustoDJViernesSKharraziH. Barriers and facilitators to clinical decision support systems adoption: a systematic review. JBAR. (2014) 3:36. doi: 10.5430/jbar.v3n2p36

  • CrossRef
  • Google Scholar

• 89.

  TsASreelakshmiP. Revisiting public health challenges in the new millennium. Ann Med Health Sci Res. (2013) 3:7. doi: 10.4103/2141-9248.117923

  • CrossRef
  • Google Scholar

• 90.

  MillerRA. A history of the INTERNIST-1 and quick medical reference (QMR) computer-assisted diagnosis projects, with lessons learned. Methods Inf Med. (2010) 19:121–36. doi: 10.1055/s-0038-1638702

  • CrossRef
  • Google Scholar

• 91.

  Jpc RodriguesJDe La TorreIFernándezGLópez-CoronadoM. Analysis of the security and privacy requirements of cloud-based electronic health records systems. J Med Internet Res. (2013) 15:e186. doi: 10.2196/jmir.2494

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 92.

  KhuboneTTlouBMashamba-ThompsonTP. Electronic health information systems to improve disease diagnosis and Management at Point-of-Care in low and middle income countries: a narrative review. Diagnostics. (2020) 10:327. doi: 10.3390/diagnostics10050327

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 93.

  OluochTde KeizerNF. Evaluation of Health IT in Low-Income Countries.

  • Google Scholar

• 94.

  PoolJAkhlaghpourSFatehiFBurton-JonesA. A systematic analysis of failures in protecting personal health data: a scoping review. Int J Inf Manag. (2024) 74:102719. doi: 10.1016/j.ijinfomgt.2023.102719

  • CrossRef
  • Google Scholar

• 95.

  IskandarKMolinierLHallitSSartelliMHardcastleTCHaqueMet al. Surveillance of antimicrobial resistance in low- and middle-income countries: a scattered picture. Antimicrob Resist Infect Control. (2021) 10:63. doi: 10.1186/s13756-021-00931-w

  • CrossRef
  • Google Scholar

• 96.

  AwayshehAWilckeJElvingerFReesLFanWZimmermanKL. Review of medical decision support and machine-learning methods. Vet Pathol. (2019) 56:512–25. doi: 10.1177/0300985819829524

  • CrossRef
  • Google Scholar

• 97.

  FoxKFoxJBexfieldNFreemanP. Computerised decision support in veterinary medicine, exemplified in a canine idiopathic epilepsy care pathway. J Small Anim Pract. (2021) 62:911–7. doi: 10.1111/jsap.13345

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 98.

  FrostARSchofieldCPBeaulahSAMottramTTLinesJAWathesCM. A review of livestock monitoring and the need for integrated systems. Comput Electron Agric. (1997) 17:139–59. doi: 10.1016/S0168-1699(96)01301-4

  • CrossRef
  • Google Scholar

• 99.

  RuttenCJVelthuisAGJSteeneveldWHogeveenH. Invited review: sensors to support health management on dairy farms. J Dairy Sci. (2013) 96:1928–52. doi: 10.3168/jds.2012-6107

  • CrossRef
  • Google Scholar

• 100.

  KawamotoKHoulihanCABalasEALobachDF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. (2005) 330:765. doi: 10.1136/bmj.38398.500764.8F

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 101.

  WrightAAshJSAaronSAiAHickmanTTTWiesenJFet al. Best practices for preventing malfunctions in rule-based clinical decision support alerts and reminders: results of a Delphi study. Int J Med Inform. (2018) 118:78–85. doi: 10.1016/j.ijmedinf.2018.08.001

  • CrossRef
  • Google Scholar

• 102.

  McKendrickIJGettinbyGGuYReidSWJRevieCW. Using a Bayesian belief network to aid differential diagnosis of tropical bovine diseases. Prev Vet Med. (2000) 47:141–56. doi: 10.1016/S0167-5877(00)00172-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 103.

  MusenMAShaharYShortliffeEH. Clinical decision-support systems.

  • Google Scholar

• 104.

  EzeUOluigboI. The Relevance of Effective Database System for Efficient Office Management in the 21 st Century Institutions. (2014); 2:614–624

  • Google Scholar

• 105.

  AshJSGuapponeP. Some Unintended Consequences of Clinical Decision Support Systems. 5 p.

  • Google Scholar

• 106.

  RawsonTMMooreLSPHernandezBCharaniECastro-SanchezEHerreroPet al. A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately?Clin Microbiol Infect. (2017) 23:524–32. doi: 10.1016/j.cmi.2017.02.028

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 107.

  DomingosPPazzaniM. On the Optimality of the Simple Bayesian Classifier under Zero-One Loss.

  • Google Scholar

• 108.

  CosmaGAcamporaGBrownDReesRCKhanMPockleyAG. Prediction of pathological stage in patients with prostate Cancer: a neuro-fuzzy model. PLoS One. (2016) 11:e0155856. doi: 10.1371/journal.pone.0155856

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 109.

  LatonJVan SchependomJGielenJDecosterJMoonsTDe KeyserJet al. Single-subject classification of schizophrenia patients based on a combination of oddball and mismatch evoked potential paradigms. J Neurol Sci. (2014) 347:262–7. doi: 10.1016/j.jns.2014.10.015

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 110.

  FerreiraDOliveiraAFreitasA. Applying data mining techniques to improve diagnosis in neonatal jaundice. BMC Med Inform Decis Mak. (2012) 12:143. doi: 10.1186/1472-6947-12-143

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 111.

  XhemaliDHindeCJStoneRG. Naïve Bayes vs. decision trees vs. neural networks in the classification of training web pages, vol. 4 (2009).

  • Google Scholar

• 112.

  GuptaBRawatAJainAAroraADhamiN. Analysis of various decision tree algorithms for classification in data mining. Int J Comput Appl. (2017) 163:15–9. doi: 10.5120/ijca2017913660

  • CrossRef
  • Google Scholar

• 113.

  TrevethanR. Sensitivity, specificity, and predictive values: foundations, Pliabilities, and pitfalls in research and practice. Front Public Health. (2017) 5:307. doi: 10.3389/fpubh.2017.00307

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 114.

  HelmonsPJSuijkerbuijkBONannan PandayPVKosterinkJG. Drug-drug interaction checking assisted by clinical decision support: a return on investment analysis. J Am Med Inform Assoc. (2015) 22:764–72. doi: 10.1093/jamia/ocu010

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 115.

  JiaPZhangLChenJZhaoPZhangM. The effects of clinical decision support systems on medication safety: an overview. PLoS One. (2016) 11:e0167683. doi: 10.1371/journal.pone.0167683

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 116.

  DavisDATaylor-VaiseyA. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. Can Med Assoc J. 9:593–602. PMID:

  • Pubmed Abstract
  • Google Scholar

• 117.

  CabanaMDRandCSPoweNRWuAWWilsonMHAbboudPACet al. Why Don’t physicians follow clinical practice guidelines?: a framework for improvement. JAMA. (1999) 282:1458–65. doi: 10.1001/jama.282.15.1458

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 118.

  VonbachPDubiedAKrähenbühlSBeerJH. Prevalence of drug–drug interactions at hospital entry and during hospital stay of patients in internal medicine. Eur J Intern Med. (2008) 19:413–20. doi: 10.1016/j.ejim.2007.12.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 119.

  LiptonJABarendseRJSchinkelAFLAkkerhuisKMSimoonsMLSijbrandsEJG. Impact of an alerting clinical decision support system for glucose control on protocol compliance and glycemic control in the intensive cardiac care unit. Diabetes Technol Ther. (2011) 13:343–9. doi: 10.1089/dia.2010.0100

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 120.

  BenhamouPY. Improving diabetes management with electronic health records and patients’ health records. Diabetes Metab. (2011) 37:S53–6. doi: 10.1016/S1262-3636(11)70966-1

  • CrossRef
  • Google Scholar

• 121.

  KunhimangalamROvallathSJosephPK. A clinical decision support system with an integrated EMR for diagnosis of peripheral neuropathy. J Med Syst. (2014) 38:38. doi: 10.1007/s10916-014-0038-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 122.

  Preventing Medication Errors. Quality chasm series. Washington, D.C.: National Academies Press (2007).

  • Google Scholar

• 123.

  Barriers to Use of Health Data in Low- and Middle-Income Countries—A Review of the Literature—MEASURE Evaluation. Available at:https://www.measureevaluation.org/resources/publications/wp-18-211.html

• 124.

  MoucheraudC.Sustainability of health information systems: A three-country qualitative study in southern Africa. (2017)

  • Google Scholar