The potential of virtual triage AI to improve early detection, care acuity alignment, and emergent care referral of life-threatening conditions

Objective To evaluate the extent to which patient-users reporting symptoms of five severe/acute conditions requiring emergency care to an AI-based virtual triage (VT) engine had no intention to get such care, and whose acuity perception was misaligned or decoupled from actual risk of life-threatening symptoms. Methods A dataset of 3,022,882 VT interviews conducted over 16 months was evaluated to quantify and describe patient-users reporting symptoms of five potentially life-threatening conditions whose pre-triage healthcare intention was other than seeking urgent care, including myocardial infarction, stroke, asthma exacerbation, pneumonia, and pulmonary embolism. Results Healthcare intent data was obtained for 12,101 VT patient-user interviews. Across all five conditions a weighted mean of 38.5% of individuals whose VT indicated a condition requiring emergency care had no pre-triage intent to consult a physician. Furthermore, 61.5% intending to possibly consult a physician had no intent to seek emergency medical care. After adjustment for 13% VT safety over-triage/referral to ED, a weighted mean of 33.5% of patient-users had no intent to seek professional care, and 53.5% had no intent to seek emergency care. Conclusion AI-based VT may offer a vehicle for early detection and care acuity alignment of severe evolving pathology by engaging patients who believe their symptoms are not serious, and for accelerating care referral and delivery for life-threatening conditions where patient misunderstanding of risk, or indecision, causes care delay. A next step will be clinical confirmation that when decoupling of patient care intent from emergent care need occurs, VT can influence patient behavior to accelerate care engagement and/or emergency care dispatch and treatment to improve clinical outcomes.


Introduction
Life-threatening illness requiring emergency care accounts for 14.0% of emergency department (ED) visits in the US, and benefits from the earliest possible detection and care referral to improve patient outcomes and reduce hospital length of stay and care costs (1,2).If patient understanding/self-perception underestimates actual acuity and risk indicated by symptoms and past medical history, patients may delay care seeking (3)(4)(5).Delayed care results in avoidable mortality and drives higher acuity care utilization that is more costly than might have been needed had the patient sought care earlier (3,6).
Symptomate is a current generation digital virtual triage (VT) engine or "symptom checker" that leverages artificial intelligence (AI) to provide access to authoritative information services beyond physician office hours on a 24/7/365 basis from any device with internet connectivity.VT can avert long ED waits or travel, and helps when individuals are uncertain if symptoms warrant medical attention or what kind of care to seek, or if symptoms worsen.The VT interview asks questions about age, sex, symptoms, medical history, risk factors, and medications and, processing responses on a current basis, selects the next most relevant question via Bayesian probabilities, simulating the manner in which a human clinician processes information.After evaluating all patient data, the VT inference engine computes the probabilities of likely conditions using a statistical algorithm for advanced symptom assessment informed by an underlying medical knowledge base.The VT AI advises the patient-user of probable causes, illness severity, and recommended acuity level of medical care.VT is superior to internet browser search because patients are evaluated systematically to identify/convey a care acuity level using clinical AI algorithms validated by physicians.VT AI improves upon traditional clinical triage protocols or decision trees in its ability to dynamically calculate the probability of possible conditions and adjust the patient interview accordingly.However, VT results are not diagnostic and do not constitute medical consultation, providing information and guidance only.Users are advised to seek immediate medical attention when the VT engine determines that a threat to life exists.
Critical to maximizing the clinical and public health impact of VT AI is identifying functional areas where it can enhance existing diagnostic processes and capabilities.It is unknown whether AI-based VT can augment existing healthcare system capabilities for early identification, care acuity alignment, and rapid care referral of individuals whose self-perception of disease acuity/risk is inaccurate when serious morbidity is actually evolving.Patient delay in seeking urgent care across life-threatening diseases increases mortality (3,7).When patient perception-understanding of symptoms/risk is decoupled from high actual risk of a severe clinical outcome, if identified reliably by VT in real-time, ED care engagement can be accelerated.VT may thus offer a new opportunity for early life-threatening disease detection, enabling improved, more rapid care acuity alignment and ED referral.VT could enhance existing healthcare system capabilities and efforts to reduce preventable disease sequelae and death.This study evaluates decoupling of actual care acuity need from patient care intent/risk selfperception as detected by a clinically validated VT engine available costfree online.We examine data on patient perception-actual risk decoupling during severe symptom onset of five leading causes of U.S. hospital admission and death.

Objective
Evaluate the extent to which patient-users reporting symptoms of severe, acute conditions classified by an AI-based VT engine as warranting emergency care had no pre-VT triage intention to engage appropriate ED care.The extent to which decoupling of patient care intent from actual clinical need occurs for five prevalent diseases could imply a potential real-time use of VT to improve early detection and care acuity alignment and accelerate more effective care referral for emergent conditions.

VT technology platform and data source
The Infermedica Symptomate VT engine is designed for general public use and completes evidence-driven analyses informed by over 800 diseases, 1,500 symptoms, and 200 risk factors.Leveraging AI, machine learning, and natural language processing, VT evaluates symptoms reported by patient-users, suggesting the most probable conditions matching the presentation and history, and refers to the most clinically appropriate and safest possible care.There are no prescribed interview pathways, and in light of new information reported, the VT AI explores various clinical queries and hypotheses (as physicians do).Before VT, patient-users are asked about their care intention.The VT interview concludes with an analysis of the reported symptoms and a recommendation to pursue one of three levels of care acuity: self-care, consult a primary care or specialist physician on an outpatient basis, proceed to an ED or call an ambulance for ED transport.
Virtual triage technologies (including Symptomate) are considered medical device class I in Europe, according to Medical Device Directive (93/42/EEC) and in the US, under the Food, Drug & Cosmetic Act.The FDA currently exercises enforcement discretion, which means that virtual triage technology is not required to comply with FDA regulations related to medical devices.
Data were extracted from VT episodes engaged by patient-users of the Symptomate VT application, a stand-alone VT engine not integrated with/implemented within a health system, and available through the Infermedica website or as a mobile application.Symptomate is available in 24 languages.Over 17 million Symptomate evaluations have been completed since 2012.

VT engine clinical validity
AI-based VT engines require rigorous validation to ensure safety and minimize mistriage, and focus on common diseases by design, with AI built to overtriage to emergency care rather than possibly miss a patient with acute care need.VT accuracy varies across clinical specialties and settings, as determined by the depth of disease-specific data used to train the triage AI.VT validity has been evaluated using clinical vignettes prepared by physicians of various patient symptomatic presentations in different clinical settings (8)(9)(10)(11).Symptomate provides safe recommendations in 97.8% of instances (9).VT engines may perform poorly detecting low prevalence conditions or presentations in atypical clinical settings (8).Published studies, while providing a point in time comparison, become quickly outdated due to the rapid evolution of AI-based VT.For example, since the analysis conducted by 2019 Gilbert et al., (9) there have been 29 releases of the Symptomate medical content model and 17 updates to core functionality, including new epidemiological models and triage algorithms, each improving accuracy and safety.In a recent study of 8,008 patient-users of the Infermedica virtual triage engine in an ambulatory care network, 2,830 or 35.0%changed their healthcare seeking action from their initial pre-VT care intent as a result of the VT evaluation and recommendation, indicating that VT output can alter the care seeking behavior of patient-users (12).

Sample selection and eligibility criteria
Symptomate VT interviews were selected using the following criteria: (1) interview completed during a 16-month period of March 2022-June 2023; (2) first ranked condition classified as requiring emergency care; (3) condition probability equal or greater than 0.90 as the first ranked by VT; (4) conditions with minimum 300 interviews; and (5) conditions with crude death rate of 1 per 100,000 and above in CDC Wonder (13).COVID-19 was excluded.Only five conditions with high potential severity/mortality met these eligibility criteria, including myocardial infarction (MI), stroke, asthma, pneumonia, and pulmonary embolism (PE).While the VT engine uses the term/variable "myocardial infraction" in its logic, language used in the VT patient interface was adapted to ensure public comprehension (e.g., "heart attack").Ischemic stroke (including cerebellar strokes) and hemorrhagic stroke (both subarachnoid and intracerebral) were collapsed into a single category for analysis.Figure 1 conveys the sequential application of eligibility criteria to patient-user interview selection.

Data captured and analyses completed
All data was collected through the Symptomate VT engine database.VT interviews were completed anonymously with no individual identifiers recorded beyond age, gender, and language.
Patient-users conveyed informed consent for using their anonymized data in the aggregate devoid of personal identifiers.A dataset of 3,022,882 patient-users was evaluated to investigate the extent of decoupling of patient-user pre-triage care intent and VT-identified high acuity conditions warranting emergency care.The pre-triage care intention survey provided response options of doing nothing; selfcare/recover at home; consult a doctor; emergency care with or without ambulance; and not sure/do not know/none.While care intent after VT guidance was queried, completion rates were insufficient to support meaningful analysis.
Pre-interview intent responses of doing nothing, self-care/ recovering at home, were grouped and classified as non-professional  1 encompassing various patient care intents across all conditions included in the analysis, weighted arithmetic means were calculated using each condition's respective prevalence as its weighted value.Weighted means were calculated across the five diseases by multiplying the number of VT interviews for each condition by the percentage of interviews with stated intent for that condition, then summing and dividing by the sum of interviews.Because data were derived from general public Symptomate use and not patient-users within a healthcare system, it was impossible to follow patients' clinical course to confirm acuity and clinical outcomes.

Adjustment for overtriage to emergency care
VT technology errs on the side of over-caution in care referral because of the potentially serious harm/injury to patients if VT under-refers to ED/ambulance, favoring care referral sensitivity over specificity by design and ethical imperative.Analyses of Symptomate over-referral have indicated that approximately 13% of ED recommendations are over-referrals, which has been declining as its VT AI advances (14).Symptomate performance was compared with widely used teletriage clinical decisioning protocols using clinical vignettes.VT over-referred 12% of patients to ED versus 26% for a leading telephone triage product (Schmitt-Thompson) (15).To enhance precision of estimates of decoupling of patient pre-triage care intent from VT acuity/ referral output, intention rates were adjusted for overtriage to ED by a factor of 13%.

Decoupling of patient pre-triage care intent and VT acuity assessment
Intent data were obtained for 12,101 VT patient-user interviews.Across all five conditions, a weighted mean of 38.5% of individuals whose VT indicated a condition requiring emergency care had no pre-triage intent to consult a physician.Furthermore, 61.5% intending to possibly consult a physician had no intent to seek emergency medical care (Table 1).The low frequency of patient-users intending before VT to seek emergency care is striking and may reflect the use of VT quite early in symptom evolution, presumably increasing with an experience of more severe symptoms.In addition, individuals with severe symptoms may bypass VT use in favor of telephonic consultation or in-person visit to an ED.
Table 2 presents adjusted rates factoring in VT safety over-triage to ED and shows that after allowing for a 13% over referral to ED, a weighted average of 33.5% of patient-users across conditions had no intent to seek professional care.One half (53.5%) had no intent to seek emergency care.

Demographic analyses of pre-triage care intent by age, gender, and language
With respect to demographics, all age groups were statistically significantly different from one another (p = 0.05).The youngest (under 18 are completed by a parent or adult caretaker) and the eldest age groups were more likely to express intent to seek appropriate emergency care for symptoms, with those 18-44 years least likely to, as shown in Table 3.
Regarding respondent gender (Table 4), men were more likely (40.7%) than women (37.6%) to indicate a non-professional care intent (p = 0.05).Men were also more likely to state an intent to seek emergency care (9.3%) than women (7.1%) (p = 0.05).The sample breakdown of patient-user respondents by gender, with women comprising over two-thirds of VT interviews, reflects the pattern of all Symptomate users (16).Overall, the lack of perceived urgency of symptoms is similar between genders.Patient-users in each major language grouping (English, Spanish, French, German, Polish, Russian and other) were statistically significantly different from each other in every intent category (p = 0.05) (Table 5).Those completing VT in Spanish were more likely to express appropriate emergency care intent than patient-users in other language groups (10.2%).Those completing in French were lowest in reporting intent to engage emergency care at 4.7%, but were most inclined to engage non-emergent care including consulting an outpatient physician.

Discussion
Delays in care seeking during medical emergencies negatively impact patient outcomes, and may result from patients failing to understand, dismissing or underestimating serious symptoms, and/or attempting to self-treat serious clinical problems at home (3).Table 6 summarizes causes and impact of care delay for the five conditions evaluated.Early detection and prompt, acuity appropriate treatment are crucial in improving outcomes of MI, which killed 108,610   In 2019, stroke accounted for 1 in 19 US deaths, the fifth leading cause of mortality (22,33).Prehospital suspicion of stroke compressed time to computer tomography by 219 minutes and to stroke unit care by 556 minutes, reducing mortality (23,34).However, 42.0% of patients experienced pre-hospital delays due to little knowledge of stroke symptoms (21).In the present study, 45.6% of patient-users with stroke symptoms intended to pursue non-emergency care, including 24.0% who planned to recover at home or take no action (ED overtriage adjusted).Reducing pre-hospital stroke care delay and acuity of care mismatch is a priority (21,23).VT can potentially reduce delays in emergency stroke care.
In 2019, asthma caused 1,835,901 US ED visits, 169,330 hospital admissions, and 3,524 deaths (35-37).Many deaths and admissions were preventable and related to care delays (5,7).In total, 45% of patients did not seek medical care during the final fatal asthma attack (25).
In 2021, 1.4 million US cases of pneumonia were diagnosed in EDs, 47,956 of which were fatal (13).Pneumonia was a concomitant cause of death in 148,498 deaths (13).Timely antibiotic treatment reduced adjusted 30-day hospital mortality by 16.0% (27).
Pulmonary embolism caused 8,809 US deaths in 2018 ( 13), with untreated mortality of 30.0%(30).In-hospital mortality was 1.6% for early versus 43.2% for delayed PE diagnosis (31).Treatment was delayed for 4.2 days due to patient procrastination (29), with 18-38% of patients symptomatic a week or longer before diagnosis (32).VT well integrated with healthcare system triage and intake can potentially reduce patient underestimation of symptom severity and delays in care seeking for these deadly respiratory illnesses.
Our findings that 53.5% of patients (ED overtriage adjusted) who reported symptoms of these five life-threatening conditions had no pre-triage intent to engage urgent care, and 33.5% had no intent to seek any medical care, aligns with national data.Users of VT may wish to assess their symptoms without engaging local healthcare for convenience and/or to save time and expense.Patients may seek reassurance that their symptoms are relatively benign.This sampling of 12,000 interviews from over three million total VT episodes demonstrates that in a significant percentage, possible existence of a severe underlying cause of symptoms should prompt the patientuser to seek professional medical attention sooner than they would have otherwise planned.The potential to reduce morbidity and mortality by more timely assessment of ambiguous/unrecognized signs and symptoms has significant public health implications.We suspect that use of AI-driven VT may become a valuable addition to under-resourced healthcare systems unable to assess all patients who seek care for uncertain symptoms but who may be at elevated risk for more serious disease.
With respect to limitations of the study, it is pertinent to acknowledge that the data collected may have been affected by certain inaccuracies.For instance, patient-users may have clicked through initial questions regarding their pre-VT care intention or conveyed random responses in order to get to the part of the encounter they were interested in, for example the VT evaluation of symptoms presented.While we have no evidence that this behavior occurred in substantial numbers of VT patient-users, the question warrants further research consideration and detection measures if a prevalent behavior.
In the effort to change patient care seeking behavior when confronted with potentially life threatening symptoms, interventions are needed that improve patient engagement, education, understanding of the clinical signs and symptoms of these acute conditions, and which are effective in activating patients to immediately seek emergency care.Given that many lives are avoidably lost, and the quality and extent of others are diminished, our current inability to effectively engage and educate patients to seek urgent care when confronted with life threatening symptoms is an area demanding far greater behavior change research, along with associated research funding.Indeed, one wonders what could be achieved in understanding and affecting patient care seeking behavior change were this research to be funded at the level of pharmaceutical development and marketing of a new therapeutic to treat patients whose severity of illness could have been lessened, if we were more capable of identifying and activating these patients to earlier care seeking and reduction of care delay.Clearly, current approaches to achieving these behavioral and care seeking objectives have been largely inadequate.Public awareness campaigns addressing the importance of seeking prompt medical care in cases of acute, life-threatening cerebrovascular symptoms, for example, demonstrate a positive impact (39) but are often episodic rather than continuous, which limits their long-term impact and benefit.Reliable online medical resources exist which educate about the importance of early recognition and care seeking for serious cardiac symptoms as well, but provide general, non-personalized medical information, which can lessen the impact on individual behavior.
Subsequent research should systematically assess patient outcomes in order to fully evaluate the potential of VT to earlier detect severe evolving pathologies, such as those reported here, and effectively motivate and accelerate patient care seeking.Prospective validation in a real world clinical setting could track outcomes of patients whose selfperception and healthcare intent prior to VT is decoupled from actual acuity and urgent care need.Using objective clinical parameters and endpoints, this would assess the role and clinical value of AI-enabled VT within a care delivery and patient engagement ecosystem.If thus validated, VT may offer payers, health systems and providers a real-time vehicle for earlier detection and engagement of at-risk patients to enable better care acuity alignment and rapid ED referral/ambulance dispatch for life-threatening conditions.

Conclusion
The impact of advanced diagnostics and therapeutics is blunted if patients do not rapidly engage appropriate acuity care to receive their benefits.This study suggests that VT AI may help accelerate the moment of disease detection and appropriate acuity level care engagement.VT lets a user know their condition may be more serious than they perceive and they should seek ED care.Integrated within a digital front door, VT can potentially accelerate early detection, care acuity alignment and live engagement/intake to save lives, reduce morbidity, and better utilize healthcare resources by reducing care delays due to patient uncertainty and/or misunderstanding.Future research should validate that VT can influence patient care seeking behavior to enable accelerated ED care referral or ambulance dispatch.If validated, VT may become an integral component in an Currently, when patient care intent is decoupled from actual clinical need by patient misunderstanding or indecision, earlier care detection and accelerated care engagement to reduce care delay is a complex challenge.As remote diagnostic monitoring and telemedical capabilities advance in coming years, so too will AI-based VT.Rapid development in generative AI and large language models will deliver improved user experience of VT AI.The use of natural language to enable patient-users to communicate in a more effective and nuanced way with VT AI will enable greater sensitivity, engendering trust in the results and greater impact on patient-user behavior.Exploiting these emerging potentialities will require investment and multisector scientific expertise converging private industry and publicgovernmental interests, assets, and capabilities.
The VT industry is not sufficiently resourced to drive the research needed to explore the potential beneficial impact and value of AI-based VT in early detection and acuity-aligned care referral.Furthermore, population databases of what will soon be hundreds of millions of VT episodes across the industry are too valuable a data and research resource to be applied exclusively for purely commercial interest.The VT industry should enable access to its anonymized aggregate data by the US National Institutes of Health and Centers for Disease Control and Prevention, international agencies, academic medical centers, and healthcare organizations to investigate and progress the full potential value of VT in the emerging 21st century health IT and AI ecosystem.

TABLE 1
Decoupling of patient care intent from virtual triage care referral for five causes of us morbidity and mortality.
2Percentages in bottom row totals are weighted mean values.

TABLE 3
Patient-user pre-VT intent by age.
Age stratum percentages are calculated relative to the particular age group, where the denominator is the number of patient-users expressing that intent or intentions within each age group.

TABLE 2
Patient care intent-virtual triage care referral decoupling adjusted for AI safety overtriage to emergency department.
3Percentages in bottom row totals are weighted mean values.

TABLE 4
Patient-user pre-VT intent by gender.All percentages are calculated relative to the gender group, where the denominator is the number of patient-users expressing that intent or intentions for each gender.

TABLE 5
Patient-user pre-VT intent by interview language.All percentages are calculated relative to the language group, where the denominator is the number of patient-users expressing that care intent or intentions by each language.

TABLE 6
Causes and impact of care delay for five life-threatening conditions.AI-enhanced clinical information ecosystem that expedites detection and delivery of appropriate urgent medical care.