Applying artificial intelligence to predict falls for inpatient

Objective Falls are adverse events which commonly occur in hospitalized patients. Inpatient falls may cause bruises or contusions and even a fractures or head injuries, which can lead to significant physical and economic burdens for patients and their families. Therefore, it is important to predict the risks involved surrounding hospitalized patients falling in order to better provide medical personnel with effective fall prevention measures. Setting This study retrospectively used EHR data taken from the Taichung Veterans General Hospital clinical database between January 2015 and December 2019. Participants A total of 53,122 patient records were collected in this study, of which 1,157 involved fall patients and 51,965 were non-fall patients. Primary and secondary outcome measure This study integrated the characteristics and clinical data of patients with falls and without falls using RapidMiner Studio as an analysis tool for various models of artificial intelligence. Utilization of 8 differ models to identify the most important factors surrounding inpatient fall risk. This study used the sensitivity, specificity, and area under the ROC curve to compute the data by 5-fold cross-validation and then compared them by pairwise t-tests. Results The predictive classifier was developed based upon the gradient boosted trees (XGBoost) model which outperformed the other seven baseline models and achieved a cross-validated ACC of 95.11%, AUC of 0.990, F1 score of 95.1%. These results show that the XGBoost model was used when dealing with multisource patient data, which in this case delivered a highly predictive performance on the risk of inpatient falls. Conclusion Machine learning methods identify the most important factors regarding the detection of inpatients who are at risk of falling, which in turn would improve the quality of patient care and reduce the workloads of the nursing staff when making fall assessments.


Introduction
There are two important clinical indicators of care: the falls indicator and the pressure injury indicator (1,2), most of these studies have looked at the causes of the events prior to their occurrence (3).However, Patient falls which occur during hospitalization can cause serious injury and are one of the most difficult patient safety issues that hospitals face.When a patient falls, family members often feel that the hospital staff has not properly fulfilled its care responsibilities, while the hospital feels both aggrieved and powerless.Therefore, preventing patients from falling during hospitalization has always been the most concerning issue within every hospital (4,5).According to statistical analysis of the Taiwan Patient Safety Reporting System (TPR), 1 out of every 4 medical safety incidents involves a fall.In 2018, the number of falls reported by various medical institutions reached as high as 17,360 patients, the incidence of falls is 0.6%.That statistic translates into 2 patients falling every hour, with the ranking of this safety concern in hospitals being only lower than drug-related incidents (6).Inpatients who fell and caused injuries to themselves extended their length of hospital stay by 6.4 days when compared with those who did not fall.In turn, medical care costs increased for them by 18,257 Taiwan New Dollars (TWDs), causing an annual increase in medical care costs to reach as much as 300 million TWDs (6,7).
In the United States, approximately 1 million patients fall in hospitals each year, with approximately one-third of those falls considered preventable (8).Therefore, if high-risk groups and fall risk factors can be screened early, this preventative approach could provide both the necessary measures and educational tools needed for medical personnel.This early screening process would help to reduce both the incidence of falls and injury rates of inpatients while also eliminating any subsequent medical costs (9).Presently, many fall risk assessment scales are being used in medical care institutions worldwide, such as the St. Thomas's risk assessment tool in falling elderly inpatients (STRATIFY) and Hendrich's High-Risk Fall Model (10)(11)(12), which require manual assessment.However, these tools require a clinician's time for performing the assessment and have a low specificity, which makes it difficult to determine how to focus on fall prevention tactics in a hospital setting (13).
Compared with Western societies, the nurse-patient ratios had a high percentage in Taiwan.Hence, applying machine learning models will increasingly assist in early disease diagnoses and targeted prevention in the medical fields (14).The use of these models is a powerful technique that can accurately predict clinical outcomes and identify important predictors.Artificial intelligence (AI) offers tremendous potential as a tool for improving both safety and predictive performance.Future advances in computing technology will be able to increase the application of electronic health records (EHRs) and electronic administrative data in order to better identify hospitalized patients who are at risk of falling.
Clinicians usually spend a lot of time using fall assessment tools to assess high-risk patients for falls.Moreover, the few features which are captured in these assessments focus primarily on intrinsic risk factors.Several studies have reported predicting high-risk falls using various machine learning algorithms, including the decision tree (10), logistic regression, linear discriminate analysis, naive bayes, kernel support vector machine, random forest, and neural networks (12,15,16).However, few studies have shown an accuracy rate of at least 80%.In addition, most studies have been based on Western populations with a limited number of samples.
Since most fall risk assessments in the past have been performed manually and caregivers have only been able to communicate patients' fall risk, this study applies an automated machine learning approach to help better identify important inpatient fall risk factors in Taiwan, and to validate the predictive efficacy of the model on a training and testing dataset.The goal of this study is to utilize artificial intelligence to predict who is at high risk of falling in hospitalized patients, while also replacing the use of manual fall assessment tools.Additionally, this study was able to demonstrate the effectiveness of the XGBoost model when working with multisource patient data, in which case the model could provide higher prediction performance for inpatient fall risk.However, using machine learning methods does have the potential to identify the most important factors for detecting inpatients at risk of falling, thereby improving the quality of patient care and reducing the workload of caregivers when performing fall assessments.This study may provide a reference for the development of AI-based fall prediction models for hospitalized patients.

Ethical approval
This study was approved by the Institutional Review Board of Taichung Veterans General Hospital (IRB No. CE20256B).All the data were anonymized data, and informed consent was hence waived.

Patient and public involvement
This research was performed without patient involvement.Patients were not invited with regards to design of study, measurement of outcome, and interpretation of results.

Study setting, design, and ethical considerations
This study retrospectively used EHR data taken from the Taichung Veterans General Hospital clinical database between January 1, 2015 and December 31, 2019, for patients who were at least 20 years of age.An exclusion conditions condition was that the collected data would not include the Hospice Center of the hospital, because the medical records of the Hospice Center are incomplete.Due to the government's emphasis on falls and the promotion of the Taiwan Patient Safety Reporting System, the chance of unreported or unidentified falls during data collection is very rare and less than 0.05.A total of 53,122 patient records were collected for this study, of which 1,157 involved fall patients and 51,965 did not.A review of the available literature shows that many manuscripts use the Morse Fall Scale to measure the risk of falls, which has been demonstrated to be a reliable method.However, this study sought to know whether patient characteristics and clinical data could each be useful in predicting falls and non-falls.This case-control study collected fall and non-fall patient characteristics and clinical data during hospitalization.In this study, since this nursing data processing is very important, this study was conducted manually to find out whether the data of 46 eigenvalues were correct or not.The data cleaning process was used to manually process the data in order to exclude unreasonable data such as, BMI > 80, blood pressure < 30, respiration >100, heart rate < 10, body temperature > 50, along with others.

Model training
This study uses the RapidMiner Studio version 9.8 Enterprise edition as an analysis tool for various models of artificial intelligence.RapidMiner Studio is a visual analysis process design software, which allows analysts to fully understand the process, where the results taken from the software can be used with full confidence.The purpose of using this tool is to quickly assist in the training mode phase.This study tested the performance of various models in order to find the ideal model for use in the prediction phase.Overall, this study tried to use the eight models, the Naive Bayes (NB), Generalized Linear Model (GLM), Logistic Regression, Decision Tree (DT), Random Forest (RF) (17,18), XGBoost and Neural Network (NN).This study adopted the method of upsampling for the recession group, attempting to obtain as much as possible of two types of data in order to achieve a balance for training the model.This study used the Bayesian optimization strategy to obtain the hyperparameters on the eight models used in the training, and used the 5-fold cross-validation process to train the model.Finally, in order to ensure the usability of the model, we chose the accuracy as the most valuable indicator of the classification model, the sensitivity and specificity as the most commonly used indicators in the medical field, the AUC as the predictive ability of the model ( 19), the higher the index means that the model predicts well, and the F1-score as the indicator calculated by considering both the precision rate and the recall rate.

Statistical analyses
Basic demographic data, biochemical examinations, and continuous variables are shown as the mean (standard deviation, SD), with categorical variables shown as the number (percent).The Mann-Whitney U test and Chi-square test were used to compare variables between the nonfaller and faller groups.All data were tabulated and analyzed using Microsoft Excel 2010 and SPSS for Windows, version 21.Statistical significance was set at a p value lower than 0.05 for all tests.

Results
This study identified a total of 1,157 hospitalized patients who had fallen and 51,965 patients who had not during their hospitalization periods.The definition of "fall" is: an unexpected change in body position, when where the center of gravity is out of balance, one cannot make a timely and effective response, causing the whole body to collapse and fall to the ground or a lower place (20).
After screening for candidate variables using the chi-square test or Mann-Whitney U test, 46 features were selected for the machine learning model, as shown in Table 1.Based on Bayesian optimization and 5-fold cross-validations of the training set, the parameters eventually collected for the models included ACC (accuracy), AUC (area under the ROC curve), sensitivity, specificity, and F1 score.As shown in Table 2, when comparing the performance of different imputation methods, the accuracy rate is between 93.74 and 98.91%.The AUC as a measure of the performance of the classifier was between 0.867 and 0.959, while the F1 score was between 49.57 and 69.52%.In terms of AUC, the deep learning methods showed the highest discriminatory ability (0.959), while the GLM performed the best in regards to ACC and specificity.These results may be due to the data being imbalanced, where the proportions of the declining groups were very different from those of the nondeclining groups.Therefore, the upsampling method was adopted for the declining groups, and attempts were made to use the two types of data as much as possible in order to achieve balance.
RapidMiner was chosen as the tool for data balancing because (1) it was first read in the original data set, (2) it first randomly selected 60% of the data in the undeclined ethnic group, (approximately 31,250 subjects), and (3) it used the Synthetic Minority Oversampling Technique (SMOTE) (4,5).The SMOTE method was used to upsample the descended ethnic groups, where it generated the 31,250 pieces of data from the descending ethnic groups for output used in model training.As shown in Table 3, the area under the receiver operating characteristic (ROC) curves, sensitivity, specificity, accuracy, and F1 scores were all compared and analyzed between the training sets.XGBoost achieved the best performance among the machine learning models.The ACC, sensitivity, specificity and F1 scores in the training set were 95.11, 95.37, 94.86 and 95.10%, respectively.This study selected the 25 best prediction features for retraining the XGBoost model where the training set sensitivity was 95.37%, with a specificity of 95.37% and an ACC of 0.99.

Discussion
From the years 2016 to 2019, according to Taiwan Medical Center, the incidence rate of falls in Taiwan hospitals was about 0.6% (21).In our hospital, the incidence rate of falls was only 0.4%.Reducing the incidence and injury rates of falls has always been an important responsibility with regards to patient safety in our hospital.Therefore, this study has aimed to use machine learning methods to better identify the most important factors necessary in the detection of inpatients who are at risk of falling.This study can both improve the quality of patient care, and reduce the workload of nursing staff by improving fall assessment methods.This method gives clinicians more time to prepare and prevent patients from falling.It will also decrease the number of injuries in patients due to falls, reduce the length of hospital stays by 6.4 days, and save up to 18,257 million TWD per inpatient in health care costs (6,7).
Fall risk assessment tools were developed as part of an evidencebased fall safety initiative.For instance, Chen et al. ( 6) used Taiwan's National Health Interview Survey to develop an elderly fall risk assessment plan and then verified fall predictive factors through the survey.A total score higher than 6 shows a high risk, with a sensitivity of 75.16% and a specificity of 52.75% in detecting falls.In addition, after studying the factors related to patient falls in Taiwan in the year 2001, some researchers found that there were six risk factors; agitation, poor vision, poor walking frequent urination/diarrhea, fall history, dizziness, and drug-related causes.
In addition to dizziness, drug-related falls combined with STRATIFY and the Hendrich II all risk model, which involves 14 selfdeveloped risk factor assessment scales, the risk factors include being a male an age greater than 65 years (9,21).Recently, in the "Balance Assessment and Fall Prevention Care and Guidance Guidelines, " formulated by the Ministry of Health and Welfare in 2019, it has been recommended that health care professionals use fall risk assessment to systematically and comprehensively identify factors that increase the risk of falls for the elderly, while also developing an individualized fall prevention program (9).
This study applied 46 eigenvalues that were reported related to the contribution to the prediction of falls.Among these features, pain score, fall history, hypnotics, diuretics, and osteoarthritis were the most relevant factors correlated with inpatient falls (10,12,16).While some of these factors have been identified as predictors of patient falls in previous studies, there is still room to learn whether a patient's cognition test, Mini-mental state exam score and Morse Fall scale assessment are all valid predictors of falls when hospitalized in a medical center or surgical nursing unit.It is worth noting that existing fall risk assessments do not contain all the items identified in our report regarding important features surrounding falls.As a result, an accurate prediction model that integrates simple and interpretable assessment tools involving high-performance contemporary machine learning methods could provide valuable clinical decision support in the medical field, enhance the quality of patient care, and reduce the workload of nursing staff when making fall assessments.This study used the SMOTE method to insert a small number of samples to artificially synthesize new samples and add them in to the data set.This causes the problems of overlap between classes (overlapping) and oversampling (oversampling), where technology modifies unequal data classes to create a balanced data set, obtaining the advantage of having no information loss while reducing both oversampling and overfitting (20).
This study applied to machine learning methods that were able to determine the major predictive factors surrounding hospitalized patient falls, while also identifying the most important risk factors related to inpatient falls.Additionally, the study applied crossvalidated prediction models from EHRs, as well as administrative data that identified the risk of falls based on that easily obtainable clinical data.This study also used a prediction model for patients who fell during hospitalization with a relatively large sample size according to the machine learning algorithms and electronic health records.This study found that the XGBoost algorithm achieved the best performance among seven machine learning models.The sensitivity value of the XGBoost model was 95.37%, with a specificity of 94.86% in the training set.The AUC and ACC were 0.99 and 95.11%, respectively.This prediction model has the potential to assist healthcare providers and organizational leadership decision-making which would improve the quality of care provided to patients.
This study be able to determine that, the XGBoost model showed an outstanding ability to solve overfitting, imbalanced samples and misclassification cost issues through regularization and pruning strategies (22).The study involved an optimized combination of decision tree algorithms and linear regression analyses under a gradient boosting framework (22).The XGBoost model was able to effectively reduce the irrelevant features.Moreover, the XGBoost classifier model was easier to extend to include new views of data, as this study just needed to train subclassifiers of the model on the new data rather than retraining the entire model from scratch.
Finally, this study implemented the trained model immediately in the clinic with limited patient information including age, gender, ethnicity and diagnosis.The predicted outcomes are presented on a dashboard and ward nursing staff can be informed more quickly of those patients who require special care, so the available advantage in serious falls prevention depends on the lead time of the assessment, which is important in situations such as deliberate falls and behavioral falls.In practice, the XGBoost model can be used in the clinical setting to predict severity after fall assessment using one of the fall risk assessment tools mentioned above (e.g., the Morse Falls Scale (MFS) (23), the STRATIFY Scale), and the Hester-Davies Scale (24).This additional layer of alerts for healthcare providers will allow for more efficient and cost-effective implementation, while also reducing the time required to prevent falls.

Limitations
This study has certain limitations.Although it offers advantages over existing fall risk assessment tools, the XGBoost model would need certain advantages.First, one a subset of inpatient data for the period from January 2015 to December 2019 was extracted from the  nursing record system and adverse event reporting system of the Taichung Veterans General Hospital.Testing our models on data taken from other hospitals would still be needed to establish external validity.Second, this study was conducted within Taichung Veterans General Hospital, and our model requires validation of its accuracy through clinical testing.Although this study identified models that have a relatively stable performance, sensitivity, specificity ACC and ROC, the estimates were subject to a case-control study design.The model performance tests would be best performed with a population sample; hence our ongoing study is continuing to conducting a validation of the XGBoost model in hospitalized patients for a period of one year.The XGBoost tool will be used to predict a high risk of fall injuries for fall patients, which will show the true calibration and discrimination of XGBoost.

Conclusion
Falls are an important issue in Taiwan's medical institutions, and hospitals have a well-established patient safety reporting system in place.This study showed that the XGBoost model is a useful tool for prediction because it achieved an accuracy of 95.11%, and the use of machine learning methods has the potential to identify the most important factors to be considered when detecting the risk of falls in hospitalized patients.The clinical application of this study can be faster for the caregivers to know that there should be fall patients as those, therefore this model and application can be effective in improving the quality of personalized care and also reduce the workload of the caregivers in performing fall assessment.Therefore, this study can be used as a future reference to better promote the development of fall prevention interventions.

TABLE 1
Baseline characteristics of hospitalized patients.

TABLE 3
Model performance comparison with different machine learning methods after data balancing.

TABLE 2
Model performance comparison with different machine learning methods.