Artificial Intelligence for COVID-19: A Systematic Review

Background: Recently, Coronavirus Disease 2019 (COVID-19), caused by severe acute respiratory syndrome virus 2 (SARS-CoV-2), has affected more than 200 countries and lead to enormous losses. This study systematically reviews the application of Artificial Intelligence (AI) techniques in COVID-19, especially for diagnosis, estimation of epidemic trends, prognosis, and exploration of effective and safe drugs and vaccines; and discusses the potential limitations. Methods: We report this systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched PubMed, Embase and the Cochrane Library from inception to 19 September 2020 for published studies of AI applications in COVID-19. We used PROBAST (prediction model risk of bias assessment tool) to assess the quality of literature related to the diagnosis and prognosis of COVID-19. We registered the protocol (PROSPERO CRD42020211555). Results: We included 78 studies: 46 articles discussed AI-assisted diagnosis for COVID-19 with total accuracy of 70.00 to 99.92%, sensitivity of 73.00 to 100.00%, specificity of 25 to 100.00%, and area under the curve of 0.732 to 1.000. Fourteen articles evaluated prognosis based on clinical characteristics at hospital admission, such as clinical, laboratory and radiological characteristics, reaching accuracy of 74.4 to 95.20%, sensitivity of 72.8 to 98.00%, specificity of 55 to 96.87% and AUC of 0.66 to 0.997 in predicting critical COVID-19. Nine articles used AI models to predict the epidemic of the COVID-19, such as epidemic peak, infection rate, number of infected cases, transmission laws, and development trend. Eight articles used AI to explore potential effective drugs, primarily through drug repurposing and drug development. Finally, 1 article predicted vaccine targets that have the potential to develop COVID-19 vaccines. Conclusions: In this review, we have shown that AI achieved high performance in diagnosis, prognosis evaluation, epidemic prediction and drug discovery for COVID-19. AI has the potential to enhance significantly existing medical and healthcare system efficiency during the COVID-19 pandemic.


INTRODUCTION
Coronavirus Disease 2019 , caused by severe acute respiratory syndrome virus 2 (SARS-CoV-2) was first detected in December 2019, and spread rapidly to most cities and countries around the world (1)(2)(3). During face-to-face contact, SARS-CoV-2 is mainly transmitted through respiratory droplets (4). The infection may cause mild symptoms of upper respiratory tract infections, as well as extremely severe sepsis and shock. It may lead to serious and lethal complications in vulnerable populations, especially in the elderly with comorbidities (4)(5)(6). As of 16 March 2021, SARS-CoV-2 has affected more than 200 countries and led to enormous losses, causing more than 120 million confirmed cases and 2.6 million identified deaths. The rising incidence and massive casualties caused by COVID-19 exert considerable pressure on limited healthcare resources. Effective tools are needed to streamline the diagnosis, treatment and surveillance of COVID-19 and increase the clinical efficiency of healthcare systems (7). Recent studies have shown that artificial intelligence is a promising technology as they can achieve better scale-up, accelerate processing power, and even outperform humans in specific healthcare tasks (8).
Artificial intelligence (AI) is a field of algorithm-based applications that enable machines to solve knowledge problems and use algorithms to simulate human decision-making, and continuously improves performance by applying inputted data to perform specific tasks (9)(10)(11). The advantages of AI are reflected in high sensitivity and specificity in identifying the object, the speed of reporting and consistency of results (9). In recent years, AI has made significant progress, especially in predictive machine learning models for medical care. Deep learning is a method of ML, based on the complex architectures of Artificial Neural Networks (ANN). Deep learning reveals significant discriminative performance after providing sufficient training data sets and is essential for making predictions (12). In medicine, technologies based on Artificial intelligence and machine learning (AI/ML) aim to improve the quality of medical care, increase diagnostic accuracy and reduce potential errors and predict outcomes by discovering new insights from the enormous amount of data produced by the experience of many patients (10).
Researchers have made significant contributions to the campaign against COVID-19, and new COVID-19-related AI models in the literature are rapidly increasing. Well-trained artificial intelligence models can ensure accurate and rapid diagnosis or assist doctors to streamline the diagnosis and reduce manual labor (13,14). AI models could early detect the patients at higher risk and characterize the epidemiology of COVID-19 and model disease transmission by training data (15,16). Artificial intelligence-based methods could assist in the discovery of novel drugs and vaccines, such as repurpose exist drugs, screen targets as vaccines based on the potential mutation model to SARS-CoV-2, as well as screen compounds as potential adjuvants for vaccines (3,17). AI-powered chatbots have been used with success in clinical scenarios and can advise many more people than a manned call center and ease the stress placed on medical hotlines (18). AI could manage the pandemic by using thermal imaging to scan public spaces for people potentially infected, and by enforcing social distancing and lockdown measures (3,17).
Artificial intelligence has been widely used in COVID-19, including diagnosis, public health, clinical decision making, social control, therapeutics, vaccine development, surveillance, combination with big data, operation of other core clinical services, and management of patients with COVID-19 (3,18,19). In order to solve the significant pressure of the limited medical resources caused by the pandemic of COVID-19, rapid diagnosis, accurate prediction, enhanced monitoring, and effective treatments are the most important measures to control the spread of the pandemic. Many related review articles have been published. However, the results of these studies are inconsistent and there is little research systematically assessing the application of AI for COVID-19 in accordance with PRISMA, and most of them only discuss aspects such as diagnosis or treatment. Therefore, we conducted this review to assess the performance of AI for COVID-19 systematically, and to describe the main categories of AI use, the potential benefits and limitations and future directions for AI.

Search Strategy and Eligibility Criteria
This systematic review is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (20) (Supplementary Material 2). We searched PubMed, Embase and the Cochrane Library for published studies from the inception of these resources to 19 September 2020 using the following terms related to artificial intelligence and COVID-19: "Artificial intelligence, " "Machine Intelligence, " "Machine learning, " "Deep learning, " "Predictive model, " "2019 novel coronavirus disease, " "COVID-19, " "2019 novel coronavirus infection, " "coronavirus disease-19, " and "2019-nCoV disease." The details of the search strategy are in the Supplementary Material 1.
We included original studies fulfilling the following criteria: (I) research topic was focused on the application of AI for COVID-19, (II) participants had a confirmed diagnosis of COVID-19 by reverse transcription-polymerase chain reaction (RT-PCR) testing or other laboratory examination (where appropriate), and (III) article was published in English.
We excluded studies if: (I) insufficient data were available, (II) we were unable to access the full text or complete data, or (III) the report was a review, case-report or comment.
Two trained researchers (Lian Wang, Dongguang Wang) screened titles, abstracts and the full text of potentially eligible studies independently using Endnote X8.2 software, Thomson Reuters. Discrepancies were resolved through consultation with a third researcher (Xiang Tong, Tao Liu).

Data Abstraction and Quality Assessment
We extracted data and recorded the following information for each study: basic information for the article (title, first author, date of publication), experimental design (algorithm, sample size) and primary outcome (sensitivity, accuracy and specificity of AI for diagnosis and prognosis evaluation; prediction of epidemic; drug repurposing and development). If a study used multiple models, we extracted the most discriminative one.
Three researchers (SZ, JH, LZ) used PROBAST (prediction model risk of bias assessment tool) to assess the risk of bias in the included studies (21). The PROBAST statement was divided into four domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signal questions to help structure judgment of risk of bias for prediction models, such as the range of included patients, whether the same predictors and results were defined for all participants, whether the clinical decision rule was determined prospectively, and whether a relevant measure of accuracy was reported (22). The details of PROBAST are in the Supplementary Material 3.
We divided the included studies into four categories: diagnosis, prognosis, epidemic prediction, and drug discovery. In the diagnosis and prognosis domains, we evaluated the classification performance using AUC, accuracy, sensitivity, and specificity. For epidemic trends and drugs and vaccines discovery, we listed the results only because there are no suitable evaluation indicators.

Role of the Funding Source
This study was supported by National Key R&D Program of China (2017YFC1309703) and 1·3·5 project for disciplines of excellence-Clinical Research Incubation Project, West China Hospital, Sichuan University (2019HXFH008). The funders of this research did not contribute to the study design, data analysis, data interpretation or preparation of the manuscript.

Chest CT Images
Deep learning with a convolutional neural network (CNN) has gained increasing attention for its outstanding image recognition performance (98). Several of the studies (n = 18) we identified had developed AI models based on CNN and these showed excellent ability to discriminate COVID-19 and non-COVID pneumonia by automatically detecting chest CT images with an accuracy of 70.00 to 99.87%, sensitivity of 73.00 to 100.00%, specificity of 25 to 100.00%, and AUC of 0.732 to 1.000 ( Table 2) (14, 23, 24, 26, 27, 29-31, 33-40, 43, 44). Mei et al. (29) developed a joint CNN model that diagnoses COVID-19 patients rapidly by combining chest CT findings with clinical symptoms, exposure history, and laboratory tests. Moreover, Mishra et al. (30) proposed a decision-fusion approach, which combined the predictions of each Deep CNN model and achieved results above 86% for all the performance metrics under consideration. Three studies (14,41,42) found that AI models had higher test accuracy, sensitivity, specificity than radiologists, and with the assistance of AI, the radiologists made diagnosis with much faster speeds and achieved a higher diagnostic performance.

Chest X-Ray Images
Although CT images have high sensitivity in detecting COVID-19, costs and radiation doses are relatively high. On the contrary, chest X-ray is a low-cost and rapid detection method, which might be used in initial screening of suspected cases of COVID-19 infection, supporting the timely application of quarantine measures in positive patients (45). Several studies have developed AI techniques to automatically detect and extract features from chest X-rays to assist in the diagnosis of COVID-19 with high accuracy (71.90 to 99.92%), sensitivity (75.00 to 99.44%), specificity (71.80 to 100.00%), and AUC (0.81 to 0.999) (45-67) ( Table 3).

Predicting the Prognosis of COVID-19
The ability to identify a patient's risk of deterioration during their hospitalization is critical for effective medical resource allocation and to ensure that patients receive appropriate management during the COVID-19 pandemic. We identified several AI models built on the chest CT images that accurately quantified lung abnormalities related to COVID-19 and evaluated the severity and prognosis of the disease (70,76,79,80). Some studies showed that deep learning models could predict the risk of COVID-19 patients developing critical illness, based on clinical characteristics at hospital admission, such as clinical, laboratory and radiological characteristics (68, 71, 73-75, 77, 78).   Table 4). Accurately determining the prognosis of COVID-19 patients as early as possible and starting early treatment may improve their prognosis and reduce mortality from COVID-19.

Predicting the Epidemic Trend of COVID-19
COVID-19 has spread globally and had a substantial impact. It was defined as a pandemic by the World Health Organization (WHO) in March 2020. As the COVID-19 pandemic evolves, it is vital to focus on building prediction models to help policymakers and health managers to allocate healthcare resources and prevent or limit outbreaks (82). We identified 9 studies that sought to predict the epidemic trend of COVID-19 (81-89) ( Table 5). Of these, 6 studies used long short-term memory (LSTM) models with or without other models to predict the   (84) proposed prediction models including support vector regression (SVR), autoregressive integrated moving average (ARIMA), long short-term memory (LSTM) and Bi-directional long short-term memory (Bi-LSTM) to predict confirmed cases, deaths and recoveries in ten major countries affected by COVID-19. Zheng et al. (85) proposed an improved susceptible-infected (ISI) model to estimate the variety of the infection rates and to analyze the transmission laws and development trend. Ribeiro et al. (88) used several machine learning models to forecast the cumulative confirmed cases of COVID-19 in the ten Brazilian states with a high daily incidence, and rank the models based on their accuracy. The results of these studies may bring broad benefits by helping to control and prevent COVID-19.

Drug Discovery and Vaccine Development for COVID-19
With the spread of COVID-19 showing no signs of slowing and there are few proven effective therapeutics for COVID-19, thousands of people continue to die from the disease every day. It is essential to develop antiviral drugs and vaccines against SARS-CoV-2. It usually needs a long time to develop a drug or vaccine using traditional methods but to try to accelerate this process, several studies have applied AI techniques to identify potential drugs and develop effective and safe vaccines for COVID-19. We identified 9 studies that developed models to find potential drugs and vaccines for COVID-19(17, 90-97) ( Table 6).

Drug Repurposing
Drug repurposing refers to the application of approved drugs to new therapeutic indications, which has become a successful drug development strategy for reducing development costs and increasing the simplicity of drug approval procedures (99). AI algorithms could be trained and then be used to screen existing drugs that may prove effective in the treatment of COVID-19. Ke     library and identify top 1,000 potential ligands for SARS-CoV-2 Mpro protein.

Vaccine Development
Without an existing effective medical therapy, the development of an effective and safe vaccine is an important method to deal with this highly infectious disease caused by the SARS-CoV-2 coronavirus. Ong et al. applied a ML tool to predict the S protein, nsp3, 3CL-pro, and nsp8-10 were crucial to the viral adhering and host invasion by investigating the entire proteome of SARS-CoV-2. SARS-CoV-2 S protein has the highest protective antigenicity score and was identified as the most favorable vaccine candidate, besides, the nsp3 protein was selected for further investigation (17). The predicted vaccine targets have the potential for COVID-19 vaccine developed, however, they need to be further evaluated in clinical studies.

DISCUSSION
Our systematic review includes 78 articles on the application of AI for COVID-19. These spanned radiological diagnosis, prediction of prognosis, estimation of epidemic trends and drugs and vaccines discovery for COVID-19. The gold standard for diagnostic tests for COVID-19 is realtime reverse-transcriptase polymerase chain reaction (RT-PCR). However, RT-PCR does produce false negatives or fluctuating results (100). A study compared the diagnostic performance of chest computed tomography (CT) scan with RT-PCR and found that the chest CT is more sensitive than RT-PCR (98 vs. 71%, respectively, P < 0·001) (101), suggesting that Chest CT could be a supplementary diagnostic measure to help physicians make faster and more accurate decisions. AI technique is used for identifying or classifying images, recognizing speech and processing natural language (102). It is well-suited to developing tools to assist with the use of chest CT to diagnose COVID-19 (103). Advanced AI-based algorithms can learn the typical CT image signs, such as bilateral and subpleural ground-glass opacities (GGO), vascular thickening, spider web, and even crazy-paving patterns (104). In addition, the algorithms can also learn some high-dimensional features that radiologists cannot handle, such as texture and wavelet information, thereby allowing pneumonia caused by SARS-CoV-2 to be distinguished from that caused by other pathogens, through advanced AI-based  algorithms (36). Several studies have shown that deep learning can automatically differentiate COVID-19 from non-COVID-19 or other pneumonia diseases through extracting features from chest CT and X-rays images. As shown in Tables 2, 3, most of the studies achieved over 90% accuracy, sensitivity and specificity. The performance of AI-assisted diagnosis was comparable to radiologists with significant clinical experience and could assist and improve the performance of radiologists. This means that AI-assisted diagnosis is a useful screening tool, which might shorten patient waiting time, simplify the workflow, reduce the workload of radiologists and allow them to respond more quickly and effectively in emergencies (38). AI techniques have recently shown great potential in the real-time diagnosis of COVID-19 by using images. However, the severity of disease, comorbidities, and the proportion of asymptomatic patients have an impact on the diagnostic sensitivity of chest CT (105). Chest CT have a relatively high sensitivity in symptomatic COVID-19 patients, but low specificity (106). The Italian Society of Medicine and Interventional Radiology recommends that CT should be used as a screening tool only for symptomatic patients with specific indications (107). AI should be used to assist diagnosis, not an independent diagnostic tool. Second, the evaluation of patients based on a single data type may be biased, therefore, AI-assisted diagnosis needs to be used in combination with other laboratory tests and a multimodal AI framework was required to analyze different data types (108). Third, several studies used a relatively small amount of data to train the deep learning models, and the testing data set had the same sources as the training data set. This may cause the problem of overfitting of the models (108,109). Fourth, there is little evidence to directly compare the performance of humans and machines or the performance of AI in actual clinical work. Only Bai, Song, Ni, and a latest research show that AI assistance improved radiologists' performance in identifying COVID-19 (13,14,41,42). In addition, confounding factors can influence the internal validity of researches and the accuracy of AI-based radiological interpretation, such as the variation of respiratory effort, image contrast, technique, and resolution of radiological images (110).
In regard to the prognosis of patients with COVID-19, information available at hospital admission, typically 6 days (median) before the patient developed severe COVID-19, can be used by AI for early detection of patients at higher risk, allowing adjustments to their in-hospital allocation and management (68). AI can evaluate the prognosis of COVID-19 patients by clinical manifestations, laboratory and radiological characteristics and identify potential predictive biomarkers related to the disease's severity. The significantly elevated LDH levels reflects the severity of pneumonia, and increased serum CRP predicts the risk of death in patients with severe COVID-19 (72). Age and comorbidities may be risk factors for severe COVID-19 after hospitalization, such as diabetes, hypertension and cardiovascular diseases (74). As well as chest CT images being a powerful tool to assist clinical diagnosis because of its high sensitivity and the ability to quantify the COVID-19 associated lung abnormalities, they also help assess the severity and prognosis of the disease and monitor the development of the disease (70,76,80). Accurate risk prediction of patients with critical COVID-19 may help to optimize patient triage and in-hospital allocation, monitor disease progression and treatment response, prioritize medical resources and improve the overall management of the COVID-19 pandemic (68). However, several of the studies we identified were retrospective singlecenter studies, which reduces their external validity. Therefore, the results in this review may not be generalizable to other environment and healthcare systems, especially considering the high variability of COVID-19 in different countries and populations (68,72). In addition, prospective studies with a larger number of patients from multiple locations are required to verify the predictive ability of a model (81).
Many statistical and numerical models have been used to predict the trend of the COVID-19 pandemic, such as the epidemic peak, transmission and development trend, the SEIR model is one of the most popular models (81). Alsayed et al. combined the SEIR model with machine learning to characterize the epidemic dynamics and to predict possible contagion scenarios of COVID-19 in Malaysia (81). Long shortterm memory (LSTM) is a recurrent neural network that is an effective model for the prediction of time series where data are sequential (82). Therefore, LSTM has been widely used to predict the confirmed cases, death and recovery, development trend of COVID-19 through time (82,(84)(85)(86)(87)89). Embed the NLP and LSTM into the Improved susceptible-infected (ISI) model is more accurate and reliable than the traditional epidemic model, providing a basis for estimating the law of virus transmission (85). In addition, AI models, such as SVR, stacking-ensemble learning, ARIMA, CUBIST, RIDGE, RF and MLP also play an important role in estimating the epidemic of COVID-19 (83,88). AI techniques provide useful tools to help policy makers make decisions and take actions to prevent diffusion at the early stage of the epidemic and to minimize the subsequent impact of COVID-19. However, we cannot verify and validate the database, so it is difficult to compare and calibrate results with other studies.
Traditional drug repurposing design methods are based on repeated trials and there is no systematic way to screen the enormous drug-dose parameter space (111). AI is an effective approach to quickly detect potential drugs as antiviral therapeutics for COVID-19 (91). Deep learning, using the relationship between drug targets and diseases can be used as a helpful tool to assist drug repurposing and minimize the possibility of failure in clinical trials (92). Chymotrypsin-like protease is a major therapeutic target, and several studies used it to identify potential therapeutic drugs against COVID-19 by performing drug screening over protein-ligand or proteinpeptide among existing drugs (90,94,112). In addition, using AI techniques to conduct virtual screening of biologically active compounds can support new drug discovery. However, all predicted drugs must be tested in randomized trials before being used in COVID-19 patients (92).
During the COVID-19 pandemic, an effective and safe vaccine is essential to prevent infection and reduce deaths. The development of vaccines was a complicated process with many difficulties, such as the complexity of the human immune system and the variability between different populations (113). Research organizations in many nations and multinational companies are developing various vaccines, including whole virus vaccine, subunit vaccine, nucleic acid vaccines. Researchers are trying to use AI techniques to explore the vaccine development. Ong et al. (17) predict 6 vaccine candidates, including S protein, nsp3, 3CLpro, and nsp8-10, S protein was identified as the most favorable vaccine candidate. Nsp3 has a high antigen protection score and has not been used for vaccine development, therefore it was also selected for further investigation. Currently, S protein has been widely used in subunit vaccines, and other proteins are expected to be used in vaccine development. In addition, the latest research analyzed the entire SARS-CoV-2 proteome via AI and identified some of the epitope hotspots that can be used in vaccine formulations (114).
AI has the potential to be an important tool in the fight against COVID-19 and similar pandemics. However, there were many problems in using AI to predict and diagnose COVID-19, and rigorous clinical trials were required before drugs and vaccines developed by AI are approved, so the use of AI has so far been rather limited. It requires continuous efforts by researchers. But recent studies have shown that AI tools such as computer vision and robots have the potential to be widely promoted and used in the short term, such as infrared thermal cameras have been paired with AI-powered facial recognition systems to determine if the individual are wearing masks, using camera images to observe whether social distancing rules are complied, AI-based dialogue chatbots can complete symptom screening and patient education (19). Robotics, AI, and digital technology have been implemented in sanitation for hospitals and public areas, delivery in hospitals and public spaces, patrolling, screening, health consulting, and virus tracking (115).
Our systematic review has several limitations that should be noted. First, although we conducted a systematic search, we only included articles published in English, introducing the possibility of publication bias. Second, in the included studies, the models in 60 studies were at high risk of bias according to assessment with PROBAST ( Table 1), and the remaining 18 studies were not evaluated due to lack of suitable evaluation tools. Therefore, the predictive performance of these AI models when used in practice is probably lower than that reported, which means that the predictions of these models may be unreliable. Third, many studies had small sample sizes, and the testing data set had the same sources as the training data set, which leads to an increased risk of overfitting. Fourth, over one fifth of the studies were retrospective single-center studies, which might limit their applicability to the specific center or the same geographical region. This means that results may not be generalizable to other settings and places.

CONCLUSIONS
Artificial intelligence has been widely explored in the medical field, especially for enhancing medical and healthcare capabilities. At present, many countries continue to struggle to contain the spread of COVID-19. Facing limited medical resource and increasing healthcare pressure, the use of AI techniques to assist with diagnosis, treatment, prediction of prognosis, evaluation of epidemic trends, surveillance and public health decision-making may improve the efficiency and ability of humans to fight the COVID-19 pandemic.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

AUTHOR CONTRIBUTIONS
YZ and LW conceived of the study. LW and DW screened the literature for relevancy and did the data extraction. SZ, JH, and LZ did the quality appraisal. XT and TL resolved any disagreements in study relevancy, extraction, and quality appraisal. LW and LC drafted and revised the manuscript. HF, YZ, and MC directed and revised the manuscript. All authors participated in data interpretation and revised the manuscript for intellectual content. The funders of this research did not contribute to the study design, data analysis, data interpretation or preparation of the manuscript.