Concordance Study Between IBM Watson for Oncology and Real Clinical Practice for Cervical Cancer Patients in China: A Retrospective Analysis

Watson for Oncology (WFO) is a artificial intelligence clinical decision-support system with evidence-based treatment options for oncologists. WFO has been gradually used in China, but limited reports on whether WFO is suitable for Chinese patients. This study aims to investigate the concordance of treatment options between WFO and real clinical practice for Cervical cancer patients retrospectively. We retrospectively enrolled 300 cases of cervical cancer patients. WFO provides treatment options for 246 supported cases. Real clinical practice were defined as concordant if treatment options were designated “recommended” or “for consideration” by WFO. Concordance of treatment option between WFO and real clinical practice was analyzed statistically. The treatment concordance between WFO and real clinical practice occurred in 72.8% (179/246) of cervical cancer cases. Logistic regression analysis showed that rural registration residences, advanced age, poor ECOG performance status, stages II-IV disease have a remarkable impact on consistency. The main reasons attributed to the 27.2% (67/246) of the discordant cases were the substitution of nedaplatin for cisplatin, reimbursement plan of bevacizumab, surgical preference, and absence of neoadjuvant/adjuvant chemotherapy and PD-1/PD-L1 antibodies recommendations. WFO recommendations were in 72.8% of concordant with real clinical practice for cervical cancer patients in China. However, several localization and individual factors limit its wider application. So, WFO could be an essential tool but it cannot currently replace oncologists. To be rapidly and fully apply to cervical cancer patients in China, accelerate localization and improvement were needed for WFO.


INTRODUCTION
Artificial intelligence (AI) is the frontier and dominating terrain of Information Technology which able to simulate human mental status and cognitive function (Jiang et al., 2017). With the development of AI and medical diagnosis technology, clinical decision-support systems (CDSS) with intelligent diagnostic function has become one of the important issues of science for medical information (Meyer et al., 2018). Watson for Oncology (IBM) is a representative AI CDSS that developed by IBM Co.Ltd in United States. WFO can provide a reasonable individualized treatment plan for cancar patients by obtaining valuable information from medical records. WFO first officially landed in China in 2016, until now, more than 80 hospitals use WFO as an important medical diagnostic tool for individualized treatment of tumor (IBM, 2017). WFO can provide counseling services for almost all cancer patients. However, whether WFO was fit for Chinese cancer patients, especially cervical cancer patients.
Cervical cancer is common in the female genital tract malignant tumors, and the incidence of which is second only to that of breast cancer among women worldwide, making it the second-most serious cancer threatening the health and lives of women (Jassim et al., 2018). Compared to breast cancer, cervical cancer is more common in developing countries due to poor health status, and it is the most common in China (Gu X. Y. et al., 2018). And rural and remote areas are also a prevalent regions for cervical cancer in China. But the current problem of the medical service is that the main hospitals hold too many premium resources, but in the meantime, the primary health agencies are excessively lack of resources (Bao et al., 2018). Cervical cancer patients in rural and remote areas can not reach the effective treatment recommendation, especially at centers where cancer expert resources are limited. So, WFO is of great significance for Chinese patients with cervical cancer, especially patients in rural and remote areas with limited medical resources.
Therefore, we conducted a retrospective and observational study on cervical cancer at The Second Xiangya Hospital Cancer Center to explore consistency between WFO and clinical treatment recommendations supported by an expert panel of cancer specialists for Cervical cancer patients.

Study Population
This retrospective study was reviewed and approved by the Medical Ethics Committee of The Second Xiangya Hospital of Central south university (approval number was 2017-S104). We retrospectively and randomly selected 300 cases of cervical cancer patients from 05/2016 to 08/2018. All patients with cervical cancer confirmed by pathology at The Second Xiangya Hospital Cancer Center. Untreated Patients and recurrent tumors, rare histology that not yet trained to offer treatment options by WFO system were excluded. A total of 18% (54/300) cases excluded from our study and 82% (246/300) cases were included in our study. The detailed patient selection process is shown in Figure 1.

Watson for Oncology
Watson for Oncology (IBM Corporation, United States, version 18.1R) used in our study were provided by Baheal Intelligent Technology Co., Ltd 1 . The clinicopathologic data of supported cases were extracted from medical records and entered into the WFO system. Treatment options recommended by WFO were presented in three categories: Blue represents "Recommended" with a strong evidence supported, Orange represents "For consideration" with a potentially suitable evidence-based alternative considered by oncologists based on their clinical judgment, and Red represents that is "Not recommended" that a treatment with contraindications or strong evidence against its use.

Real Clinical Practice for Cervical Cancer
The Second Xiangya Hospital Cancer Center one of the biggest and best oncology departments in the Hunan Province of China. Gynecological Oncology Center is the most important part of The Second Xiangya Hospital Cancer Center and mainly serves cervical cancer, ovarian cancer, endometrial cancer, and other gynecological malignant tumors. Gynecological Oncology Center has a multidisciplinary team (MDT) composed of oncologists, gynecologists, radiologists, pathologists, and nutritionists, et al. MDT forms and implements a comprehensive regimen based on NCCN guidelines and the patient's specific conditions. This comprehensive regimen was considered to be a real clinical practice for cervical cancer.

Data Acquisition and Concordance Judgment
The available clinicopathologic data of 246 patients included a registered residence, age, performance status, pathological type, differentiation degree, FIGO stage, lymphatic and distant metastasis, HPV status, and detailed clinical treatment plan were collected from Second Xiangya Hospital Cancer Center clinical electronic medical records and inputted into WFO system by 2 oncologists manually. Treatment options generated by WFO and recorded through two trained oncologists. It should be noted that in the data analysis process, real clinical practice were categorized as concordant if treatment options were designated "recommended" or "for consideration" by WFO. And if the real clinical practice was not recommended by WFO or if WFO did not provide the same treatment options, the recommendations were considered as discordant. The discordant cases were reevaluated by two senior oncologists provided their    Table 3.

DISCUSSION
From 2013, concordance studies between WFO and physicians have been performed in various countries and cancer types. A double-blind study showed that 93% concordance rate for 638 breast cancer patients (Kaur and Singh Mann, 2018;Somashekhar et al., 2018). A retrospective study from India for 1000 consecutive cases showed 80% concordance between multidisciplinary team (MDT) (Baek et al., 2017).
A observational study from Korea showed a 73% concordance rate for colon cancer and a 49% concordance rate for gastric cancer (Somashekhar et al., 2016;Suwanvecho et al., 2017). And, a comparative Study from Korea indicated that WFO without the gene expression assay has limited clinical utility (Kim et al., 2018). It appears that the concordance results varies by countries and cancer types (Zhou et al., 2018). For China, a huge population and regional differences created a different therapeutic experiences and considerations for cancer patients,  Neoadjuvant/adjuvant chemotherapy 6 (9.1) PD-1/PD-L1 antibodies 2 (2.8) as well as large differences with Western countries. Also, a retrospective study (Liu et al., 2018) reported by our center revealed that treatment concordance between WFO and MDT occurred in 65.8% (98/149) of lung cancer. Another retrospective study (Zhou et al., 2019) from China showed that Ovarian cancer, lung cancer and breast cancer obtained a high concordance, the concordance of gastric cancer was very low, Incidence and pharmaceuticals may be the major cause of discordance. However, limited reports on whether WFO is suitable for Chinese cervical cancer patients, Zhou et al. reported 14 cervical cancer patients in this study, but the sample size is too small. Our retrospective study provides the first evidence that accelerates localization and improvement were needed for WFO before comprehensive application in cervical cancer patients in China. Although treatment options generated by WFO were mostly concordant with real clinical practice, there are still unresolved issues. Firstly, as mentioned in the manual , some clinical settings are not yet supported by WFO system. In our study, of 73.7% (28/38) unsupported cases were recurrent tumors patients. But compare with our center, grass-roots hospitals have a greater proportion of patients with recurrent tumors. So, the cases that cannot be supported by WFO system are very large for cervical cancer patients in China. Secondly, localization factors such as physical of patients, medical reimbursement plan, economic condition, and patient preferences of China were different from western countries, and they ultimately affect the inconsistency. In our study, of 46.4% (31/67) cases select nedaplatin due to cannot tolerate gastrointestinal reactions of cisplatin, of 26.9% (18/67) patients reject bevacizumab therapy for financial burden. of 19.4% (13/67) patients prefer surgical therapy instead of concomitant radiochemotherapy. Moreover, registered residence, age, performance status, FIGO stage have a remarkable impact on consistency. Urban registration patients, low age group, good performance status, and stage II/III disease exhibiting higher concordance than rural registration patients, advanced age group, poor performance status, and stage I/IV disease. These personal factors make WFO unable to achieve individualized treatment and affect the consistency significantly in China. Finally, neoadjuvant/adjuvant chemotherapy . Chemotherapeutic drugs Goffin et al. (2010) such as gemcitabine, docetaxel, mitomycin, irinotecan, pemetrexed, vinorelbine, and PD-1/PD-L1 antibodies (Kim et al., 2017) drugs recommendations that performed in real clinical practice are not included in the WFO system.
Compared with previous research, our study provides the first evidence that WFO is not suitable for Chinese cervical cancer patients currently, and the sample size of this study was the largest among all cervical cancer studies performed. Also, we not only reported the consistency between WFO and real clinical practice, but also analyzed several influence elements and offered certainly advises for the improvement of WFO to better suit Chinese patients. But, our study contains some limitations. Firstly, this was a retrospective and observational study with control groups lacked, several unmeasured elements may influence the outcome. Secondly, treatment preferences among different experts also affect consistency. Thirdly, the distribution of clinicopathological characteristics among patients is imbalanced, for example, fewer patients were stage IV diseases may lead to a large disagreement for Stage IV tumors. Finally, molecular parameters, such as mutations, gene expression or protein localization can affect the treatment decision. But, in China, unlike lung cancer and breast cancer, gene detection were lacked for cervical cancer. Although there are some targeted drugs that may be effective for cervical cancer, such as PARP inhibitors (for BRCA1 or BRCA2 mutations patients), EGFR tyrosine kinase inhibitors (for EGFR mutations patients), gene detection is still not widely used in China. So, in our study, Because of the lack of gene detection datas, we cannot observe the effect of molecular parameters on treatment decisions.
For WFO, WFO could be an essential tool for clinicians, provides good references and literature for medical students, or even give some treatment advice to non-specialist (Malin, 2013;Werner et al., 2016). However, we believe that human physicians will not be replaced by AI in the foreseeable future, WFO still has a long way to go to replace oncologists. Medicine is not just a science, but also a social and psychological subject. Any tool and guidelines can only be used as a doctor's reference, localization factors and individual elements should considered for different patients, especially for cancer patients with large heterogeneous (Kemin et al., 2017). Therefore, WFO must be significantly improved to adapt the real clinical practice in different countries. Patient's physical and mental state, economic situation, complications, patient's treatment preference and medical reimbursement plan in different countries should be taken into account and not just provide advice based on existing knowledge. For China, a unique medical database with Chinese characteristics should be created by WFO to adapt and serve Chinese cancer patients.

CONCLUSION
In conclusion, WFO recommendations were in 72.8% of concordant with real clinical practice for cervical cancer patients in China. However, several localization and individual factors limit its wider application. So, WFO cannot replace oncologists for cervical cancer patients in China currently. WFO could be an effective decision-support tool in cancer therapy for Chinese physicians, it also helps to standardize the treatment of cervical cancer. To be rapidly and fully apply to cervical cancer patients in China, accelerate localization and improvement were needed for WFO.

DATA AVAILABILITY STATEMENT
The datasets generated for this study are available on request to the corresponding author.

ETHICS STATEMENT
This retrospective study was reviewed and approved by the Medical Ethics Committee of The Second Xiangya Hospital of Central South University (approval number was 2017-S104).

AUTHOR CONTRIBUTIONS
CH was responsible for overall planning for research. FZ was responsible for data collection and statistical analysis. YT and CL were involved in data analysis. JM was participated in the preparation of manuscript.