AUTHOR=Chai Hua , Xia Long , Zhang Lei , Yang Jiarui , Zhang Zhongyue , Qian Xiangjun , Yang Yuedong , Pan Weidong TITLE=An Adaptive Transfer-Learning-Based Deep Cox Neural Network for Hepatocellular Carcinoma Prognosis Prediction JOURNAL=Frontiers in Oncology VOLUME=Volume 11 - 2021 YEAR=2021 URL=https://www.frontiersin.org/journals/oncology/articles/10.3389/fonc.2021.692774 DOI=10.3389/fonc.2021.692774 ISSN=2234-943X ABSTRACT=Background: Predicting hepatocellular carcinoma (HCC) prognosis is important for treatment selection, and it’s increasingly interesting to predict prognosis through gene expression data. Currently, the prognosis remains of low accuracy due to the high dimension but small sample size of liver cancer data. In previous studies, transfer learning strategy has been developed by pre-training models on similar cancer types and then fine-tuning the pre-trained models on the target dataset. However, transfer-learning has limited performance since other cancer types are similar at different levels, and it’s not trivial to balance the relations with different cancer types. Methods: Here, we proposed an adaptive transfer-learning based deep Cox neural network (ATRCN), where cancers are represented by 12 phenotype and 10 genotype features, and suitable cancers were adaptively selected for model pre-training. In this way, the pre-trained model can learn valuable prior knowledge from other cancer types while reducing the biases. Results: ATRCN chose the pancreatic and stomach adenocarcinomas as the pre-training cancers, and the experiments indicated that our method improved the C-index of 3.8% by comparing with traditional transfer-learning methods. The independent tests on three additional HCC datasets proved the robustness of our model. Based on the divided risk subgroups, we identified 10 HCC prognostic markers including one new prognostic marker TTC36. Further wet experiments indicated TTC36 is associated with the progression of liver cancer cells. Conclusion: These results proved our proposed deep learning-based method for HCC prognosis prediction is robust, accurate, and biologically meaningful.