AUTHOR=Wang Chong Ming , Wang Xing Jian , Chen Yang , Wen Xue Mei , Zhang Yong Heng , Li Qing Wu 

TITLE=Deep learning based on self-supervised pre-training: Application on sandstone content prediction

JOURNAL=Frontiers in Earth Science

VOLUME=Volume 10 - 2022

YEAR=2023

URL=https://www.frontiersin.org/journals/earth-science/articles/10.3389/feart.2022.1081998

DOI=10.3389/feart.2022.1081998

ISSN=2296-6463

ABSTRACT=Deep learning has been widely used in various fields and performs well in recent years. Therefore, deep learning is the future trend to realize the intelligent and automatic interpretation of seismic data. However traditional deep learning only uses labeled data to train the model which wastes a large amount of unlabeled data. Self-supervised learning, widely used in natural language processing (NLP) and computer vision, is an effective method of learning information from unlabeled data. In this paper, a pretext task is designed with reference to BERT and MAE, through the pretext task we can realize self-supervised pre-training of unlabeled seismic data. After pre-training, we fine-tune the model to the downstream task. Experiments show that the model can effectively extract information from unlabeled data through the pretext task, and the pre-trained model has better performance in downstream tasks.