AUTHOR=Müller Martin , Salathé Marcel , Kummervold Per E. 

TITLE=COVID-Twitter-BERT: A natural language processing model to analyse COVID-19 content on Twitter

JOURNAL=Frontiers in Artificial Intelligence

VOLUME=Volume 6 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2023.1023281

DOI=10.3389/frai.2023.1023281

ISSN=2624-8212

ABSTRACT=In this work, we release COVID-Twitter-BERT (CT-BERT), a transformer-based model, pretrained
on a large corpus of Twitter messages on the topic of COVID-19. Our model shows a 10–30%
marginal improvement compared to its base model, BERT-LARGE, on five different classification
datasets. The largest improvements are on the target domain. Pretrained transformer models, such as CT-BERT, are trained on a specific target domain and can be used for a wide variety of natural language processing tasks, including classification, question-answering and chatbots. CT-BERT is optimised to be used on COVID-19 content, in particular from social media