AUTHOR=Liu Zhaolu , Peach Robert L. , Lawrance Emma L. , Noble Ariele , Ungless Mark A. , Barahona Mauricio TITLE=Listening to Mental Health Crisis Needs at Scale: Using Natural Language Processing to Understand and Evaluate a Mental Health Crisis Text Messaging Service JOURNAL=Frontiers in Digital Health VOLUME=Volume 3 - 2021 YEAR=2021 URL=https://www.frontiersin.org/journals/digital-health/articles/10.3389/fdgth.2021.779091 DOI=10.3389/fdgth.2021.779091 ISSN=2673-253X ABSTRACT=Mental health is a growing public health issue that requires a large-scale response that cannot be met with only traditional services. While digital tools are proliferating, most are not evaluated, and we know little about their users and their needs. Shout is a free mental health text messaging service run by the charity Mental Health Innovations that provides support for individuals in the United Kingdom experiencing mental or emotional distress and seeking help. The Shout dataset includes anonymised text message conversations and post-conversation surveys, providing a unique and valuable opportunity to hear at scale from those experiencing distress. This data offers the potential to better understand mental health needs, particularly for people not using traditional mental health services, and to evaluate the impact of a novel form of crisis support. In this paper, we use natural language processing (NLP) to gain insight into the demographic groups that are using the Shout service, their behavioural expressions of distress (behaviours), and to assess Shout Volunteer's adherence to the conversation techniques and formats they are taught (conversation stages). We report accurate classification of texter demographics (96%), behaviours (1-hamming loss = 95%) and conversation stages (88%), exemplifying the capabilities of NLP applied to frontline mental health datasets to offer insight into the experiences of those using digital mental health services and the quality of service provision.