Edited by: Raj Ratwani, MedStar Health Research Institute (MHRI), United States
Reviewed by: Jin Han, University of New South Wales, Australia; Ethan Larsen, Children's Hospital of Philadelphia, United States; Mihriye Mete, MedStar Health Research Institute, United States
This article was submitted to Human Factors and Digital Health, a section of the journal Frontiers in Digital Health
†ORCID: Lasse B. Sander
Eva De Jaegere
Rebekka Büscher
Eva-Maria Messner
Harald Baumeister
Yannik Terhorst
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
People with suicidal ideation and non-suicidal self-injury (NSSI) behavior face numerous barriers to help-seeking, which worsened during the COVID-19 pandemic. Mobile health applications (MHA) are discussed as one solution to improve healthcare. However, the commercial app markets are growing unregulated and rapidly, leading to an inscrutable market. This study evaluates the quality, features, functions, and prevention strategies of MHA for people with suicidal ideation and NSSI. An automatic search engine identified MHA for suicidal behavior and NSSI in the European commercial app stores. MHA quality and general characteristics were assessed using the Mobile Application Rating Scale (MARS). MHA of high quality (top 25%) were examined in detail and checked for consistency with established suicide prevention strategies. Of 10,274 identified apps, 179 MHA met the predefined inclusion criteria. Average MHA quality was moderate (M = 3.56, SD = 0.40
Suicidal behavior is a major global health challenge with more than 800,000 persons dying by suicide every year, an even higher amount of suicide attempts, and a major burden caused by suicide bereavement (
Because smartphones have become omnipresent, mobile health applications (MHA) have the potential to increase the access to evidence-based support by overcoming some barriers of traditional mental health treatment for people with suicidal ideation, such as stigmatization, perception that professional treatment is not needed, or lack of time in an acute suicidal crisis (
To enfold this potential for suicide prevention, MHA should include methods to reduce or eliminate risk factors for suicidal behavior. Despite interventions directly targeting to reduce suicidal behavior or facilitating access to crisis support, non-suicidal self-injury (NSSI) has repeatedly been discussed as a major risk factor for suicidal behavior (
MHA for NSSI and the prevention of suicidal behavior are, however, particularly prone to risks for users, due to unnoticed harmful information, frustration due to lack of functionality, or recommendations that are not guideline-directed (
In a previous systematic assessment of smartphone tools for suicide prevention, Larsen et al. (
Given the rapid developments in the app market, this study presents an up-to-date overview of the quality, functions, and features of MHA for prevention of suicidal behavior and NSSI.
The aim of this study is to provide an overview of general characteristics, a standardized quality rating, and content analysis of MHA for the prevention of suicidal behavior and NSSI in European commercial app stores (iOS Store and Google Play Store). The following questions were addressed:
What is the quality of MHA or the prevention of suicidal behavior and NSSI in European commercial app stores regarding user involvement, functionality, aesthetics, and quality of information?
Which features, functions, and suicide/NSSI prevention strategies do the best rated MHA include?
Do the used features, functions, and suicide/NSSI prevention strategies mirror the “best practice” for the treatment of NSSI/suicidal behavior?
Suicidal behavior and NSSI-related search terms were identified according to the respective relevant literature (for the complete search term, see
All apps identified by the search engine were documented (automated extraction of e.g., title, version, download link, and app description), and duplicates were removed automatically. The eligibility of the identified apps was assessed in a two-step process: first, the description and pictures of the identified MHA were screened, whether the MHA was (a) designed for the treatment of suicidal/NSSI behavior and/or suicide/NSSI prevention and (b) available in one of the aforementioned languages. In a second step, the remaining MHA were considered eligible for the assessment if they additionally met the following criteria: (c) the MHA functions well enough to allow downloading and assessment and (d) the access of the content of the MHA requires no further gadgets or information (e.g., zip code or institutional login information).
Two independent reviewers (psychologists supervised by a licensed psychotherapist who reconfirmed all final ratings) evaluated each MHA and extracted all data using the German or English version of the Mobile Application Rating Scale (MARS) (
In advance to the rating process, all reviewers underwent a standardized online reviewer-training provided by the developers. To test for accuracy of the ratings, the Intraclass correlation (ICC) between reviewers of the same MHA was calculated (
For the present work, the classification section of the MARS was modified to cover the following dimensions: (a) app name and URL, (b) platform, (c) affiliation (d) obligatory payment, (e) involvement in therapy, (f) security and privacy, (g) user rating, (h) technical features, (i) certification, and (j) emergency contact.
The quality rating of the MARS consists of 19 items, with a scale ranging from one (inappropriate) to five (excellent) (
The star ratings (one to five stars) from users were extracted from the app stores. Bivariate correlations between the user ratings and the MARS overall rating score were investigated. Only user ratings with a minimum of three user ratings were included in the analyses.
To identify potential evidence on the feasibility or effectiveness of included apps, we conducted systematic literature searches in PubMed and PsycINFO. The searches were performed in March 2021. The search string comprised the names of all suicide/NSSI MHA identified in the app store searches and additional terms on suicidal behavior and self-harm, combined with terms on mobile apps (see
To describe and evaluate specific MHA features, functions, and suicide/NSSI prevention strategies of high-quality MHA, MHA in the top quartile of rating scores of all included MHA were examined in greater detail and compared with established suicide prevention strategies.
Therefore, the following information was captured: target group (persons affected by suicide/NSSI), their affiliated environment, healthcare professionals (like psychotherapists), certification (awards or certificates), and the provision by a credible source [competitive government or research funding, governmental/university agencies, non-governmental organizations (NGOs)/institutions, or specialized commercial companies/funding agencies].
Suicide prevention strategies were reviewed and categorized according to a modified classification scheme presented by Larsen et al. (
Mobile health applications aimed at preventing suicidal behavior or NSSI were sought, screened, and selected through two distinct, but analogous processes (
Flowchart of the app inclusion process.
Detailed general characteristics of the included MHA are shown in
General characteristics of the included MHA for suicide/NSSI prevention.
iOS |
92 |
51.4 |
Commercial companies |
60 |
33.5 |
Basic version |
2 |
1.1 |
Stand-alone |
149 |
83.2 |
Password |
12 |
6.7 |
Apple app |
3 |
1.7 |
Emergency contact |
157 |
87.7 |
While the basic version of most MHA (
The majority of MHA (
Password protection was given in 12 (6.7%) MHA, and five (2.8%) MHA required a login. Most MHA provided contact information or legal notice (
A total of 157 (87.7%) MHA offered emergency functions, e.g., helpline numbers or contact information for psychological or medical assistance to be used quickly in case of an acute (suicidal) crisis. In 34 (19.0%) MHA, interaction with others (e.g.,
The overall quality rating score of MHA for the prevention of suicidal behavior and NSSI was moderate with M = 3.56 (SD = 0.39), ranging from a minimum of 1.88 to a maximum of 4.59. Regarding the four objective subscales, functionality was rated highest (M = 4.14; SD = 0.34; range 2.50–5.0), followed by information quality (M = 3.47; SD = 4.10; range 1.50–4.35), esthetics (M = 3.42; SD = 0.52; range 1.0–5.0), and engagement (M = 3.19; SD = 0.64; range 1.0–4.9). The additional subscales showed lower rating scores: therapeutic gain (M = 2.74; SD = 0.32; range 2.0–4.0), subjective quality (M = 2.72; SD = 0.50; range 1.38–4.0), and perceived impact (M = 2.99; SD = 0.45; range 0.67–4.42).
Fifty-one of the included apps had a user star rating from a minimum of three users. No significant bivariate correlations between the overall total score and the user star ratings were found (
The literature searches revealed published articles on two of the included MHA (“BackUp” and “MYPLAN—your safety plan”). We found no randomized controlled trial on the effectiveness of any included MHA. For MYPLAN, we identified the study protocol of an ongoing randomized controlled trial (
About 46 (25.14%) MHA with the highest MARS overall rating score (hereinafter referred to as “high-quality MHA”) were examined in more detail for their specific characteristics, functions, and suicide/NSSI prevention strategies. The overall MARS rating score of high-quality MHA was above average with M = 3.91 (SD = 0.24; range 4.53–3.66
Mean rating scores, features and functions of high-quality MHA for suicide/NSSI prevention.
BackUp | Suicidal behavior | iOS | 4.53 | AP |
PS, CI | WAT Label | ✓ |
Calm harm—manages self-harm | NSSI | iOS/Android | 4.43 | AP | PW, PS, CI | Award winning | ✓ |
Better stop suicide | Suicidal behavior | Android | 4.4 | AP | PS, CI | Award winning | ✓ |
AuxiliaApp | Suicidal behavior | iOS/Android | 4.32 | AP |
PW, PS, CI | Web/Aplicación de Psiquiatría-Psicología Acreditada | ✓ |
Krisen Kompass | Suicidal behaviour | iOS/Android | 4.31 | AP |
PS, CI | ✓ | |
Friend2Friend | Suicidal behavior | iOS | 4.26 | AE | PS, | ✓ | |
distrAct |
Suicidal behavior/ |
iOS/Android | 4.24 | AP | PS, CI | Certified member of the information standard | ✓ |
mhGAP-IG 2.0 App (e-mhGAP) | Suicidal behavior | iOS | 4.2 | P | PS, CI | ✓ | |
Be Safe | Suicidal behavior | iOS | 4.19 | AP | PS, CI | ✓ | |
DMHS: Interactive Suicide Prevention | Suicidal behavior | iOS | 4.16 | AP | CI | ✓ | |
Vrag Maar | Suicidal behavior | iOS/Android | 4.14 | AP |
PS, CI | ✓ | |
Stay Alive | Suicidal behavior | iOS/Android | 4.11 | AP |
PS, CI | Award winning | ✓ |
Operation Life | Suicidal behavior | iOS/Android | 4.06 | AP | PS, CI | ✓ | |
Suicide Prevention App | Suicidal behavior | iOS/Android | 4.06 | AP |
PS, CI | ✓ | |
Jewish Care | Suicidal behavior | Android | 3.92 | AP | PS, CI | ✓ | |
Prevent Suicide | Suicidal behavior | iOS | 3.89 | AP |
PS, CI | Award winning | ✓ |
Kokua Life | Suicidal behavior | iOS | 3.87 | AP |
CI | ✓ | |
MoodTools—Depression Aid | Suicidal behavior | iOS/Android | 3.87 | AP | PS, CI | ✓ | |
ReMinder Suicide Safety Plan | Suicidal behavior | iOS/Android | 3.87 | AP | PS, CI | ✓ | |
TechSafe—Mental Health | NSSI | iOS/Android | 3.86 | AP |
PS, CI | ✓ |
About 37 (80.4%) high-quality MHA were found with search terms for suicidal behavior and nine with search terms for NSSI. Nonetheless, some high-quality MHA also featured contents for the respective other condition to a notable extent (
Three target groups were addressed by the high-quality MHA, i.e., persons affected by suicidal behavior or NSSI (
The major part of high-quality MHA contained both privacy statement (
A quality labeling (e.g., “certified member of the information standard,” an NHS England quality standard that ensures that the information provided is of high quality and based on the best practice) was found for four (8.7%) MHA. In addition, MHA (
Regarding their primary purpose high-quality MHA could be divided into three main categories, i.e, providing information about suicidal behavior/NSSI or general information about mental health (
All of the high-quality MHA included at least three suicide/NSSI prevention strategies. No high-quality MHA included all prevention strategies, but four high-quality MHA were only missing two strategies, whereas 16 (34.8%) high-quality MHA featured self-screening tools to detect suicidal/NSSI risk, and only two (4.3%) included (professional) physician screening tools. The majority high-quality MHA encouraged or facilitated access to peer or family support functions (
The high-quality MHA represented a number of mental health strategies focused on preventing suicide/NSSI. All of the high-quality MHA delivered/provided some kind of psychotherapy, some more extensively than others. The psychotherapeutic contents that were provided were: tips/advice (
An individual safety plan could be created in 25 (54.3%) high-quality MHA, and 12 (26.1%) high-quality MHA gave information on restricting access to lethal means (e.g., firearms and drugs).
Out of all high-quality MHA, the majority (
All of the high-quality MHA, except one (2.2%), either allowed users to enter their own personalized coping strategies and/or provided a selection of predefined coping strategies for users to choose from. The coping strategies used were: resource orientation (
This is the first study systematically examining publicly available MHA for the prevention of suicidal behavior and NSSI in European commercial app stores (Apple App Store and Google Play). The search engine revealed a large number of available apps (
A further concern relates to the absence of an evidence base of the included MHA. For only two MHA, there are descriptive or qualitative reports on the development process and preliminary feasibility (“MYPLAN” and “BackUp”) (
Yet, 183 functioning and relevant MHA with an average moderate MARS quality (M = 3.56) were identified. The MARS subscale rating revealed that the assessed MHA typically functions well, but shows deficiencies in information quality, esthetics, and engagement. The primary focus of 40 high-quality MHA was to provide helpful resources for persons affected with suicide/NSSI ideation or who are in an acute crisis and to provide contact information to crisis support (phone numbers, addresses, or links), medical facilities, or groups or personal contacts (e.g., family and social environment). In addition, a notable number of MHA addressed specific target groups, e.g., veterans or college students, who are known to be at increased risk for suicide (
The suicide prevention tools used in high-quality MHA (
Although the majority of high-quality MHA included valuable crisis- or non-crisis support functions, these resources were often only limited to a specific geographical area. This information should be provided more consistently in the app store description. Furthermore, Martinengo et al. (
Only about half of the high-quality MHA focused on creating a safety plan as a treatment strategy. Those safety plans further differ in their features and comprehensiveness and vary in the degree of adherence to the guidelines proposed by Stanley and Brown (
The widespread presence of privacy features in high-quality MHA is positively surprising, given the results of prior MHA evaluations (
Furthermore, it is encouraging that the majority of high-quality MHA (
This study has some limitations. First, it is possible that not all relevant MHA were found. Google Play and Apple App Store limit the number of results per search term, and this limitation applies also to the search engine provided on the MHAD platform. Furthermore, some MHA are limited to certain geographic areas (e.g., by the publisher), since this search was limited to the European stores of some relevant MHA, which are only available in non-European regions, were not identifiable. Yet, a large number of the identified MHA were developed for the non-European market (e.g., USA and Canada), but also available in the European stores. This suggests that our search may be representative for the global app market. Moreover, the app market is expanding quickly, and a new search and rating process could lead to different search and rating results (
Second, consistent with other studies (
Our investigation showed that there are numerous MHA for suicide/NSSI prevention available in the European commercial app stores, some of high quality, incorporating evidence-based suicide prevention measures and thus potentially able to support affected persons, their affiliated environment, or healthcare professionals. Given the limited interpersonal contact opportunities due to the measures to contain the COVID-19 pandemic, high-quality MHA for the prevention of suicidal behavior and NSSI can be a necessary and valuable source of assistance for affected persons. However, because of an absent evidence base on effectiveness, the benefits and harms of the included MHA cannot be evaluated. Furthermore, users may have great difficulty finding an appropriate MHA with suitable content, due to a plethora of irrelevant apps including apps with potential harmful content. These results suggest that the current free availability of MHA for mental disorders may need to be more regulated to protect users. The results of this review will be made publicly available on the MHAD (
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
LS, YT, and E-MM initiated this study. LS, YT, E-MM, M-LL, RV, and HB contributed to the study design. LS, M-LL, and RV drafted the manuscript and ran the analysis. M-LL, RV, and EDJ rated MHA. LS supervised the ratings. RB performed the literature review. All authors revised the manuscript and approved the final version.
LS, E-MM, HB, and YT developed and run the German Mobile Health App Database (MHAD) project. The MHAD is a self-funded project at Ulm University without commercial interests. HB, LS, and E-MM received payments for talks and workshops in the context of e-mental-health. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
We would like to the thank Rüdiger Pryss, Michael Stach, Robin Kraft, Pascal Damasch, and Philipp Dorzenbach for their support in the development of the search engine and their support in the Mobile Health App Database project.
The Supplementary Material for this article can be found online at: