Physical Activity App Store Apps: How Can We Evaluate their Effectiveness?
-
1
University of Glasgow, Institute of Health and Wellbeing, United Kingdom
-
2
University of Glasgow, School of Computing Science, United Kingdom
Rationale: Thousands of smartphone physical activity applications (apps) are readily available through app stores. However, the rapidly evolving nature of apps presents challenges for evaluating their effectiveness. Traditional methodologies, such as randomised control trials, are slow and often impractical. Alternative methodologies that can accommodate rapid evaluation of smartphone apps are required. App store platforms can be used to automate evaluation procedures, and potentially reach many users. Single Case Designs (SCDs) may be a compatible approach to evaluate the effectiveness of app store apps. However, SCDs involve rigorous methodological criteria, and the feasibility of implementing these in an app store environment has not yet been tested.
Aim: To explore whether SCD criteria can be implemented in an app store release of a smartphone app-based intervention (Quped), designed to promote physical activity through self-monitoring, goal setting and social comparison.
Methods: A mixed methods approach, including review and synthesis of SCD methodological criteria from established checklists to inform the design of the Quped app. Following deployment on the Apple App Store in February 2016, six months of smartphone-logged data was collected from users (aged≥18 years downloading Quped on an iPhone 5S or above), and 13 semi-structured interviews conducted.
Analysis: Descriptive analyses (frequencies and visualisations) of logged data and framework analysis of interview transcripts.
Results: Five methodological criteria were identified as essential to the application of SCDs in evaluating the effectiveness of app store apps: (i) repeated measurement of the outcome over time; (ii) at least three outcome measurements in both baseline and intervention phases; (iii) the intervention introduced at different time points to provide overlap between baseline and intervention phases across users; (iv) replication across users with different characteristics; and (v) users’ perspectives on social acceptability (e.g. data privacy). These criteria were implemented during the deployment of Quped as follows: step counts (the outcome) were retrieved daily from internal smartphone sensors; retrieving steps retrospectively from the week prior to download provided baseline data; intervention timing was manipulated by delaying the introduction of goal-setting. 145 users downloaded Quped and consented to the study, and 80 (55.2%) were eligible for inclusion. 55/80 (68.8%) provided repeated measurement of steps during both baseline and intervention phases, but ≥3 data points per phase were only available for 43/80 (53.8%). Visualizations of download patterns revealed overlap between different users’ baseline and intervention phases. 62 (77.5%) users provided information on their age and gender through the app’s social comparison feature. Interviews suggested that participants were happy to provide their personal step count data to the research team, but were surprised that it could be accessed retrospectively.
Conclusion: The largely successful implementation of SCD methodological criteria in the design and deployment of Quped demonstrates that SCDs are a feasible approach for evaluating the effectiveness of app store PA apps. However, length of use and missing data may limit sample size considerably, and more work is needed to ensure users are fully informed of how (and when) their personal data will be collected.
Keywords:
physical activity,
app store evaluations,
Single case design,
smartphone app,
Research Methods
Conference:
3rd UCL Centre for Behaviour Change Digital Health Conference 2017: Harnessing digital technology for behaviour change, London, United Kingdom, 22 Feb - 23 Feb, 2017.
Presentation Type:
Research abstract
Topic:
Digital Health
Citation:
McCallum
C,
Asadzadeh
P,
Rooksby
J,
Chalmers
M and
Gray
C
(2017). Physical Activity App Store Apps: How Can We Evaluate their Effectiveness?.
Front. Public Health.
Conference Abstract:
3rd UCL Centre for Behaviour Change Digital Health Conference 2017: Harnessing digital technology for behaviour change.
doi: 10.3389/conf.FPUBH.2017.03.00059
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
22 Feb 2017;
Published Online:
22 Feb 2017.
*
Correspondence:
Ms. Claire McCallum, University of Glasgow, Institute of Health and Wellbeing, Glasgow, United Kingdom, c.mccallum.2@research.gla.ac.uk