Edited by: Brian W. Jack, Boston University, United States
Reviewed by: Jayakanth Srinivasan, Boston University, United States; Jacqueline Anne Boyle, Monash University, Australia
This article was submitted to Digital Public Health, a section of the journal Frontiers in Public Health
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Mobile health (mHealth) interventions are ubiquitous and effective treatment options for obesity. There is a widespread assumption that the mHealth interventions will be equally effective in other locations. In an initial test of this assumption, this retrospective study assesses weight loss and engagement with an mHealth behavior change weight loss intervention developed in the United States (US) in four English-speaking regions: the US, Australia and New Zealand (AU/NZ), Canada (CA), and the United Kingdom and Ireland (UK/IE). Data for 18,459 participants were extracted from the database of Noom's Healthy Weight Program. Self-reported weight was collected every week until program end (week 16). Engagement was measured using user-logged and automatically recorded actions. Linear mixed models were used to evaluate change in weight over time, and ANOVAs evaluated differences in engagement. In all regions, 27.2–33.2% of participants achieved at least 5% weight loss by week 16, with an average of 3–3.7% weight loss. Linear mixed models revealed similar weight outcomes in each region compared to the US, with a few differences. Engagement, however, significantly differed across regions (
While initially concentrated in the United States, obesity has now also become a pervasive health concern for countries around the world (
One of the most effective treatments for obesity is lifestyle modification (
This understanding, however, inherently assumes that mHealth interventions deliver equally effective care to anyone anywhere in the world. Almost all meta-analyses, systematic reviews, and commentaries about mHealth mention that a significant benefit of mHealth is its scalability, or generalizability to other populations and even other countries. Many claims are made, such as that “with little or no loss in effectiveness,” mHealth interventions developed in one place “can transcend time, culture, and language: they can be used simultaneously anywhere in the world” as is (
It is crucial to test this widespread assumption that mHealth interventions have similar outcomes in other countries, given the enormous costs of obesity for countries worldwide and the importance of providing the most effective treatments possible. Cross-country comparisons of mHealth intervention outcomes are lacking and focused on obesity prevalence rather than intervention outcomes. In addition, meta-analyses and systematic reviews make broad claims based on heavily US-centric data. For example, in one systematic review, 15 studies were conducted in the US, 2 in the UK, and one each in Australia and China (
In particular, there is little work comparing whether an intervention primarily developed in one country (i.e., the US) can result in comparable outcomes in other similar countries around the world. Given how culturally adapted interventions can be effective (
The study also sought to advance understanding of cross-country mHealth differences by examining engagement. Engagement is a critical determinant of intervention success. Several studies have found that increased engagement, such as logging meals or steps, contributes to increased weight loss (
In this study, we compare outcomes over the course of the Noom intervention in four English-speaking regions, spanning a total of six individual countries. We explore how weight compares over time in each region in comparison to the US. We also examine whether differences exist across countries in engagement levels and consider how engagement is associated with weight over time. Based on previous research showing similarities in weight loss across regions (
The US was selected as a positive control comparator to test whether the intervention is as effective in other countries. Then, to allow for a non-biased comparison (e.g., there were no translation or food content issues preventing the full intervention from being received as intended), the following countries were chosen: Canada, Australia, New Zealand, the United Kingdom (comprising Great Britain and Northern Ireland), and Ireland. These countries are similar to the US in the baseline prevalence of overweight and obese individuals (all between 60.2 and 70.1%), the main spoken language (English), and the prominence of Western foods (
Similarly to past work, the following countries were combined in analyses based on geographic and cultural similarities: United Kingdom and Ireland (UK/IE), as well as Australia and New Zealand (AU/NZ). It is important to note that though these countries are geographically close, they can differ on cultural factors, climate, and ethnic makeup, among others. The regions included in this study were: the US, UK/IE, Canada, and AU/NZ.
Noom is an mHealth behavior change intervention that utilizes in-app tracking features such as food and weight logging, virtual 1:1 health coaches, behavior change techniques, and education on diet, physical activity, and psychology to enable multi-component healthy lifestyle modification. The intervention draws from CBT, third wave CBT such as DBT, and motivational interviewing techniques. These techniques are particularly used by coaches and the curriculum, which covers general psychological and behavior change principles as well as those related to diet, physical activity, and weight management. Coaches trained in CBT and motivational interviewing techniques help users to set effective goals, identify barriers, and oversee users' progress (
Participants in all regions had access to all of these intervention components. To ensure that participants could use logging features, imperial measurements were replaced with metric measurements. Aside from this, the intervention was not culturally adapted to each country or region in order to test whether the intervention as is could effectively be applied to other countries.
The Advarra IRB approved this study. Participants for the study were selected from a pool of individuals who had signed up for Noom's Healthy Weight (HW) program based on their own motivation to lose weight. All of these participants provided informed consent that their de-identified data could be used in a longitudinal study and were given the option to opt out during the initial sign-up process for Noom. Participants were included in the study if they met the following criteria: they were between 18 and 60 years of age, began the HW program in May or June of 2019, and had at least 1 in-app action and 1 weigh-in every week over the 16 weeks of the program as a minimum threshold of activity. Data for 32,983 potentially eligible participants was pulled from Noom's database (Noom, Inc., New York, NY) and de-identified (US: 30576, CA: 822, AU/NZ: 375, GB/IE: 1,210). Participants were considered ineligible if their initial weight classified their body mass index (BMI) as underweight (<18 kg/m2) or healthy (18.5–24.9 kg/m2); if they were using the free version of the app, meaning they did not have access to the full intervention; and/or if they did not input baseline characteristics (gender and height). In addition, as in previous work, outliers were excluded, defined as an individual whose magnitude of BMI change was >3.5 within 1 month (
The primary outcome was self-reported weight, observed at baseline and week 16. Participants are encouraged to log their weight weekly. We also measured self-reported age, gender, and baseline BMI as predictors of weight. The country of the app store used to purchase the Noom program was used as a participant's resident country. Due to inclusion criteria for modeling purposes, there were no participants with missing data for these variables.
Engagement was measured using the total number of meals and exercises logged, messages to a coach, steps recorded, articles read, and days with at least one weight measurement each week. This includes all the fundamental components of the intervention, such as the curriculum, coaching, and self-monitoring. Each engagement variable was measured over 16 weeks. Additionally, to tap into overall engagement, a composite engagement score was calculated following previous work (
Participants' baseline characteristics were calculated using descriptive statistics, with means and standard deviations for continuous variables and frequencies and percentages for categorical variables (
Descriptive statistics for baseline characteristics by region.
Male | 2,689 (14.5%) | 2,557 (14.8%) | 35 (8.1%) | 15 (7.9%) | 82 (13.7%) | <0.001 |
Female | 16,770 (85.5%) | 14,683 (85.2%) | 396 (91.9%) | 176 (92.1%) | 515 (86.3%) | |
Age (years), mean (SD) | 44.93 (9.84) | 44.99 (9.89) | 44.57 (9.16) | 43.2 (9.68) | 44.05 (8.8) | 0.008 |
Initial weight (kg), mean (SD) | 101.51 (16.96) | 101.66 (17.06) | 100.06 (14.9) | 98.49 (13.48) | 99.16 (5.86) | <0.001 |
Height (inches), mean (SD) | 66.02 (3.41) | 66.04 (3.4) | 65.35 (3.47) | 65.76 (3.17) | 66.17 (3.53) | <0.001 |
Baseline BMI (kg/m2), mean (SD) | 30.24 (4.56) | 30.27 (4.59) | 30.14 (4.18) | 29.46 (3.62) | 29.47 (4.13) | <0.001 |
To examine weight, our primary outcome and dependent variable, as a function of time, linear mixed effects models were conducted. Compared to multiple regression or ANOVA, linear mixed effects models produce more accurate parameter estimates for repeated measurements and can accommodate multiple layers of non-independence (
Descriptive baseline characteristics, along with significance tests across regions, are displayed in
Descriptive statistics for weight loss over the 16 weeks are displayed in
Mean weight (kg) and weight change (kg) by region.
18,459 | 17,240 (93.4%) | 431 (2.3%) | 191 (1.0%) | 597 (3.2%) | |
Baseline | 101.51 (16.96) | 101.66 (17.06) | 100.06 (14.9) | 98.49 (13.48) | 99.16 (5.86) |
Week 8 | 98.96 (16.73) | 99.11 (16.82) | 97.52 (14.63) | 96.18 (13.5) | 96.52 (16.09) |
Δ (Week 8 – Baseline) | −2.5 (2.73) | −2.49 (2.73) | −2.49 (2.80) | −2.3 (2.73) | −2.71 (2.87) |
Week16 | 98.09 (17.02) | 98.23 (17.11) | 96.91 (15.19) | 95.74 (13.92) | 95.65 (16.24) |
Δ (Week 16 – Baseline) | −3.56 (4.35) | −3.57 (4.36) | −3.26 (4.3) | −3 (4.12) | −3.69 (4.35) |
Participants > 5% weight loss (%) | 5,681 (30.8%) | 5,482 (31.80%) | 131 (30.4%) | 52 (27.2%) | 198 (33.2%) |
Participants > 10% weight loss (%) | 1,969 (10.7%) | 1,831 (10.6%) | 47 (10.9%) | 20 (10.5%) | 71 (11.9%) |
In the linear mixed models, we observed significant main effects predicting weight loss as well as significant interaction effects that provide more explanation for the main effects (
Summary of linear mixed model results.
Time (weeks) | −0.18 | −0.19, −0.18 | 0.002 | −76.64 | <0.001 |
Age (years) | −0.02 | −0.03, −0.01 | 0.003 | −5.85 | <0.001 |
Baseline BMI (kg/m2) | 3.45 | 3.43, 3.46 | 0.007 | 483.29 | <0.001 |
Engagement | 0.03 | 0.02, 0.04 | 0.004 | 7.22 | <0.001 |
Female | – | – | – | – | – |
Male | 8.82 | 8.64, 9.00 | 0.09 | 95.65 | <0.001 |
US | – | – | – | – | – |
CA | −0.71 | −1.15, −0.27 | 0.226 | −3.15 | 0.002 |
AU/NZ | −0.09 | −0.74, 0.57 | 0.335 | −0.26 | 0.795 |
UK/IE | 0.16 | −0.22, 0.53 | 0.192 | 0.82 | 0.41 |
Time * US | – | – | – | – | – |
Time * CA | 0.03 | −0.00, 0.06 | 0.015 | 1.8 | 0.072 |
Time * AU/NZ | 0.07 | 0.03, 0.12 | 0.023 | 3.18 | 0.001 |
Time * UK/IE | 0.01 | −0.02, 0.03 | 0.013 | 0.62 | 0.533 |
Time * Engagement | −0.02 | −0.02, −0.01 | 0.0004 | −33.07 | <0.001 |
Engagement * US | – | – | – | – | – |
Engagement * CA | 0.02 | −0.03, 0.07 | 0.026 | 0.7 | 0.483 |
Engagement * AU/NZ | 0.08 | −0.01, 0.15 | 0.041 | 1.79 | 0.074 |
Engagement * UK/IE | 0.04 | −0.01, 0.08 | 0.022 | 1.65 | 0.099 |
Time * Engagement * US | – | – | – | – | – |
Time * Engagement * CA | −0.002 | −0.01, 0.00 | 0.003 | −0.75 | 0.452 |
Time * Engagement * AU/NZ | 0.011 | −0.02, −0.00 | 0.004 | −2.61 | 0.009 |
Time * Engagement * UK/IE | 0.006 | −0.01, −0.00 | 0.002 | −2.39 | 0.017 |
Examining interactions with time allowed further exploration of weight loss over the course of the program, instead of averaged across time in main effects. Two-way interactions of time * region revealed that over time, Canadian, and UK/IE participants did not significantly differ in weight loss compared to US participants. The only significant region * time interaction was a AU/NZ * time interaction (
Notably, participants engaged with the intervention differently across regions. One-way ANOVAs revealed that were significant differences in the average amount of engagement on five of six factors: average number of articles read, steps recorded, days with weigh ins, number of messages to the coach, and number of times they exercised each week (all
Average weekly engagement across regions.
Articles read, mean (SD) | 14.59 (13.45) | 14.58 (13.44) | 15.23 (13.55) | 14.59 (13.57) | 14.53 (13.58) | 0.001 |
Meals logged, mean (SD) | 15.15 (6.35) | 15.15 (6.35) | 15.06 (6.36) | 14.84 (6.4) | 15.23 (6.37) | 0.069 |
Steps, mean (SD) | 30,193.83 (26,696.75) | 30,058.37 (26,716.58) | 31,792.84 (26,340.36) | 30,808.69 (26,523.88) | 32,770.26 (26,262.71) | <0.001 |
Days with weigh ins, mean (SD) | 6.53 (1.26) | 6.54 (1.25) | 6.51 (1.31) | 6.38 (1.46) | 6.42 (1.41) | <0.001 |
Coach message, mean (SD) | 1.55 (2.14) | 1.55 (2.12) | 1.84 (2.58) | 1.32 (1.86) | 1.5 (2.23) | <0.001 |
Exercise, mean (SD) | 1.88 (3.11) | 1.9 (3.12) | 1.69 (2.97) | 1.58 (2.74) | 1.65 (2.99) | <0.001 |
In the linear mixed models, a significant engagement effect (
Summary of cross-region engagement and weight loss differences.
Canada | Canada > US | None | None |
AU/NZ | None | US > AU/NZ | US > AU/NZ |
UK/IE | None | None | US > UK/IE |
To our knowledge, this is the first study to evaluate whether the same mobile intervention for weight loss is as effective across countries. Overall, as hypothesized, our findings showed that the Noom program generated comparable significant weight loss in the United States, the United Kingdom and Ireland, Canada, and Australia and New Zealand. Weight loss at 16 weeks was between 3% (3.0 kg) and 3.7% (3.69 kg) in all regions, matching the results of a previous study on this intervention that generated 3.5% weight loss by week 16. This is on track to surpass 5% weight loss by 1 year (
While no previous cross-country comparisons of an mHealth weight loss intervention existed prior, the verifiable portions of our results replicated past findings. Matching strong patterns in the literature, we found significant associations between weight and baseline BMI, gender, age, and engagement over time (
Beyond differences in demographics, there were region-level differences in how participants actively engaged with the program. For example, UK/IE participants logged the most steps and Canadian participants had the highest messages to coaches. This could be for several reasons, for example differences in sociability, time zones, and seasonality. To our knowledge, there is no prior work showing differences between the regions included in this study in terms of coach messaging. Our results with UK/IE participants and steps aligns with a study leveraging automatically recorded smartphone activity data. Results showed higher step counts in the UK than in the US (
With regard to engagement cross-culturally, only one other study compared participants from Finland and India and found that while individuals had similar attitudes toward weight loss, they had very different culturally-induced engagement behaviors in a mobile physical activity app. For instance, because Indian participants viewed health as oriented around routines and not goals, they used the app's goal setting functions of the app much less than Finnish participants (
Another reason for cross-region differences in engagement could relate to cultural tailoring. According to previous research, interventions that are specifically adapted for certain populations or locations can be effective (
With the association between engagement and weight loss found in previous research and replicated in our results, it is notable that we did not just find a two-way interaction with engagement and time. Rather, we saw an additional three-way interaction with region. With high engagement, the rate of weight loss was the fastest in the US compared to AU/NZ and the UK/IE. This aligns with rare cross-cultural work, such as a study that used self-reported survey data. Results showed that US participants had greater familiarity with mHealth apps than UK participants (
This study had several strengths. First, in using this retrospective design, the study could explore participants' behaviors with a weight loss intervention that they were using without researcher supervision, contact, or being alerted to participation requirements along study time points. This could increase external validity of real-world conditions and contribute rare data to the literature. In addition, the study analyzed intervention effects in ways that captured the complexity of the relationships of weight loss over time, engagement, and region. Rather than merely averaging across time points or conducting one multiple comparison test in the change in weight loss at week 16, we found a more sophisticated picture of results when analyzing weight loss over time and investigating interactions with engagement. Finally, the study's extensive sample size provides greater confidence in generalizability than typical intervention studies.
The study has a few limitations. First, because the intervention was developed in the US, the sample was heavily skewed toward US participants. Seventeen thousand two hundred and forty out of 18,459 participants (93.4%) were from the US; in comparison, 431 (2.3%) were from Canada, 375 (1.0%) from Australia/NZ, and 597 (3.2%) from UK/IE. While significant interaction effects were still found with these sample sizes and they are larger than typical intervention samples, our analyses may not have captured the full range of variation in these regions. The US skew in our sample also limits generalizability of our findings. Future research should extend our findings to samples that are more equally distributed. Another limitation of the study is that no active control group was used, which means causal interpretations of the effect of Noom cannot be made, particularly in relation to the superiority of Noom compared to usual care. Future studies should use active control groups to assess the difference between the intervention and control groups across countries. Additionally, only participants in these regions who were interested in a mobile health intervention for weight loss signed up for the program and were included in the study. This same self-selection bias would apply to all four regions and the population of individuals who participate in interventions, but future studies should determine if this bias manifests differently in each of the regions. The study also used the user population of one program. While the number of participants was larger than many typical studies, the population was nevertheless naturally restricted to those who self-selected into this particular program. Future research should examine whether the results generalize to other interventions in other regions beyond these four. In addition, only one time period of 16 weeks was used. Future work should incorporate multiple time periods to explore whether cross-country effectiveness remains consistent over different periods of time and seasons. Further, the sample only included participants who met a minimum criteria of in-program activity for 16 weeks, which limits generalizability to individuals who meet this criteria. Finally, within each individual country, individuals may differ greatly on culture, primary language, country of birth, and ethnicity, which could affect engagement in ways we could not capture in this study; future studies should explore these differences and how they affect engagement.
This study provided a necessary test of a widespread implicit assumption in the literature, with important implications for how digital weight loss interventions are applied, understood, and designed worldwide. Our results provide preliminary evidence that the same mobile intervention can induce comparable weight loss in other regions or countries. In addition, even though weight loss was similar, there were country-level differences in the way people engaged with the intervention. Even with similarities of language and Western cultures, there are differences that need to be understood better in applying mHealth interventions across countries or regions. This reveals important areas of investigation for future obesity research, such as the factors determining cross-country differences in engagement, and the impact of these engagement differences over time in predicting weight.
Data were obtained from Noom, and legal and proprietary restrictions apply to the availability of these data. Requests to access the data should be directed to Siobhan Mitchell,
The study was approved by Advarra Institutional Review Board. The participants provided informed consent to participate in this study.
QY conducted data analysis, data interpretation, and contributed to study conception and study design. EM conceived the study and contributed to study design and editing. AH performed literature searches and wrote and edited the manuscript. LD, HB, and AM contributed to study conception, study design, and editing. All authors contributed to the article and approved the submitted version.
QY, AH, EM, LD, HB, and AM are employees at Noom Inc.