ORIGINAL RESEARCH article
A Mixed Methods Process Evaluation of a Clustered-Randomized Controlled Trial to Determine the Effects of Community-Based Dietary Sodium Reduction in Rural China
- 1Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China
- 2The George Institute for Global Health at Peking University Health Science Center, Beijing, China
- 3Department of Family Medicine and Mixed Methods Program, School of Medicine, University of Michigan, Ann Arbor, MI, United States
- 4School of Public Health, Peking University Health Science Centre, Beijing, China
- 5Peking University Clinical Research Institute, Beijing, China
- 6George Institute for Global Health, University of New South Wales, Newtown, NSW, Australia
- 7Global Health Research Center, Duke Kunshan University, Suzhou, China
Purpose: A clustered-randomized controlled trial was conducted to determine the effects of a sodium reduction program in 120 rural villages in Northern China. This mixed-methods process evaluation was used to investigate the implementation and to evaluate the feasibility of the complex intervention to translate the findings from clinical study to the real world.
Methods: A convergent mixed-methods process evaluation design was used in this study. Quantitative data were collected from activity logs and routine study records. Qualitative data were collected from 53 project stakeholders and 45 villagers from 10 intervention villages. Thematic analysis of qualitative interviews facilitated integration with the descriptive quantitative data analysis based on theory-informed domains of fidelity, delivery, reach, receipt, and contextual factors of intervention from a process evaluation framework.
Results: The intervention was implemented with high fidelity, delivery, reach, and receipt. A total of 5,450 sheets of posters, 31,400 calendars, and 78,000 sheets of stickers were delivered as planned, and 11 promotion activities were conducted in each village. Contextual factors hindering full uptake of the intervention included preference for salty taste, higher cost of low-sodium salt, and low education levels of villagers. Other contextual factors, positive policy support, administrative support, and staff enthusiasm were the facilitators for implementation.
Conclusions: This multifaceted intervention was implemented well and effectively in rural China. This process evaluation has indicated that conducting health education interventions in rural areas requires policy and administrative support, enthusiastic staff, easy-to-understand health education materials and activities, and key persons, but tempered expectations as behavior change requires time. This project demonstrates the feasibility and benefits of using mixed-methods process evaluation in large-scale studies.
Stroke is the leading cause of death in China, responsible for about 1.7 million deaths each year (1, 2). Excess sodium intake is a key determinant of high blood pressure (3), the leading cause of stroke (4). The magnitude of the effect of sodium on blood pressure is such that each 75-mmol difference in daily salt intake translates into an ~5.4 mm Hg difference in systolic blood pressure among individuals with hypertension, and 2.4 mm Hg among individuals without hypertension (3). Chinese people, especially those living in northern rural areas, have the highest sodium intake levels in the world, where hypertension and the incidence of stroke are also all very high (5). In Western populations, most dietary sodium derives from processed and restaurant foods, but in rural China the major source comes from salt and condiments added in home cooking (6).
Given this hypertensive crisis in rural China, a clinical trial was conducted to identify a novel, low-cost, scalable, and sustainable, community-based strategy for the prevention of blood pressure-related diseases in rural China (7). The trial was registered with clinicaltrial.gov in December 2010, registration number NCT01259700.
The main study was published (8). The overall intervention was designed on the basis of the health belief model (HBM). The HBM is a theoretical model that has been constructed from six domains, which are perceived susceptibility, severity, barriers, benefit, cues to action, and self-efficacy. According to the framework of the HBM and proved cost-effective strategy for sodium reduction (9), the complex intervention contained three parts, health education materials with key message about salt reduction (posters, calendars, and stickers were pasted, respectively, on the wall outside, indoors, and in salt containers), health education activities (launch events, activities organized in consideration of local context, activities for individuals at elevated risk of cardiovascular disease, and student-to-parent education activities), and low-sodium salt substitute supply (Table 1). The intervention is further depicted in the tables in the appendix (Supplementary Material in Appendix 1).
After complex intervention being conducted for 18 months, 1,903 people had valid 24-h urine collections. The mean urinary sodium excretion in intervention compared with control villages was reduced by 5.5% (−14 mmol/day, 95% confidence interval −26 to −1; p = 0.03). In the intervention group, potassium excretion was increased by 16% (+7 mmol/ day, +4 to +10; p < 0.001), and the sodium to potassium ratio declined by 15% (−0.9, −1.2 to −0.5; p < 0.001). Between the intervention and control groups, the mean blood pressure differences were −1.1 mm Hg systolic (−3.3 to +1.1; p = 0.33) and −0.7 mm Hg diastolic (−2.2 to +0.8, p = 0.35). The difference in the proportion with hypertension was −1.3% (−5.1 to 2.5, p = 0.56) (8). The absence of effects on blood pressure reflects the moderate changes in sodium and potassium intake achieved (Supplementary Material in Appendix 1).
However, alongside intervention efficacy, some questions are still unclear, these are as follows: how are the components of the complex intervention implemented; what are the barriers or facilitators within its context; and how can implementation be optimized in future practice. Therefore, the mixed-methods approach was used to integrate qualitative and quantitative aspects of this study, to investigate the fidelity, delivery, reach, receipt, and context of intervention. The current study could provide us a comprehensive view of implementation and effectiveness and promoted evidence-based practice for village doctors who provide the basic medical service as a family doctor to prevent and control chronic disease in local village in rural China.
Theoretical Framework of Process Evaluation
“How-to Guide” (10) and REAIM frameworks (11) were used for the process evaluation to examine the level of implementation comprehensively. This involves understanding the implementation of intervention in terms of (1) fidelity, defined as to what extent the intervention was implemented consistently with the underlying theory as planned; (2) delivery, defined as to what extent all of the intended activities, training, and materials were provided to program participants; (3) reach, defined as the absolute number, proportion, and representativeness of individuals who are willing to participate in a given initiative, intervention, or program; (4) receipt, defined as to how participants reacted to specific aspects of the intervention; and (5) context, defined as what contextual factors influence implementation or the intervention outcome.
A convergent mixed-methods design (12) was used in the current study that was guided by the process evaluation theoretical framework. While data were collected prospectively, the analysis of the data occurred at the same time after the intervention. The impetus for the mixed-data evaluation was to provide a comprehensive view of the extent the intervention worked, and to understand why the interventions conducted in this study were effective (Figure 1).
Study Settings and Participants
The main trial was conducted in five Northern Provinces of China, including Hebei, Liaoning, Ningxia, Shanxi, and Shaanxi (7). Two counties from each province were selected for participation; twelve townships of each county were engaged with total 120 to wnships. A typical township in the study comprised ~17 villages with a total population of 25,000. There was one village doctor in each village who provides basic health services there. The average population of included villages was 1,867 with 512 households. The annual per capita income was $884, and 91.5% participants have <9 years of education. The village doctors, who conducted intervention in this study, have played a significant role in providing basic health services to villagers when the area had great shortage of health resources.
The health education activities and process evaluation were launched in May 2011, and health education activities ended on September 2012, while the process evaluation continued throughout the project and afterward until March 2013. The study data sources are represented in Supplementary Material in Appendix 1.
Data about delivery of the materials were collected from 150 activity logs and 91 routine records recorded prospectively by the county project officers and project assistants.
The strategy of purposeful sampling was used to select participants. Using a semi-structured, face-to-face depth interview (interview guides are shown in Appendix 2 in Supplementary Material), the qualitative evaluation was conducted in 10 intervention villages from five provinces (two villages intentionally sampled in each county, one county from each province). Interviews were conducted among 48 key stakeholders including five provincial project investigators, five province project coordinators, five county project officers, five county project governors, five county health educators, eight town health educators, 15 village doctors; and 45 villagers from intervention villages including 19 housewives, 13 general villagers, and 13 villagers with a high risk of cardiovascular disease (Supplementary Material in Appendix 1).
Intramethod Data Analysis
The number and percentage of the education materials distributed and health education events held were calculated for each component on three dimensions including fidelity, delivery, and dose reached to villagers. SPSS 24.0 (IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp)software was used to analyze the quantitative data. An immersion/crystallization approach was used to analyze the qualitative data based on the dimension of receipt and contextual factors (13). The transcribed data were analyzed using Nvivo 11 (QRS, Australia) software of Chinese version. Three researchers (CH, LH, and GW) coded and analyzed the transcripts by making sense of the transcribed data, developing codes, categorizing the data, and abstracting. We discussed every discrepancy regarding the finding interpretation until reaching consensus. The analysts calibrated the coding process after reviewing one independently coded interview. Models were constructed through the analysis and then confirmed by reviewing the interview transcripts. The final agreement was achieved through the review of entire research team.
Mixed Data Analysis
The mixed data were analyzed based on the study process evaluation framework using the process of joint display analysis (14). This iterative process of juxtaposing links quantitative and qualitative data together, examining the implications of each other and looking for new facilitates drawing conclusions in light of both types of findings.
Interviews were completed among 93 stakeholders, and 150 records were collected prospectively. We present the quantitative findings for the three dimensions of process evaluation of fidelity to protocol, delivery and reach, and qualitative findings for two dimensions of receipt and contextual factors. The interview responses have been summarized in the results and specific quotations can be found in the related sections.
Fidelity was determined by comparing the activities outlined in the strategic action plan with delivered activities. This comparison showed that all health education materials and activities were implemented as planned, with high fidelity (Supplementary Material in Appendix 1). A total of 5,450 sheets with five different types of posters, 62,800 sheets of yearly calendars, and 78,000 pieces of special designed stickers were delivered to intervention villages. Seven hundred and twenty health education events were held in total, and 115, 228 packs of salt substitute were delivered to the village stores.
In terms of student-to-parent education activities, the numbers of planned and delivered were 60 and 120, respectively (Table 2). The reason for this was when we completed the first round of student education activities (60 events), we found that the students were willing to participate in activities and the effect of health education was relatively good. Therefore, the second round of activities (60 events) was conducted. From qualitative interviews, some interviewees mentioned that children and primary school kids paid more attention to salt reduction, “I found that children were interested in those pictures, stories on the calendar, as well as the activities.” (Villager).
Table 2. Joint display of fidelity, delivered, reach, receipt, and meta inferences by each component of intervention.
The data was transformed from the interviews. It showed that the proportion of attended health education activities was low (61.5%) in general villagers, while the number was relatively high (92.3%) for people with high risk of cardiovascular diseases (Table 3). From the qualitative interviews, we found that a dearth of young adults participated in activities due to out-migration to cities. “There are many middle-aged and young people moved into the urban cities or temporarily work there, who were unable to participate.” (VHE). In addition, some villagers felt that the content later during implementation became repetitive. “There was no new ideas added with passing of time.” (CHE). These two reasons might explain a low reach of the health education activities in the general villagers.
Receptivity of Intervention
Generally, the salt reduction project was accepted and the local people were satisfied. The health education materials, including posters, calendars, and stickers, were accepted by the local people because they were popular, simple, easy-to-understand, and practical. However, posters placed outdoors were easily lost and less effective than posters placed indoors. Some interviewees suggested writing slogans on walls.
The health education activities created a specific and suitable atmosphere important during program launch and provided the detailed information and cues necessary to reduce salt intake. Some interviewees mentioned that children and primary students paid more attention to salt reduction. However, some villagers felt that the content later during implementation became repetitive and wanted a greater variety of information. Interviewees had many valuable suggestions, such as leading the salt reduction activities by enthusiastic volunteers who are the elders or children, and that activities should avoid busy farming times.
In terms of low-sodium salt substitute supply, most interviewees deemed that low-sodium salt substitute is acceptable because the village doctor told them that it is healthier than regular salt, even though some felt that the taste was less salty than regular salt and that it was more expensive. Supplementary Material in Appendix 3 provides typical interview quotes regarding receipt of the intervention.
Contextual factors influencing implementation included policy support, administrative support, staff enthusiasm, and out-migration of young adults to cities. At present, some national chronic disease policies have been implemented in certain areas and incorporated into public health service projects. From the interviews, we serendipitously found that some areas with policy support for a chronic disease project improved the conduct of this project due to interactive and mutual support of both projects. Administrative support, in general, can improve the quality of project implementation. However, in some cases, there are some staff who conducted project work only because of administrative pressures—this has potentially negative implications for the quality of the project. Health educators paid time and effort on the project varying from 10 to 70%. In most counties, the health educators had more than one job. Thus, in addition to the program work, they already had a very heavy workload. This led to time conflicts and hampered the quality of the program's implementation. Still, some health educators prepared for this project in the evening or on weekends. This included drafting reports and other administrative activities. Some experienced health educators conducted a variety of health education activities. There was a dearth of young adults participating in activities due to out-migration to cities.
Contextual factors influencing the intervention's effect included the following: adding too much salt when cooking and eating pickled foods as long-standing behaviors, change is a long-term process that requires gradual adaptation; and as the education level of villagers is generally low, it takes a long time to change and maintain healthy behaviors. The Supplementary Material in Appendix 4 provides quotes of contextual factors from the study interviews.
To integrate the quantitative and qualitative results, we created Table 2 based on the process evaluation framework and the structure of a joint display.
The clustered-randomized controlled trial was a rigorously conducted trial. The process evaluation found that the fidelity, delivery, reach, and receipt of the intervention were high. Posters were recommended for the value of creating a positive atmosphere; calendars were practical as they could be posted on the wall of the house for a long time and provide reminder information. Stickers for placing on salt containers were recommended as they prompted caution when using salt during cooking. This is especially important, since sodium intake primarily comes from added salt during cooking in the study population (6).
A program launch event is strongly recommended as it captured the attention of most villagers and implementers and created an atmosphere conducive to salt reduction among the whole village. Activities organized in consideration of the local context are also strongly recommended as they maximized the involvement of villagers in the sodium reduction intervention by holding a variety of activities based on local culture. Activities for high-risk individuals were only moderately recommended as they are very time consuming and covered only fewer topics about health, while the targeted people felt less benefit. The principle of “simple and repetitive” in health education in this case was not found suitable for activities (15). Villagers need more health-related knowledge, not simply a repetition of the same information. Interestingly, the student-to-parent education activities are strongly recommended because students played an influential role in families and they readily adapted the healthy behavior. From the interview, we also find that schoolchildren could take a leadership role in the sodium reduction intervention because of their key role in the family. This finding is seen also in a sequential school-based education program to reduce salt intake in children and their families (16, 17).
Notably, there were contextual factors influencing implementation and/or intervention effectiveness. Policy support, administrative support, and staff enthusiasm were all found to be facilitators, as they enhanced implementing this complex intervention in rural China. In contrast, lack of an effective governance mechanism to implement in low- and middle-income countries (LMICs) has been identified as an ongoing challenge in previous research (18, 19). The current project involves village doctors who conduct implement intervention as they easily gain trust from local villagers.
Moreover, there were three main barriers found from process evaluation. The first one is that general villagers were involved in the survey at the end of the intervention, although general villagers had a low reach in health education activities during intervention. This might make the evaluated effect lower than the actual effect. Secondly are adding too much salt when cooking, and eating pickled foods, which were long-standing dietary habits. It is well-known that behavior change is a long-term process requiring gradual adaptation. Many studies have identified that the strong cultural importance of feasting and people's perception of salty food taste were barriers of sodium intake reduction (20). Third, because of the low educational level of villagers, it might take a longer time to change and maintain healthy behavior (21). This also explains the secondary outcome that over 18 months' intervention, blood pressure did not differ significantly between the intervention and control groups (8).
Limitations and Strengths
As for limitations, a. the evaluation would have been further strengthened if some process measurements could have been collected after completion of investigation, e.g., reach for each component of the intervention; b. data were not intentionally collected to show why differences in the extent of implementation had varied outcomes among different villages, because the process evaluation was conducted before the study outcomes were analyzed; and c. the insights about receptivity of intervention and context of this study from different populations including health educators, villagers, and implementers will be further analyzed for future research. As for strengths, the mixed-data evaluation provided a comprehensive understanding of the extent of the implementation and mechanism of effectiveness. It also promoted evidence-based practices in primary healthcare systems in rural China.
When conducting health education interventions for salt reduction in rural China, the design of easy-to-understand educational materials including posters, calendars, and stickers proved to be effectively implemented ways to attract the attention of villagers. The findings suggest that interventions deeply involving primary school students or enthusiastic volunteers can promote the implementation of health education activities. Stocking salt substitutes in stores appears to provide convenient access for villagers. Meanwhile, policy and administrative support and staff with enthusiasm were important for the success of the intervention. These recommendations may be relevant in other low-middle-income countries. This project also demonstrates the feasibility and benefits of using a mixed-methods process evaluation in large-scale clinical trials and promotes evidence-based practices for village doctors who provide the basic medicine care and promote behavior change as a family doctor to prevent and manage chronic disease in rural China. This successful example may also help in the adoption of mixed-methods health service research in China.
1. The 18th National Annual Conference of Clinical Epidemiology and Evidence-based Medicine, Aug. 24, 2019, Dalian, China.
2. MMIRA Asia Regional Conference, Sept. 15, 2019, Hamamatsu, Japan.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
The studies involving human participants were reviewed and approved by Ethics Committee of the Peking University Health Science Center. The patients/participants provided their written informed consent to participate in this study.
YWu, MF, and YWa: conceptualization. NL, LY, YWu, HL, WN, MF, and JZ: writing—review and editing. HC, JZ, MF, and YWa: writing—original draft. HC: software. HC, JZ, and HL: project administration and formal analysis. YWa, WN, YWu, LY, NL, HC, and HL: methodology. YWu, LY, NL, WN, and JZ: investigation. YWu: funding acquisition. All authors contributed to the article and approved the submitted version.
This project was funded in part by the National Heart, Lung, and Blood Institute, National Institutes of Health, Department of Health and Human Services, and National Center for Chronic Disease Prevention and Health Promotion (CDC), under Contract No. HHSN268200900027C. Additional support was received from the United Health Group Chronic Disease Initiative. NL was supported by an Australian National Health and Medical Research Council Overseas Fellowship. MF' participation was made possible through support of the Mixed Methods Program of the University of Michigan.
The views expressed are those of the authors and not necessarily those of any sponsors.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors express their appreciation to all the participants and their family members of this research.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2021.646576/full#supplementary-material
1. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the global burden of disease study 2010. Lancet. (2012) 380:2095–128. doi: 10.1016/S0140-6736(12)61728-0
3. He FJ, Li J, Macgregor GA. Effect of longer term modest salt reduction on blood pressure: cochrane systematic review and meta-analysis of randomised trials. BMJ. (2013) 346:f1325. doi: 10.1136/bmj.f1325
4. Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ. Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet. (2006) 367:1747–57. doi: 10.1016/S0140-6736(06)68770-9
5. Tan M, He FJ, Wang C, MacGregor GA. Twenty-four-hour urinary sodium and potassium excretion in China: a systematic review and meta-analysis. J Am Heart Assoc. (2019) 8:e12923. doi: 10.1161/JAHA.119.012923
6. Anderson CA, Appel LJ, Okuda N, Brown IJ, Chan Q, Zhao L, et al. Dietary sources of sodium in China, Japan, the United Kingdom, and the United States, women and men aged 40 to 59 years: the INTERMAP study. J Am Diet Assoc. (2010) 110:736–45. doi: 10.1016/j.jada.2010.02.007
7. Li N, Yan LL, Niu W, Labarthe D, Feng X, Shi J, et al. A large-scale cluster randomized trial to determine the effects of community-based dietary sodium reduction–the China rural health initiative sodium reduction study. Am Heart J. (2013) 166:815–22. doi: 10.1016/j.ahj.2013.07.009
8. Li N, Yan LL, Niu W, Yao C, Feng X, Zhang J, et al. The effects of a community-based sodium reduction program in rural China - a cluster-randomized trial. PLoS ONE. (2016) 11:e166620. doi: 10.1371/journal.pone.0166620
10. Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. (2005) 6:134–47. doi: 10.1177/1524839904273387
11. Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health. (2019) 7:64. doi: 10.3389/fpubh.2019.00064
16. He FJ, Wu Y, Ma J, Feng X, Wang H, Zhang J, et al. A school-based education programme to reduce salt intake in children and their families (School-EduSalt): protocol of a cluster randomised controlled trial. BMJ Open. (2013) 3:e003388. doi: 10.1136/bmjopen-2013-003388
17. He FJ, Wu Y, Feng XX, Ma J, Ma Y, Wang H, et al. School based education programme to reduce salt intake in children and their families (School-EduSalt): cluster randomised controlled trial. BMJ. (2015) 350:h770. doi: 10.1136/bmj.h770
18. Webster J, Pillay A, Suku A, Gohil P, Santos JA, Schultz J, et al. Process evaluation and costing of a multifaceted population-wide intervention to reduce salt consumption in Fiji. Nutrients. (2018) 10:155. doi: 10.3390/nu10020155
19. Wolfenden L, Wyse R, Nichols M, Allender S, Millar L, McElduff P. A systematic review and meta-analysis of whole of community interventions to prevent excessive population weight gain. Prev Med. (2014) 62:193–200. doi: 10.1016/j.ypmed.2014.01.031
Keywords: hypertension, clustered-randomized trial, salt reduction, complex intervention, process evaluation, mixed methods
Citation: Chu H, Zhang J, Fetters MD, Niu W, Li H, Li N, Yan LL, Wang Y and Wu Y (2021) A Mixed Methods Process Evaluation of a Clustered-Randomized Controlled Trial to Determine the Effects of Community-Based Dietary Sodium Reduction in Rural China. Front. Med. 8:646576. doi: 10.3389/fmed.2021.646576
Received: 27 December 2020; Accepted: 15 March 2021;
Published: 28 May 2021.
Edited by:Vinaya Manchaiah, Lamar University, United States
Reviewed by:Segundo Mariz, European Medicines Agency, Netherlands
Eldre Wiida Beukes, Lamar University, United States
Copyright © 2021 Chu, Zhang, Fetters, Niu, Li, Li, Yan, Wang and Wu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.