An Implementation Evaluation of A Group-Based Parenting Intervention to Promote Early Childhood Development in Rural Kenya

Early childhood development (ECD) parenting interventions can improve child developmental outcomes in low-resource settings, but information about their implementation lags far behind evidence of their effectiveness, hindering their generalizability. This study presents results from an implementation evaluation of Msingi Bora (“Good Foundation” in Swahili), a group-based responsive stimulation and nutrition education intervention recently tested in a cluster randomized controlled trial across 60 villages in rural western Kenya. Msingi Bora successfully improved child cognitive, receptive language, and socioemotional outcomes, as well as parenting practices. We conducted a mixed methods implementation evaluation of the Msingi Bora trial between April 2018 and November 2019 following the Consolidated Advice for Reporting ECD implementation research (CARE) guidelines. We collected qualitative and quantitative data on program inputs, outputs, and outcomes, with a view to examining how aspects of the program's implementation, such as program acceptance and delivery fidelity, related to observed program impacts on parents and children. We found that study areas had initially very low levels of familiarity or knowledge of ECD among parents, community delivery agents, and even supervisory staff from our partner non-governmental organization (NGO). We increased training and supervision in response, and provided a structured manual to enable local delivery agents to successfully lead the sessions. There was a high level of parental compliance, with median attendance of 13 out of 16 fortnightly sessions over 8 months. For delivery agents, all measures of delivery performance and fidelity increased with program experience. Older, more knowledable delivery agents were associated with larger impacts on parental stimulation and child outcomes, and delivery agents with higher fidelity scores were also related to improved parenting practices. We conclude that a group-based parenting intervention delivered by local delivery agents can improve multiple child and parent outcomes. An upfront investment in training local trainers and delivery agents, and regular supervision of delivery of a manualized program, appear key to our documented success. Our results represent a promising avenue for scaling similar interventions in low-resource rural settings to serve families in need of ECD programming. This trial is registered at ClinicalTrials.gov, NCT03548558, June 7, 2018. https://clinicaltrials.gov/ct2/show/NCT03548558.

Introduction

An estimated 43 percent of children under age 5 in LMICs experience compromised cognitive and socioemotional development due to poverty, poor nutrition, and inadequate psychosocial stimulation (1). Early childhood development (ECD) interventions have been shown to improve child development outcomes in low- and middle-income (LMIC) settings among children under 3 years of age (2, 3). Evidence from the most recent Lancet series (1, 4, 5) highlights the critical importance of nurturing care, a term coined to characterize the necessary inputs for ECD, namely good health, adequate nutrition, security and safety, and responsive caregiving with opportunities for early learning. The Nurturing Care Framework (NCF) (6) was created in 2018 by the World Health Organization, UNICEF and the World Bank to recommend the promotion of responsive care, early learning activities, and nutrition support for children under 3 years of age (7). Health Ministers from around the world have pledged to implement the NCF.

Governments and organizations in LMICs are now asking for operational guidance on how to implement effective and scalable programs to improve ECD outcomes. Though there is now consistent evidence of effectiveness of ECD interventions to improve child outcomes, there is strikingly very little information about how these programs were implemented to demonstrate how such results were achieved. There are many ways to implement ECD programs, and decisions must be made regarding: program content, duration and intensity, delivery mode –whether group-based meetings, clinic or home visits– and the necessary qualifications of delivery agents. Some designs may be more suitable for low-resource settings where cost is an issue (8) and where adult literacy and awareness of ECD may be low (4, 9).

To our knowledge, only two published effectiveness studies from Uganda (10) and Pakistan (11) provide information on the process of implementing an ECD parenting program in an LMIC setting that addresses NCF practices. However, they did not identify contextual factors considered when developing or adapting their programs, such as how the needs of parents influenced the program content, and how the capacity of providers influenced the training offered. They also did not examine the determinants and consequences of implementation outputs. Since then, a more structured guideline has been published for reporting implementation of early childhood development programs called the Consolidated Advice for Reporting of ECD implementation research (CARE), (12) along with options for measuring implementation processes (13).

Using the CARE guidelines, this study describes the implementation of a successful parenting intervention that compared the effectiveness of two group-based potentially scalable delivery models for an integrated ECD intervention for families with young children in rural Kenya (see Supplemental Material). Because individual home visits in dispersed rural settings like ours would be prohibitively labor intensive and expensive for scaling, one delivery model featured only group meetings at the village level, while the other combined a small number of home visits with group meetings, as advocated by some (3). The responsive stimulation and nutrition education intervention was named Msingi Bora (“Good Foundation” in Swahili). In collaboration with a Kenyan non-governmental organization (NGO), the Safe Water and AIDS Project (SWAP), we conducted a community-based multi-arm cluster-randomized trial to evaluate the effectiveness of Msingi Bora, implemented across 60 villages in three rural sub-counties using Kenya's network of community health volunteers (CHVs) (14). In 20 intervention villages, CHVs delivered a total of 16 fortnightly group-based sessions within their villages. In another 20 villages, CHVs delivered the same program, but combined 12 group sessions with 4 home visits. A third group of 20 villages served as a comparison arm. Among the 40 villages assigned to an intervention arm, 20 villages (10 from each arm) invited only mothers and children, while the remaining 20 villages also invited fathers to the 16 sessions. On average, each village had 19 eligible children invited to attend sessions with their mothers. Our results showed that parenting interventions delivered by trained para-professional CHVs in large mother-child groups can be effective in promoting child cognitive, language and socio-emotional development, as well as parental stimulation practices in low-resource settings with potential for scalability (14).

The current manuscript reports results from an implementation evaluation of the Msingi Bora trial, with a view to examining determinants and consequences of implementation outputs, such as acceptance of the program and fidelity of delivery, and their association with parent and child outcomes at endline (15, 16). Outcomes were reported in the impacts paper but are examined here in relation to implementation indicators. Using a logic model framework (Figure 1), we conducted a mixed methods study to assess Inputs (e.g., workforce and parenting context, program resources such as curriculum, and design), Outputs (e.g., post-training competence of workforce, fidelity of delivery, attendance by parents, adoption by parents), and Immediate Outcomes (e.g., parental behaviors, child outcomes).

FIGURE 1

Figure 1. Logic model. Child and parent outcome indicators are reported in Luoto et al. (14).

Our research questions were:

• What program resources were input ahead of implementation? Was the program content and delivery acceptable to parents and delivery agents?

• How did Outputs such as delivery fidelity and parental attendance and enactment (i.e., adoption of promoted practices) change during the intervention, and what determined these Outputs?

• What features of Inputs and Outputs were, in turn, associated with child and parent Outcomes, and how did this differ by delivery mode?

Materials and Methods

Setting and Participants

We conducted a mixed methods implementation evaluation of the Msingi Bora trial between April 2018 and November 2019 in three sub-counties outside Kisumu. These predominantly rural areas are characterized by high rates of poverty, child mortality, stunting (23%), and spousal violence (60%); (17) 56% of children under 2 years of age had at most one plaything, (18) and few parents understand the need for stimulation from birth (19). Spoken languages include Swahili, Luhya, and Luo. The majority of villagers are subsistence farmers or unskilled informal workers. There is no ECD policy pertaining to ages under 3 years in this region.

Study Parents

The eligibility criteria for parent participants within villages were mothers or other female primary caregivers aged 15 and over with a child between 6 and 24 months without signs of severe mental or physical impairment. Baseline demographic data from 1,152 households across 60 villages are available in Luoto et al. (14).

Delivery Agents

Program delivery agents were existing village Community Health Volunteers (CHVs), which is a part-time and voluntary position under the Ministry of Health (MOH) tasked with improving community health through home visits (20).

Supervisors

We worked with three Kisumu-based staff and seven sub-county supervisors (4 males, 6 females) from the local NGO to oversee the daily operations of Msingi Bora and to provide training and supervision to delivery agents.

Ethical Approvals

All mothers provided written informed consent at the time of data collection. Selected CHVs and SWAP supervisors provided consent to be interviewed at the end of the intervention. Ethics approval was obtained from Maseno University in Kisumu, Kenya, and RAND. The trial is registered at ClinicalTrials.gov, NCT03548558, and the study protocol has been published (21).

Implementation Strategy

The implementation evaluation was guided by the CARE guidelines and organized by the logic model in Figure 1.

Content and Delivery Strategy

Msingi Bora's parenting intervention was based on a structured curriculum adapted from previous successful parenting trials in LMICs, and expanded to include more activities around responsive play and talk with children (22, 23). Documented evidence on low awareness about ECD among parents and CHVs (18, 19) informed the need for a structured and manualized set of activities for each session, along with time for guided discussions among parents.

Six sessions were piloted in May-June 2018 in six villages not included in the main trial. The pilot confirmed the need for a structured manual of activities and that CHVs were not familiar with hosting group sessions on ECD. It also helped us recognize the need to reduce lengthy speeches by the CHV in favor of practical activities, and to provide a streamlined manual that could be easily navigated by CHVs. The main trial tested group-only delivery against a mixed model combining group sessions with home visits, but in both arms group sessions were the main format for delivery of messages (8).

The finalized curriculum included a total of 16 fortnightly sessions with session-specific activities and materials, and Luo or Swahili and English manuals for each CHV. The structured curriculum was based on five “key” practices: responsive play, responsive communication, hygiene, nutrition, and love and respect (22). The Responsive Play and Communication curriculum was informed by research on responsive stimulation (24). The Love and Respect curriculum used a range of culturally-adapted interpersonal, cognitive and behavioral treatment elements as the basis for promoting maternal well-being and healthy family dynamics (22, 25). Every fourth session served as a review session, which aimed at consolidating knowledge and new practices learned in the preceding sessions. For these sessions, households in the group-only arm continued with group meetings while households in the mixed-delivery arm received individual home visits from their CHV during the same week that group review sessions were held. During these home visits, CHVs delivered identical review messages to those in the group reviews, but the focus was personalized on that family.

Mothers and children were invited to attend every session. All sessions emphasized parents learning new practices with their child or spouse through demonstration and coached practice, group-based problem solving and peer support, along with homework to play and talk with their child.

A poster with each of the five practices illustrated was used in every session. To help families recall the practices, they received a small version of the poster to take home after the first session. An illustrated Kenyan story book in Luo or Swahili was given to each child in Session 14, to be used during following sessions and at home to encourage responsive communication. For the seven responsive play and communication sessions, parents were taught how to play games using materials available in homes and adapt the games to make them more challenging as the child aged. Parents were not given toys or playthings, because they would be costly and not appropriate as the child aged. Instead, they were encouraged to have a playbag with play objects that can be found at home such as bottle caps, ball, sticks, colored cloths, and pictures. Parents were reminded to add things monthly. CHVs also collected materials for a play bag and brought this to sessions.

Mothers received a small incentive for timely attendance, namely a small bar of soap, based on piloting. In later sessions, as all children were above 12 months, small pouches of milk were distributed instead of soap (each valued at USD $0.15). Sessions took place in local community centers or churches. Households in villages assigned to the comparison group did not receive any interventions besides information about child feeding during the baseline survey.

Delivery Personnel

CHVs are members of their own communities and the position is generally part-time. Minimum qualifications include being literate, permanent residents of the village, and accepted in their roles by the community members. Upon selection, CHVs undergo roughly 10 days of training held by county MOHs on topics related to basic health promotion; (20) ECD topics are not addressed. For our study, CHVs in villages randomly assigned to an intervention arm were invited to deliver the program; none refused. The research project paid CHVs a monthly stipend of USD $20 for their duties according to local policy.

CHVs were trained on Msingi Bora in two parts. Training of sessions 1–8 took place at the start of the program over 8 days; training of sessions 9–16 took place at midline over 8 days. The first training was conducted in English by international researchers (JL, IL, FA, and DS) with all CHVs together, though they had an English and local language version of the manual. The second training was conducted in the local languages by SWAP staff following a train-the-trainers model in which international staff first trained SWAP staff, who then trained the CHVs. Beginning with session 4, monthly 1-day refresher trainings were also performed in each sub-county to help CHVs prepare ahead of each session.

Training focused on three critical skills: (a) knowing what children need to develop; (b) demonstrating and coaching caregivers as they engaged in the new practices; and (c) facilitating group discussions about problems and solutions. Trainings followed the sequence of activities in the manual. They emphasized practicing how to demonstrate the activities, coach parents and facilitate group discussions, first among themselves and then with groups of mothers and children from villages not participating in the study.

CHVs underwent tests on knowledge and competency to deliver sessions at the end of each training, the former with a paper-and-pencil test and the latter with mock sessions scored using a fidelity rating sheet (described below). Those who were weaker at the end of training were required to attend the session of a peer and undergo extra practice before delivering their own group session.

To obtain information from CHVs concerning their perspectives of the program content and delivery format, as well as on their training, supervision, and curriculum acceptance, semi-structured interviews were conducted with eleven randomly selected CHVs at the end of the program. Two local researchers individually interviewed selected CHVs, male and female, from different sites in a private, neutral community setting.

Supervisory Personnel

SWAP (www.swapkenya.org) is a local NGO that has operated since 2005 in western Kenya and served as the local implementing partner overseeing the training and supervision of the CHVs. Since topics of stimulation and responsive parenting were new to SWAP staff, the pilot training was delivered by international researchers in English, and attended by six SWAP staff and six CHVs from the selected pilot villages who were trained to deliver the pilot program. For the full trial, four of the six pilot CHVs were promoted to “mentor” supervisory roles to help with local monitoring of the sessions. As mentioned above, SWAP staff and mentor CHVs participated in the two training courses for the main trial, and led the second one under a train-the-trainers model.

Local SWAP staff and mentor CHVs supervised CHVs in the field. This entailed observing and monitoring each group session and a portion of home visits from each review session in assigned villages using a checklist (see section Fidelity). After the session, they asked CHVs to complete their own self-evaluation form and then provided feedback on their performance.

The same local qualitative researchers conducted two focus groups at the end of the program with SWAP staff – one with the three lead supervisory staff, and a second with three sub-county staff. They were asked the same questions and probed about four key components of the program: (a) acceptance of the program content and delivery; (b) helpful and unhelpful aspects of training and supervising CHVs; (c) how easy and difficult it was for CHVs to demonstrate practices and to facilitate discussion; and (d) benefits and problems associated with group sessions and home visits.

Fidelity

SWAP supervisors utilized a checklist of fidelity items during sessions and provided CHV delivery agents with feedback immediately after each session. In addition, CHVs completed self-evaluation forms following each session and attendance sheets of parent participants. The supervisor and self-evaluation forms were similar in content, and included 5-point scales, from “poor” (1) to “excellent” (5), to rate 13 dimensions of the CHV's fidelity to session content, demonstration of activities, facilitation of group discussions, coaching, use of visual aids and play materials, and other delivery qualities. At the end of the forms, respondents were asked to rate the overall session quality on a 1–10 scale, as well as how much “fun” participants had on a 1–10 scale to capture the level of engagement by participants.

Caregiver Attendance, Acceptance, and Enactment of Practices

Qualitative interviews of eight pairs of participant mothers from the three sub-counties were conducted at the end of the program, in a private neutral setting. Mothers were interviewed in pairs in order to collect more information and to make them feel more comfortable by being in the presence of a familiar person. They were selected by SWAP staff to meet criteria of representing each of the different delivery formats (group-only meetings, mixed group-home meetings, with and without fathers invited in their villages), and we requested that they include mothers with both high and low attendance in the program. The same two independent qualitative researchers used a semi-structured guide of questions. The questions focused on participants' attendance and acceptance of the program, enactment or adoption of the practices promoted during sessions and how easy and difficult practices were to enact; they also evaluated the group sessions and home visits.

Data Collection and Analysis Plan

Table 1 summarizes our qualitative and quantitative data sources and how they were analyzed to answer the three research questions.

TABLE 1

Table 1. Research questions and data sources crosswalk and analysis plan.

All qualitative implementation data were collected by a team of two local independent researchers, one male and one female, trained by FA and JL. All interviews obtained informed consent prior to beginning, were conducted in the local language of choice, and were audiorecorded, translated to English and transcribed. Interviewers probed for elaborations. Transcriptions and translations were verified between the two independent researchers.

Content analysis was performed to analyse all qualitative data using a pre-arranged list of codes on desired information. For the CHV interviews, two co-authors (FA and RZ) independently coded four interviews after agreeing on a coding scheme and found high inter-rater reliabilities (kappa 0.774, 95% CI 0.68–0.87, z = 13.71, p < 0.0001). For the parent interviews, the same two researchers independently coded four after agreeing on a coding scheme (kappa 0.766, 95% CI 0.68–0.85, z = 15.76, p < 0.0001). Because there were only two supervisor FGD, two researchers (FA and DS) together coded their responses.

Quantitative data were collected using SurveyCTO on Android tablets. All supervisory staff were supplied tablets and filled out the session monitoring forms directly. Following each session, CHVs completed attendance and self-evaluation forms on paper, and the supervisor photographed the paper forms and transfered them to the tablet. A US-based research assistant cross-checked a random subset of photographed paper forms with SurveyCTO forms periodically to ensure accuracy and inquired if there were any inconsistencies to resolve them.

Final outcomes were collected at baseline and endline by a team of trained local enumerators using SurveyCTO on Android tablets. Child cognitive and language outcomes were measured with the Bayley Scales of Infant Development third edition (Bayley III), (26) and socioemotional development was measured with the Wolke Scale (27). Parent outcomes included stimulation practices measured with the Family Care Indicators (FCI) (28) at baseline, and the more comprehensive Home Observation for Measurement of the Environment (HOME) at endline (29). More details on outcome measures listed in Figure 1 are provided in the impact manuscript (14).

Quantitative analyses used multivariate linear regressions by estimating ordinary least squares (OLS) models for continuous outcomes as a function of program inputs or outputs, as appropriate, and combining different data sources including CHV self-evaluation forms, supervisor monitoring forms, maternal attendance records, and household survey data. More details can be found in Table 1 or in notes to the individual tables in Results. For many analyses we used a composite single index of overall delivery quality that we estimated using Principal Component Analysis (PCA) models on the 13 distinct aspects of delivery rated by supervisors on a 1–5 scale. Similarly, when we relate program inputs and outputs to final project outcomes to address our third research question, we estimated a single index of child development using PCA models on four different child development outcomes measured at endline including cognitive, receptive language, expressive language, and socioemotional outcomes.

Results

Question 1

What program resources were input ahead of implementation? Was the program content and delivery acceptable to parents and delivery agents?

Table 2 contains illustrative quotes from a content analysis of qualitative interviews with supervisors, CHVs, and mothers on program adaptations and delivery. All three groups of participants indicated that initially they knew very little about child development and the need for stimulating play and communication, consistent with our expectations. Similarly, low baseline scores on the FCI showed that children had on average 1.4 out of 6 playthings and parents performed 3.3 out of 6 stimulating activities with them. They also had low dietary diversity scores (3.1 out of 7 food categories consumed in past 24 h). Thus, there was a need for messages about playing and talking with the child, and including fish and eggs in their diet. The interviews also made clear that the program's five practices were acceptable to implementers and parents, and that the use of a structured manual with practical activities for CHVs and parents to follow was appreciated.

TABLE 2

Table 2. Illustrative participant responses regarding need for and acceptance of the designed program.

Question 2

How did Outputs such as delivery fidelity and parental attendance and enactment (i.e., adoption of promoted practices) change during the intervention, and what determined these Outputs?

To answer this research question, we discuss aspects of fidelity, attendance and enactment of practices. Within each topic, we first present quantitative results followed by analyses of their determinants, and under determinants we include relevant quotes from qualitative interviews with parents, CHVs, and Supervisors.

Fidelity

Monitoring by supervisors took place as intended during 88% of the 12 (non-review) group sessions across all arms and villages, and 91% of group review sessions were supervised. Fewer home visit review sessions were supervised by necessity (9%). Monitoring forms (n = 686) showed gradual and statistically significant improvement across all sessions in all delivery measures (Figure 2). Similar upward trends were seen in self-evaluation ratings made by CHVs (n = 2,143; results available upon request). The median session lasted 90 minutes.

FIGURE 2

Figure 2. Mean CHV Delivery Scores by Session. Figure plots mean scores from supervisor monitoring forms across all CHVs by session for items measured on a 1–5 scale from “poor” to “excellent” for six of 13 delivery measures using solid lines. N = 686 supervisor rating forms across 16 sessions and 40 villages. Some earlier sessions had more than 1 supervisor present. For session “quality” and “fun,” figure plots mean scores on 1–10 scale on right-hand-side axis using dashed lines. All 13 delivery measures show similar upward trends and the full list is in Figure 3. Red vertical lines represent review sessions when sessions were home visits for mixed-delivery arm and group sessions for group-only arm.

Most aspects of session delivery were rated good or excellent in 60% of sessions or more by supervisors (Figure 3). The most difficult skills involved facilitating group discussions, and discussing problems and solutions to enacting the new practices. CHVs performed better at following session content, discussing homework, expressing acceptance, being prepared and using visual aids. CHV self-evaluations were in general agreement, if slightly inflated, relative to supervisor ratings for aspects of delivery quality (Figure 3).

FIGURE 3

Figure 3. Delivery Ratings: Supervisor and Self-Evaluations. Findings are mean ratings from supervisor or self-assessment monitoring forms across all 16 sessions and 40 intervention villages. N = 686 for supervisor monitoring forms; N = 2,043 for CHV self-assessment forms that include each home visit.

Determinants of Fidelity

The 40 CHVs from the intervention villages who underwent training and delivered the intervention included 10 males and 30 females, were on average 44 years old (range 26–69), had 11 years education (range 7–14), and 9 years experience as a CHV (range 1–24). All CHV characteristics were balanced across intervention arms. CHVs who scored higher on a knowledge test of ECD after the first training delivered better quality and more engaging sessions, and scored higher in all fidelity outcomes on average (Table 3). CHV age, education, and sex were not consistently related to performance.

TABLE 3

Table 3. Determinants of delivery fidelity, supervisor, and self evaluations.

During post-intervention interviews, CHVs and supervisors were asked to comment on training, supervision and monitoring of performance. Their answers reveal important determinants of fidelity. Illustrative quotes are summarized in Supplementary Table 1. CHVs responses showed a need for more practice demonstrating responsive play and communication, and facilitating a group discussion on problems enacting the messages. They were more familiar with the key messages on nutrition and hygiene. CHVs became more proficient and confident in their session delivery after the midline training when SWAP staff trained them in smaller groups and in the language of delivery, and with refresher meetings in the sub-counties twice a month (Supplementary Table 1). Their post-training knowledge scores from the first to the midline training increased from a mean of 57.2% to 60.8% correct, though this was not statistically significant (p = 0.17). Their performance scores in delivering a mock session after training served as a way to identify 30% of CHVs who needed extra training.

All 11 interviewed CHVs mentioned ways the supervisor feedback and opportunity to evaluate themselves after sessions was helpful, and reported having prepared themselves ahead of sessions. CHV descriptions of how they demonstrated responsive talk were thorough: 8 out of 11 CHVs described at least 3 key components such as letting the child show what he/she is interested in, and then the parent repeats or expands verbally. However, descriptions of how they demonstrated responsive play were incomplete and none was able to name three key components of responsive play; they often correctly mentioned it was letting the child choose playthings, but then incorrectly required the child to follow the parent's lead. CHVs said they initially found it difficult to facilitate group discussions, but later developed a style to encourage shy mothers and sum up the best solutions at the end. Interviews with supervisors showed they recognized early on that CHVs would benefit from periodic refresher trainings ahead of sessions (Supplementary Table 1).

Maternal Attendance

Among the 776 intervention households, 97% of mothers attended at least one Msingi Bora session, with no differences by intervention arm (p = 0.142). The median mother attended 13 of 16 sessions (IQR 8–15 sessions) with a median group size of 13 mothers at a group meeting (IQR 10–15 mothers). The mixed-delivery model had higher overall attendance, with an average of 74 percent of mothers attending a given session, compared to 64 percent of mothers in group-only villages. The bulk of this difference comes from the review sessions, which were home visits for the mixed-delivery arm (and thus, mothers did not have to travel to these sessions). These sessions were the only ones in which attendance was statistically different across arms (Figure 4).

FIGURE 4

Figure 4. Mean maternal attendance by study arm and session. From attendance monitoring data. Dashed faded lines represent 95% confidence intervals; red vertical lines represent review sessions when sessions were home visits for mixed-delivery arm and group sessions for group-only arm. The only sessions with statistically significant differences in attendance at 95% level or higher are the four review sessions (4, 8, 12, and 16).

Determinants of Maternal Attendance

Maternal attendance to the group sessions was higher in households with a father present, and with higher baseline FCI stimulation scores (Table 4). More educated mothers and those with a greater distance to walk had lower attendance. Mothers also attended more sessions in villages where CHVs were rated higher on fidelity measures by supervisors (Table 4).

TABLE 4

Table 4. Determinants of maternal attendance to common group sessions.

During post-intervention interviews, both Mothers and CHVs were asked what encouraged mothers to attend. The frequencies of commonly mentioned reasons and representative quotes are seen in Table 5. Benefits to the child were recognized by both groups of participants, but CHVs frequently mentioned incentives and their reminders to the mothers, whereas mothers frequently said that socializing with others drew them to sessions. An equal number mentioned the content of the program. No mother mentioned reminders or incentives as spurring their attendance.

TABLE 5

Table 5. Barriers and enablers for caregiver attendance—Mother and CHV perspectives.

Parental Enactment

Mothers and CHVs were asked about barriers and enablers to enacting the new program practices (see Supplementary Table 2). Both mothers and CHVs reported it was sometimes difficult for mothers to find new play things and to find 15 min daily to play with their children, but learning games with sticks and learning how to talk about pictures in a book facilitated responsive play and talk practices.

Many of the responses to enactment of specific practices by mothers, especially about play and communication (Supplementary Table 2), triangulated well with answers by CHVs and with relevant HOME Inventory items (Table 6). For example, 8 out of 8 mother pairs said they had a playbag, and 7 out of 8 said they added new playthings. These responses were largely corroborated by CHVs, and by the full sample's responses to related items on the HOME Inventory at endline (Table 6). Similarly, 6 out of 8 interviewed mother pairs said they look at a book with their child daily; on the HOME, 82% of intervention mothers vs. 23% in the control group reported looking at a picture book with their child in the past week.

TABLE 6

Table 6. Maternal enactment of new practices.

Question 3

What features of inputs and outputs were in turn associated with child and parent outcomes, and how did this differ by delivery mode?

After combining endline survey and program implementation data, we found that aspects of both the delivery agents and their performance were often related to parent and child outcomes (Table 7). For example, CHV age and initial-training knowledge of ECD were positively related to better HOME scores as well as to child developmental outcomes, but not to dietary diversity. CHVs with higher performance ratings by supervisors were associated with better HOME scores, but we did not see a similar relation with any child outcomes.

TABLE 7

Table 7. Determinants of final outcomes.

While both intervention arms saw improvements in key outcomes relative to the control arm, the group-only arm generally outperformed the mixed-delivery arm in child impacts, though differences were rarely statistically significant (14). The only practical difference across intervention arms were the four review sessions, which were delivered in groups in the group-only arm and through home visits in the mixed-delivery arm. Interviews with CHVs, supervisors and mothers asked their perspectives on group meetings and home visits (Table 8). CHVs and supervisors had positive things to say about both meeting formats, but appeared to have a preference for home visits, where they were able to maximize attendance, provide greater direct supervision of the mother's practices, and appreciated that “you will know the truth, whether she is doing it or not.” Conversely, mothers clearly preferred the group meetings, where they felt their children were able to socialize more and learn how to play with others, and they felt better supported by other mothers. Mothers viewed home visits somewhat as an inspection and a test of their uptake, using phrases such as “The CHV would come inspect the play bag,” and “They would ask me questions from the poster and I would answer.”

TABLE 8

Table 8. Mother, CHV, and supervisor perceptions about home visit vs. group meetings.

Discussion

This implementation evaluation of our cluster randomized effectiveness trial testing an ECD parenting program in rural Kenya demonstrated acceptability of the program content and format, high maternal attendance to group sessions, improved fidelity over time, and links with final outcomes. These findings demonstrate the importance of implementation quality and delivery performance. We did not present results on the inclusion of fathers, who attended sessions at low rates, and their inclusion had no measurable impacts. Further details on including fathers will be discussed in another paper.

Regarding the first research question, the curriculum, which was focused on mothers' interaction with their children to provide responsive play and communication, addressed the needs of both parents and CHVs who knew little about the role of stimulation in mental development starting at birth (18, 19). This was confirmed by low baseline scores on the FCI, as well as by post-intervention qualitative interviews with parents, CHVs and program supervisors, which revealed that mothers were not accustomed to playing with or providing playthings for their children, and that CHVs were not used to demonstrating games for children. CHVs found acceptable the Manual that laid out a structured set of activities to be conducted each session. CHVs and caregivers liked the practical approach to demonstrating and coaching new play and communication interactions.

Other programs use a different curriculum, focusing on teaching specific skills to the child to be repeated by the mother and child as homework (30, 31), or counseling individual mothers after identifying gaps in their responsive stimulating interactions with children (23). Teaching specific age-appropriate skills would not be suitable for a large group of mothers with children of different ages. Likewise, counseling does not always suit group sessions. Instead, coaching after a CHV demonstration or having some mothers demonstrate while others watch was appropriate for our setting. Other program developers and implementers have also adapted the content and delivery of well-known parenting programs such as Reach Up (32) and Care for Child Development (CCD) (33) to deliver them in groups in rural areas with low-skilled parents and providers, including making the program more structured (32). We therefore attribute acceptance of the content and delivery format of the Msingi Bora program to these aspects of the curriculum and its delivery.

Regarding implementation fidelity, CHVs' experience delivering sessions and frequent supervisory feedback over the 8-month intervention led to significant improvements in CHV delivery performance and enjoyment of mothers and CHVs with the program. A key finding from our implementation evaluation is that CHVs who grasped the content more thoroughly from the first training tended to score better on measures of delivery performance, and were associated with significantly improved parenting practices and child development at endline. Similarly, CHVs who delivered better sessions had greater attendance, and their villages had higher HOME scores at endline, though not better child developmental outcomes. These results are highly encouraging given that our study trained low-skilled delivery agents and used the existing local infrastructure in a disadvantaged setting. It is also encouraging that both male and female CHVs were able to deliver these sessions, and CHV sex was not a consistent determinant of program quality or final outcomes.

As mentioned above, the quality of CHV session delivery increased as the program unfolded. This is potentially a result of a course-correction in our training program with the introduction of monthly refresher trainings beginning with session 4. The initial plan for a baseline and midline training to cover half of the 16 total sessions apiece proved to be insufficient for our delivery agents. Because CHVs were unfamiliar with the practice of responsive play and communication, they initially found it difficult to demonstrate and coach these actions. Extensive practice during training and the introduction of monthly refresher trainings was required to overcome the habit of instructing and directing children. Likewise, CHVs were unfamiliar with facilitating open discussions with groups of parents, probing them for more descriptions of their experiences and encouraging them to solve problems together. What helped was to have them watch each other, especially those more skilled, practice facilitating small group discussions. Interviews with parents, CHVs and supervisors recognized the improvement in quality that developed over time. Other implementers have similarly highlighted the importance of provider competence and the need for more training and supervision than initially expected (11, 19, 34–36).

Parental attendance to the group sessions was an anticipated challenge in our dispersed and disadvantaged rural setting. Indeed, mothers with more education and who lived farther from a meeting place had lower attendance on average, suggesting an opportunity cost of attendance. However, the effect of distance was small: each additional kilometer to travel resulted in −0.7 fewer group sessions attended, and the median distance mothers had to travel was 0.65 kilometers. Although maintaining high attendance is an issue with many group programs (9), perceived benefits became a motivator over time. Until mothers saw the benefits of the program, reminders and incentives were likely required, but their importance was not recognized by mothers by the end of our program.

Mothers' attendance and receptiveness to the program content translated into high reported uptake of the recommended behavioral practices as measured by the HOME Inventory at the endline survey. Interviews with a subsample of mothers indicated that most had a playbag for their child's playthings, and spent time playing with them, though initially they found these practices difficult (e.g., due to competing priorities such as household chores). Less frequently than requested, they added new materials to the bag. After learning how to engage in responsive two-way talk with their child, they found this relatively easy. When given a picture book in a later session, they were able to apply the responsive talking method to it rather than simply reading. Quotes from mothers and CHVs (Supplementary Table 2) confirm the gradual uptake of these important practices, especially as mothers noticed positive benefits for their children.

Perhaps the most surprising result of our study is the relative advantages shown by the group meetings to improve children's final outcomes. The interviews with mothers clearly highlight group camaraderie and opportunities for peer learning among the children as relative benefits of the group format, and caution against a model that utilizes home visits if they engender a feeling of mothers being under inspection.

A key limitation of our study is that, though interviews point to convincing reasons, we cannot say with certainty why children in the group arm outperformed the mixed-delivery arm. Another study limitation is that we have only coarse measures of CHV characteristics and expertise, which constrains our ability to predict who might make the best delivery agents.

Our results show that a responsive stimulation and nutrition education intervention featuring large group sessions delivered by para-professional community health workers can benefit multiple child and parent outcomes. An upfront investment in training local trainers and delivery agents, and regular supervision of delivery of a manualized program, appear key to our documented success. Our results represent a promising avenue for scaling similar interventions in low-resource rural settings to serve families in need of ECD programming.

Data Availability Statement

The datasets presented in this article are not readily available because the de-identified datasets generated during the study along with statistical plan and analytic code will be available from the corresponding author on reasonable request at the end of the 5-year project after all planned articles have been accepted for publication. We will make the data without identifiers available to users only under a data-sharing agreement that provides for: (1) a commitment to using the data only for research purposes and not to make the attempt of identifying any individual participant; (2) a commitment to securing the data in case there are still some sensitive variables after the identifiers have been removed, by using appropriate computer technology; and (3) a commitment to destroying or returning the data after analyses are complete; and (4) a commitment to not publish any information that is not treated at the aggregate level so that no specific characteristics can be linked to small communities. Requests to access the datasets should be directed to jluoto@rand.org.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethics approval was obtained from Maseno University in Kisumu, Kenya, and RAND. Written informed consent to participate in this study was provided by the participants' mothers and fathers, or legal guardian/next of kin.

Author Contributions

JL and IL secured the funding, designed the study, and performed the quantitative analyses. JL, IL, and FA wrote the first draft of the manuscript. FA, DS, JL, and IL trained SWAP's trainers into the intervention content. RO and EA became lead trainers for CHVs and managed the study's implementation throughout. FA and JL trained the qualitative researchers. FA, DS, and RZ analyzed the qualitative data. FA and DS reviewed and provided critical input to study design and conceptualization. All authors reviewed and contributed to writing the final draft.

Funding

This work was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development (NICHD) with award number R01HD090045 (PI: Luoto). NICHD had no role in study design, data collection, data analysis, data interpretation, or writing of the report. DS was supported by an Academic Scholars Award from the Department of Psychiatry at the University of Toronto. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Alie Eleveld, Alex Mwaki, Ursula Liona, Frederick Museveni, Caroline Odhiambo, George Simbiri, Arieda Joab Ochieng, the CHVs, and the entire team at the Safe Water and AIDS Project (SWAP) in Kenya for invaluable support in the carrying out of this study. We thank the County Health Management Teams and Sub-County Health Management Teams.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2021.653106/full#supplementary-material

Additional Files. Supplementary Material file includes CARE Guidelines to show where in manuscript each CARE item is described, as well as Supplementary Tables 1, 2 that are too large for inclusion here.

Abbreviations

CARE, Consolidated Advice for Reporting ECD implementation research; CHV, Community Health Volunteer; ECD, Early Child Development; FCI, Family Care Indicators; HOME, Home Observation for Measurement of the Environment; LMIC, Low and Middle Income Country; MOH, Ministry of Health; NGO, Non-Governmental Organization; NCF, Nurturing Care Framework; SWAP, Safe Water and AIDS Project.

References

1. Black MM, Walker SP, Fernald LC, Andersen CT, DiGirolamo AM, Lu C, et al. Early childhood development coming of age: science through the life course. Lancet. (2017) 389:77–90. doi: 10.1016/S0140-6736(16)31389-7

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Jeong JEF, Ramos de Oliveira CRK, AK Y. Parenting interventions to promote early child development in the first three years of life: a global systematic review and meta-analysis. PLoS Med. (in press).

3. Aboud FE, Yousafzai AK. Global health and development in early childhood. Annu Rev Psychol. (2015) 66:433–57. doi: 10.1146/annurev-psych-010814-015128

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Richter LM, Daelmans B, Lombardi J, Heymann J, Boo FL, Behrman JR, et al. Investing in the foundation of sustainable development: pathways to scale up for early childhood development. Lancet. (2016) 389:103–18. doi: 10.1016/S0140-6736(16)31698-1

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Britto PR, Lye SJ, Proulx K, Yousafzai AK, Matthews SG, Vaivada T, et al. Nurturing care: promoting early childhood development. Lancet. (2017) 389:91–102. doi: 10.1016/S0140-6736(16)31390-3

PubMed Abstract | CrossRef Full Text | Google Scholar

6. World Health Organization United Nations Children's Fund World Bank. Nurturing Care for Early Childhood Development: A Framework for Helping Children Survive and Thrive to Transform Health and Human Potential. Geneva (2018).

7. World Health Organization. Improving Early Childhood Development: WHO Guideline. World Health Organization. Geneva (2020).

Google Scholar

8. van Ravens J. ECDAN and Its SDG: Back to Philanthropy. New Haven, CT: Yale University (2019).

9. Cavallera V, Tomlinson M, Radner J, Coetzee B, Daelmans B, Hughes R, et al. Scaling early child development: what are the barriers and enablers? Arch Dis Child. (2019) 104(Suppl. 1):S43–50. doi: 10.1136/archdischild-2018-315425

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Singla DR, Kumbakumba E. The development and implementation of a theory-informed, integrated mother-child intervention in rural Uganda. Soc Sci Med. (2015) 147:242–51. doi: 10.1016/j.socscimed.2015.10.069

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Yousafzai AK, Rasheed MA, Siyal S. Integration of parenting and nutrition interventions in a community health program in Pakistan : an implementation evaluation. Ann N Y Acad Sci. (2018) 1419:160–78. doi: 10.1111/nyas.13649

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Yousafzai A, Aboud F, Nores M, Kaur R. Reporting guidelines for implementation research on nurturing care interventions designed to promote early childhood development. Ann N Y Acad Sci. (2018) 1419:26–37. doi: 10.1111/nyas.13648

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Aboud FE, Prado EL. Measuring the implementation of early childhood development programs. Ann N Y Acad Sci. (2018) 1419:249–63. doi: 10.1111/nyas.13642

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Luoto JE, Lopez Garcia I, Aboud FE, et al. Group-based parenting interventions to promote child development in rural Kenya: a multi-arm, cluster-randomised community effectiveness trial. Lancet Glob Heal. (2020) 9:E309–19. doi: 10.1016/S2214-109X(20)30469-1

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Nilsen P. Making sense of implementation theories, models, and frameworks. In: Implementation Science. (2015) 10:53–79. doi: 10.1007/978-3-030-03874-8_3

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research : conceptual distinctions, measurement challenge, and research agenda. Adm Policy Ment Heal. (2011) 38:65–76. doi: 10.1007/s10488-010-0319-7

CrossRef Full Text | Google Scholar

17. DHS. Kenya Demographic and Health Survey 2014 Final Report (2015).

18. Statistics KNB. Nyanza Province Multiple Indicator Cluster Survey 2011 Final Report (2013).

19. Lindland E, Mwoma T, Owino GE, Ngutu M, Okeng'o L, Nelima D. Expanding the Basics: Mapping the Gaps between Expert, Decision-Maker and Public Understandings of Early Childhood Development in Kenya, Washington, DC: FrameWorks Institute. (2018).

Google Scholar

20. Kenya Ministry of Health - Division of Community Health Services. Community Health Volunteers (CHVs): Basic Modules Handbook. Nairobi (2013).

21. Luoto JE, Lopez Garcia I, Aboud FE, Fernald LCH, Singla DR. Testing means to scale early childhood development interventions in rural Kenya: the Msingi Bora cluster randomized controlled trial study design and protocol. BMC Public Health. (2019) 19:259. doi: 10.1186/s12889-019-6584-9

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Singla DR, Kumbakumba E, Aboud FE. Effects of a parenting intervention to address maternal psychological wellbeing and child development and growth in rural Uganda: a community-based, cluster-randomised trial. Lancet Glob Heal. (2015) 3:e458–69. doi: 10.1016/S2214-109X(15)00099-6

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Yousafzai AK, Rasheed MA, Rizvi A, Armstrong R, Bhutta ZA. Effect of integrated responsive stimulation and nutrition interventions in the Lady Health Worker programme in Pakistan on child development, growth, and health outcomes: a cluster-randomised factorial effectiveness trial. Lancet. (2014) 384:1282–93. doi: 10.1016/S0140-6736(14)60455-4

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Bradley R, Corwyn R. Caring for children around the world: a view from HOME. Int J Behav Dev. (2005) 29:468–78. doi: 10.1177/01650250500146925

CrossRef Full Text | Google Scholar

25. Rahman A, Malik A, Sikander S, Roberts C, Creed F. Cognitive behaviour therapy-based intervention by community health workers for mothers with depression and their infants in rural Pakistan: a cluster-randomised controlled trial. Lancet. (2008) 372:902–9. doi: 10.1016/S0140-6736(08)61400-2

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Bayley N. Bayley Scales of Infant and Toddler Development, 3rd edn. San Antonio, TX: Pearson Psychological Corporation (2006). doi: 10.1037/t14978-000

CrossRef Full Text

27. Wolke D, Skuse D, Mathisen B. Behavioral style in failure-to-thrive infants: a preliminary communication. J Pediatr Psychol. (1990) 15:237–54. doi: 10.1093/jpepsy/15.2.237

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Hamadani JD, Tofail F, Hilaly A, Huda SN, Engle P, Grantham-McGregor SM. Use of family care indicators and their relationship with child development in Bangladesh. J Health Popul Nutr. (2010) 28:23–3. doi: 10.3329/jhpn.v28i1.4520

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Caldwell BM, Bradley RH. Home Inventory Administration Manual. Little Rock, AR: University of Arkansas for Medical Sciences (2003).

30. Hamadani JD, Mehrin SF, Tofail F, et al. Integrating an early childhood development programme into Bangladeshi primary health-care services: an open-label, cluster-randomised controlled trial. Lancet Glob Heal. (2019) 7:e366–75. doi: 10.1016/S2214-109X(18)30535-7

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Aboud FE, Yousafzai AK. Scaling up child psychosocial stimulation programmes for young children. Lancet Glob Heal. (2019) 7:e294–5. doi: 10.1016/S2214-109X(19)30018-X

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Attanasio O, Baker-Henningham H, Bernal R, Meghir C, Pineda D, Rubio-Codina M. Early stimulation and nutrition: the impacts of a scalable intervention. NBER Work Pap. (2018) p. 1–48. doi: 10.3386/w25059

CrossRef Full Text | Google Scholar

33. Gladstone M, Phuka J, Thindwa R, Chitimbe F, Chidzalo K, Chandna J, et al. Care for child development in rural Malawi: a model feasibility and pilot study. Ann N Y Acad Sci. (2018) 1419:102–19. doi: 10.1111/nyas.13725

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Hill Z, Dumbaugh M, Benton L, Källander K, Strachan D, ten Asbroek A, et al. Supervising community health workers in low-income countries – a review of impact and implementation issues. Glob Health Action. (2014) 7:24085. doi: 10.3402/gha.v7.24085

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Kohli-Lynch M, Ponce Hardy V, Bernal Salazar R, Bhopal SS, Brentani A, Cavallera V, et al. Human resources and curricula content for early child development implementation: multicountry mixed methods evaluation. BMJ Open. (2020) 10:e032134. doi: 10.1136/bmjopen-2019-032134

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Francis T, Baker-Henningham H. Design and implementation of the irie homes toolbox: a violence prevention, early childhood, parenting program. Front Public Heal. (2020) 8:1–21. doi: 10.3389/fpubh.2020.582961

PubMed Abstract | CrossRef Full Text | Google Scholar