Can home visits for early child development be implemented with sufficient coverage and quality at scale? Evidence from the SPRING program in India and Pakistan

Abstract

Introduction:

There is limited evidence from low and middle-income settings on the effectiveness of early child development interventions at scale. To bridge this knowledge-gap we implemented the SPRING home visiting program where we tested integrating home visits into an existing government program (Pakistan) and employing a new cadre of intervention workers (India). We report the findings of the process evaluation which aimed to understand implementation.

Methods and materials:

We collected qualitative data on acceptability and barriers and facilitators for change through 24 in-depth interviews with mothers; eight focus group discussions with mothers, 12 with grandmothers, and 12 with fathers; and 12 focus group discussions and five in-depth interviews with the community-based agents and their supervisors.

Results:

Implementation was sub-optimal in both settings. In Pakistan issues were low field-supervision coverage and poor visit quality related to issues scheduling supervision, a lack of skill development, high workloads and competing priorities. In India, issues were low visit coverage - in part due to employing new workers and an empowerment approach to visit scheduling. Coaching caregivers to improve their skills was sub-optimal in both sites, and is likely to have contributed to caregiver perceptions that the intervention content was not new and was focused on play activities rather than interaction and responsivity - which was a focus of the coaching. In both sites caregiver time pressures was a key reason for low uptake among families who received visits.

Discussion:

Programs need feasible strategies to maximize quality, coverage and supervision including identifying and managing problems through monitoring and feedback loops. Where existing community-based agents are overstretched and system strengthening is unlikely, alternative implementation strategies should be considered such as group delivery. Core intervention ingredients such as coaching should be prioritized and supported during training and implementation. Given that time and resource constraints were a key barrier for families a greater focus on communication, responsivity and interaction during daily activities could have improved feasibility.

Introduction

The importance of early child development (ECD) interventions to improve short and long term outcomes and disrupt the intergenerational cycle of poverty is well recognized (1–5). ECD holds a central place in the Sustainable Development Goals, with multilateral support for implementation and scale up (6–9). Global guidelines recommend that all infants and children should receive responsive care and have early learning activities with their caregivers in the first 3 years of life (7). Interventions to achieve this include providing support for caregivers through individualized home visits and/or holding group sessions at home or in community spaces or through contacts at health facilities (5). In low and middle income settings interventions are most commonly community based and delivered by peer educators or community based agents (CBAs) who have some high school education and 1 to 2 weeks of ECD training (10).

We know from efficacy trials that community ECD interventions can improve child development outcomes in diverse settings, and can offset developmental risks such as poverty and malnutrition (5, 11, 12). However, evidence is lacking on effectiveness at scale, with a major gap in research on the ‘how to’ of ECD interventions (13–17). This lack of implementation evidence has been cited as a reason for the slow roll out of ECD programs in low and middle income countries (18). To support scale up we need to share implementation successes and failures and understand what processes and factors contribute to positive impacts, or lack of impact, in relation to the workforce, infrastructure needs, quality, coverage and demand (13, 16, 17, 19, 20).

To bridge the scalability knowledge-gap we implemented the SPRING program, which was designed from the outset to be feasible, affordable and appropriate for delivery at scale. SPRING designed and tested the impact of monthly home visits by CBAs to promote and support caregivers with responsive care, early learning and nutrition practices in India and Pakistan using a cluster randomized controlled trial design (21, 22). SPRING adapted WHO and UNICEF ECD and nutrition content (23, 24), and tested two delivery models for scale-up using CBAs: In Pakistan the intervention was called Roshan Kal (brighter tomorrow) and was integrated within the government Lady Health Workers program (25). In India the intervention was called Kilkaari (gurgling of a young child) and was delivered by a non-governmental organization (SANGATH) who employed a new cadre of intervention specific CBAs called Kilkaari workers (KWs). We refer to the Roshan Kal and Kilkaari programs collectively as the SPRING Program.

As reported in our companion paper, the SPRING Program did not achieve significant impacts on ECD or growth outcomes at 18 months of age (22). It is important to understand why this occurred, so that others can learn lessons for delivering community based ECD and nutrition interventions at scale. Embedded in the SPRING program was a process evaluation that aimed to understand implementation in relation to training, supervision, coverage, quality, acceptability and barriers and facilitators to behavior change among CBAs and families. In this paper we focus on the process evaluation findings that relate to ECD as this is where evidence is most lacking.

Materials and methods

The intervention

In Pakistan, SPRING was embedded in the government Lady Health Worker (LHW) program. The LHW program was established in 1994 and LHWs are well known and credible health workers in their communities. The LHW role comprises over 20 maternal and child health services including health education on breastfeeding and complimentary feeding, child growth monitoring, immunizations, family planning and basic curative care. Within the government program LHWs receive 15 months training, are aged 18–45 years, resident in the community they serve, have a minimum 8 years of education, are preferably married and are acceptable to the community. Each Union Council has 15–20 LHWs, each serving 150–200 households, they are supervised and monitored by a Lady Health Supervisor (25). LHWs conduct approximately 7 home visits per day and the SPRING content was integrated into these home visits.

In India the KWs were lay workers specifically identified, recruited, trained and supervised to only deliver the SPRING intervention. KWs were recruited by SANGATH with renumeration and characteristics similar to government frontline workers. They were resident in the community, had a minimum of 8th grade education, were married and had good communication skills. Each served a maximum of 100 eligible households. As new workers they had to identify eligible families so were asked to engage with local frontline workers and attend local community events such as Village Health Days to identify new pregnancies and mothers with young children.

In Pakistan SPRING was implemented in 10 predominantly rural Union Councils in Rawalpindi District by 150 LHWs; 70 of these LHWs and the children in their catchment areas were included in the SPRING outcome evaluation. In India, SPRING was implemented in three blocks of the predominantly rural Rewari district in Haryana State by 54 KWs. The intervention was delivered to all pregnant women and families with children 0–2 year of age. The content (Table 1) was adapted based on formative research into current behaviors, and barriers and facilitators for adopting the desired ECD and feeding practices (26). In pregnancy, home visits focused on maternal health and sensitization about breastfeeding. Postnatal visits focused on caregiver-child interaction and play activities, breastfeeding, complementary and responsive feeding, with new messages introduced each month dependent on the age if the child.

SPRING was delivered using a counseling approach based on Cognitive Behavior Therapy techniques (27) which includes: family support; guided discovery using pictures; behavioral activation; empathic listening; problem-solving; and praise. Structured pre-tested counseling cards provided pictures for families to look at and instructions and key messages for the CBAs. In each visit approximately 4 messages were given – e.g. in postnatal visits this was two on nutrition and two on play and interaction with the infant. In Pakistan routine program messages were integrated into these counseling cards and families were also provided with a reminder calendar. The visits targeted the primary caregiver, and included providing information, problem solving and family engagement. The primary caregiver practiced play activities and received coaching and demonstrations from the CBA to enhance skills, self-efficacy and recall in relation to play activities and responsivity. Community and health systems sensitization was also conducted.

CBAs received 8 days of skills based training, delivered in two phases to groups of 14–29 CBAs by SPRING supervisors – female social science graduates trained by a child development expert. Training included classroom sessions, participatory exercises, role play, discussions of videos and practical sessions with caregivers and children.

Monthly group supportive supervisory meetings were held in groups of 11–25 CBAs who shared experiences and supported each other, problem-solved and developed their skills. In Pakistan this was integrated into the LHW’s routine monthly supervision meetings. During monthly one-to-one supportive field supervision, supervisors observed home visits and used a supervisory checklist to feedback on performance, problem solve and make suggestions on areas for improvement. The CBA to supervisor ratio was around 1:10 in India and 1:15 in Pakistan.

Process evaluation data collection

Quantitative data

Details of the quantitative process data collection can be found in Table 2. In brief, training knowledge gain was determined using a pre- and post-training test, group supervision coverage from program attendance record and field supervision from completed supervisory checklists. Data on visit coverage data came from the SPRING surveillance system (22), implemented as part of the outcome evaluation, under which resident surveillance workers visited households every 8–10 weeks to collect outcome data. Data on visit quality were obtained from a supervisory checklist of 33–37 items, dependent on site and whether a pregnancy or child visit was being observed. The checklist covered counseling, family support, problem-solving and child stimulation with between 3 and 15 items for each of these domains. The checklist questions were not identical in each site, and to aid comparison we present a selection of overlapping indicators for each domain. Information on visit duration and the use of job aids was collected in a cross-sectional survey with caregivers.

Qualitative data

Data on acceptability of the intervention for families and the CBAs, and barriers to and facilitators for change were collected toward the end of implementation from four intervention clusters in each country. Data were collected by trained bilingual interviewers, who were not part of the implementation team. The clusters were selected to ensure a range of home visit coverage, and to reflect the diversity of the study area. Data were collected through narrative in-depth interviews (IDIs) with mothers and focus group discussions (FGDs) with mothers, grandmothers, and fathers. Supervisors were interviewed through FGDs in Pakistan and IDIs in India. Sample size was approximated based on when saturation had been reached in our previous studies in the area, and additional interviews and FGDs were added if needed. The methods, sample size and content are shown in Table 3.

IDI mothers were selected from the surveillance database to ensure that the sample had a variety of socio-demographic characteristics (age, education, parity, gender of the child, and socioeconomic status). Community FGDs participants were selected through community informants to have a range of parities, education levels and socio-economic status and stratified for homogeneity where mixed groups could inhibit participant interaction. Topic guides were developed in local languages and pilot tested before data collection. All Interviews and FGDs were audio recorded in addition to notes taken. Narratives and IDIs were turned into expanded notes using the audio recordings to add verbatim quotes (28). Debriefs were held regularly during data collection to provide feedback to interviewers and discuss themes and saturation. These discussions and the topic guides were used to develop a deductive coding template, with inductive codes and themes added during analysis. A framework analysis approach was used for the narrative IDIs, and thematic analysis using a phenomenological approach for the other IDIs and FGDs (29, 30). In the results when reporting on the individual countries we refer to LHWs and KWs and when reporting combined results we refer to CBAs.

Results

Training and supervision

Training increased overall ECD and nutrition knowledge scores among LHWs from 50 to 91%, and among KWs from 70 to 85%. Areas where knowledge was suboptimal post-training were targeted during individual and group supervision sessions. Group supervision coverage was high in both sites, with more than 75% of the monthly groups occurring and high attendance (see Table 4). In India mean monthly individual field supervision coverage was 91%, but only 23% in Pakistan, where each LHW had field supervision an average of 5.5 times (range 2–7 times) over the 24 months of implementation. Field supervision was made difficult in Pakistan as the number of days LHWs were available for supervision was limited due to polio activities, official meetings, training, maternal and child health week activities, measles campaigns, strikes and public holidays; with a new supervision round initiated only when all LHWs had been seen.

In both sites the CBAs valued SPRING supervision because it focused on feedback rather than on “checking,” allowed for collective problem solving and enhanced relationships with each other and their status in the community:

“In Roshan Kal [SPRING] supervision, LHWs are like a family and there is no hesitation, where as in the Lady Health Supervisor meetings they feel fear from ways of checking” [LHW FGD Pakistan]

“[The] Supervisor’s way of giving feedback is good. It does not hurt us. In fact we like it…. in group meeting supervisor asks everyone...so we get the solution from each other” [KW FGD India]

“Actually feedback was appropriate…they [KWs] never defended…..they have taken the feedback in a positive way” [Supervisor IDI India]

Coverage of visits

Based on data from the caregiver surveillance system, most caregivers were visited by a CBA at least once (>95%) (see Table 4), indicating that CBAs were able to locate families. In Pakistan the mean monthly visit coverage was 92% (monthly range 78–98%), but this was for any LHW visits including, for example, polio visits. In India only 30% of caregivers reported a visit in the last month (monthly range 20–45%), with coverage similar by age of the child and by socio-economic status but varied by KW cluster (ranging from 12 to 50%).

Supervisors in India identified 43% of KWs as either generally poor performers in relation to coverage or of facing issues such as a high load of migrant population, being replacement workers, lack of family support, taking sick leave and long travel distances. The KWs themselves did not perceive coverage as an issue, but when asked about reasons for low coverage they reported isolated households, the lack of safety in certain neighborhoods and a large workload reducing their ability to make frequent visits. Some households refused visits either because a household elder did not want the visit to occur, the mother themselves did not want it or because visits were not seen as needed. It was harder to find mothers during harvest time:

“Respondent 8: Maybe her [KWs] family refuses her or she has less time

Respondent 6: She doesn’t get family support or her mother in law is upset

Respondent 3: Maybe she is busy in other work

Respondent 8: Maybe she has young children…. She isn’t able to manage the time” [KW FGD India]

“The mother does not allow outsiders [to] visit her house. She doesn’t go anywhere- always stays at home” [KW FGD India]

“During harvesting, mother goes in field – some don’t go but many of them had to go… fewer visits happen” [KW FGD India]

KWs reported that they enjoyed conducting visits, particularly because of the identity and status that the role gave them: “We have got a good identity in the houses. Earlier we did not used to go out, no one knew us. Now everyone knows” [KW FGD India]. For some, it was their first paid employment outside the home and they described the role as transformative in terms of increasing their knowledge of the outside world and their ability to navigate it: “I never went to the village alone but after I joined here…. I do not hesitate to go anywhere in the village” [KW FGD India]. The financial independence the role gave them was also key for some workers: “My husband drinks a lot. With the salary I pay children’s school fee and manage household expenses. …Earlier I used to cry and stayed at home….. Now I am managing everything by my own” [KW FGD India].

Quality of visits

In India, home visits were SPRING specific, so if a home visit occurred we can assume that SPRING content was covered. In Pakistan SPRING content was added to existing visits and the high visit coverage may not reflect high SPRING content coverage. In the Pakistan endline survey (n = 120) 36% of mothers reported that their last visit was under 10 min long and only 29% that counseling cards were used, suggesting that coverage of SPRING content was most likely suboptimal. The SPRING protocol was more closely followed in India with women reporting longer visit lengths and 78% reporting that counseling cards were used in every visit.

Table 5 shows data from the field supervision observation checklist. Observed counseling skills were high in both settings, with the exception in Pakistan of using positive words and gestures, which was only observed in 53% of the supervision visits. In Pakistan the LHWs reported in the qualitative interviews an improvement in their overall approach to counseling: “In the past, we just asked about a behavior, like whether mother has breastfed or not, now we explain in detail and tell the advantages of it and discuss the difficulties as well” [LHW FGD Pakistan]. For the KWs their new counseling skills were also valued with impacts on their self-confidence: “We have learnt a way of talking, developed self-confidence among people. So, I really liked to work here” [KW FGD India].

The use of counseling cards during observed supervision visits was high in both settings, but visit fluency was much lower in Pakistan than India (48% observed as fluent compared with 88% in India), and in Pakistan all relevant topics were covered in only 28% of observations. Family engagement and problem solving were also lower in Pakistan than India as were ECD and coaching skills, with guiding, prompting and modelling particularly low in Pakistan (31–52%). In India there were striking improvements in ECD related skills over time. For example, guiding the mother to be responsive increased over the course of the intervention from 24% in the first supervision round to 75% in the final round, guiding to follow the child’s lead increased from 19 to 47% and modelling behaviors from 35 to 79%. In Pakistan only observing the caregiver trying the activity (increased from 85 to 91%), prompting the caregiver to encourage the child (40–66%) and modelling behaviors (20–49%) increased over time, while skills related to family involvement, showing counseling cards and problem solving decreased over time in Pakistan.

Mirroring the quantitative results, caregivers in the qualitative interviews reported better use of counseling cards and the provision of messages in India compared to Pakistan. For example, in India most caregivers reported that counseling cards were used in the visits, that they were asked to describe the pictures which were then explained by the KW and they recognized most of the counseling cards. Caregivers less frequently reported that they were asked to try play activities with the KW present. In Pakistan caregivers reported that counseling cards were rarely used and that the play component was often limited to general advice to play more or a verbal description of an activity to try:

“LHW didn’t tell any proper play activity .... told them [family] to give time to child” [Mother FGD Pakistan]

“LHW told us to play with the child to make her intelligent.... LHW also advised us to give blocks to the children.” [Mother IDI Pakistan]

Both the LHWs and the supervisors corroborated the mothers’ descriptions of visits, reporting that the use of counseling cards and coaching/demonstrations were rarely done. There were some exceptions where mothers described being told about new activities with counseling cards and/or being guided to try them:

“I gave cups and glasses to my son and taught him to recognize things. This was new for me and I didn’t know about it when I had my older child. LHW explained these activities to me through pictures” [Mother narrative Pakistan]

Barriers to conducting high quality visits

In the qualitative interviews, CBAs in both settings recognized the importance of appropriate counseling, family engagement, praise, and coaching and could describe high quality visits, reflecting that they knew what they should be doing: “LHW should pay greetings, introduction of Roshan Kal [SPRING], family involvement, health calendar follow up, follow up of the play activity, which benefits they got from these messages, show pictures, ask about the pictures, tell about the picture, asking about problems, keep involving family and to conduct the play activity” [LHW FGD Pakistan].

The CBAs felt that new skills, especially ECD skills, took time to develop: “We like the child play activities, but we only came to know about these through Roshan Kal [SPRING] trainings, this is something new and we will take some time to do these perfectly” [LHW FGD Pakistan]. The lower quality visits in Pakistan were mainly attributed to a lack of LHW time due to competing activities such as polio and dengue, delivery referrals, mother and child weeks and trainings and meetings: “We want to, but we cannot deliver Roshan Kal [SPRING] visits as planned because of program activities” [LHW FGD Pakistan]. SPRING content was less of a priority than activities that had a direct impact on their salaries or were monitored by the program: “We have to bring delivery referrals to the health unit, which takes a lot of our time. If we do not do that our salaries are deducted” [LHW FGD Pakistan]. There were also issues around the LHWs assuming they did not need to focus on play in educated families or where families reported they already played with their children, caregivers being busy or uninterested, children being asleep, absent or not in the right mood and LHWs being embarrassed to coach:

“LHWs don’t conduct play activities because of workload and because they think verbal messages are enough” [LHS FGD Pakistan]

“If mother says that she is doing, they believe in her and end the visit” [LHW FGD Pakistan]

“Respondent 1: People make fun of her, mothers says that they are already doing all this

Respondent 2: People start laughing when LHW tell them about play....

Respondent 3: People say what is this new thing?” [LHW FGD Pakistan]

In India, the KWs were SPRING specific and so had no competing activities. The main quality issues were related to inconsistently demonstrating, modelling and promoting ECD behaviors, mainly attributed to caregivers not having time or the KW perceiving them as being busy and to the KWs being poorly received:

“We [KWs] give enough time to the mother, but mother is in hurry” [KW FGD India]

“If mother was doing some work …and left it - then mother-in-law gets angry for not completing the chore first. Sometimes they get angry in front of us” [KW FGD India]

“Her husband saw us and went outside. Next time he came in…. he started grumbling. The husband got angry that she [mother] hadn’t cooked the chapatti yet.” [KW FGD India]

When mothers describe the KW visits, they more frequently reported that the KW asked them to show them the activity they had been doing previously rather than trying and being coached on the new activity, this was seen as “checking” what they had done. This “checking” was not included in the KW counseling cards, which focuses on trying and coaching around the new activity, but it was on the supervisor checklist.

Acceptability of visits

Overall, the CBAs were accepted and trusted by most families they visited. In both sites they were described as “patient,” “helpful,” and “supportive” and gave ‘good’ and ‘right’ information ‘nicely’, ‘calmly’ and ‘jovially’. Most mothers described the visits as being pleasurable and valued that someone else was concerned about their child: “There was another person to care about upbringing the child. We look after the child but when she comes, I feel different – it feels good” [Mother FGD India].

In Pakistan the LHWs were long standing workers who were well know and trusted in their communities. In India the KWs were newly recruited and trust and recognition took time to develop: “Initially when we used to go for a visit, mother did not pay attention…they used to ignore [what we said]. But now… they listen to us carefully and also give respect saying sit for a while, have tea…” [KW FGD India]. By the end of the intervention acceptance was high, but both community respondents and the KWs reported that it was not universal with some families finding their KW too young and inexperienced and that visits imposed on their time.

Families liked the counseling cards and, in Pakistan, the calendar. Families found them salient and felt they enhanced understanding: “We had some knowledge and some we gained through pictures… It was easy to understand through pictures” [Mother FGD India]…. “When I looked at it, I said well this is my situation” [Mother IDI Pakistan]. The benefit of using counseling cards was echoed by the KWs, especially for illiterate women: “Without picture they will not have understood so much. They understood more through pictures” [KW FGD India].

Grandmothers often looked after the child and SPRING aimed to encourage fathers and other family members to support the mother and interact with the child, but as seen in the supervisor checklist data family engagement in the visits was low in both sites. Family members were not always at home, but the qualitative data also showed that some grandmothers and fathers were reluctant to be involved in the visits due to time constraints or because they did not feel the visits were important for them, and for fathers in India because it would mean the KW would have to wear a veil during the visit:

“Sometimes it is only the mother-in-law who plays with the child….. Mother ….. says, tell mother-in-law, let mother-in-law try the activity …. mother-in-law says, tell her [mother] only. I am uneducated. I don’t know” [KW FGD India]

"Has to veil her face from male members of the house … that’s why male members also don’t join" [Mother narrative India]

In Pakistan LHWs said another reason they did not involve families was because this added time to their visits and because not all of the content of their routine visits was appropriate for other family members: “They could not involve other family members because of workload …. they just told messages to mothers mostly and do not involve family members” [LHW FGD Pakistan].

Barriers and facilitators to family uptake

In both sites most caregivers reported that the visits had increased the amount of attention they gave their child and their knowledge of the importance of play, but families rarely reported that they had done the play activities routinely and frequently. Most reported that they did each activity once or twice or for a few days.

Among those who did the activities regularly the main drivers were trusting the CBAs, knowledge of the benefits of play and positive experiences/outcomes such as enjoyment by child and mother and pride that a child could to the activity:

“We thought if KW is saying then we can try..... they are saying for some benefit” [Mother narrative India]

“The child will be more intelligent and will pick quickly when she will join school” [Mother narrative Pakistan]

“She plays 2-3 times a day with child and the child enjoys a lot and she can also feel the child’s happiness from his face” [Mother IDI Pakistan]

“When I make her learn and she does, then I feel like doing more things with her. I feel very happy” [Mother narrative India]

Barriers to family uptake are shown in Table 6, with illustrative quotes. In both sites all respondent groups reported caregiver time pressures as a key reason ECD activities were not done. Caregivers were often in charge of most domestic chores, including cooking, heating up water for baths, hand washing laundry, care of the domestic livestock and harvesting twice a year. Time pressures lead to other household members looking after the child who, as described above, were infrequently engaged in the visits: “After doing all chores, I think to make child sleep and take rest. Mother-in-law and father-in-law are there. They play a lot with the child” [Mother FGD India].

Few families reported the potential for play and interaction to have long term effects, instead focusing on the child learning a new ability and the activity keeping them happy or distracted. Few felt that the CBAs was suggesting a new behavior, with play something that was already done, and caregivers focused their descriptions of the visits on the play and play activities being promoted rather than the responsivity and interaction that doing the activity was aiming to enhance. Other barriers were beliefs that the child would learn in their own time without interactive play, that families desired children to play independently so they can occupy themselves, and a perception among some caregivers that their child was too young for the suggested activity, or conversely that their child could do the activity easily.

The key barriers to coverage quality and family uptake are summarized in Figure 1.

Figure 1

Discussion

The SPRING intervention was based on existing global guidelines, was designed using best practice in relation to formative research and counseling practices and included elements considered key to the success of ECD interventions such as a structured curriculum, actionable messages, opportunities for caregivers to practice activities and get feedback and coaching, demonstration, problem solving, skills based training and supportive supervision (10, 23, 24, 26, 27). SPRING was designed from the outset to be feasible, affordable and appropriate for delivery at scale, and thus to answer key questions about effectiveness at scale and implementation processes. Using a rigorous cluster RCT design we found no impact on ECD or growth outcomes neither when SPRING was integrated into the Lady Health Worker program in Pakistan nor when it was delivered by intervention specific NGO workers in India (22). It is essential to identify what could have caused this lack of impact given the consistent impact seen in efficacy trials (5) and the call for ECD interventions to be scaled up (7).

Key implementation issues were low supervision coverage and poor fidelity in Pakistan, and low visit coverage in India. In addition, the coaching element of the intervention was sub-optimal in both sites, which is likely to have contributed to caregivers feeling that the intervention was not promoting anything new and that visits focused on play and play activities rather than on the interaction and responsivity - which was to be the focus of the coaching. This is important as interventions that promote responsivity have a greater impact than those that focus only on play and learning materials (5).

Supervision of community health workers is often weak and under-supported with low and irregular coverage and an administrative focus (31). We were able to implement a supportive group supervision system in both settings which was well received by the CBAs. Individual field supervision focused on improving the quality of the visits through feedback and problem solving, but coverage was very low in Pakistan, which may in part explain the low content fidelity. In India individual supervision coverage was high and we saw ECD skills starting low and improving over time, this was seen to a lesser extent in Pakistan. Other studies have also found that skills take time to develop as workers need to become familiar with the new approach and be supported through supervision in order to complete all the objectives and cover key concepts in their visits (13, 32). The low supervision coverage in Pakistan was in part due to the realities of the LHW schedules and commitments, which needed to be taken into account more fully when designing the supervision system. Other home visit ECD interventions in Pakistan using LHWs have had good supervision coverage, so with the right system high coverage is achievable (13). SPRING field supervision in Pakistan required a better balance between ensuring that all LHWs were seen in a supervision round (leading to potentially long gaps between rounds), this could have been achieved through conducting monthly supervision for all available LHWs, but with a special focus on those missed the previous month.

In many settings it may be impossible for existing CBAs to cover the population in need (33–35) and new workers may need to be deployed. Utilizing an NGO model, with project specific CBAs who have no competing work tasks and carefully delineated catchment areas, does not guarantee adequate coverage with only 30% of eligible women in India reporting a home visit in the last month. The low KW visit coverage in India was in contrast to the 96% coverage achieved by the SPRING Surveillance workers in the same sites. KW and Surveillance workers were similar in their education, pay, catchment area and access to transport. Their management structures differed with the Surveillance workers having independent checks on their coverage and performance and a more structured approach to scheduling their visits with mothers. The KWs, following an empowerment model, had more flexibility in scheduling their visits and fewer independent checks. A program review of 10 country experiences of scaling up CBA postnatal home visits demonstrate the difficulty of achieving high coverage at scale, with most countries achieving less than 10% coverage and none achieving more than 20% coverage (36); the magnitude of this challenge should not be overlooked. Programs need strategies to maximize coverage, including having a strategy for scheduling visits and identifying and managing poor performers through monitoring and feedback loops for course correction.

The experience of scaling up Crianca Feliz in Brazil, which found no impact on a range of ECD outcomes (37), was that rapid scale up was a barrier to achieving quality and consistency; this is an important example that coverage should not be prioritized at the expense of quality (38). In Pakistan LHW coverage was impressively high, but visit quality was low. As described above this was likely in part due to poor field supervision, but interacting with this were issues of integrating additional ECD tasks into the LHW workload. Resonating with our findings other studies have found that workforce constraints such as limited staff, high workloads, competing tasks seen to be of higher priority (e.g., immunizations), incentives for other work, large catchment areas and lack of supervision and remuneration structures inhibited coverage and quality visit by CBAs (32–35). There are clear synergies of integration in relation to efficiencies, having a worker who can provide holistic care and understands the needs of the whole child and family (20), but where existing CBAs have low coverage, are overstretched and systems strengthening is unlikely then alternative implementation strategies included in the WHO global guidelines should be considered such as group delivery, primary health care contacts or a mixture of these (7).

The LHW program was going through a difficult period during SPRING implementation which was likely to have influenced LHW motivation and ability to take on new tasks. This included fallout from the abolition of the Federal Ministry of Health and the devolution of activities to provinces in 2011, and regularization of LHWs in 2012 (39–42). The SPRING study team recorded resulting changes in the program that affected implementation which included: an increase in the catchment population of LHWs from 1,500 to 2000 after regularization, 24-h delivery services at Basic Health Units leading to increased workload for LHW as they were to accompany women to the facility, difficulties for SPRING staff engaging with the provincial government, and delays in the disbursement of salaries to LHWs.

In both sites the qualitative data suggest that coaching and modelling of behaviors was low, and that caregivers saw the intervention to be related to play and play activities rather than to responsivity and interaction. Core intervention ingredients need to be made clear to programs for prioritization during training and implementation and steps put in place to fully support these, especially where they are time consuming and may be the first thing to be dropped by busy workers. For example, the practical coaching of families in responsive care has been identified as a critical component of successful interventions, but this task takes time and skills and may be new to CBAs. We saw that in India, with frequent field supervision, KWs ECD skills increased over time and that acceptability also took time to develop–KW workers would have benefitted from a longer embedding period before the child outcomes were evaluated.

Mothers in the qualitative study reported a change in the attention they gave their children and as has been found in other studies most mothers liked the visits and enjoyed the play activities (13, 32, 34, 38). But, they rarely reported doing the play activities routinely and frequently. Time and resource constraints were a key barrier and although several activities were based around everyday activities a greater focus on communication, responsivity and interaction during daily activities could have improved feasibility. Few caregivers reported any perceived long-term benefit of play or saw the intervention as promoting anything new – perhaps related to the lack of coaching and modelling – and thus they had inadequate motivation to engage in the activities given their busy lives. Future programs need to ensure that interventions include a strong motivational element, ensure interaction and responsivity come across strongly and clearly communicate how the behaviors being promoted are different from existing behaviors. Utilizing approaches such as Trials of Improved practices (43), could provide tangible evidence of the impact of barriers such as time and resources and how these could be overcome.

Although the interventions were based on formative research, we adopted a relatively linear approach to implementation. Given the complexities of the contexts a pilot and having on-going adaptation, improvement and responsive feedback built into implementation would have given us the opportunity to refine the intervention and maximize impacts and should be built into future programs (44, 45).

The study had several strengths in that we could utilize coverage data from a rigorous surveillance system and use this to select a wide range of respondents for the qualitative component. The results of the supervision checklist show skills under observation which are likely to be CBAs’ best rather than their routine way of working, but this data provides evidence of the CBAs’ core competencies and highlights changes over time. The low supervision coverage in Pakistan meant we had fewer checklists than we had for India, and CBAs with more supervisory visits are over-represented in the data. There was good agreement around what typical home visits looked like both between data collection methods and between respondent groups which suggests our overall findings are valid. We were not able to accurately determine the content of the LHW home visits to identify the extent to which SPRING messages were delivered, but evidence from multiple sources suggest that fidelity was low. We utilized several qualitative methods and a variety of respondents enabling us to gain different points of views and experiences and to triangulate our findings. In the qualitative component transferability was enhanced by having multiple study sites, purposive sampling, sampling to saturation and reflexivity among those collecting and analyzing the data. Despite this the findings may not be transferrable to settings with very different contexts and there is likely to have been some social desirability bias (46, 47).

In conclusion, we identified several factors that could have improved implementation including: maximizing coverage and supervision by improving how visits and field supervision are scheduled and monitored through the use of feedback loops and course correction, using alternative delivery strategies such as group meetings when delivery by existing CBAs is unlikely to be feasible, identifying and focusing on core intervention components such as coaching during training and supervision, acknowledging that ECD skills are new for CBAs and take time and support to develop and a focus on barriers to family uptake such as integrating ECD into daily activities and highlighting the long term impact of the behaviors and how they are different from existing interaction and play. This can be achieved by building in on-going adaptation, improvement and responsive feedback into implementation to refine the intervention and maximize impacts rather than taking a linear approach to implementation.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1.

  LuCBlackMMRichterLM. Risk of poor development in young children in low-income and middle-income countries: an estimation and analysis at the global, regional, and country level. Lancet Glob Health. (2016) 4:e916–22. Epub 2016/10/09. doi: 10.1016/s2214-109x(16)30266-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 2.

  BlackMMWalkerSPFernaldLCHAndersenCTDiGirolamoAMLuCet al. Early childhood development coming of age: science through the life course. Lancet. (2017) 389:77–90. doi: 10.1016/S0140-6736(16)31389-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3.

  Grantham-McGregorSCheungYBCuetoSGlewwePRichterLStruppBet al. Developmental potential in the first 5 years for children in developing countries. Lancet. (2007) 369:60–70. doi: 10.1016/S0140-6736(07)60032-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  RichterLMDaelmansBLombardiJHeymannJBooFLBehrmanJRet al. Investing in the Foundation of Sustainable Development: pathways to scale up for early childhood development. Lancet (London, England). (2017) 389:103–8. doi: 10.1016/S0140-6736(16)31698-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5.

  JeongJFranchettEERamos de OliveiraCVRehmaniKYousafzaiAK. Parenting interventions to promote early child development in the first three years of life: a global systematic review and meta-analysis. PLoS Med. (2021) 18:e1003602. doi: 10.1371/journal.pmed.1003602

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  United Nations. Transforming our world: The 2030 agenda for sustainable development [online]. New York: United Nations (2015).

  • Google Scholar

• 7.

  World Health Organization. Improving early childhood development: Who guideline. Geneva: World Health Organization (2020).

  • Google Scholar

• 8.

  KuruvillaSBustreoFKuoTMishraCKTaylorKFogstadHet al. The global strategy for Women's, Children's and Adolescents' health (2016-2030): a roadmap based on evidence and country experience. Bull World Health Organ. (2016) 94:398–400. doi: 10.2471/blt.16.170431

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9.

  World Health Organization. Nurturing Care for Early Childhood Development: A framework for helping children survive and thrive to transform health and human potential. Geneva: World Health Organization (2018).

  • Google Scholar

• 10.

  YousafzaiAKAboudF. Review of implementation processes for integrated nutrition and psychosocial stimulation interventions. Ann N Y Acad Sci. (2014) 1308:33–45. Epub 2014/01/01. doi: 10.1111/nyas.12313

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11.

  AboudFEYousafzaiAK. Global Health and development in early childhood. Annu Rev Psychol. (2015) 66:433–7. doi: 10.1146/annurev-psych-010814-015128

  • CrossRef
  • Google Scholar

• 12.

  BrittoPRLyeSJProulxKYousafzaiAKMatthewsSGVaivadaTet al. Nurturing care: promoting early childhood development. Lancet. (2017) 389:91–2. doi: 10.1016/s0140-6736(16)31390-3

  • CrossRef
  • Google Scholar

• 13.

  YousafzaiAKRasheedMASiyalS. Integration of parenting and nutrition interventions in a community health program in Pakistan: an implementation evaluation. Ann N Y Acad Sci. (2018) 1419:160–8. doi: 10.1111/nyas.13649

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  BanerjeeABrittoPRDaelmansBGohEPetersonS. Reaching the dream of optimal development for every child, everywhere: what do we know about 'How To'?Arch Dis Child. (2019) 104:S1–2. doi: 10.1136/archdischild-2019-317087

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15.

  YousafzaiAKAboudFENoresMKaurR. Reporting guidelines for implementation research on nurturing care interventions designed to promote early childhood development. Ann N Y Acad Sci. (2018) 1419:26–37. doi: 10.1111/nyas.13648

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  BrittoPRSinghMDuaTKaurRYousafzaiAK. What implementation evidence matters: scaling-up nurturing interventions that promote early childhood development. Ann N Y Acad Sci. (2018) 1419:5–16. doi: 10.1111/nyas.13720

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  LombardiJ. What policymakers need from implementation evaluations of early childhood development programs. Ann N Y Acad Sci. (2018) 1419:17–9. doi: 10.1111/nyas.13661

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18.

  TomlinsonMDarmstadtGLYousafzaiAKDaelmansBBrittoPGordonSLet al. Global research priorities to accelerate programming to improve early childhood development in the sustainable development era: a Chnri exercise. J Glob Health. (2019) 9:020703. doi: 10.7189/jogh.09.020703

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  DuaTTomlinsonMTablanteEBrittoPYousfzaiADaelmansBet al. Global research priorities to accelerate early child development in the sustainable development era. Lancet Glob Health. (2016) 4:e887–9. doi: 10.1016/S2214-109X(16)30218-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  HurleyKMYousafzaiAKLopez-BooF. Early child development and nutrition: a review of the benefits and challenges of implementing integrated interventions. Adv Nutr. (2016) 7:357–3. doi: 10.3945/an.115.010363

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21.

  ClinicalTrials.gov. Spring Cluster Randomised Controlled Tria. Available at: https://clinicaltrials.gov/ct2/show/NCT02059863 (2018). Last accessed March 31st 2023).

  • Google Scholar

• 22.

  KirkwoodBSikanderSRoyRSoremekunSBhopalSAvanBet al. Effect of the Spring Home Visits Intervention on Early Child Development and Growth in Rural India and Pakistan: Parallel Cluster Randomised Controlled Trials. Frontiers in Nutrition. (In press)

  • Google Scholar

• 23.

  LucasJERichterLMDaelmansB. Care for Child Development: an intervention in support of responsive caregiving and early child development. Child Care Health Dev. (2018) 44:41–9. doi: 10.1111/cch.12544

  • CrossRef
  • Google Scholar

• 24.

  World Health Organization. Global strategy for infant and young child feeding. Geneva: WHO (2003).

  • Google Scholar

• 25.

  HafeezAMohamudBKShiekhMRShahSAJoomaR. Lady health workers Programme in Pakistan: challenges, achievements and the way forward. J Pak Med Assoc. (2011) 61:210–5. Epub 2011/04/07.

  • Google Scholar

• 26.

  LingamRGuptaPZafarSHillZYousafzaiAIyengarSet al. Understanding care and feeding practices: building blocks for a sustainable intervention in India and Pakistan. Ann N Y Acad Sci. (2014) 1308:204–7. Epub 2014/01/08. doi: 10.1111/nyas.12326

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  ZafarSSikanderSHaqZHillZLingamRSkordis-WorrallJet al. Integrating maternal psychosocial well-being into a child-development intervention: the five-pillars approach. Ann N Y Acad Sci. (2014) 1308:107–7. Epub 2014/02/28. doi: 10.1111/nyas.12339

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  HillZTawiah-AgyemangCKirkwoodBKendallC. Are verbatim transcripts necessary in applied qualitative research: experiences from two community-based intervention trials in Ghana. Emerg Themes Epidemiol. (2022) 19:5. doi: 10.1186/s12982-022-00115-w

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29.

  BraunVClarkeV. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3:77–1. doi: 10.1191/1478088706qp063oa

  • CrossRef
  • Google Scholar

• 30.

  GaleNKHeathGCameronERashidSRedwoodS. Using the framework method for the analysis of qualitative data in multi-disciplinary Health Research. BMC Med Res Methodol. (2013) 13:117. doi: 10.1186/1471-2288-13-117

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31.

  HillZDumbaughMBentonLKällanderKStrachanDten AsbroekAet al. Supervising community health Workers in low-Income Countries--a Review of impact and implementation issues. Glob Health Action. (2014) 7:24085. doi: 10.3402/gha.v7.24085

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  SmithJABaker-HenninghamHBrentaniAMugweniRWalkerSP. Implementation of reach up early childhood parenting program: acceptability, appropriateness, and feasibility in Brazil and Zimbabwe. Ann N Y Acad Sci. (2018) 1419:120–12. doi: 10.1111/nyas.13678

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33.

  GladstoneMPhukaJThindwaRChitimbeFChidzaloKChandnaJet al. Care for child development in rural Malawi: a model feasibility and pilot study. Ann N Y Acad Sci. (2018) 1419:102–9. doi: 10.1111/nyas.13725

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34.

  JeongJBliznashkaLAhunMNKaruskina-DrivdaleSPicoloMLalwaniTet al. A pilot to promote early child development within health Systems in Mozambique: a qualitative evaluation. Ann N Y Acad Sci. (2022) 1509:161–3. doi: 10.1111/nyas.14718

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35.

  BrentaniAWalkerSChang-LopezSGrisiSPowellCFinkG. A home visit-based early childhood stimulation Programme in Brazil-a randomized controlled trial. Health Policy Plan. (2021) 36:288–7. doi: 10.1093/heapol/czaa195

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 36.

  McPhersonRHodginsS. Postnatal home visitation: lessons from country programs operating at scale. J Glob Health. (2018) 8:010422. doi: 10.7189/jogh.08.010422

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  SantosISMunhozTNBarcelosRSBlumenbergCBortolottoCCMatijasevichAet al. Evaluation of the happy child program: a randomized study in 30 Brazilian municipalities. Cien Saude Colet. (2022) 27:4341–63. doi: 10.1590/1413-812320222712.13472022

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38.

  BucciniGVenancioSIPérez-EscamillaR. Scaling up of Brazil's Criança Feliz early childhood development program: an implementation science analysis. Ann N Y Acad Sci. (2021) 1497:57–73. doi: 10.1111/nyas.14589

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  PervaizFShaikhBTMazharA. Role of development Partners in Maternal, newborn and child health (Mnch) programming in post-reform times: a qualitative study from Pakistan. BMJ Open. (2015) 5:e008665. doi: 10.1136/bmjopen-2015-008665

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40.

  ClosserSJoomaR. Why we must provide better support for Pakistan’s female frontline health workers. PLoS Med. (2013) 10:e1001528. doi: 10.1371/journal.pmed.1001528

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41.

  ZaidiSABigdeliMLangloisEVRiazAOrrDWIdreesNet al. Health Systems Changes after Decentralisation: Progress, Challenges and Dynamics in Pakistan. BMJ Global Health. (2019) 4:e001013. doi: 10.1136/bmjgh-2018-001013

  • CrossRef
  • Google Scholar

• 42.

  ZhuNAllenEKearnsACagliaJAtunR. Lady health Workers in Pakistan: Improving access to health Care for Rural Women and Families. Boston, MA: Maternal Health Task Force (2014).

  • Google Scholar

• 43.

  The Manoff Group. Trials of improved practices (tips): Giving participants a voice in program design. Washington: The Manoff Group (2005).

  • Google Scholar

• 44.

  ChambersDAGlasgowREStangeKC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. (2013) 8:117. doi: 10.1186/1748-5908-8-117

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45.

  ViswanathKSynowiecCAghaS. Responsive feedback: towards a new paradigm to enhance intervention effectiveness. Gates Open Res. (2019) 3:781. doi: 10.12688/gatesopenres.12937.1

  • CrossRef
  • Google Scholar

• 46.

  Finfgeld-ConnettD. Generalizability and transferability of meta-synthesis research findings. J Adv Nurs. (2010) 66:246–4. doi: 10.1111/j.1365-2648.2009.05250.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47.

  PolitDFBeckCT. Generalization in quantitative and qualitative research: myths and strategies. Int J Nurs Stud. (2010) 47:1451–8. doi: 10.1016/j.ijnurstu.2010.06.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar