Considerations for Monitoring School Health and Nutrition Programs

Schultz, Linda; Ruel-Bergeron, Julie

doi:10.3389/fpubh.2021.645711

COMMUNITY CASE STUDY article

Front. Public Health, 16 July 2021

Sec. Public Health Education and Promotion

Volume 9 - 2021 | https://doi.org/10.3389/fpubh.2021.645711

This article is part of the Research Topic Covid-19: School and Community Feeding Programs for Children and Young People View all 6 articles

Considerations for Monitoring School Health and Nutrition Programs

$\nLinda Schultz$ Linda Schultz

Julie Ruel-Bergeron^*

Global Financing Facility, Health, Nutrition and Population Global Practice, World Bank, Washington, DC, United States

School health and nutrition (SHN) interventions are among the most ubiquitous public health investments and comprise a key mechanism for reaching populations that are otherwise difficult to reach through the health system. Despite the critical role of monitoring these multisectoral programs to enable data-informed adaptive programming, information to guide program implementers is scant. This manuscript provides an overview of how monitoring indicators can be selected across a SHN program's logical framework, with specific examples across five SHN implementation models. Adaptation of SHN programs in times of school closures, such as those currently being experienced globally due to the emergence of COVID-19, is also addressed. Key aspects of SHN program monitoring are explored, including: (1) why monitor; (2) what to measure; (3) how to measure; and (4) who measures. In situations of school closures, strategies to shift both program activities and corresponding monitoring mechanisms are critical to understanding the rapidly evolving situation and subsequently guiding policy actions to protect vulnerable populations.

Introduction

School health and nutrition (SHN) interventions are among the most ubiquitous public health investments worldwide, with more than 100 countries offering school-based or school-linked routine health and nutrition services (1). SHN has become a common intervention due to the broad recognition that healthy students learn better. In addition, school-based service delivery is compelling to both the health and education sectors: this approach reaches populations that traditionally have little contact with health facilities and targeted health interventions create a more level playing field for vulnerable students to benefit from the existing investments in education (2). Although SHN services have historically targeted primary schools, variations in age at school entry and late enrollment enable this platform to reach at a minimum, young adolescents as well (3). In addition, rising rates of secondary school enrollment globally (4) make the school platform yet again an attractive mechanism for reaching adolescents with health and nutrition services as they enter critical periods of physical and socio-emotional development, and sexual debut.

Well-designed SHN interventions align with critical periods of development and address social determinants of health for school-age children and adolescents (5). SHN interventions can include routine health service delivery and should be complemented with improved water, sanitation, and hygiene (WASH) infrastructure, and related messaging within existing curricula. The benefits and breadth of SHN interventions to improve human capital formation have been well-summarized elsewhere (1, 2, 6). Delivering school health services alone, however, is not sufficient to improve health or learning outcomes; rather, the quality, consistency, and relevance of services and delivery modalities is critical to achieve their intended outcomes. Furthermore, the COVID-19 pandemic has revealed the importance of having “back-up mechanisms” for the delivery of school health and nutrition services that can quickly pivot to meet the needs of school-going children and adolescents in periods of prolonged school closures.

As with any development program, it is important to monitor the implementation of SHN services, particularly as these represent significant investments. As an example, an estimated USD 41–43 billion is spent annually on school feeding programs alone (7). Program monitoring is deeply engrained in programmatic preparation and implementation, serving as a tool to track implementation progress against its goals through frequent or routine collection of process and output-based indicators throughout the life of the project's implementation. To set the stage for program monitoring, implementers should agree on: (1) the problem the project is trying to solve; (2) how the project inputs will lead to desired outcomes; (3) the type of evidence needed to assess progress toward program results; and (4) the existing data sources and instruments available in the country.

There are varying approaches to deliver and monitor school health and nutrition services, and the range of relevant interventions to deliver can vary widely across contexts. As a result, there is no blueprint for delivering and monitoring high quality and equitable SHN programming and each monitoring system will be unique to the programs, interventions, and context. Carefully selected and measurable indicators help project planners and implementers assess intervention quality, inform mid-course revisions where needed, and ensure resources are being used effectively.

The multisectoral nature of SHN interventions requires further careful planning and coordination to meaningfully capture monitoring data, as there are two or more, as is often the case for school feeding, sectors and stakeholders (such as parents and pupils themselves) to engage. This principle remains true even for monitoring interventions that have long engaged multiple sectors for school-based delivery, such as deworming, as there may be limited coordination between information systems to facilitate the transfer of information from one to the other. These governance and accountability challenges mean that countries may have limited ability to monitor implementation progress to inform future service delivery (8).

This manuscript presents key principles for SHN program monitoring and provides broad guidance aimed at supporting the development and implementation of monitoring mechanisms within SHN programs. The discussion includes specific examples of indicators used to monitor the delivery of SHN interventions and considers monitoring responsibility and information flows between education and health sectors at the national, subnational, and individual level. Emphasis is also placed on governance and accountability of monitoring processes, such as responsibility of collecting data, and information flows between the health and education sectors to maximize data-informed decision-making. Although program monitoring is often complemented by program evaluation, this manuscript focuses on monitoring, given the paucity of information on the topic.

Monitoring SHN in Countries and Innovation During and Following Prolonged School Closures

Monitoring Responsibility

Data collection for SHN services are often led by either the education or health sector. Baltag and colleagues categorized five delivery modalities for SHN services (1), with monitoring responsibilities typically aligning with the cadre who are providing the intervention as presented in Figure 1. For example, in a delivery model where health services are provided by on-site personnel, the monitoring responsibility falls to those cadres (educators/teachers), who then must produce monthly reports of service delivery to front line health workers who push the information up the health management information system (HMIS). In delivery models that involve health service delivery by visiting or permanent health staff, the monitoring and reporting responsibility is taken on fully by the health sector, with a communication channel extended to the education sector. Alternatively, collection methods may reflect the sector-specific technical considerations for delivery of standalone interventions, for example in the case of HPV vaccine delivery, which is almost exclusively delivered and monitored by the health sector.

FIGURE 1

Figure 1. Information flow between schools and ministries by school health and nutrition delivery modality. The encircled cross icon represents the health sector and the pencil icon represents the education sector. Information for Malawi and Mozambique were contributed based on personal communication with World Bank staff. Sources for the summary on Maryland, United States: (9); Sources for the Netherlands: (3, 10–13); Sources for Tajikistan: (10, 13–15).

The Role of Monitoring in Program Adaption

Program monitoring is just as important for documenting the reach of an intervention as it is for identifying when adjustments to the service delivery design are necessary to maximize their intended impact. For this reason, program monitoring should include a feedback loop that facilitates the flow of information from the subnational to the national level and vice versa, to guide policies and enable adaptive programming that promotes course correction. Table 1 provides a granular summary on the process select countries take when collecting and reporting school health and nutrition data, and includes information when data is moving from schools to the lead ministry. The summaries captured within both Figure 1 and Table 1 are specific to the data collection and reporting process for these programs, and it is important to note that the roles for delivering and collecting data can vary within and across countries depending on the program design.

TABLE 1

Table 1. Monitoring considerations and methods for selecting ASHN interventions.

Though not specific to SHN programming, a successful example of adaptive programming based on a well-functioning feedback loop was seen in Malawi, when a community-based nutrition program successfully adapted its programming by using real time monitoring data, adding service delivery sites in areas with low attendance, following-up with participants who had repeated absences, and improving targeting of services to improve procurement efficiencies and achieve cost savings (21). Monitoring of school-based programming can similarly use real-time data to identify whether there are trends among students who have routinely missed the delivery of interventions and serve as the starting point for reconsidering how best to reach those individuals going forward.

Pandemics, such as the one facing the world in 2020, and other types of socio-political or environmental crises that cause school closures, have further demonstrated the utility in adapting both the service delivery mechanism and the modality for collecting monitoring data in extenuating circumstances. Emerging evidence from the COVID-19 pandemic suggest that public health emergencies make accessing social services more difficult, and this is particularly true for those who depend on the education system as their delivery mechanism. An unprecedented number of countries worldwide closed schools to slow the transmission of COVID-19, with more than 192 countries mandating some form of school closures, impacting at least 1.6 billion children and youth and an estimated 63 million teachers (22, 23). In response, program implementers have pivoted service delivery to continue the provision of health services that were previously school-based to reach vulnerable populations.

School meals and cash transfers constitute one example: recognizing that school meals provide immediate benefit for undernourished students, Thailand began to distribute shelf-stable milk to the homes of students who would otherwise receive meals in schools (24); Guatemala engaged Parent-Teacher Associations to deliver 2-week lunch rations (25); and the World Food Programme piloted digital food vouchers, contactless cash transfers, and delivery of take-home rations to families, among other approaches (26). India, which provides the largest school feeding program worldwide, similarly adapted its delivery and monitoring mechanisms. In Uttar Pradesh State, schools provided grains for over 75 days, directly transferred a stipend for cooking costs to beneficiary bank accounts, and used an interactive voice response system to confirm that both the funds and foodstuffs were received. Meghalaya State, on the other hand, engaged District School Education Officers to confirm that parents had received their entitlements for the duration of school closures. Schools in Meghalaya State retained the data and submit reports to their respective District School Education Officers, Sub-Divisional School Education Officers, and the Directorate of School Education and Literacy. Data collected through these various monitoring mechanisms suggest that despite each states' best efforts, the alternative delivery platforms reach only half of its intended beneficiaries (27). This data is valuable in that it guides program implementers in each state on which populations need additional and varied forms of targeted outreach to ultimately benefit from the provisions provided by their respective states.

Key Program Elements: Indicators, Reporting, and Monitoring Mechanisms

As discussed, given variations in program design, implementation, and monitoring, this manuscript emphasizes key principles to designing robust and effective monitoring systems, as opposed to prescriptive guidance on specific indicators or monitoring mechanisms. These principles include: (1) Selection of indicators to adequately measure the intended final impact and identification of appropriate data sources; (2) simplicity and feasibility for data collectors; (3) integration of monitoring data into other sources of information; and (4) clarity in all stages of the data collection, reporting, analysis, referrals and feedback processes. This section provides an overview of how to apply these principles to the specific considerations that program implementers must weigh when monitoring a multi-sector effort, such as a SHN program.

Indicators to Monitor School Health and Nutrition Programming

Indicators drive resource allocation as well as all subsequent data collection, analysis, and reporting. As with the design and implementation of any program or project, the selection, monitoring, and validation of appropriate indicators specified within the results framework is critical to incentivizing not only the inclusion, but also the implementation of SHN services. The selection of specific indicators that make up the results framework are largely dependent on and closely aligned with the program's logical framework. Figure 2 provides an illustrative example of a program logical framework and a broad set of actions that could be considered in SHN programming.

FIGURE 2

Figure 2. Illustrative logical framework for adolescent school health and nutrition interventions.

In a country programmatic scenario, this logical model would require adjustments based on the specificity of interventions included in the basic package, activities needed to deliver them, and intended results of those. Indicators can then be categorized according to which programmatic process they measure along the program's logical framework, such as inputs, processes, outputs, outcomes, and impacts (refer to Figure 2). The selected indicators will subsequently serve as the basis for the programmatic monitoring plan.

The wide variation in SHN programs and delivery models render it difficult to be prescriptive about specific indicators to use. Table 2 provides an illustrative example of a limited number of proposed indicators under the categories of processes traditionally outlined in a logical framework and from the perspective of a program planner from a development agency. For additional indicators on SHN, program implementers can reference the Focusing Resources on Effective School Health (FRESH) Monitoring and Evaluation Thematic Indicators (28) and the WHO/UNESCO Global Standards for Health Promoting Schools and their Implementation Guidance (29) for examples of indicators across relevant intervention areas. In addition, the Global School-Based Student Health Survey (30) tracks health behaviors among students and the WHO Global Action for Measurement of Adolescent Health (GAMA) (31) consolidates and tracks health and policy indicators relevant for this cohort.

TABLE 2

Table 2. Examples of school health and nutrition indicators by intervention.

Program implementers must weigh several considerations when defining which indicators to measure and which data sources to utilize, frequency of data collection, roles for collecting and submitting data, and how to ensure the findings enable program adaptation. These include:

Considerations Related to the Selection of Indicators

1. Determine indicators along the results chain based on the intended final impact

Program planners are encouraged to develop a theory of change to define the expected results among key target groups, and intervention components that will enable their achievement. This mapping will also be critical to identifying the relevant indicators that need to be measured (32), which will be heavily dependent on the intended final impact of the program. There is much debate around whether certain indicators are output, outcome, or impact indicators, but this will ultimately depend on the final outcomes expected in each program. For example, if the desired impact of an investment is to reduce adolescent pregnancy, then the delivery of comprehensive sexuality education to adolescents in secondary schools would be an outcome indicator on the pathway to get you to that impact. However, if increasing the access to and use of modern contraceptives among adolescent populations is the desired impact, then the outcome indicator could be related to the coverage of a family planning program that targets adolescents. In the context of time-bound investments in which school health and nutrition may be one activity within a larger investment, indicators should be selected carefully and represent not only inputs and outcomes but also program processes to track implementation, enable program adaptation, and subsequently increase the likelihood of achieving the intended objectives within the project period. Again, this is where the critical role of alignment with the program's logical framework comes to light.

2. Clarify the details of how indicators will be measured and assessed, with special considerations for defining your target population.

Equally important to selecting the indicators of interest is ensuring the ability to analyze data along key equity dimensions (such as gender and age groups), as success is dependent on the quality, adequacy, and the reach of the interventions. As with any SHN program that includes delivery of commodities to specific population groups, like school-based HPV vaccination programs for girls ages 9–14 years, determining the total number of targeted beneficiaries is critical. When targeting is meant to be age-based, program planners may consider calculating their targets using grade-based eligibility instead, as determining age is difficult in contexts where birth records are not routinely available, school registers are inaccurate, and/or census data is outdated (33).

3. Reflect on and clarify who is implementing and monitoring the program

The actors initiating and implementing SHN programs also influences the definition of indicators within each category of the logical framework. For example, for a government, an input-level indicator could be the number of supplies or commodities procured for program functioning, while for a donor agency, it may be the development of a SHN policy. National programs and monitoring generally operate under one ministry, while implementation may be led by technical bodies at the national or sub-national level, as is seen with school feeding (18, 19). It is important to note that SHN programming may be led by both state and non-state actors, thereby requiring considerable coordination and human resource costs to ensure consistency across operational frameworks as well as data collection and reporting throughout the period of program development and implementation.

Beyond the considerations about the program implementer, it is equally important to specify the actors who will support programmatic monitoring and reach agreement on the process for collecting data and reporting against each indicator. Data collection should be standardized across the program regardless of service delivery point to facilitate data aggregation and analysis (34). This is particularly important as multisector investments like school health and nutrition tend to engage representatives across ministries and organizations. Lack of clarity on these roles can create duplications and inconsistencies in reporting of key program monitoring indicators. In addition, capacity gaps related to programmatic monitoring and reporting similarly presents a common challenge.

Considerations Related to Information Systems for Program Monitoring, and Frequency of Data Collection

1. Identify available data sources and whether these require further strengthening

Recognizing that national health management information systems may be too weak to be the sole source of data for programmatic monitoring, implementors can also consider complementary forms of data collection, such as through SMS or phone-based surveys and through innovative geo-enabled data collection mechanisms. Where needed, project funding can also include resources for support with data analysis, visualization, and use, or to strengthen the information system. To facilitate strengthened cross-sector collaboration and evidence-based decision making, data collected should routinely be shared, reviewed, and discussed between relevant sectors and institutions.

2. Ensure an appropriate balance between minimum and ideal program monitoring and indicator selection standards to improve feasibility and minimize monitoring costs.

Minimum standards are those which are necessary to achieve the desired outcome or to achieve country requirements, while ideal standards are more ambitious. In many cases, project teams might determine which indicators to prioritize based on what data collection systems are already in place or based on what can organically complement other efforts in the same space, rather than establishing siloed but more burdensome and costly approaches to collect an ideal indicator. As with any program, quality assurance mechanisms are essential for ensuring robust data collected against each indicator during collection, transfer, compilation, and analysis (35). This is especially true when routine program monitoring data is collected by different partners with different capacities and working at different levels.

3. Explore opportunities for integrating monitoring into other school-based sources of information.

Monitoring by development partners is often program specific and not necessarily aligned to government programs or data sources, which places a significant data reporting burden on implementing entities and misses an important opportunity to build monitoring capacity at the central and local levels. Therefore, where possible, program indicators should be aligned to those of national programs and/or programs implemented by other development partners, as they will already be collected through existing monitoring systems. Not only should indicators be aligned, but ideally so would data sources and frequency of data collection. For example, in countries with schools providing comprehensive sexuality education, integrating questions into annual, national, and standard assessments administered to test students' knowledge of key topics covered can be a useful and independent source of data to monitor the program's impact. This approach allows for standardized, routine and repeated learning assessments, which can generate independent and longer-term program feedback and can be used to assess intervention feasibility and effectiveness.

Considerations Related to Feedback and Referral Mechanisms

1. Ensure feedback mechanisms exist to enable program course correction

Recognizing that SHN programs engage actors across sectors, it is important to build in a feedback loop that connects education and health information systems, especially in cases where the monitoring responsibility is with the education system (Figure 1, see Malawi example specifically). Data sources and reporting responsibility will thus vary based on the delivery model that is used, so clarity in these protocols and how the data will eventually reach the Ministry of Health, is essential. Protocols should clearly specify data sources, collection methods (including who collects and what monitoring tool is used), frequency, how routine monitoring data will be analyzed and used as it moves from one administrative level to another, and how it will be communicated, disseminated, and used for program- and implementation-related decision-making. This feedback loop should also flow back from the national to sub-national level for decisions around resource allocation and program course correction. Concerted efforts to build capacity in data management, analysis, and reporting will likely be needed at all levels of the system to ensure that this feedback loop functions properly.

2. Identify how referrals from schools to facilities will be tracked and monitored.

Schools should have an established and well-disseminated referral mechanism between the school and health facilities and other appropriate institutions/agencies. Staff, be it health or education, who teach health and/or nutrition education should be equipped to refer students who present with symptoms and monitor whether the student received care, including establishing necessary linkages to support a student who faces a situation of abuse (ex. with local police, mental health counseling, etc.). It is important to ensure that there is a mechanism to track whether referred pupils were ultimately seen by a health or other relevant specialist (Figure 3). Linkages with the health system can also entail redistributing excess materials from SHN interventions to health facilities. School-based deworming campaigns, for example, often have agreements to redistribute unused tablets to health facilities and the number of redistributed tablets should be recorded as part of the deworming campaign.

FIGURE 3

Figure 3. Example of referral forms used in Liberia for school-based deworming and vision screening.

Discussion (INCL. Lessons Learned for Future Applications)

Program monitoring is both a management tool and a powerful feedback mechanism. It is best practice to integrate project monitoring from the earliest stages of project design, in alignment with the program's logical framework. The selection of indicators drives resource allocation toward the areas being assessed and incentivizes their implementation.

There are known gaps in the monitoring of SHN programs. At the global level, there is no repository of the health and nutrition status of children and adolescents nor of the breadth and coverage of SHN services provided in countries (36). Since coverage from school-based service delivery programs may not be captured within administrative surveys, SHN monitoring systems may need to rely on multiple systems to collate and monitor the relevant program indicators. While there is no internationally accepted standardized framework for the monitoring of SHN service delivery, there are a wide range of indicators that can be drawn upon and further contextualized to meet specific programmatic needs and activities.

There are key principles to develop a robust monitoring system that apply whether the service delivery is standalone or integrated with multiple services. In the context of school health and nutrition service delivery, multiple sectors will likely be engaged, and as such, each monitoring system will be unique to the programs, interventions, and setting. It is prudent that all engaged actors agree in advance on the process for collecting and transmitting data between levels and sectors, including, for example, interoperability between systems.

The COVID-19 pandemic, and the resulting prolonged school closures, has forced innovations in education, health, and nutrition service delivery mechanisms, and similarly, has necessitated varied approaches to monitoring the reach and quality of adapted interventions. There are emerging examples of how service providers have pivoted to reach these populations, with examples of innovative school meal distribution and tracking methodologies described above. The current pandemic shows the importance of developing and routinely updating emergency preparedness plans that outline contingency protocols for service delivery during school closures and localized disease outbreaks. These protocols should include guidance for which metrics to prioritize and which actors to task with collecting the information. In the case of prolonged school closures, community members could be called upon to prioritize indicators to monitor and to engage in data collection activities to inform adaptive measures (37).

Beyond the school closure period and as a country moves toward reopening, monitoring mechanisms may continue to need shifting and adaptation to respond to the new reality under which schools operate to comply with public health guidelines. For example, in the case of COVID-19, this includes coordination across sectors to implement mechanisms to monitor, report, and trace cases among students and teachers alike, monitoring and procurement of personal protective equipment and sanitation supplies to minimize spread within the school setting, and ability to track and respond to the broad spectrum of changing physical and mental health needs of adolescents as they return to schools after periods of extended closure. Many of the key principles outlined in this manuscript are increasingly relevant in such instances of sudden school closures, and again emphasize the need for rapid action, coordination, and innovation.

Acknowledgment of Conceptual or Methodological Constraints

The authors acknowledge the relative paucity of country-specific information published on this topic. As such, the illustrative country examples and the key principles presented within this manuscript represent the best attempt and most complete information that the authors were able to collate through a desk review. The authors recognize that there is wide variation in approaches and compositions of SHN services within and across countries, and as such, it is impossible to be prescriptive around SHN monitoring methodology. Instead, the authors deemed it more efficient to guide program implementers with key principles for program monitoring.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author Contributions

LS and JR-B worked in tandem to equally contribute to the conceptual framing, research, writing, and editing of this manuscript. All authors contributed to the article and approved the submitted version.

Funding

This manuscript was developed as part of the Global Financing Facility-funded Advisory Services and Analytics (ASA) entitled Strengthening Evidence to Improve Adolescent Sexual and Reproductive Health in Support of Human Capital (ID: P169994).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors extend their gratitude to Brendan Hayes and Supriya Madhavan for their steadfast support in elevating the importance of multisector entry points to deliver health and nutrition services to adolescent populations and for their invaluable comments on drafts of this manuscript. The authors also recognize Genesis Samonte for imparting her technical expertise on monitoring public health initiatives at the country-level.

References

1. Baltag V, Pachyna A, Hall J. Global overview of school health services: data from 102 countries. Heal Behav Policy Rev. (2015) 2:268–83. doi: 10.14485/HBPR.2.4.4

CrossRef Full Text | Google Scholar

2. Bundy D, Schultz L, Sarr B, Banham L, Colenso P, Drake L. The school as a platform for addressing health in middle childhood and adolescence. In: Bundy DAP, de Silva N, Horton S, Jamison DT, Patton G, editors. Disease Control Priorities, Third Edition (Volume 8): Child and Adolescent Health and Development. Washington, DC: IBRD/World Bank (2017). p. 269–85. doi: 10.1596/978-1-4648-0423-6_ch20

CrossRef Full Text | Google Scholar

3. Baltag V, Saewyc E. Pairing children with health services: the changing role of school health services in the twenty-first century. In: Cherry A, Baltag V, Dillon M, editors. International Handbook on Adolescent Health and Development: The Public Health Response. Cham: Springer (2017). p. 463–77. doi: 10.1007/978-3-319-40743-2_24

CrossRef Full Text | Google Scholar

4. UNESCO. Gross enrolment ratio by level of education [Internet]. Available online at: http://data.uis.unesco.org/index.aspx?queryid=142 (accessed March 13, 2020).