Critical review of indicators, metrics, methods, and tools for monitoring and evaluation of biofortification programs at scale

Introduction

Micronutrient malnutrition affects ∼2 billion people worldwide (1, 2); it contributes to poor child growth, intellectual impairment, increased risk of morbidity and mortality, and is highly prevalent in food-insecure settings in low-and middle-income countries (LMIC) (3). Biofortification, an agriculture-based method of increasing the density of micronutrients of staple crops through selective plant breeding and agronomic techniques, can play a crucial role in addressing micronutrient malnutrition (4, 5) since it reaches populations in remote rural areas and could potentially reach urban consumers as well (6, 7).

Under the leadership of HarvestPlus, crop-breeding programs at CGIAR centers, national agricultural research systems (NARS), private, public, and community-based seed producers, and farmers have co-developed, tested, and released more than 400 biofortified varieties of 11 different crops worldwide (8). The efficacy of biofortified food to improve nutritional and health outcomes has been demonstrated consistently in the past 15 years, especially for vitamin A enriched crops, iron beans, and iron-pearl millet (4, 6), and cost-effectiveness has been demonstrated for vitamin A enriched crops (9).

Given its promise for improving diets, biofortification is at a tipping point to go to scale to replace currently grown staples with low nutrient density (10), contributing to food system transformation without changing consumer eating behaviors (7). For this, national governments, non-governmental organizations (NGOs), the private sector, the UN, and international financial institutions will need to invest more in context-specific biofortification programs (10, 11) with an increased diversity of stakeholders across geographies. As biofortification scaling efforts gain momentum (10), so does the need for a monitoring and evaluation (M&E) framework with a set of harmonized indicators metrics, methods, and tools (IMMT) that allow the measurements of key results of biofortification programs across countries, regions, and organizations.

Sound M&E systems are needed to generate quality data on program performance to inform learning and adaptive program management for effective implementation and evidence-based policymaking (12). Despite the existence of comprehensive M&E systems for many of the current biofortification programs, their focus has primarily been on project-level management, and the IMMTs are currently not harmonized across countries, crops, and implementors. Furthermore, the existing M&E frameworks for biofortification interventions are primarily reported in gray literature, such as donor reports and institutional publications. Hence, there is a need to review and document the tried-and-tested, found-to-work, and common elements of the existing M&E frameworks to facilitate harmonization across programs, crops, countries, and organizations.

In a multi-phase, iterative process, we aimed to review M&E frameworks of biofortification programs implemented to date, and to recommend a subset of high-level indicators (HLI) for a harmonized global M&E framework to track program progress along a generic implementation pathway. We conducted (1) a mapping review to compile IMMT for biofortification programs; (2) semi-structured interviews (SSIs) with biofortification programming experts (and other relevant stakeholders) to contextualize findings from step 1; and (3) compiled a generic biofortification program Theory of Change (ToC) to use it as an analytical framework for selecting the HLI (Figure 1). This study responds to the collective call to strengthen monitoring and evaluation efforts to support healthy and sustainable food systems—“The Accountability Pact” (13)—and to the need to use rigorous monitoring to guide food system transformation in the countdown to the 2030 global goals (12). We expect to fill an important gap in the literature concerning best-bet M&E frameworks for biofortification programming.

FIGURE 1

Figure 1. Overview of the multi-phase, iterative process research approach.

Step 1—Mapping review

We focused on published reviews and official program documents that contained detailed descriptions of biofortification programs, M&E frameworks, and IMMT. MEDLINE, Cochrane Library, Web of Science, and Scopus were searched to retrieve reviews that met the inclusion criteria (Table 1). Six building blocks of strings were developed for MEDLINE by adapting the search strategy of Garcia-Casal et al. (14). The search syntax included MeSH terms in the title and abstract fields (Supplementary Data). The building blocks and strings created for MEDLINE were adapted for searches in the other databases. The final search results were imported into EndNote; duplicates were removed automatically. We also accessed project reports from the Inclusive and Sustainable Value Chains and Food Fortification projects (EuropeAid/151093/DH/ACT/Multi) shared by the European Commission’s Food Fortification Advisory Service. Based on the inclusion and exclusion criteria (Table 1), two investigators (TCA and SRM) independently selected reviews and program documents for further use. Any disagreements between the investigators were resolved through consultation with another researcher from the team (CL).

TABLE 1

Table 1. Inclusion and exclusion criteria for the mapping review.

To guide the identification of indicators for M&E from the published and gray literature, we adapted the WHO/CDC logic model for implementing micronutrient interventions in public health (15) to represent the underlying implementation processes of biofortification programs (Supplementary Figure 1) and identified relevant input, output, outcome, and impact indicators. Next, two authors (TCA and SRM) developed two data charting forms in Microsoft Excel 2016 to extract the IMMT from the reviews and gray literature. CL pilot-tested the charting forms. No changes were made after the pilot test, and input, activity, output, outcome, and impact indicators were extracted accordingly.

Step 2—Semi-structured interviews with biofortification programming experts and other relevant stakeholders

The SSIs with biofortification programming experts aimed to contextualize the findings from the mapping review by obtaining a generic description of biofortification programs, key factors limiting/enabling their success, M&E frameworks used, and IMMT, as well as methods for assessing coverage and consumption of biofortified foods. SSI guides were developed accordingly with guiding semi-structured questions and detailed probes to gain in-depth information on key topics of inquiry (16).

Nine biofortification programming experts and one food systems expert were recruited through snowball sampling (17) and were invited for an (online) interview. Two experts, EB and BM, are also coauthors of this paper. The participants’ expertise in biofortification programming covered Africa, Asia, Latin America, and the Caribbean regions, and their collective programming expertise included global to regional, and local levels of biofortification programming. In addition to the biofortification (n = 9) and food systems (n = 1) experts, two experts with extensive food intake assessment experience were interviewed to obtain further details on the challenges and opportunities of conducting dietary intake assessments in LMIC. The latter interviews enabled an understanding of the most suitable methods for assessing consumption of biofortified foods and potential adaptations that could be made to existing diet quality scores. Table 2 shows participants’ affiliations.

TABLE 2

Table 2. Characteristics of SSI participants.

On average, each interview lasted 75 min and was conducted in English. The interviews were digitally recorded and transcribed verbatim by two research assistants for subsequent thematic analysis. The content of the transcripts was spot-checked by FMG and SRM with the original audios to ensure content fidelity. Data were collected over five weeks from March to June, 2021 until data saturation was reached among key themes (16, 18).

During the conduct of the SSIs, some participants from HarvestPlus shared additional program documents: an unpublished generic biofortification ToC from 2016 (19), three unpublished country- and crop-specific ToCs (20–22), and the published ToC for the Commercialization of Biofortified Crops Program led by HarvestPlus and the Global Alliance for Improved Nutrition (GAIN) (23). The latter was adapted from an unpublished harmonized ToC developed by a collective working on developing a harmonized monitoring, evaluation, learning, and impact assessment system (MELIAS) for large-scale biofortification programs from HarvestPlus, CIP, and GAIN. SSI participants from HarvestPlus also shared a set of 19 M&E indicators anchored to the MELIAS biofortification ToC. This was part of an internal HarvestPlus M&E manual (24). Finally, a CIP participant shared an M&E tool for orange-fleshed sweet potato (OFSP) interventions in sub-Saharan Africa (25), from which one indicator was extracted. These and other documents reviewed for further analysis are presented in Table 3.

TABLE 3

Table 3. List of additional documents received from SSI participants from HarvestPlus and CIP.

Data analysis of semi-structured interviews and program documents

Thematic analysis followed procedures suggested by Huberman et al. (18). First, SRM and FMG read the program documents and interview transcripts several times to gain a holistic view of the data set. Second, a codebook with 25 categories of information that served as a framework for analysis was developed and cross-checked by another researcher from the team (SRK). The initial codes were applied to the text in program documents and SSI transcripts to tag content based on meaning (first coding cycle). FMG coded all program documents, and the interview transcripts were coded in duplicate by SRM and FMG. Third, through a second coding cycle, the 25 codes were clustered and merged into eight pattern codes (i.e., thematic areas). Interrelations between pattern codes were then examined, and three primary themes aligned with the study aim were extracted for theory building. Exemplar quotations that best described the topic of interest were selected to illustrate the findings. Dedoose (26) software version 8.3.47 was used for data management and analysis of secondary sources of information and interview transcripts.

Step 3—Compiling a generic Theory of Change and selecting indicators, metrics, methods, and tools for biofortification programs

The implementation processes of biofortification programs described by the SSI participants were compiled into a narrative structure and subsequently represented in an initial draft ToC. This draft was then compared to the ToCs from Table 3, adapted to add detail and ensure that common elements of biofortification interventions were represented in the compiled ToC. Next, the indicators selected from the mapping review (36 indicators) were merged with the new set of 19 indicators (24) anchored to the ToC developed by the MELIAS group (27), one indicator extracted from the CIP M&E manual (25), and one indicator recommended by three HarvestPlus participants in SSIs into a final list of 57 indicators (Table 4). Indicators that, from our perspective, had similar or overlapping definitions with those of the list of 19 indicators (24) were eliminated (Table 4, 13 indicators marked in red). Furthermore, as we aimed to keep only high-level indicators that reflected critical implementation stages of biofortification programs across the ToC, input and activity level indicators were excluded (four input and six activity indicators marked in orange). A total of 34 indicators (marked in black, blue, and green) were kept to select a final subset of HLI. Using the ToC of this study as an analysis framework, the authors selected the final list of HLI.

TABLE 4

Table 4. Total number of indicators (N = 57), extracted from the mapping review (n = 36 orange, black and red indicators) merged with the 19 indicators (in green) anchored to the generic ToC from the MELIAS group, one indicator recommended by three HarvestPlus experts, and one indicator extracted from the CIP monitoring and evaluation manual (in blue).

Ethical considerations

Participation in the study was voluntary. Before the interviews, potential participants received a detailed explanation (verbal and written) about the purpose of the study and were informed that the data would be used for research purposes. SSI participants gave verbal consent to record the interviews. As data collection methods were primarily desk-based and contact with participants was minimal (online interviews), approval from an institutional review board was not sought.

Mapping review (Step 1)

The mapping review resulted in three published reviews fulfilling the eligibility criteria (4, 28, 29) and four program documents (30–33). An additional scientific paper (34) recommended by experts was also included (Table 5). Although not a review, this study aimed to develop and test methods and indicators for assessing awareness and household coverage of biofortified foods in Rwanda. It yielded five outcome indicators with their respective data collection methods. From the three reviews (4, 28, 29), we identified four outcome indicators and two output indicators.

TABLE 5

Table 5. List of documents included in the mapping review.

From the gray literature, three reports that focused on tracking the progress of biofortification programs in tackling micronutrient deficiencies in the Gambia (30, 31) and Ethiopia (32) were reviewed, from which we extracted four outcome, one output, and two activity indicators. In addition, from the HarvestPlus Indicator Reference Manual (33), we extracted four input, four activity, seven output, seven outcome, and three impact indicators. Indicators with similar definitions were grouped into a single indicator citing the sources. When available, the indicators were extracted with their metrics, methods, and tools for constructing their numeric values. Overall, the mapping review yielded 36 unique indicators presented in Table 4 (indicators marked in orange, black, and red) with their respective sources.

Semi-structured interviews with biofortification programming experts and other stakeholders (Step 2)

The findings of the SSIs were grouped into three main themes, as described below.

Theme 1—Generic Description of biofortification programs

Most participants described the biofortification programs in four stages. Those descriptions were also comparable to the biofortification interventions described in the reviews (4, 28, 29) and program documents listed in Tables 3, 5. A generic description of biofortification programs centered around common elements of successful programs is presented below in four stages.

Stage 1: Breeding and releasing biofortified crop varieties—All participants indicated that the initial development of biofortified varieties of crops was primarily carried out at the CGIAR centers at global and regional levels. In LMIC, the CGIAR shares biofortified parental lines with NARS for inclusion in national breeding programs for further adaptive breeding and field testing. Subsequently, the NARS, or other national varietal release authority, may approve the release of tested biofortified varieties with competitive agronomic traits if they meet predetermined threshold micronutrient levels and other agronomic characteristics as required by the national authorities. The released varieties may then be licensed to public and private companies (small and medium scale) for multiplication and subsequent distribution to farmers. Most participants described biofortification breeding as a “dynamic,” “never-ending process,” “constantly searching for high-yielding, pest-resistant, and climate-resilient varieties where breeding for higher micronutrient levels—enough to have a significant impact on micronutrient deficiencies—is mainstreamed into the existing breeding programs.” Most participants considered investing in capacity strengthening at the NARS level for breeding and testing new biofortified varieties crucial for successfully sustaining biofortification in public and private breeding programs over time.

Stage 2: Multiplying, introducing, and distributing planting material of biofortified crop varieties to farmers—Most participants explained that multiplication and distribution of planting material are carried out by private, public, or community-based planting material producers, depending on the type of crop. Grain crop seeds for hybrid and open-pollinated varieties (OPV), such as maize and pearl millet, are usually licensed to commercial private and/or public seed companies for subsequent multiplication and marketing. In contrast, vegetatively propagated crops (VPC), such as sweet potato and cassava, are usually multiplied and distributed by community-based stem/vine multipliers.

Depending on the competitiveness of the planting material, there is a mixed seed production and distribution system for OPVs and self-pollinating biofortified varieties of crops (SPV), such as beans, rice, and wheat. Private seed companies prefer to produce and market highly competitive varieties (e.g., hybrid and OPVs), while public sector and NGO support are needed for less competitive varieties with high levels of micronutrients. All participants explained that the crop type and seed systems also define the seed commercialization pattern. Depending on the crop, private seed companies, public multipliers, NGOs, farmer organizations, including women’s groups, or a combination of two or more of these are crucial to scaling the multiplication of biofortified planting material and hence availability thereof to smallholder farmers.

Stage 3: Scaling production and utilization of biofortified foods—All participants indicated that scaling will be the focus of most national and global programming in the coming decade. Strengthening all value chain actors, from breeders to farmers, aggregators, processors, retailers, and other service providers, is a crucial cross-cutting aspect of program implementation activities to integrate biofortification into seed and food systems in a sustainable way. Implementing behavior change communication strategies to accelerate awareness and adoption of biofortified crops, fostering good pre and post-harvesting practices, and facilitating access to and promoting utilization of biofortified planting material among farmers are crucial in this stage. Subsidies, partial or complete, and free or low-cost demonstration kits/trial packs can effectively foster the adoption of biofortified crops by farmers. Some biofortification programs also establish linkages between farmers and different market options, such as food processors, to help farmers and aggregators sell their products as ingredients for processed foods.

Most participants also highlighted the role of the public sector, NGOs, and humanitarian organizations in scaling access to biofortified planting material and foods and using biofortified foods for emergency response. For example, CIP participants indicated that they work with the World Food Programme to include OFSP in food security programs in refugee camps in northern Ghana, northern Uganda, and northern Kenya. Other examples include involving the public sector in distributing OFSP in school feeding programs in Nigeria and Ghana. Most participants also indicated that behavior change strategies to promote the utilization of biofortified crops and foods among potential consumers in rural and urban settings are crucial in the scaling phase.

Stage 4: Integrating biofortification in local food systems—A participant described the mainstreaming of biofortification into food systems as follows:

“… biofortification needs to be mainstreamed into national policies, plans, and breeding programs. The new release of competitive biofortified varieties is actively ongoing; the varieties are tested by NARS, released for production by national authorities, and adopted by farmers, and the production of biofortified crops increases continuously. As a result, biofortified foods are mainstreamed into the agricultural and food sectors; consequently, consumption of biofortified food increases, contributing to relieving micronutrient deficiencies and improving the quality of diets.”

Interview with senior expert, Washington, DC, March 2021

Furthermore, some participants indicated that defining standards with ranges of micronutrient content for biofortified planting material and food manufactured with biofortified ingredients is crucial to ensure that biofortified crop varieties and foods meet the expected micronutrient content in the future.

Theme 2—M&E frameworks for biofortification programs

From the SSIs, two application scenarios emerged for M&E of outcome (adoption) and coverage of biofortification programs and consumption of biofortified foods: nationally representative surveys and sentinel site monitoring surveys. All participants indicated that nationally representative surveys are warranted only when the program has reached sufficient maturity and coverage and recommended the integration of biofortified crops and foods into production and consumption modules of existing nationally representative data collection systems (e.g., National Crop Surveys, National Demographic and Health Surveys) to optimize costs and sustainability of M&E of production, coverage, and consumption of biofortified crops and foods. On the contrary, some participants indicated that sentinel site surveys could be used for monitoring pilot stage or “nascent programs” or in cases of budget constraints for M&E. In a sentinel surveillance system, data are regularly reported from a sample of pre-selected sites (units of programming such as district or local government authority) purposefully selected to represent the population of the areas where the programs are implemented (35). The required sample size for sentinel site surveys is smaller and can prioritize program intervention areas. Therefore, these are less costly and simpler to conduct than nationally representative surveys (35).

Theme 3—Compiling methods, metrics, and tools for constructing indicators

The following sections discuss the methods and tools to assess the coverage and consumption of biofortified crops described in the SSIs. This section specifically concerned diversity and diet quality scores, as used by CIP and HarvestPlus in biofortification programs (and others in generic nutrition-specific and nutrition-sensitive interventions).

Methods for assessing coverage and consumption of biofortified crops/foods

A lack of agreement emerged among participants on the most appropriate methods for assessing coverage and consumption of biofortified foods. Some participants indicated a preference for traditional consumption surveys such as 24-hour recalls (24hR) or Food Frequency Questionnaires (FFQ). In contrast, others suggested alternative methods, such as mathematical modeling of Household Consumption and Expenditure Surveys (HCES) data to estimate apparent food consumption and micronutrient intake (before and after biofortification of the staple[s] consumed) based on the adult male equivalent formula, as proposed by the Food and Agriculture Organization (FAO) (36, 37).

Concerning HCES, some participants voiced concern that these may produce inaccurate estimates of individual food consumption and nutrient intakes, primarily because they do not consider intra-household food sharing practices or foods consumed away from home. In contrast, others argued that considering the complexity of conducting 24hR or FFQ, their estimates are also inaccurate. The most salient examples of sources of inaccuracy mentioned for 24hR were:

• standardized portion sizes are seldom available for specific LMIC contexts;

• extensive training and experience are required to collect, process, and analyze data from 24hR and FFQ. Such training is often lacking, leading to inaccurate data collection processes and poor data quality.

• For FFQ, appropriate food lists tailored to the context and population group are needed, requiring initial formative research and validation against 24hR and is therefore even more rare in LMIC.

Other mentioned barriers to undertaking 24hR and FFQ were the time required for data collection, processing, and analysis of data, cost, and complex logistics and the availability of up-to-date and accurate food composition tables in LMIC.

In light of the above limitations, one participant explained that apparent food consumption estimated from HCES is sufficient to estimate the coverage and consumption of biofortified foods, as long as biofortified foods are included in the surveys as specific categories of food. The participant also indicated that the advantage of HCES is that food acquisition or consumption data are regularly collected at the household level in multiple LMIC and that the data are publicly available in most cases. Another participant indicated that including biofortified foods in the food acquisition and consumption lists of HCES would allow making sufficiently robust estimates of coverage and consumption of biofortified foods for routine program monitoring.

Diet diversity and diet quality scores

Given the complexity and cost of undertaking dietary intake surveys, some metrics have been developed to simplify diet quality assessment. One of them is the minimum dietary diversity score for women (MDD-W), a dichotomous indicator that assesses dietary diversity and estimates nutrient adequacy among women of reproductive age (WRA) (38). The MDD-W’s score ranges from 0 to 10 (0–10 food groups consumed in the past 24 h), with a cut-off of <5 indicating the inadequacy of micronutrient intake (39). However, some participants pointed out that the MDD-W was not designed to assess the consumption of specific foods, such as biofortified foods, nor to quantify nutrient intake from food consumption. Hence, the MDD-W would provide limited actionable information for monitoring and adjusting biofortification programs:

“…Diet indicators such as the MDD-W elicits a score: 2, 3, 6. Positive changes in these numbers attributable to a certain intervention will tell at the macro level whether things are improving. However, this information will not contribute to improving the quality of implementation of a program or identify specific actions needed to improve diet quality of target groups…”

Interview with biofortification expert, USA, March 2021.

Regardless of these limitations, efforts have been made to adapt the MDD-W to capture the contribution of biofortified foods to nutrient intake. CIP participants indicated that they use the MDD-W to assess the change in dietary diversity among WRA attributable to their interventions with OFSP. CIP modified the MDD-W’s scoring system by adding biofortified crops as a unique and independent food group contributing to diet diversity and extending the maximum MDD-W score to 11. Although this modification has not been validated as a proxy measure for assessing vitamin A adequacy, it enables CIP to assess their interventions’ contribution to dietary diversity in specific geographies.

Another potential method to assess the coverage and consumption of biofortified foods described in the SSIs was the Global Diet Quality Score (GDQS), a novel food-based metric that calculates nutrient adequacy and the risk of non-communicable diseases associated with dietary intake. The GDQS is calculated based on the consumption of 25 food groups (16 healthy, seven unhealthy, and two classified as unhealthy when consumed in excess (i.e., red meat and high-fat dairy)) (40). Total scores ≥23 are associated with a low risk of nutrient inadequacy and non-communicable disease risk, whereas scores ≥15 and <23 indicate moderate risk, and scores <15 indicate high risk (40).

As mentioned by one participant, the GDQS metric and its data collection tool [the GDQS mobile application (41)] could be adapted by adding a specific food group, such as biofortified tubers (e.g., OFSP and yellow cassava), and an extra point to the DGQS metric to account for foods consumed from that food group. Another participant explained that the GDQS data collection method uses a 24hR format; thus, it can capture the consumption of any food, including biofortified foods. The participant explained:

“… Data are collected with a mobile application that stores dietary intake data in an Excel file. The Excel sheet could be easily modified by adding new rows with the foods of interest, uploading the new file to the app, and it would be ready to capture the consumption of specific foods. However, this modification should be validated. Also, the GDQS data collection method is not designed to capture portion sizes of single foods but to estimate quantities of food intake at the food group level.”

Interview with an expert on dietary assessment methods, USA, June 2021.

The two experts on dietary intake methods also explained that modifications to the GDQS metric and data collection method could be used to assess the contribution of biofortified foods to diet quality and to estimate the coverage of biofortified foods, but not to estimate the contribution of foods to micronutrient intake. Yet, some biofortification experts observed that, given the potential variation in micronutrient content of biofortified foods eventually consumed by households, and their coverage, the use of modified diet quality indicators could lead to over-or underestimation of the contribution of biofortified foods to diet quality.

Compiling a generic ToC for selecting HLI with their metrics, methods, and tools (Step 3)

Based on the generic description of biofortification programs from the SSI participants (Theme 1) and the internal program documents provided by HarvestPlus and CIP participants (19–25, 33, 42, 43), we compiled the ToC presented in Figure 2. The ToC in Figure 2 deviates from the harmonized MELIAS ToC (23) in two aspects: first, The ToC from Figure 2 visualizes the mainstreaming of biofortification into national policies and investment plans; second, it includes the expected outcome of behavior change communication (BCC) strategies at two levels of the ToC. BCC fosters farmers’ adoption and consumption of biofortified crops at the first level. At the second level, it promotes the consumption of biofortified foods in rural and urban households.

FIGURE 2

Figure 2. Theory of Change representing the generic biofortification program impact pathway; the numbers in white circles correspond to the indicators shown in Table 6.

We used the ToC from Figure 2 as an analysis framework to identify 17 HLI for measuring key results along the implementation pathway of biofortification programs. The 17 HLI are presented in Tables 6, 7, along with their definitions and associated metrics, methods, tools, and data sources.

TABLE 6

Table 6. High-level indicators and metrics for M&E of biofortification programs.

TABLE 7

Table 7. Summary of methods and tools for constructing the set of high-level indicators.

Actionable recommendations

A generic or global M&E framework for biofortification programs with a comprehensive set of 17 high-level indicators is presented with their respective metrics, methods, and tools for data collection. The framework is developed based on the commonalities of biofortification programs implemented in multiple LMICs in Africa, Asia, Latin America, and the Caribbean regions by collaborations between national and international agricultural research systems and other public, private, NGO, and UN stakeholders. The 17 HLI presented in this paper may not be used all at once. Their use is flexible and can be prioritized, depending on the implementation stage and stakeholders’ information needs, as illustrated in Figure 3. Like any other program, biofortification programs mature through their different implementation stages, e.g., from the breeding phase to the introduction of biofortified crops and the scaling phase. Along this pathway, the indicators in use may shift from an initial focus on program outputs to a later focus on outcomes and impact. The required number of indicators may also change, as the scaling phase will require cost-efficient prioritization of M&E activities.

FIGURE 3

Figure 3. Evolution of the M&E framework across biofortification implementation phases.

This study highlights crop-related differences in biofortification programming. Competitive planting materials such as grain crop seeds for hybrid and OPV are usually licensed to commercial private or public/private seed companies. They are distributed to farmers through formal commercial channels. VPCs such as OFSP or cassava belong to decentralized informal seed systems, where the planting material is exchanged/shared with other farmers or sold in local markets by traders and vendors (44). Yet, the crop-related differences in biofortification programming should not be an obstacle to using a generic or global M&E framework, particularly at the outcome level. However, M&E of biofortification programs with informal seed systems will require active involvement and investment of the private sector and NGOs, while monitoring programs with formal seed systems will require joint efforts of private seed companies, NGOs, and governments (45).

Sentinel site surveys may be a convenient alternative for M&E of biofortification interventions of formal and informal seed systems. They can generate timely actionable information to improve program implementation at a relatively low cost (46, 47). Sentinel site surveys have long been used in multiple nutrition and health interventions (46–48). The government of Costa Rica, for example, uses them to track changes in iron deficiency and anemia attributable to their food fortification programs (47). It is undertaken in strategically selected urban and rural areas (47). HarvestPlus SSI participants also indicated they use sentinel site surveys as an option to monitor program outcomes in targeted geographies. Nevertheless, for mature programs with high coverage, most participants recommended the integration of biofortified crops and foods into production and consumption modules of existing nationally representative data collection systems, e.g., National Crop Surveys, Nutrition, and Health Surveys, HCES, or Demographic and Health Surveys. This would ensure cost-effective, regular, and sustainable data collection on the production, coverage, and consumption of biofortified crops and foods and contribute to monitoring diet quality (12).

Concerning the 17 HLI, Most of them have already been tested globally in multiple biofortification projects (indicators 1, 3-7, 12, 13, and 17). Additionally, indicators 1, 3, 4, and 6–16 were anchored to the ToC developed by the MELIAS collective, and hence their definitions are now harmonized across biofortification programs led by HarvestPlus, CIP, and GAIN. Furthermore, the indicators on awareness (indicator 12), coverage, and consumption (indicator 13) were built on approaches previously used to assess large-scale food fortification programs (LSFF) (34, 49). Indicator 2 for monitoring the proportion of biofortified varieties released with minimum micronutrient content was recommended by three SSI participants to establish a parameter for monitoring the quality of biofortified planting material. Some countries have already included minimum micronutrient levels as criteria for releasing biofortified varieties; e.g., the government of India established a minimum standard for levels of iron (42 ppm) and zinc (32 ppm) for all released pearl millet varieties (10). However, standards for seed biofortification levels still need to be defined/implemented in most countries. Although this indicator has not been tested, we believe that it will be crucial to enforce compliance of private and public seed companies with biofortification levels in the near future. Monitoring compliance is also crucial in ensuring the quality of fortified food in LSFF programs (6, 50).

Likewise, indicators 8–11 for assessing market availability of raw, prepared, and processed biofortified foods were recently developed by HarvestPlus and GAIN for the scaling phase of biofortification programs and are not tested. Proposals for studies to test these indicators are being prepared. Indicator 17 (i.e., change in the prevalence of dietary diversity or diet quality scores between baseline and endline) can be constructed using a diet quality indicator, e.g., the MDD-W or GDQS. These indicators are most useful once significant coverage of biofortified crops and foods is attained and when consistent data on the quality of biofortified crops and foods (i.e., level of biofortification meeting target ranges) are available. Using this indicator with uncertain data on coverage and biofortification quality may lead to under-or overestimating the contribution of biofortified foods to diet quality.

The generic ToC presented in Figure 2 permits visualizing crucial stages of biofortification programs that can be monitored with the 17 HLI. However, it should be noted that other indicators may be needed to track specific inputs and activities at project-level management along the implementation cycle. We aimed to identify HLI to enable common data collection across geographies, institutions, and programs to enable aggregation and comparison of results. Further research is recommended to test, revise, and harmonize this framework with indicators across programs, implementing institutions, and countries as biofortification programs are scaled across and within LMIC.

Discussion

This study presents a generic M&E framework for biofortification programs, including a ToC for a generic or ‘global’ biofortification program impact pathway and a set of 17 HLI and their associated methods, metrics, and tools. The study is based on a thorough review of a wide range of available frameworks from published and gray literature and ground-truthed through interviews with experts in biofortification programming and dietary intake assessment.

The generic implementation pathway of biofortification programs described in Theme 1 and represented in Figure 2 is a compilation of descriptions of current biofortification programs described by the participants interviewed for this study and a harmonized version of the following: generic ToC for biofortified crops (19), a generic ToC for cassava (20), two country-specific ToCs for cassava and maize for Nigeria (21) and Zambia (22), and the ToC developed by the MELIAS group (23). The latter represents a generic description of the implementation pathways of biofortification programs at scale, harmonized across HarvestPlus, CIP, and GAIN-led biofortification programs, and it is grounded in cross-country, multi-year program experiences of these institutions and their partners in Africa, Asia, and Latin America and the Caribbean.

Compared to the ToC of the MELIAS collective (23), the ToC from Figure 2 differs in two aspects. First, it visualizes the integration of biofortification into national policies and investment plans, which can be instrumental for the sustainable scaling of biofortification programs. Second, it visualizes the expected changes in behavior toward adopting biofortified crops and foods among farmers and consumers—of urban and rural settings—attributable to BCC. From our perspective, BCC strategies will be a crucial element of biofortification programming to foster the adoption and consumption of biofortified crops and foods at all levels of the value chain.

This study has limitations and strengths. A limitation is that most of the information on the M&E framework and biofortification programming was obtained from program documents provided by HarvestPlus and CIP. Though these two organizations have been spearheading biofortification programming in the past decade, relying on only their points of view may represent a narrow scope. Although we searched for biofortification programs implemented by other organizations, we couldn’t find many, and those we found did not provide significant relevant information for the aims of this study. The joining of GAIN of HarvestPlus’s efforts in 2018 to scale biofortified foods (51, 52) and the most recent national uptake by governments, NARS, NGOs, and associated partners will likely broaden the number and types of institutions implementing biofortification programs in the coming years. Once that happens, the M&E framework for biofortification programs presented warrants a revisit, and until then, the framework presented here can be used by biofortification programmers.

The main strengths of this study include (1) compilation of a generic ToC for biofortification programming based on a review and triangulation of tried-and-tested ToCs of various biofortification programs implemented globally; (2) identification of the most pertinent HLI used by biofortification programs and anchoring of these to the ToC above; and (3) review of available IMMT to recommend a sub-set of HLI and ground-truthing of these with experts on biofortification programming. This multi-method iterative approach allowed us to identify rich information on biofortification programs to address our research aims.

Further work is required to determine how IMMT for biofortification programs can best be incorporated into existing national data collection systems and integrated into other nutrition and health indicator-based high-level reporting such as Demographic and Health Surveys. Furthermore, implementation research (e.g., formative research, process evaluation) is warranted to explore barriers and enablers to using the proposed HLI in ongoing biofortification programs, including programs implemented in remote areas with difficult access. This will help to further harmonize this framework across programs, countries, geographies, crops, and institutions; and to optimize planning and funding of monitoring and evaluation activities to ensure that biofortification programs reach the neediest.

Another area of research that we suggest implementing is in the artificial intelligence field. Participants unanimously described numerous limitations to implementing food consumption surveys using traditional methods such as 24hR and FFQ. Artificial intelligence applications could help simplify these assessments, for example, by using pictures captured with a mobile phone integrated into an artificial intelligence-based food recognition system and linking them to mega-databases of food composition tables (55) to estimate food intake and nutrient composition, respectively. This area of research is in line with the United Nations’ 17 Sustainable Development Goal and the Artificial Intelligence for Social Good movement (56). Yet any artificial intelligence initiative should follow ethical principles and guidelines for developing innovative and trustworthy technologies.

Author contributions

SR-M participated in research design, data collection and analysis, and conceptualizing the manuscript. FG participated in data collection and analysis and wrote the manuscript with SR-M, BM, and EB shared the gray literature, ToCs, and indicators used by HarvestPlus globally, participated in SSIs, and critically reviewed the manuscript. SK and CL advised on the study design and critically revised the manuscript. TA developed the syntax for the mapping review, participated in the mapping review data analysis, and critically revised the manuscript. AM-B was the current project manager and contributed to the manuscript. KH was the former project manager and contributed to implementing the project. IB and SO developed the proposal to obtain funding for this study. EF critically reviewed the research proposal and manuscript. SR-M and FG had primary responsibility for the final content. All authors read and approved the final version of the manuscript.

Funding

Funding for the research was provided by Wageningen University, recipient of a Food Fortification Advisory Service (2FAS) grant from the European Union, Landell Mills, and the Global Alliance for Improved Nutrition (GAIN). The funder had no role in the study design or implementation.

Acknowledgments

We especially thank all the participants from the CGIAR centers, the experts ib dietary intake assessment, and the expert on food systems for participating in this study and sharing published and gray literature for this study.

Conflict of interest

BM was the current head of Monitoring and Evaluation at HarvestPlus. EB was the Director of Impact and Strategy at HarvestPlus when we conducted the study. KH was affiliated with Wageningen University and Research when she was involved in this project management. Currently, she was employed by Pepsico Inc. TA was affiliated with Wageningen University and Research when she was involved in this project. She was currently employed by Amsterdam UMC.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnut.2022.963748/full#supplementary-material

Abbreviations

24hR, 24-hour Recalls; 2FAS, Food Fortification Advisory Services funded by the European Union; CIP, International Potato Center; FAO, Food and Agriculture Organization of the United Nations; FFQ, Food Frequency Questionnaire; GAIN, Global Alliance for Improved Nutrition; GDQS, Global Diet Quality Score; HCES, Household Consumption and Expenditure Surveys; HH, Household; HLI, High-level indicators; IMMT, Indicators, Methods, Metrics and Tools; LMIC, Low- and Middle-Income Countries; LSFF, Large-Scale Food Fortification; M&E, Monitoring & Evaluation; MDD-W, Minimum Dietary Diversity Score for Women; NARS, National Agricultural Research System; NGO, Non-Governmental Organization; OFSP, Orange Fleshed Sweet Potato; OPV, Open Pollinated Variety; SPV, Self-pollinating Variety; VPC, Vegetatively Propagated Crop; SSI, Semi-Structured Interview; ToC, Theory of Change; WRA, Women of Reproductive Age.

References

1. Development Initiatives. 2018 Global Nutrition Report: Shining a Light to Spur Action on Nutrition. Bristol: Development Initiatives (2018).

Google Scholar

2. Council for Agricultural Science and Technology [CAST]. Food Biofortification—Reaping the Benefits of Science to Overcome Hidden Hunger— A paper in the series on The Need for Agricultural Innovation to Sustainably Feed the World by 2050. (2020). Available online at: https://www.cast-science.org/publication/food-biofortification-reaping-the-benefits-of-science-to-overcome-hidden-hunger/ (accessed Dec 20, 2021).

Google Scholar

3. de Pee S, Taren D, Bloem MW. Nutrition and Health in a Developing World. 3rd Edn. Totowa, NJ: Humana Press (2017).

Google Scholar

4. Bouis HE, Saltzman A. Improving nutrition through biofortification: a review of evidence from HarvestPlus, 2003 through 2016. Glob Food Secur. (2017) 12:49–58. doi: 10.1016/j.gfs.2017.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Bouis HE, Hotz C, McClafferty B, Meenakshi JV, Pfeiffer WH. Biofortification: a new tool to reduce micronutrient malnutrition. Food Nutr Bull. (2011) 32:S31–40.

Google Scholar

6. Osendarp SJM, Martinez H, Garrett GS, Neufeld LM, De-Regil LM, Vossenaar M, et al. Large-Scale food fortification and biofortification in low- and middle-income countries: a review of programs, trends, challenges, and evidence gaps. Food Nutr Bull. (2018) 39:315–31. doi: 10.1177/0379572118774229

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Mkambula P, Birol E, Friesen VM, Munyua HM, Alberts D, Aytekin D, et al. GAIN Discussion Paper n°10 Transforming Food Systems to Deliver Nutritious Foods: The Vital Roles of Fortification and Biofortification. (2022). Available online at: doi: 10.36072/dp.10 (accessed May 23, 2022).

CrossRef Full Text | Google Scholar

8. HarvestPlus. HarvestPlus Database of Biofortified Crops Released. (2022). Available online at: https://biocropshplustst.ciat.cgiar.org/ (accessed Mar 23, 2022).

Google Scholar

9. Hotz C, Loechl C, De Brauw A, Eozenou P, Gilligan D, Moursi M, et al. A large-scale intervention to introduce orange sweet potato in rural Mozambique increases vitamin A intakes among children and women. Br J Nutr. (2012) 108:163–76. doi: 10.1017/S0007114511005174

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Foley JK, Michaux KD, Mudyahoto B, Kyazike L, Cherian B, Kalejaiye O, et al. Scaling up delivery of biofortified staple food crops globally: paths to nourishing millions. Food Nutr Bull. (2021) 42:116–32. doi: 10.1177/0379572120982501

PubMed Abstract | CrossRef Full Text | Google Scholar

11. HarvestPlus. Scaling Up Biofortified Crops and Foods for Healthier, Inclusive, and Resilient Food Systems Calls to action for the UN Food Systems and Tokyo Nutrition for Growth Summits. (2021). Available online at: doi: 10.2499/p15738coll2.134438 (accessed Jun 7, 2021).

CrossRef Full Text | Google Scholar

12. Fanzo J, Haddad L, Schneider KR, Béné C, Covic NM, Guarin A, et al. Viewpoint: Rigorous monitoring is necessary to guide food system transformation in the countdown to the 2030 global goals. Food Policy. (2021) 104:102163.

Google Scholar

13. Garton K, Kraak V, Fanzo J, Sacks G, Vandevijvere S, Haddad L, et al. A collective call to strengthen monitoring and evaluation efforts to support healthy and sustainable food systems: “The Accountability Pact.”. Public Health Nutr. (2022) 16:1–13. doi: 10.1017/S1368980022001173

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Garcia-Casal MN, Peña-Rosas JP, Pachón H, De-Regil LM, Centeno Tablante E, Flores-Urrutia MC. Staple crops biofortified with increased micronutrient content: effects on vitamin and mineral status, as well as health and cognitive function in the general population. Cochrane Database Syst Rev. (2016) 8:CD012311.

Google Scholar

15. De-Regil LM, Peña-Rosas JP, Flores-Ayala R, Del Socorro Jefferds ME. Development and use of the generic WHO/CDC logic model for vitamin and mineral interventions in public health programmes. Public Health Nutr. (2014) 17:634–9. doi: 10.1017/S1368980013000554

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Patton MQ. Qualitative Research and Evaluation Methods: Integrating Theory and Practice. Fourth Edn. Thousand Oaks, CA: SAGE Publications Ltd (2002).

Google Scholar

17. Coyne IT. Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? J Adv Nurs. (1997) 26:623–30.

Google Scholar

18. Huberman AM, Miles M, Saldana J. Qualitative Data Analysis: A Methods Sourcebook. Thousand Oaks, CA: Sage Publications (2014).

Google Scholar

19. HarvestPlus. Generic biofortified crop Theory of Change. Washington, DC: HarvestPlus (2016).

Google Scholar

20. HarvestPlus. Vitamin A Cassava Generic Theory of Change. Washington, DC: HarvestPlus (2015).

Google Scholar

21. HarvestPlus. Vitamin A Cassava Nigeria Theory of Change. Washington, DC: HarvestPlus (2015).

Google Scholar

22. HarvestPlus. Vitamin A Maize Zambia Theory of Change. Washington, DC: HarvestPlus (2016).

Google Scholar

23. Global Alliance for Improved Nutrition [GAIN]. HarvestPlus. The Commercialisation of Biofortified Crops Programme: Monitoring Reference Manual. Geneva: GAIN and HarvestPlus (2020).

Google Scholar

24. HarvestPlus. Biofortification Indicator Definition Tables (IDTs). Washington, DC: HarvestPlus (2021).

Google Scholar

25. International Potato Center [CIP]. Tools and Techniques for Monitoring and Evaluating Sweetpotato Interventions in Sub-Saharan Africa: A Practitioner’s Toolkit – Prepared by the CIP Monitoring, Learning, and Evaluation Team. Nairobi: CIP (2017).

Google Scholar

26. Adu P. A Step-by-Step Guide to Qualitative Data Coding. 1st Edn. New York, NY: Routledge (2019).

Google Scholar

27. HarvestPlus, CIP, GAIN. Harmonized Theory of Change for Scaling up Biofortification. Geneva: GAIN (2021).

Google Scholar

28. Laurie S, Faber M, Adebola P, Belete A. Biofortification of sweet potato for food and nutrition security in South Africa. Food Res Int. (2015) 76:962–70.

Google Scholar

29. Lockyer S, White A, Buttriss JL. Biofortified crops for tackling micronutrient deficiencies – what impact are these having in developing countries and could they be of relevance within Europe? Nutr Bull. (2018) 43:319–57.

Google Scholar

30. European Union, FAO, United Purpose. Baluu Tim-Maring-Ngo Project – Reducing Micronutrient Deficiencies of Women and Children in The Gambia through Sustainable and Integrated Approaches to Food Fortification. in Final Narrative Report. Rome: FAO (2018).

Google Scholar

31. European Union, FAO, United Purpose. Improving Food Security and Nutrition in the Gambia through Food Fortification. in Interim Project Report (FOOD/2016/380-042). Rome: FAO (2018).

Google Scholar

32. CIP, People in Need, Emory University, European Union. Sustained Diet Quality Improvement by Fortification with Climate-smart, Nutrition-Smart Orange-fleshed Sweetpotato in Southern Nations, Nationalities and Peoples’ Region (SNNPR). in Annual Project Report (FOOD/2016/380-038). Hawassa: SNNPR (2018).

Google Scholar

33. HarvestPlus. Indicator Reference Manual (IRM). Washington, DC: HarvestPlus (2018).

Google Scholar

34. Petry N, Wirth JP, Friesen VM, Rohner F, Nkundineza A, Chanzu E, et al. Assessing the coverage of biofortified foods: development and testing of methods and indicators in Musanze, Rwanda. Curr Dev Nutr. (2020) 4:1–8. doi: 10.1093/cdn/nzaa107

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Tuffrey V, Hall A. Methods of nutrition surveillance in low-income countries. Emerg Themes Epidemiol. (2016) 13:4.

Google Scholar

36. Coates J, Rogers BL, Blau A, Lauer J, Roba A. Filling a dietary data gap? Validation of the adult male equivalent method of estimating individual nutrient intakes from household-level data in Ethiopia and Bangladesh. Food Policy. (2017) 72:27–42.

Google Scholar

37. Fiedler JL. Towards overcoming the food consumption information gap: Strengthening household consumption and expenditures surveys for food and nutrition policymaking. Glob Food Secur. (2013) 2:56–63.

Google Scholar

38. Madzorera I, Isanaka S, Wang M, Msamanga GI, Urassa W, Hertzmark E, et al. Maternal dietary diversity and dietary quality scores in relation to adverse birth outcomes in Tanzanian women. Am J Clin Nutr. (2020) 112:695–706.

Google Scholar

39. FAO. Minimum Dietary Diversity for Women – An Updated Guide for Measurement: From Collection to Action. Rome: FAO (2021).

Google Scholar

40. Bromage S, Batis C, Bhupathiraju SN, Fawzi WW, Fung TT, Li Y, et al. Development and validation of a novel Food-Based Global Diet Quality Score (GDQS). J Nutr. (2021) 151:75S–92S. doi: 10.1093/jn/nxab244

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Moursi M, Bromage S, Fung TT, Isanaka S, Matsuzaki M, Batis C, et al. There’s an app for that: development of an application to operationalize the global diet quality score. J Nutr. (2021) 151:176S–84S. doi: 10.1093/jn/nxab196

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Baral A, Birol E. Strategic Brief: Catalyzing the Scale-up of Crop Biofortification. Washington, DC: International Food Policy Research Institute (2020).

Google Scholar

43. HarvestPlus. HarvestPlus varietal development and delivery models for biofortified crop varieties. Washington, DC: HarvestPlus (2021).

Google Scholar

44. Gatto M, Le PD, Pacillo G, Maredia M, Labarta R, Hareau G, et al. Policy options for advancing seed systems for vegetatively propagated crops in Vietnam. J Crop Improv. (2021) 35:763–89.

Google Scholar

45. Tadesse Y, Almekinders CJM, Schulte RPO, Struik PC. Tracing the seed: seed diffusion of improved potato varieties through farmers’ networks in Chencha, Ethiopia. Exp Agric. (2017) 53:481–96.

Google Scholar

46. Levinson FJ, Herforth A. Monitoring and Evaluating the Food Security and Nutrition Effects of Agricultural Projects. (2013). Available online at: www.fao.org/publications (accessed Sep 10, 2021).

Google Scholar

47. Martorell R, Ascencio M, Tacsan L, Alfaro T, Young MF, Addo OY, et al. Effectiveness evaluation of the food fortification program of Costa Rica: impact on anemia prevalence and hemoglobin concentrations in women and children. Am J Clin Nutr. (2015) 101:210–7. doi: 10.3945/ajcn.114.097709

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Pena-Rosas JP, Parvanta I, Van Der Haar F, Chapel TJ. Monitoring and evaluation in flour fortification programs: design and implementation considerations. Nutr Rev. (2008) 66:148–62. doi: 10.1111/j.1753-4887.2008.00019.x

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Aaron GJ, Friesen VM, Jungjohann S, Garrett GS, Neufeld LM, Myatt M. Coverage of large-scale food fortification of edible oil, wheat flour, and maize flour varies greatly by vehicle and country but is consistently lower among themost vulnerable: results from coverage surveys in 8 countries. J Nutr. (2017) 147:984S–94S. doi: 10.3945/jn.116.245753

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Ebata A, Thorpe J, Islam A, Sultana S, Mbuya MNN. Understanding drivers of private-sector compliance to large-scale food fortification: a case study on edible oil value chains in Bangladesh. Food Policy. (2021) 104:102127. doi: 10.1016/j.foodpol.2021.102127

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Global Alliance for Improved Nutrition [GAIN]. GAIN-HarvestPlus Partnership on Commercialisation of Biofortified Crops. (2018). Available online at: https://www.gainhealth.org/impact/programmes/commercialisation-biofortified-crops-cbc (accessed Feb 21, 2022).

Google Scholar

52. HarvestPlus. Commercialisation of Biofortified Crops Programme Expands Reach of Nutrient-Enriched Staples. (2018). Available online at: https://www.harvestplus.org/wp-content/uploads/2022/01/Commercialisation-of-Biofortified-Crops.pdf (accessed Feb 21, 2022).

Google Scholar

53. Stein AJ, Meenakshi JV, Qaim M, Nestel P, Bhutta ZA. Analyzing the Health Benefits of Biofortified Staple Crops by Means of the Disability-Adjusted Life Years Approach: a Handbook Focusing on Iron, Zinc and Vitamin A – HarvestPlus Technical Monograph 4. Washington, DC: HarvestPlus (2005).

Google Scholar

54. Dary O, Imhoff-Kunsch B. Measurement of food consumption to inform food fortification and other nutrition programs: an introduction to methods and their application. Food Nutr Bull. (2012) 33(3 Suppl.):S141–5. doi: 10.1177/15648265120333S201

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Adams KP, Bell W, Somé JW, Colaiezzi B, Wafa S, Rogers B, et al. The cost and cost efficiency of conducting a 24-h dietary recall using INDDEX24, a mobile dietary assessment platform, compared with pen-and-paper interview in Viet Nam and Burkina Faso. Br J Nutr. (2022) [Epub ahead of print]. doi: 10.1017/S0007114522001362

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Tomašev N, Cornebise J, Hutter F, Mohamed S, Picciariello A, Connelly B, et al. AI for social good: unlocking the opportunity for positive impact. Nat Commun. (2020) 11:2468. doi: 10.1038/s41467-020-15871-z

PubMed Abstract | CrossRef Full Text | Google Scholar

57. World Band Group. World Bank Country and Lending Groups. (2022). Available online at: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups

Google Scholar