A comparison of seven academic and nonacademic urban food system resilience assessment frameworks

Diverse food system resilience frameworks and assessment tools are being developed to measure food system performance in the face of disruptive events. The divide between academic research and gray literature can result in inaccessibility of assessment tools to communities and non-academic researchers. The authors performed a literature review and selected seven urban food system resilience assessment frameworks for comparison with resilience attributes, as well as consideration of their intended audience; spatial scope; data type; data collection; strengths; and ease of use. The frameworks were found to match between three and seven of the ten identified resilience attributes, with a range of intended audiences. Framework data collection methodologies included surveys, spatial data analysis, and mixed methods approaches to collect quantitative, semi-quantitative, and qualitative data. Most of the included frameworks include flexible indicators and metrics for investigators to collect relevant data for their planning goals. While the ability to develop unique metrics can be a strength, undefined metrics present an issue for non-academic researchers in communities seeking to effectively assess their own food system. Limitations in existing assessment tools include a wide range of intended outcomes and burdensome data collection. The comparison of the assessment tools resulted in recommendations of frameworks for academic and non-academic researchers and revealed gaps including a lack of fair labor considerations. This review allows researchers to develop effective frameworks for diverse users to prioritize resilience in food systems.

1 Introduction

The urban food system continues to face disruptive events and long-term stressors, including the COVID-19 pandemic and climate change. Frameworks and assessment tools are being developed at a quickening pace in response to disruptive events, particularly since 2000 (Ujjwal et al., 2024). Existing assessment frameworks and tools span diverse areas of the food system, including scales ranging from farm- and household-level assessments to sub-national and national-scale tools intended to measure, among other outcomes, nutrition security, governance, or social and economic capital.

Still, there exists a need for resilience-focused systems-level frameworks to measure the problem of food system resilience and sustainability (Worstell and Green, 2017). Food systems research requires a holistic approach that considers food system activities, socioeconomic and environmental drivers, outcomes, and feedback loops (Ericksen et al., 2010). Because resilience is a systems property, frameworks seeking to measure resilience should approach the food system at the system level with food security as a “non-negotiable goal” (Hodbod and Eakin, 2015). Regional approaches, including the city region food system approach, allow food systems spanning multiple proximal cities to be assessed as a system (Blay-Palmer et al., 2018).

Definitions of resilience typically include the ability of a system to respond over time to shocks and stressors (Tendall et al., 2015). Assessment tools should be able to measure changes in metrics of resilience to illustrate changes in resilience capacity over time. Existing frameworks present the challenge of determining what to measure and what data to collect, and developing common metrics would allow for measurement and comparison from community to community (Atoloye et al., 2023).

Several frameworks are validated and used by academics; other entities like governments, non-governmental organizations (NGOs), and community groups often develop and use their own frameworks. Building on previous reviews of food systems assessments, this article seeks to develop a table to facilitate the selection of the most appropriate food system assessment tool for academic and nonacademic practitioners. The goal of this manuscript is to (1) review assessment frameworks of food systems resilience that can be used to facilitate sustainable food systems redesign; (2) compare frameworks based on strengths, data requirements, and user-friendliness; and (3) provide a recommendation for academic and nonacademic researchers. By comparing and evaluating frameworks for assessment and development of resilient food systems, including frameworks published in both academic and gray literature, the researchers seek to provide an overview of the distinct arguments offered. A resource including data about each framework can allow food system actors, advocates, and governments to make informed decisions about appropriate frameworks for their food system assessment.

1.1 Review of food system resilience measurement approaches

There are varying definitions of resilience as it applies to food systems. Using the definition of resilience developed by Tendall et al. (2015), resilience is the “capacity over time of a food system and its units at multiple levels, to provide sufficient, appropriate and accessible food to all, in the face of various and even unforeseen disturbances” (p. 19). Béné et al. (2023) further refine this definition to include “the ability of the different individual and institutional actors of the food system to maintain, protect, or successfully recover the key functions of that system despite the impacts of disturbances” (p. 1438). Roosevelt et al. (2023) note that with the recent upward trend of research on measurement of resilience in food systems and related fields across disciplines, concepts could face degradation through conceptual disagreements.

Resilience can only be measured through proxy measurements of metrics that allow the researcher to predict the capacity of a system to return to full operation after a disruptive event. Thus, assessment tools seeking to measure resilience must measure metrics that allow the assessor to generalize about the system’s latent capacity for resilience (Béné et al., 2023).

There are gaps in existing research, particularly regarding resilience and sustainability. Sirdey et al. (2023) found that food systems assessment tools fall within a polar gradient, wherein one pole includes frameworks for quantitative data that are applicable to national-level food systems, while the other pole is a mixed-methods and participatory approach at sub-national levels. The authors state that “although resilience concept is considered in almost half of the methods, its articulation with sustainability is regarded in various ways: (i) as a means (or a condition) to achieve sustainability, (ii) as a dimension of sustainability (or associated to some of the dimensions, e.g., economic resilience), or (iii) linked to sustainability in an imprecise way,” especially in city-region level participatory assessment tools (Sirdey et al., 2023, p. 9). Roosevelt et al. (2023) found that resilience is often framed as a subconcept of sustainability, particularly in large-scale assessment tools. These examples illustrate the need for assessment tools that consider resilience as a concept more precisely than as an aspect of environmental or economic sustainability. Accordingly, Ujjwal et al. (2024) found through literature review that sustainability was the most-cited reason for development of food system resilience research. Lew et al. (2016) position sustainability and resilience as distinct but equally important concepts that offer communities a choice in perspective. Hodbod et al. (2024) support including both backwards and forwards thinking through increased participatory data collection in resilience assessment. Campbell et al. (2022) present a participatory method for developing shared values to include in assessment tools. Atoloye et al. (2023) propose a database be maintained for local food systems assessments to include common metrics.

There are varying qualitative, quantitative, and mixed methods approaches represented in food system assessment tools, including food system mapping; statistical analysis of values derived from metrices; case studies; and two- and three-dimensional matrices. Existing research on the inclusion of resilience concepts of food system assessment tools illustrates the variety of goals and measurement options (Table 1). Roosevelt et al. (2023) developed 8 adaptive capacities that should be included in resilience assessments and point to the need for a strong definition of resilience across assessment tools. Worstell and Green (2017) developed 8 criteria to place frameworks in context of a sustainability/resilience index. Béné et al. (2023) propose five emergent properties to include in resilience analysis and argue that resilience of the system cannot be separated from the well-being and resilience capacities of local households, populations, and all actors of the food system, underscoring the importance of a systems approach to resilience assessment. The first three sets of attributes are intended to guide the development of resilience assessment frameworks (Béné et al., 2023; Roosevelt et al., 2023; Worstell and Green, 2017). The final set of attributes describes existing indicators (Atoloye et al., 2023).

Table 1

Table 1. Resilience assessment attributes suggested by existing research.

Development of food system tools for resilience will be crucial to plan and develop food systems that can respond and adapt to disasters. Risk identification and management is a preliminary step in determining to what disruptive events food systems may need to be resilient (Roosevelt et al., 2023). Measures of resilience typically seek the capacity of a system to return to normal functioning after a disruptive event, Béné et al. (2023) suggest a one-analysis-to-one-event assessment to collect data directly related to resilience responses by individual actors within the food system.

2 Methods

The researchers focused this review on assessment frameworks of regional, urban, and peri-urban food systems to illustrate the connection between food systems and community resilience. We have narrowed the literature search to include only assessment frameworks of regional food systems that facilitate holistic views of the food system with a focus on resilience.

Academic frameworks were found using a PRISMA framework. The researchers used the search term ““food system*” AND framework or assessment or indicator or metric AND resilience* AND urban” in ProQuest, Scopus, Academic Search Premier, and SpringerLink. Results were limited to peer-reviewed articles in the English language published from 2000 to 2024. Papers were rejected for being about anything besides food systems and resilience; for prioritizing agricultural production or sustainability over resilience; for being national-scale assessment tools; and for strictly concerning food system governance. Duplicate results were excluded. Searches were performed in ProQuest (8,440 results); Academic Search Premier (29 results); Scopus (88 results); and SpringerLink (1,148 results).

Grey literature assessment tools were found using the same search terms in with results narrowed to non-academic journal sources published from 2000–2024: audio and video works; blogs, podcasts, and websites; conference papers and proceedings; dissertations and theses; magazines; and newspapers. Document types were limited to annual report; blog; conference, conference paper, and conference proceeding; dissertation or thesis; or essay. Searches were performed in Google Scholar (12,200 results) and ProQuest (1745 results). Further grey literature assessment tools were identified through research of other food system assessment reports. In both cases, papers that provided a framework for measuring resilience in food systems with indicators and metrics were retained. Most papers were rejected for prioritizing sustainability over resilience. Search biases in our review include the small number of databases searched. Bias in the gray literature search includes difficulty in finding gray literature through a PRISMA review, leading to possible researcher bias.

The frameworks were compared with the local food system resilience attributes found in “Food System Resilience: A Planning Guide for Local Governments” by the Johns Hopkins Center for a Livable Future (Moore et al., 2022). The researchers chose this set of attributes because the 10 attributes overlapped generally with the other three sets of attributes presented in Table 1; the inclusion of 4 equity attributes facilitate assessment tool analysis in accordance with principles of equity and food justice. Frameworks were labelled with intended audience; ease of use; spatial scopes; strengths; and adherence to resilience attributes (Table 2).

Table 2

Table 2. Comparison of food system assessment tools with resilience attributes and relevant descriptors.

3 Results

Three academic frameworks were considered. “Resilient Food Systems: A Qualitative Tool for Measuring Food Resilience” (Toth et al., 2016) is a modeling approach with nodes (food system activities) and links (flows of food) to find R values, wherein a lower R value equals higher resilience. “A Framework to Assess the Resilience of Farming Systems” (Meuwissen et al., 2019) has been used throughout Europe and measures both specific and general resilience, including resilience of what; to what; for what purpose; what resilience capacities exist; and what enhances resilience. “Food System Resilience Measurement: Principles, Framework and Caveats” (Béné et al., 2023) includes resilience capacities and emergent properties for system analysis and assessment of individual and food system response to one disruptive event.

Four gray literature frameworks were considered. Community & Agriculture Resilience Audit Tool (CARAT), North American Food Systems Network (NAFSN) (2023) is a community self-assessment tool to gather data about food system asset use. The Resilience of America’s Urban Food Systems: Evidence from Five Cities (Zeuli and Nijhuis, 2017) is intended to identify vulnerabilities and analyze at-risk areas of the food system for disaster preparedness planning. Self-Evaluation and Holistic Assessment of Climate Resilience of Farmers and Pastoralists (SHARP) (Hernández Lagana et al., 2022) is a self-administered survey that includes questions in 13 behavior-based indicators to assess the resilience of households and agroecosystems. The City Region Food System (CRFS) Indicator Framework (Carey and Dubbeling, 2017) is an assessment and planning tool with a whole-system approach to gather data, identify weaknesses, and inform planning decisions.

Table 2 illustrates the review of assessment tools. No tool matched all 10 resilience attributes. Scores ranged from 3 to 7 resilience attributes, with diversity being the most used and structural and intergenerational equity being the least. The intended audiences included planners and policy advocates; academic and nonacademic researchers; pastoralists; and community and food system actors. The tools represent a diversity of methods and approaches, ranging from modeling frameworks to self-administered surveys. A strength in existing tools is an emphasis on resilience and identifying vulnerabilities within food systems. The frameworks are applicable on multiple scales and appropriate for assessment at the systems level. SHARP+, Meuwissen et al. (2019), and Béné et al. (2023) allow for assessment of individual households, farms, and actors as part of systems assessment.

Each framework contains indicators that may align with resilience attributes and frames resilience differently, as observed by varying attributes included (Table 2). Indicators are described by metrics, which are relevant data points. Metrics can be used to measure change over time and allow for the interpretation of indicators to determine the resilience of the food system. SHARP+ describes indicators as “overarching principles” that are useful to contextualize metrics and provide consistency and a systemic approach to assessment (Hernández Lagana et al., 2022). The frameworks take varying approaches to indicators and metrics. The modeling approach developed by Toth et al. (2016) uses attributes, such as length, fragility, and capacity, and factors, such as cost, time, and redundancy, to describe food system links; these are analogous to indicators and metrics. Béné et al. (2023) include the collective resilience capacity indicator “level of agency, inclusion and empowerment” in the food system; the indicator meets the resilience attribute of capital reserves, particularly social capital. The metric is unspecified semi-quantitative data suggested to be based on community-level resilience research of food system actors. The CRFS Indicator Framework includes the indicator “number of food education services that involve other food system actors (farmers, cooks, food vendors, policy makers) as educators,” which meets the resilience attribute of connectivity. The authors suggest 7 metrics to collect data for this indicator, including “reports from city food networks or community food organizations or NGO’s” and “school food education surveys” (Carey and Dubbeling, 2017).

4 Discussion

While frameworks and assessment tools are often developed to be broadly applicable, variations in each food system preclude any tool from fitting every situation perfectly (Campbell et al., 2022). Many farm resilience assessments do not include adequate consideration of farm labor conditions. Perrin et al. (2024) found that resilient farms are not necessarily associated with fair working conditions, and otherwise resilient farms can have poor conditions for labor. Of the frameworks included in this study, labor is generally considered from an economic or social capital perspective; the CRFS Indicator Framework includes the largest focus on fair labor conditions. Urban agriculture has benefits for sustainable cities, but there is a research gap in planning and governance, pointing to the need for inclusion of governmental, planning, and policy support in assessments (Tapia et al., 2021).

Though adjusting the assessment area to include a city region can provide system-level data, increased data collection presents a challenge (Meuwissen et al., 2019). There is also a significant time and resource commitment in practical application of the included tools that increases with scale. The meso or regional level of food system is also open to changes and feedback from systems at both the micro (for example, farm-level) and macro (global) scale, and each scale faces different constraints and challenges (Ge et al., 2016). Ujjwal et al. (2024) found that assessment is important at granular levels to capture interactions between systems and actors, in addition to coarser sub-national and national levels. Each included tool allows users the option to scale data collection up or down at a minimum from city to city-region, with SHARP+ allowing for collection of household-level data and Toth et al. (2016) being applicable to subnational levels.

Gaps in existing frameworks include disparity of expected outcomes; disagreement on concept definitions; and the burden of data collection, particularly when data are suggested but not prescribed. Researchers note the importance of customizing indicators and metrics for specific food systems. Campbell et al. (2022) describe a methodology for developing metrices that are broadly applicable to a food system through gaining approval of a panel of local food system experts. Toth et al. (2016) state that “Indicators provide insight into the characteristics of a system, but do not reveal the gaps or missing links in that system.” Existing assessment tools offer starting points for interested parties to begin collecting data about food system resilience; gaps can be filled through development of shared or unique metrics to collect system-specific data. This can pose a limitation for some users, but it also allows significant freedom for users to identify their own data collection needs.

The process of conducting a food system assessment increases the need for stakeholder and community participation, pointing to the possibility of tools facilitating collaboration for participatory food system redesign. Results of the assessment can provide needed data and evidence for municipal policy and planning advocacy, funding opportunities, and program partnerships to foster resilience capacities.

Academic researchers may effectively use Béné et al. (2023), which matches 5 resilience attributes, to collect system-wide resilience data; however, the extensive methodology at play may exclude non-academic researchers. Both CARAT and Zeuli and Nijhuis (2017) match 7 resilience attributes; provide defined indicators; and allow for customization of metrics, making them appropriate for both academic and non-academic researchers. Researchers and stakeholders seeking an overview of community resilience strengths and vulnerabilities can use the former, while researchers seeking to improve disaster planning can use the latter.

5 Conclusion

Developing resilient urban food systems with validated assessment frameworks aligns with several of the UN’s Sustainable Development Goals, including 1: No Poverty; 2: Zero Hunger; 3: Good Health & Wellbeing; and 11: Sustainable Cities & Communities. Appropriate resilience assessment tools are vital to show existing strengths and weaknesses; to create equitable food systems with fair labor practices; provide food security and food sovereignty; and support strong communities that respond to and recover from disruptive events.

We identified seven assessment frameworks that can be used to study food systems resilience at the community or regional scale. Table 2 presents concise information for interested parties to choose an appropriate assessment tool. Examining gaps in existing food systems assessment tools supports the goal of developing unified measures of resilience and answers the call to collect quantitative data of resilience metrics (Worstell and Green, 2017). Development and deployment of resilience assessments will allow researchers to answer questions about food system redesign for long-term resilience (Hodbod and Eakin, 2015), decrease inequality and develop general and specific resilience (Campbell et al., 2022), and identify priorities for redesign (Roosevelt et al., 2023). Continued development and comparison of frameworks and assessment goals will help to drive research on resilient food systems. Future research should develop shared resilience definitions and indicators specifically for urban food systems to produce effective and user-friendly tools for diverse food system researchers and practitioners.

Author contributions

KW: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. TJ: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Writing – review & editing. DK: Conceptualization, Investigation, Methodology, Writing – review & editing, Data curation. RS: Funding acquisition, Writing – review & editing, Conceptualization, Methodology. EP: Writing – review & editing, Funding acquisition, Supervision. PB: Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This work was supported by Kansas State University’s “Game-changing Research Initiation Program.” Project title: “Development of Resilient Urban Food Systems That Ensure Food Security in the Face of Climate Change”.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Atoloye, A., Schouboe, S., Misiaszek, C., Harding, J., Cooksey Stowers, K., Bassarab, K., et al. (2023). Developing a food system indicators database to facilitate local food systems assessments: using a scoping review approach. J. Agri. Food Syst. Commun. Develop 13, 1–14. doi: 10.5304/jafscd.2023.131.008

Crossref Full Text | Google Scholar

Béné, C., Frankenberger, T. R., Nelson, S., Constas, M. A., Collins, G., Langworthy, M., et al. (2023). Food system resilience measurement: principles, framework and caveats. Food Secur. 15, 1437–1458. doi: 10.1007/s12571-023-01407-y

Crossref Full Text | Google Scholar

Blay-Palmer, A., Santini, G., Dubbeling, M., Renting, H., Taguchi, M., and Giordano, T. (2018). Validating the City region food system approach: enacting inclusive, Transformational City region food systems. Sustain. For. 10:1680. doi: 10.3390/su10051680

Crossref Full Text | Google Scholar

Campbell, C., Papanek, A., DeLong, A., Diaz, J., Gusto, C., and Tropp, D. (2022). Community food systems resilience: values, benefits, and indicators. J. Agri. Food Syst Commun. Develop. 11, 1–25. doi: 10.5304/jafscd.2022.114.006

PubMed Abstract | Crossref Full Text | Google Scholar

Carey, J., and Dubbeling, M. (2017). "City region food system Indicator framework." RUAF foundation, food and agriculture Organization of the United Nations, and Wilfred Laurier university. Available online at:https://ruaf.org/document/city-region-food-system-indicator-framework/.

Google Scholar

Ericksen, P., Stewart, B., Dixon, J., Barling, D., Loring, P., Anderson, M., et al. (2010). "The value of a food system approach," in food security and global environmental change. Oxford: Taylor & Francis Group.

Google Scholar

Ge, L., Anten, N. P. R., van Dixhoorn, I. D. E., Feindt, P. H., Kramer, K., Leemans, R., et al. (2016). Why we need resilience thinking to meet societal challenges in bio-based production systems. Curr. Opin. Environ. Sustain. 23, 17–27. doi: 10.1016/j.cosust.2016.11.009

PubMed Abstract | Crossref Full Text | Google Scholar

Hernández Lagana, M., Phillips, S., and Poisot, A. S. (2022). Self-evaluation and holistic assessment of climate resilience of farmers and pastoralists (SHARP+)--a new guidance document for practitioners. Rome Food Agri. Organ. United Nations. doi: 10.4060/cb7399en

Crossref Full Text | Google Scholar

Hodbod, J., and Eakin, H. (2015). Adapting a social-ecological resilience framework for food systems. J. Environ. Stud. Sci. 5, 474–484. doi: 10.1007/s13412-015-0280-6

PubMed Abstract | Crossref Full Text | Google Scholar

Hodbod, J., Goralnik, L., Vicari, L., and White, S. (2024). From theory to transdisciplinary practice: community-based resilience visioning in urban agriculture. Soc. Nat. Resour. 37, 143–167. doi: 10.1080/08941920.2023.2228264

Crossref Full Text | Google Scholar

Lew, A., Ng, P., Ni, C., and Wu, T.-C. (2016). Community sustainability and resilience: similarities, differences and indicators. Tour. Geogr. 18, 18–27. doi: 10.1080/14616688.2015.1122664

Crossref Full Text | Google Scholar

Meuwissen, M. P. M., Feindt, P. H., Spiegel, A., Termeer, C. J. A. M., Mathijs, E., de Mey, Y., et al. (2019). A framework to assess the resilience of farming systems. Agric. Syst. 176:102656. doi: 10.1016/j.agsy.2019.102656

PubMed Abstract | Crossref Full Text | Google Scholar

Moore, E., Biehl, E., Burke, M., Bassarab, K., Misiaszek, C., and Neff, R. (2022). "Food system resilience: a planning guide for local governments. John Hopkins Center for a Livable Future." Available online at: https://clf.jhsph.edu/publications/food-system-resilience-planning-guide-local-governments

Google Scholar

North American Food Systems Network (NAFSN). (2023). "Community & Agriculture Resilience Audit Tool (CARAT)." Available online at: https://carattool.org/

Google Scholar

Perrin, A., Cournut, S., and Martin, G. (2024). Further consideration of working conditions is needed in farm resilience assessment. Agric. Syst. 214:103845. doi: 10.1016/j.agsy.2023.103845

PubMed Abstract | Crossref Full Text | Google Scholar

Roosevelt, M., Raile, E. D., and Anderson, J. R. (2023). Resilience in food systems: concepts and measurement options in an expanding research agenda. Agronomy 13:444. doi: 10.3390/agronomy13020444

Crossref Full Text | Google Scholar

Sirdey, N., David-Benz, H., and Deshons, A. (2023). Methodological approaches to assess food systems sustainability: a literature review. Glob. Food Sec. 38:100696. doi: 10.1016/j.gfs.2023.100696

PubMed Abstract | Crossref Full Text | Google Scholar

Tapia, C., Randall, L., Wang, S., and Aguiar Borges, L. (2021). Monitoring the contribution of urban agriculture to urban sustainability: an Indicator-based framework. Sustain. Cities Soc. 74:103130. doi: 10.1016/j.scs.2021.103130

PubMed Abstract | Crossref Full Text | Google Scholar

Tendall, D. M., Joerin, J., Kopainsky, B., Edwards, P., Shreck, A., Le, Q. B., et al. (2015). Food system resilience: defining the concept. Glob. Food Sec. 6, 17–23. doi: 10.1016/j.gfs.2015.08.001

PubMed Abstract | Crossref Full Text | Google Scholar

Toth, A., Rendall, S., and Reitsma, F. (2016). Resilient food systems: a qualitative tool for measuring food resilience. Urban Ecosyst. 19, 19–43. doi: 10.1007/s11252-015-0489-x

Crossref Full Text | Google Scholar

Ujjwal, K. C., Campbell-Ross, H., Godde, C., Friedman, R., Lim-Camacho, L., and Crimp, S. (2024). A systematic review of the evolution of food system resilience assessment. Glob. Food Sec. 40:100744. doi: 10.1016/j.gfs.2024.100744

Crossref Full Text | Google Scholar

Worstell, J., and Green, J. (2017). Eight qualities of resilient food systems: toward a sustainability/resilience index. J. Agri. Food Syst Commun. Develop. 7, 1–19. doi: 10.5304/jafscd.2017.073.001

Crossref Full Text | Google Scholar

Zeuli, K., and Nijhuis, A. (2017). “The resilience of America’s urban food systems: evidence from five cities. The Initiative for a Competitive Urban City.” Available online at:https://icic.org/wp-content/uploads/2017/01/ROCK_Resilient_Food_f2.pdf

Google Scholar