A curricular model to train doctoral students in interdisciplinary research at the food-energy-water nexus

1. Introduction

Food, energy and water (FEW) systems are critically stressed worldwide. Global challenges related to economic development from the perspective of linkages among food, energy, and water, known as the FEW nexus, were formally recognized at the 2011 World Economic Forum [World Economic Forum (WEF) Water Initiative, 2011]. Since then, phenomena such as climate change, food insecurity, droughts, and public health crises including the COVID-19 pandemic are increasingly viewed as being highly interconnected, representing “wicked” challenges that require transformative science, engineering and policy solutions (Hoff, 2011; Calder et al., 2021). FEW nexus research, to date, recognizes the inextricable linkages between FEW systems and emphasizes an interconnected approach to science, policy and practice focused on FEW nexus solutions [Food and Agriculture Organization of the United Nations (FAO), 2014; Dodds and Bartram, 2016]. This approach is based on the awareness that these systems are interdependent, and it is impossible to address problems of any individual component of the nexus without considering the impacts on the other two (Hoff, 2011; De Laurentiis et al., 2016).

Nevertheless, research universities typically focus on traditional science, technology, engineering, or mathematics (STEM) education models that emphasize expertise in highly specialized fields (Begg et al., 2015; Bosch and Casadevall, 2017). Specifically, graduate training often takes place in academic silos in which students are trained in discipline-specific theory, methods, and applications (Esler et al., 2016). However, problems at the FEW nexus span complex geographic, temporal, socioeconomic, and governance scales, requiring integration of physical, biological, and social sciences, engineering and engagement with multiple stakeholders (Rodríguez et al., 2019). Furthermore, the majority of STEM graduate programs do not have formal professional training for skills such as technical writing, communicating to diverse audiences, budget and project management, leadership, mentorship, and conflict resolution, leaving many graduates poorly prepared for success across diverse career pathways (Bosch and Casadevall, 2017; Denecke et al., 2017). Hence, there is an urgent need to develop educational models that focus on the interdependencies among FEW systems (D'Odorico et al., 2018), train the next generation of FEW nexus professionals in interdisciplinary research and systems thinking (Aboelela et al., 2007; Klein, 2014; Bosque-Pérez et al., 2016), and arm these future leaders with the transferable professional skills that will support success across multiple sectors.

This need for the integration of diverse perspectives requires innovative STEM graduate education models that focus on interdisciplinary training. The integration of systems thinking approaches in interdisciplinary curricula (Mandinach and Cline, 1993; Mayer and Kumano, 1999; Meadows, 2008; Orgill et al., 2019) represents a particularly important advancement in educating future leaders to be poised to address many of the global challenges currently facing humanity. The application of systems thinking within FEW nexus training programs, particularly at the graduate student level, is imperative to the success of future FEW nexus researchers.

To address this need, University of Maryland (UMD) faculty obtained funding from the National Science Foundation (NSF) to develop an interdisciplinary, experiential graduate education program focused on innovations at the FEW nexus. This program, the UMD Global STEWARDS (STEM Training at the Nexus of Energy, WAter Reuse and FooD Systems) NSF Research Traineeship (NRT), enrolls a cohort of doctoral students from multiple schools/colleges annually over a five-year period. As part of the program, we offer a two-course sequence over a calendar year: the first course provides an experiential introduction to broad food, energy, water topics and systems thinking at the FEW nexus; and the second course is a data practicum. The two courses are linked through an interdisciplinary FEW systems research project conducted in teams of three students that begins during the first course in the spring semester and is completed at the end of the second course in the fall semester.

Here, we explore how the two courses complemented each other to train PhD students to be collaborative interdisciplinary scientists at the FEW nexus. Specifically, we ask the following research questions: 1) To what extent did students report that they acquired skills and areas of confidence that were promoted over the two-course sequence? 2) What products resulted from the interdisciplinary FEW systems research project completed throughout the two courses? and 3) What improvements have been made to the project experience?

2. Materials and methods

2.1. Context of study

The University of Maryland (UMD) is a public, research-intensive university located on the east coast of the United States with over 40,000 students enrolled in more than 200 undergraduate and graduate programs. The UMD Global STEWARDS NSF NRT was founded in 2018, and the program aims to train UMD PhD students from a wide array of disciplines with the interdisciplinary research, communication and professional skills needed to translate research discoveries into actionable science at the FEW nexus. The program has multiple elements, including the two-course sequence that is the focus of this study, weekly seminars, outreach and mentoring, an optional domestic internship, an optional short-term faculty led study abroad trip, and an annual intensive professional development workshop series (Figure 1). Specifically, the program focuses on developing students’ skills in interdisciplinary research to address challenges at the FEW nexus. The program also emphasizes refining students’ written and oral communication skills, with a focus on communication to diverse disciplines and audiences.

FIGURE 1

Figure 1. Overview of the UMD Global STEWARDS major program activities.

2.2. The two-course sequence

The first course is a 3-credit course taught in the Spring semester that provides an experiential introduction to broad FEW nexus topics, focusing on how integration across the biological, physical, social, behavioral, computer and engineering sciences will be critical in solving FEW systems challenges. The course also emphasizes the development of interdisciplinary research skills and communication skills appropriate for diverse audiences (Murray et al., 2021). The course consists of lectures, expert guest speakers, student-led discussions, field trips, and case studies focused on domestic and international FEW challenges (Supplementary Table S1). Students gain an appreciation for different writing styles in science communication through class assignments such as writing policy memos, Op-Eds, and short research papers. In addition, oral science communication skills are honed through five-minute lightning rounds, short research presentations and group project presentations. The first iteration of the course in 2019 was taught completely in person. Due to the COVID-19 pandemic, the latter half of the second iteration and entire third iteration of the sequence were taught online.

The second course, also a 3-credit course, is taught in the following Fall semester and is designed to provide hands-on practice in working on an interdisciplinary team to address problems at the FEW nexus. The course consists of lectures, expert guest speakers, class discussions, and group work time with live instructor support. The course components support PhD student growth in working with interdisciplinary teams, conflict management, and enhancing presentation and communication skills.

The two courses are linked by the interdisciplinary FEW systems research project which begins in the first course, is completed in the second course, and results in an actionable deliverable. Students from different disciplines form interdisciplinary groups who work together on a project deliverable, such as a scientific manuscript, grant proposal, OpEd, or outreach materials. The students form their groups and choose their research topic at the beginning of the introductory course and work together throughout the two courses and summer break. Motivation for creating this two-course sequence is outlined in section 3. The overarching goals of this specific two-course sequence are to:

1. Enhance interdisciplinary knowledge at the FEW nexus

2. Promote communication skills appropriate for diverse audiences, including multiple scientific and academic disciplines, the general public, and varying career sectors (such as academia, industry, government, nonprofit)

3. Broaden interdisciplinary research skills (e.g., data collection, analysis, and interpretation) to explore problems and generate solutions relevant to the FEW nexus

4. Increase collaborative skills with a range of professionals (including individuals in academia, industry, government and nonprofit) and scientists outside of the students’ primary academic discipline

2.3. Study participants

We collected data from the first three iterations of the courses over three consecutive years. Overall, 33 PhD students participated in the sequence thus far (12 in 2019; 11 in 2020; 10 in 2021). The participants were diverse in terms of race/ethnicity, gender, year in their doctoral program, career goal, and academic discipline (Table 1). Most of the participants were female (70%), White (52%), and more than half of the students have been in their doctoral programs for 2 years or less. Almost half of the students identify as non-White (48%), and 12% as being Hispanic or Latino. Students came to our program from 10 different departments/units on our campus, with the majority being from public health (24%), environmental sciences (21%) and plant sciences (12%). Before beginning the program, students were asked to indicate the career options that they were interested in pursuing after graduation, with the ability to select more than one option. Most selected multiple options (Table 1). A career in government was the most popular career sector (54%), followed by academia (52%), non-profit (45%), then industry (30%).

TABLE 1

Table 1. UMD Global STEWARDS demographics, cohorts 1–3.

2.4. Data collection and analysis

We draw upon the ongoing internal evaluation program which utilizes a mixed method, multi-informant evaluation that includes surveys, focus groups, observations, and artifact collection. We used an adapted version of a validated pre-post-survey (O’Meara and Culpepper, 2018; McKee et al., 2021) to collect baseline data and end-of-semester data. Face validity of the adapted survey was established through our science education faculty member, graduate assistant (both members of the evaluation team), the course instructor, and the program manager. Prior to distribution of the survey, a science education graduate student completed the survey to verify content validity. Validity evidence based on content is focused on the relationship between the content of a survey and the construct it is intended to measure (American Educational Research Association, 2014). Such validity evidence ensures a match between the domain measured (e.g., skills acquired during the two-course sequence) and the content of the test (e.g., the specific items on the survey). The interviews and focus groups provided evidence of validity based on response processes, a concept described by the American Educational Research Association (AERA) as “the fit between the construct and the detailed nature of the performance or response actually engaged in by test takers” (American Educational Research Association, 2014). The focus groups also offered an opportunity to gather evidence on instrument validity respondent think-aloud procedures. Reeves and Marbach-Ad (2016) noted that during think-aloud, respondents can “verbally explain and rationalize their thought processes and responses” (Reeves and Marbach-Ad, 2016, 4), allowing for recording, transcription, analysis, and interpretation of validity by focus group administrators.

The surveys were administered online via Qualtrics survey software (Qualtrics Software, 2016) and included questions about the students’ experience in the courses, self-assessment of their own skills, and level of confidence using a variety of question formats including scale-response and open-ended questions. An example of a post-course survey is provided in the Supplemental Material 2 section. All fellows in all cohorts (n = 33) completed the surveys. Informed consent (written for surveys and oral for interviews) included a disclaimer that only the evaluation team would have access to identifiable data, and the leadership team would have access to aggregated, de-identified data. Focus groups were conducted with all students from the three cohorts following their completion of each course, audio recorded, and transcribed for analysis. Individual interviews (n = 6) were conducted only with students from the first cohort, who reflected the diversity of the program.¹

For the scale questions on the survey, we calculated means and standard deviations of student reports (Figures 2, 3). Students ranked their confidence in mastering skills on a five-level scale (1 = Not at all, 2 = Not much, 3 = Somewhat, 4 = To a good extent, 5 = To a great extent) before and after the two-course sequence. The Wilcoxon signed rank test was used to compare pre- and post-means. Error bars represent ± SD, which corresponds to a 95% confidence interval for each item. Students also rated whether or not the course activities were presented at the appropriate frequency on a three-level scale (1 = Not enough, 2 = Sufficient, 3 = Too much), and we report on the number of students who chose each level for each activity (Figure 4).

FIGURE 2

Figure 2. Average and standard deviation of students’ reported gains (n = 33) divided to cohorts on a 5-level scale (1 = Not at all, 2 = Not much, 3 = Somewhat, 4 = To a good extent, 5 = To a great extent).

FIGURE 3

Figure 3. Average and standard deviation of student reports of their own confidence (1 = Not at all, 2 = Not much, 3 = Somewhat, 4 = To a good extent, 5 = To a great extent) in interdisciplinary research skills, collaboration, communication and cultural competence, before and after the two-course sequence. The Wilcoxon signed rank test was used to compare pre- and post-means. Error bars represent ± SD, which corresponds to a 95% confidence interval for each item; *** p < 0.001; ** p < 0.01; * p < 0.05. Most data were collected from all three cohorts (n = 33). Several items were only collected from cohort 2 and 3 (n = 21), and they are represented in the figure with †.

FIGURE 4

Figure 4. Frequency of student responses to the survey prompt: “Please rate (1 = Not enough, 2 = Sufficient, 3 = Too much) whether or not the course activities were presented at the appropriate frequency”.

The focus group interviews were done at the end of each course (Spring and Fall) as part of the last class session, so the participation rate was 100%. Only the evaluation team was present, and the focus group was 1 hour long. Prior to the focus group, the evaluation team collected the surveys, and the focus group goal was to gain a deeper understanding of the responses that were collected through the surveys through probing. Prior to conducting the focus group, two authors, who are also members of the evaluation team, separately reviewed all of the open-ended responses in the survey (e.g., list two skills that you gained from the introductory course) and coded the responses into several themes based on their context (e.g., Oral and Written Communication, collaboration; see Tables 2, 3 for themes and quotes). After initial coding was performed, there was a high agreement between the coders. Any disagreements were negotiated between the coders until they reached 100% agreement (Saldaña, 2015). Individual quotes from the open-ended survey questions and the focus group were also used to support and contextualize findings that emerged from the quantitative analysis. Quotes have been lightly edited for conciseness and clarity; verbatim quotations are available upon request. The UMD Institutional Review Board (IRB) approved all materials and procedures of the interdisciplinary FEW systems research projects.

TABLE 2

Table 2. Themes and example responses to the open-ended question on the survey regarding the most important things they gained from the introductory course (MIEH 690).

TABLE 3

Table 3. Themes and example responses to the end of the data practicum open-ended question on the survey regarding the most important things they gained from the course.

3. Results

The UMD Global STEWARDS Program was envisioned as a curriculum that would support interdisciplinary education and collaboration of PhD students working at the FEW nexus. In order to illustrate the motivation behind developing the specific course sequence, the evaluation team interviewed the course instructors. Both instructors are tenured research faculty members with extensive experience mentoring doctoral students, which they drew upon when creating the course content and course sequence. The instructor of the data practicum course highlighted the importance of engaging graduate students in interdisciplinary team research projects since this now a common practice in the workplace, and explained how the sequence of courses supported this endeavor. She explained,

…to me, the uniqueness of this series of courses is the fact that you have a group project that links the two courses, but you're provided with the tools. So the students are effectively able to engage in team research, which is really hard at the PhD level. And they have the tools to work outside their area of expertise because we give them these tools through the two courses to allow them to work in these teams.

A major emphasis of the sequence design was to bring to the courses different stakeholder (from academia, industry, policy, non-profit organizations) that serve as guest speakers and project mentors, and allow students to experience different types of research products (policy memo, academic paper, grant proposal). The instructor of the experiential course stated,

The other unique thing is the fact that we bring stakeholders in so they might have an output. It might be a proposal, it might be an academic paper, or a forward-facing website, for example, or storyboard. But it's based on identifying stakeholders outside of their area of expertise and working with those stakeholders to produce a unique product that's a team product that's outside their area of expertise. So it's just a unique experience that most PhD students don't get because Ph.D. students tend to just work inside their own framework… But here they're working not only outside of their area of expertise, but they're working with outside stakeholders… They're producing something that they wouldn't normally produce inside their own program, and they're working as a part of an interdisciplinary team.

This instructor continued to explain how working on the projects expose students to real life situations with the pros and cons, benefit and challenges, of working with diverse group of students and stakeholders,

[We are] trying to get the students more exposure to real life situations where you need to engage with all the good and bad that come with stakeholder engagement. Sometimes it's really challenging because people disagree. I mean, people disagree about the way that the project should be worked. So I really do think there's value in just talking about this …

The experiential course instructor also noted the importance of exposing students to career opportunities that they are not usually exposed to in their PhD program, as explained in the following quote:

We are trying to engage with these outside groups and nonprofits, folks at other institutions to basically expand the net. It effectively expands the network of our students as well, and they get to engage with people at nonprofits, and that might be a career pathway for them. Another place that I'm aware that they do this is the Yale Environmental Law Clinic, where it's the same concept that I mean, you're bringing interdisciplinary groups together from different schools to work with an outside group on a problem that ends up having a deliverable that is an actionable item…These students are so focused that they don't necessarily have the tools to work in an interdisciplinary environment, and this allows them to not only gain those tools but have an experience of succeeding in interdisciplinary work during their Ph.D. time.

Another goal that led to the design of the course sequence and the projects was introducing students to new research methods outside of those typically used in their specific disciplines. The instructor of the experiential course provided a specific example,

…last year, we know that [fellow name] was in our program, the fact that she met [a program stakeholder] and learned about rapid ethnographic assessment completely changed the way that she thought about her dissertation research. And now she's using that method as a huge part of her dissertation research. And she never would have known that method if she hadn't been part of this program.

Lastly, both instructors highlighted the importance of continuity of the projects and allowing later cohorts to build on previous cohort projects. The instructor of the data practicum said, “… the fact that we do it for every year it brings some of the projects have continuity, from year to year and they are able to build off of each other.”

3.1. Research question 1: to what extent did students report that they acquired skills and areas of confidence that were promoted over the two-course sequence?

To streamline our responses to this research question, we considered the results in terms of overall program goals.

3.1.1. Goal 1: enhance interdisciplinary knowledge at the FEW nexus

The main goal of the introductory course (MIEH 690) is to enhance PhD students’ interdisciplinary knowledge at the FEW nexus. Specifically, this course is designed so that at the end of the course students are able to:

1. Identify major food, energy, and water systems challenges from local to global scales.

2. Conceptualize and articulate interplays between food-energy-water systems from local to global scales.

Upon completing the course, we asked each cohort of students to rate the extent (1 = not at all, 2 = not much, 3 = somewhat, 4 = to a good extent, 5 = to a great extent) to which they gained or improved in each of the two learning outcomes from taking the introductory course. Figure 2 shows means of student ratings and standard deviation (SD) divided by cohorts. It is noteworthy that for both of the items an improvement was reported from cohort 1 to cohort 3, which could possibly be attributed to improvements made to the course as a result of feedback from students. Overall, the mean rating of these skills in the second and third cohorts were 4 and above (agree to a good/great extent).

These quantitative ratings were corroborated through qualitative data (open-ended questions) collected from students through surveys and interviews. Table 2 shows that when students were asked to list two skills that they gained from the introductory course, FEW content knowledge was the second most frequently mentioned area of skills gained (14 out of 33 students). One student explained, “[I gained] broad knowledge in the FEW nexus areas which were new to me before I joined the program.” Another student suggested that the knowledge they gained related to their field of study: “I gained better insight into the Food-Energy-Water nexus and have become more appreciative of its inter-connectedness to my current field of study.” Yet another student specified that the course provided “A great overview of FEW Nexus research both at a microscopic and macroscopic scales.”

Responding to the question “Has your view of FEW systems changed?” in an individual interview following the introductory course, one student referred to gaining an understanding about FEW systems challenges from different scale perspectives:

Yes, like before that, FEW systems for me – I always think in a big scale. Like within a country, within a region, within a whole world how it can work. But in this class, we came to know how it should start from the beginning base microscale – or from your house from your family – and then how can you change your community scale, and then state level and then others. So it was nice to think from the top to bottom to top.

Other students explained that the course helped them understand that FEW nexus topics are applicable to everyday life and learn the importance of communicating it to people without a scientific background, “… I think the FEW systems should be modeled so it is relatable to the people who are nonscientific because it’s quite important.” Yet another student mentioned, “now I’m looking at it much more as a decision-making tool for people who are trying to manage.”

Following the data practicum course (MIEH 691), students (n = 9) also emphasized gains related to FEW nexus content knowledge (Table 2). One student highlighted how “Learning about potential career options is broadening interest in the FEW nexus and making me more confident in my knowledge surrounding sustainability.” In the individual interviews, one student explained that they are now better able to mentor undergraduate students that are working in their lab to understand the systematic view of FEW nexus, sharing the following quote:

So as we have learned throughout the course all of our departments are quite siloed, we only focus on one aspect of the system, it could be climate stuff, it could be water stuff, it could be let’s say about the surface processes a lot. So through this knowledge about the FEW systems, interactions, and trade-offs, I could introduce to [the undergraduate mentees] a number of the feedback systems as well as the interaction between systems and incorporate them to climate, or surface processes, or whatever they’re working on.

Another student described how their advisor used material that they developed in the introductory course to teach the advisor’s undergraduate course, “… I basically hand over the case study that I developed to my advisor so that she can use it in her class. Because …it was an insight from actually teaching her class that sparked the case study so it fits right into her curriculum.”

3.1.2. Goal 2: promote communication skills appropriate for diverse audiences, including different disciplines, the public, and multiple sectors (academia, government, industry, nonprofit)

One of the most prominent goals—of both the UMD Global STEWARDS NSF NRT program in general and the two-sequence course in particular—is to promote PhD students’ communication skills, especially with regard to communicating their research to diverse audiences. In an effort to promote this goal, both courses involve students practicing their writing and presentation skills across multiple assignments (Supplementary Table S1). Specifically, the main course learning outcomes related to this goal state that at the end of the two-course sequence, students will be able to:

1. Explain to peers the most important aspects of your research and why it is important.

2. Explain to non-academics the most important aspects of your research and why it is important.

3. Take the results of research from your field of study and translate them for other applications.

4. Explain how your research connects to issues that are important to society.

For the first three outcomes, Figure 3 shows that students reported significant growth in confidence from before the course sequence (means: Cohort 1 = 3.78, Cohort 2 = 3.69, Cohort 3 = 3.31 out of 5) to after (means: 4.21, 4.03, 3.88 respectively). For the fourth outcome, “Explain how your research connects to issues that are important to society,” students’ rating was high (mean = 4.19 out of 5) even before the two-course sequence with no significant difference at the end of the course sequence.

Students’ high ratings and significant confidence growth in communicating their research to others were corroborated by their open ended-responses to the survey. After the experiential introduction course (Table 2), communication was the skill that was mentioned the most as one of the two most important things that students gained in the course (n = 21). One student wrote that they “[learned] how to do presentation in front of people from other discipline.” Another mentioned that they appreciated the “… communication skills in regard to drawing parallels between language/jargon of different disciplines.”

Students also referred specifically to the writing assignments and the feedback that they received from the instructors (“…writing is challenging but I love the feedback from the instructor. It is really helpful) and peers (“I appreciated the emphasis on communication, and the opportunity to practice and receive peer feedback.”). Furthermore, students highlighted how they benefited from specific course assignments, such as the policy memo, which has real-life application in society and aids the public and policy makers in everyday life decisions, “[I gained] communication skills, including oral presentation skills and translating important scientific understanding and information into something tangible (the policy memo specifically) that can be used by policy makers and the general public.”

Following the second course (the data practicum), 12 students mentioned communication as one of the skills they gained the most in the course (Table 3). They stressed that “there were numerous opportunities to present and get comfortable with delivering information.” A student from social sciences commented on how they learned about differences in scientific writing style between the social and natural sciences, “Social science has a very different approach to scholarly writing and structure. This class let me learn about how it is done with hard sciences in a practical way.”

In the interviews, one student relayed the importance of being able to communicate across all three FEW nexus research areas that can have different jargon and research skills as well as communicating across diverse audiences in the following quote:

… obviously [it is important] to understand the complexities and interconnections of the three – food, water, energy resources, and I know that one [way to approach this] is to work on communication. Not just within those disciplines, because you know- water experts, food experts, energy experts - they all have their own jargon, they all are siloed, pretty much talking across those disciplines is difficult so building those skills but also building the skills to talk to non-experts, so that’s like policy makers and the individual household users. To sort of bridge the academic research and policy user side.

3.1.3. Goal 3: broaden interdisciplinary research skills (e.g., data collection, analysis, and interpretation) to explore problems and generate solutions relevant to the FEW nexus

Since research within the FEW nexus involves the integration of diverse research approaches, the goal of the two-course sequence is first to expose students to different research approaches, and then to provide students with the opportunity to collaborate on interdisciplinary FEW systems research projects that require them to use methods and approaches that they are not necessarily utilizing or exposed to in their own discipline or field of study. Specifically, the main course learning outcomes related to this goal stated that in the end of the two course sequence students will be able to:

1. Identify strengths and critique weaknesses of multiple disciplines.

2. Synthesize the approaches and tools from multiple disciplines to evaluate and address a research problem.

3. Understand the ethics relating to your research.

At the beginning of the course sequence (Figure 3), students reported that they were generally less than somewhat confident (mean = 2.76) in their abilities related to the first two learning outcomes. However, from the start, they were confident to a good extent (mean = 3.95) in their ability to understand the ethics relating to their research. For all three learning outcomes there was significant growth in students’ confidence from the beginning to the end of the two-course sequence.

The open-ended responses at the end of the two courses to the question “List the two most important things that you gained from taking the course” provided more context to the growth in confidence that was seen in the Likert type question about the confidence (Tables 2 3). Following the experiential course, 12 students mentioned gains related to interdisciplinary research. Students mainly referred to the importance of understanding “how other disciplines do research/analysis/methods.” One student explained how “the course provided insights on how to take my research discipline/ideas and apply them to (or within) complementary frameworks (e.g., environmental justice at the FEW Nexus).” Another student explained that they believe that interdisciplinary research approach will allow for “creativity regarding potential research endeavors moving forward.”

Following the second course, many students (n = 19) mentioned gains that were categorized under the interdisciplinary research goal. At this time, they stressed not only their growth in awareness of diverse research but also of interdisciplinary research practices, especially as a result of the final interdisciplinary FEW systems research projects. One student expressed that they gained “more confidence in approaching and accomplishing research that is outside my direct area of expertise…”

Students attributed their awareness of other research areas and methods to the group work on the project and the diverse body of guest speakers that were brought to the course, as one student said, “[I gained] exposure to different research tracks and to different career paths. Having guest speakers from a large variety of countries, backgrounds, cultures, and disciplines was extremely beneficial. Having the opportunity for Q & A was also highly valuable.”

Several students referred to the challenge of leaving their comfort zone, “Working… from multiple disciplines and stepping in a field new to one’s comfort zone are probably the most important take homes for me…” This theme was repeated in the focus group following the presentation of the research projects, especially for students who collaborated on projects that included research approaches outside of their prior research expertise and for those who believed that the project’s methods (e.g., data analysis techniques) were not applicable or relevant to their PhD research (additional details provided under Research Question 2).

In the focus group following the second course, students mentioned that there was tension between the breadth and depth of research approaches. While fellows appreciated the exposure to diverse perspectives about research methods that were brought by multiple guest speakers, they felt that it was happening at the expense of concrete opportunities to learn and practice new research skills (e.g., data analysis methods). This was especially mentioned regarding the second course, in which fellows were looking forward to the course as an opportunity to learn/apply methods in more depth. They felt that there was often too much information to be considered an overview, but not in-depth enough for them to apply the methods to their own work.

This could be seen also in their responses after the second course to the question, “Please rate whether or not the course activities were presented at the appropriate frequency” (Figure 4), where “Opportunity to learn data analysis techniques” and “Opportunities to learn new research skills” were rated by around half of the students as not addressed sufficiently in the course (n = 14, n = 15 respectively). It is noteworthy that most of the low ratings came from students from the first iteration of the course (n = 8 and n = 9 respectively). Additionally, the course instructors noted that while these categories had the lowest ratings overall, the purpose of the second course was not to provide students with these skills, but to allow students with certain existing skill sets to have the ownership necessary to shape and enhance their projects using these skills. This speaks to the challenge and complexity of an interdisciplinary program. Since it is not possible to teach students the wide range of skills required to successfully complete an interdisciplinary FEW systems research project, each team is also paired with appropriate faculty mentors who will guide students and encourage them to hone the skills necessary for project success.

3.1.4. Goal 4: increase collaborative skills with a range of professionals (including individuals in academia, industry, and government) and scientists outside of the students’ academic discipline

More than half of the students indicated that collaboration was the most important skill gained following the second course (n = 18, Table 3). Specifically, the main course learning outcomes related to this goal stated that in the end of the course students will be able to:

1. Collaborate with scientists outside of your field of expertise

2. Collaborate with a range of professionals (including non-academic scientists, industry professionals, policy makers, etc.) on issues relating to your field of study

3. Work with team members from diverse, racial, cultural, and other backgrounds

Regarding the two first goals, results showed (Figure 3) significant growth in confidence from the beginning (means: 3.41, 3.31 respectively) to the end of the course sequence (means: 4.09, 4.12 respectively). From the start of the course sequence, students reported high confidence (mean: 4.24) in their ability to work with team members from diverse racial, cultural, and other backgrounds. There was no significant difference between the beginning and the end of the course sequence regarding this ability.

Following the experiential introduction course, seven students mentioned that collaboration was one of two most important things they gained in the course. One student explained that they benefitted from “Forming connections outside of my department with professors and students who have different perspectives on FEW issues I deal with in my own work.” Another student mentioned in their interview how through collaboration with other students in the course they learned more about FEW nexus components that they were not so familiar with, “…my work is in food water nexus so through the classes and other things I came to know about the energy system and how energy is related to these … we did collaborative work with atmospheric science students. And since then, it’s become clearer.”

Following the second course, students mentioned the collaboration gained through working in interdisciplinary teams on a common goal. As one student explained, “working with groups requires a very specific set of skills and the final project gave valuable experience with improving those skills: group communication, collaboration, time management, and efficient workflows skills were all improved upon.” Another student said, “I thought the team project was really great. I thoroughly enjoyed working with my classmates and learning how to create something collaboratively. I gained a knowledge of how to work better in a team…”

Students also pointed to the benefit of allocating the first hour of the three-hour weekly meetings to small-group discussions. They thought that it was a good practice for building relationships between the group members. As one student commented,

At the beginning of each class we were supposed to meet with our groups to discuss our projects. Instead of jumping into the work we naturally developed a routine where we would just talk about whatever was going on that week and problems we have as well as the positive benefits of therapy. Then eventually we would get to the group project. Our group also met most Sundays for two hours so that additional time meant we didn't need to cram every conversation in during class on Wednesday but we still always used the full hour. And I think that was important because we were relationship building and empathetic towards each other’s experiences. Which helped us as a group in the long run, because there were no frustrations if someone couldn't contribute much one week.

3.2. Research question 2: what products resulted from the interdisciplinary FEW systems research project completed throughout the two courses?

The research products that resulted from the first three iterations of the two-course sequence (Table 4) included six academic manuscripts, two grant proposals, three sets of Extension materials (Factsheets), one policy memo, one literature review with an accompanying stakeholder survey, and one storyboard map website. These products covered topics across the spectrum of the FEW nexus including the impacts of climate change on crop yields and water availability, sustainable farming approaches, the impact of the COVID-19 pandemic on food and energy affordability, and the intersection of racial disparities and issues at the FEW nexus.

TABLE 4

Table 4. Interdisciplinary FEW systems research project topics, disciplines, and products.

Each interdisciplinary FEW systems research project was supposed to focus on two or more of the FEW nexus areas. Of the thirteen projects, six focused on food and water, three focused on food and energy, and three focused on all three nexus areas (Table 4). There was one project solely focused on energy, but due to the extent and wide-reaching implications of that team’s work (a publicly available website story map), their project was approved. While some projects are still being finalized for publication or submittal, three of the academic manuscripts have been published in scientific journals, the policy memo was submitted to the Governor of Maryland, the storyboard website is live and publicly accessible, and the stakeholder survey has been validated and will be piloted by a team of students from the subsequent cohort. Furthermore, faculty that taught the fall semester class and oversaw the group projects rated the students’ collaboration as 9.42 out of 10 on average, where 1 was not collaborative at all and 10 was extremely collaborative (Rubric available in the Supplemental Material 3). When asked to choose from four options about groups collaboration style, instructors responded that five groups had fully integrated collaboration where all students contribute equally throughout the projects. In six groups, the collaboration style was that each student contributed equally to the project, but each took ownership of an aspect of the project in which they utilized their specific expertise. In other groups, one or two students emerged as leaders for the project and were supported by the rest of the group, and there was one project that was conducted by an individual student after another group decided to part ways in order to focus on different research directions.

3.3. Research question 3: what improvements have been made to the interdisciplinary FEW systems research project experience?

Throughout all iterations of the two-course sequence we have continued to make adaptations that improve the interdisciplinary FEW systems research project experience for students. Three main changes resulted from student feedback that allow for students to gain the most benefit from the project. Table 5 shows the improvements, examples of the student feedback that prompted the change, description of the improvement, and intended and/or observed benefit of the improvement. We elaborate on these changes since the lessons learned from student feedback following each iteration could be relevant to other course sequences that aspire to promote content knowledge and research experiences in a systemic interdisciplinary topic, such as FEW nexus, as well as communication and collaboration skills.

TABLE 5

Table 5. Improvements to the interdisciplinary FEW systems research project experience.

3.3.1. Timing of group formation

Following the first iteration of the two-course sequence, students felt that the courses were largely isolated from each other. In the focus group interview after the first iteration of the sequence, one student raised the question,

Is there a reason that the two courses can't be combined? So that we take more practical application in [the fall] semester and mix it in with kind of more that literature, background side of the last semester and work on them concurrently throughout the entire year rather than spending one whole semester just kind of learning theories and backgrounds and then one whole semester trying to do a project… because we gave what, three presentations in the span of like in just 10 weeks in here. And so it was really hard to not keep regurgitating kind of similar stuff, and it would have been better to spread that out.

In response to this question, another student suggested that since the first course is in the Spring and the second is in the Fall, fellows should utilize the summer for working on the projects.

You can do the same thing over the year. In the Spring you think about the concept, in the summer, you can get in the data or start analyzing things, and in the [Fall] semester you can write the results. So just to distribute these things all over the year.

Another student also pointed out that it takes time to develop a strong and healthy collaboration, especially if one wants to extend these collaborations beyond the courses. This was emphasized in the following quote:

We need to build trust and rapport with one another early so that we feel comfortable exploring ideas together. Then we need the time and structures to pursue those ideas. Otherwise, we won't develop collaborative projects with one another, and we won't continue to collaborate beyond the end of the program.

Drawing from the educational “Team-First” model for interdisciplinary research described by Bosque-Pérez et al. (2016), the course instructors worked together to initiate project group formation during the first course for the second iteration of the two-course sequence in response to student suggestions (Bosque-Pérez et al., 2016). In the focus group following the second iteration, students commented that they appreciated the early start. However, they suggested including checkpoints along the summer to ensure that students are utilizing the summer to work on their projects. In the third iteration of the two-course sequence, the instructors added a required check-in point during the summer where groups reported on their project progress, thereby encouraging them to place more focus on the project during the summer break. Cheruvelil et al. (2014) noted that establishing a timeline for periodic progress updates is an essential component of effective team functioning (Cheruvelil et al., 2014).

3.3.2. Topic selection

In the first iteration of the two-course sequence, project topics were suggested by faculty of the UMD Global STEWARDS program, and students selected their group’s topic from these suggestions. In the focus group that followed, students expressed their frustrations that the topics were not related to their research fields or dissertation topics, and projects were a missed opportunity to utilize each student’s strengths and research expertise. Some students also viewed the projects as added work rather than a conduit to gaining a broader research perspective that is relevant to their dissertation. One student expressed, “I think one thing that was frustrating, at least for me, was that the idea of like the selection of projects was kind of ad hoc … like very much all over the place. And it was very difficult [to choose project] like I do not know anything about this stuff.”

In response to this feedback, ideas for project topics were proposed by both faculty and students in the next iteration of the sequence. Once again drawing from the “Team-First” model described by Bosque-Pérez et al. (2016), the specific project topics were refined from these initial ideas by consultation with stakeholders, which included UMD faculty, internal UMD groups, and external partners in nonprofit organizations. In the focus group following the second iteration, some students were happy about this process since they felt more ownership over the project selection process. As one student said,

I thought it was great and I do think that the way that it was structured where we sort of pitched projects individually and then came together on them …. I felt like I ended up with a project that was really in my wheelhouse … I was the one who pitched it and then we were doing it. It was … entirely geography [the student major], and it's all of the data analysis, things that I'm already used to doing, … and I felt like if we had more ability to converse [it would be more interdisciplinary], the actual formation of the project ideas wasn't as interdisciplinary sometimes as it could have been, just because, …we weren't coming up with the ideas collectively… So I think it would have been cool to have like a more interactive project development process.

Another student stressed that even though they decided on the topic in the Spring semester, “the way we came up with the concepts was individually, so everyone pitched ideas …” They suggested that “… maybe it would have been interesting to come up with [the project topic] collectively or we are sort of bouncing ideas back and forth from each other vs., oh, everyone is just in their own silo looking for individual things.”

Several students felt that the process of pitching project topics was too quick and there was no time to check “what are the skills that we need for [the project] and [eventually, we] realized that, like none of us had sufficient skill and like GIS² or things like that.” Another student referred to the quick process, “… it wasn’t as intentional of a process of like thinking about what skills each of us have and what skills each of us want to develop like. And if we just had a little more time like we all could have, yeah, made those decisions.” Additionally, another student specifies that they wish the project was more relevant to their dissertation topic, “… [In the future I suggest to] better integrate interdisciplinary projects with pre-existing student research where applicable – I love that the projects are interdisciplinary and go in new directions, but I wish there was some connection so that I could at least relate all this work to my dissertation.

3.3.3. Mentorship

In the first iteration of the course sequence, the instructor of the data practicum course was the main mentor for all projects, and students could seek additional help from other faculty members of the program. Following the first iteration, students of the second cohort suggested a change to having a range of faculty serve as project mentors, as explained in the following quote:

I think instead of putting [the decision about choosing topics] on us, put it back on to the STEWARDS' faculty because we're supposed to have these faculty you know, mentors are supposed to be, you know, kind of like sponsors. So I would ask that they consider asking them to really, you know, present some projects that are kind of like, you know, ripe, or very, you know, ready to kind of like launch. And that might mean, you know, a short turnaround like a semester for us to work on something that we could cling on to or maybe have them as also like a semester long advisor to kind of like walk us through. So that way, we have a little bit more of like solid footing to work on a project that's ripe, and that's able to kind of like have some meaningfulness, I guess.

The team-based interdisciplinary doctoral education model described by Bosque-Pérez et al. (2016) incorporated faculty involvement, which was important to help students to develop the skills needed to engage in interdisciplinary teamwork. We emulated this component of their model and invited UMD faculty to not only serve as mentors to project teams, but also to suggest project topic ideas related to their own work. The faculty mentors were selected based on their expertise in a FEW nexus area of research and their willingness to mentor an interdisciplinary team of students. Each interdisciplinary team was paired with a faculty mentor, and faculty participation varied across the different projects, including refining research questions, providing datasets, conducting fieldwork, mentoring students through the project process, participation in team meetings, engaging with stakeholders, and providing research seminars to the entire cohort. Students in the second and third iterations of the course indicated that faculty mentorship was important to the success of their project teams, and thanked their mentors during the final group presentations at the end of the semester.

4. Discussion

There is a clear and growing need for systems thinking approaches to solving problems at the FEW nexus (Aboelela et al., 2007), requiring educators to rethink the traditional siloed approach to teaching environmental and sustainability issues (Begg et al., 2015; Esler et al., 2016; Bosch and Casadevall, 2017). Students must be able to draw from different disciplines in order to truly understand and address issues that exist at the nexus of interconnected systems. The UMD Global STEWARDS NSF NRT program seeks to recognize this need and foster collaboration among doctoral students of different disciplines and enhance communication skills to diverse audiences.

This work demonstrates the substantial benefits yielded from pairing two graduate courses in which students work together on a research product related to an issue at the FEW nexus. The administrative and financial burden of offering this course structure is minimal as it only requires the intentional scheduling of the two courses as a sequence and enrolling the same students in both courses. These simple steps provide a curricular experience that greatly exceeds the benefits of taking each course independently with different groups of students. The skills that students reported gaining from the two courses were complementary and aligned with the specific course objectives such that gains in the introductory course were mostly focused on FEW system content knowledge and exposure to different communication styles while the gains in the second course (the data practicum) were mostly focused on collaboration and interdisciplinary research experience. Word clouds created from students’ open-ended responses after the conclusion of each course shows that the skills students reported gaining from the two courses were complementary and aligned with the specific course objectives (Figure 5).

FIGURE 5

Figure 5. Word clouds created from students’ open-ended survey responses to questions about the skills gained in the introductory course (A) and the data practicum course (B).

In future offerings of this sequence and program, we will attempt to address the concerns expressed by some students regarding the relevance of the projects to their dissertation research. In the interdisciplinary team model described by Bosque-Pérez et al. (2016), the research conducted by the student teams culminated in “dissertation sets” comprising coordinated dissertations related to an overarching research theme (Bosque-Pérez et al., 2016). These dissertations not only contained disciplinary chapters by individual authors, but also interdisciplinary co-authored chapters (Bosque-Pérez et al., 2016). The authors noted that this requirement worked well to motivate students and keep the teams together (Bosque-Pérez et al., 2016). While it is not typical for students to include co-authored chapters in their dissertation at UMD, the UMD Global STEWARDS program leadership is considering advocating for this as a way of promoting the importance of interdisciplinary teamwork to graduate student education.

There are limitations to the study that we present here. One limitation is that the study relies mainly on students’ self-reports of their experiences, gained skills and level of confidence, which may not be an accurate reflection of what they are doing in the classroom. However, class observations and students’ actual products from the course (projects and presentations) corroborate the students’ reported benefits. A further limitation is that during the semester students participated in other coursework and activities in their own PhD programs that could influence their growth of confidence from pre- and post-surveys. Nevertheless, it was obvious from the open responses and interviews that students attributed much of their gains in skills such as communication, collaboration and interdisciplinary research to the two-course sequence.

Finally, sustainability of the program after funding from the NSF concludes is a challenge faced by program leaders. Of the 33 fellows who participated in the program over three cohorts, 11 (33%) did not receive a stipend (Table 1), indicating that not all students who enagage with the program are motivated solely by the stipend. These 11 students still saw value in the program, and chose to enroll as a fellow despite not being financially compensated. We believe that each student gains something valuable from engaging with our program. Some included portions of the group project in their dissertation, some utilized a new research method that they learned in the sequence in their own research, and others gained other important skills, such as communication skills to diverse audience, to name a few. To ensure sustainability of the program we have considered modifying the course sequence and shifting to a different funding model which would provide smaller stipends during the summer. Our UMD Global STEWARDS program as described in this manuscript can serve as a model for academic institutions that seek to implement similar interdisciplinary programs for doctoral students. While our hope is that federal and state agencies will recognize the value of this program and provide additional financial support to ensure its sustainability, a pared down model will still achieve our main outcomes without substantial funding resources.

5. Conclusion

Despite the overwhelming evidence that solutions to current issues, particularly those intertwined within the FEW nexus, will require interdisciplinary and cross-boundary solutions, training programs for graduate students still mainly operate in academic silos. This work is drawn from an interdisciplinary, experiential graduate education program focused on innovations at the FEW nexus. The backbone of the program consists of a two-course sequence during which students complete an interdisciplinary FEW systems research project. The two-course sequence described here represents one successful curricular approach to this issue. There were substantial benefits from the pairing of two graduate courses in which students from different disciplines work together on a research product related to a FEW systems issue. This model provides PhD students with the opportunity to learn about the most pertinent and real-world FEW nexus issues using a system thinking framework and to practice hands-on interdisciplinary collaboration on a tangible research product. This model could be implemented in a variety of academic settings and at different levels of education.

Feedback from students shows that this model works best when students are given ample time to form their project groups and select a research topic, and when they have a specific, dedicated faculty mentor to guide the project. In addition to generating an actionable research product, completing the project helped improve students’ confidence in conducting collaborative research and improved their interdisciplinary research skills focused at the FEW nexus.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by The University of Maryland Institutional Review Board (IRB). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AS, RM, GM-A, KM, and SL: study conceptualization. RM, GM-A, KM, SL, MW, and AS: methodology and writing—review & editing. GM-A, KM, and MW: formal analysis and data curation. RM and GM-A: writing—original draft preparation. AS and RM: project administration. AS: funding acquisition. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by NRT-INFEWS: UMD Global STEWARDS (STEM Training at the Nexus of Energy, WAter Reuse and FooD Systems) that was awarded to the University of Maryland School of Public Health by the National Science Foundation National Research Traineeship Program, Grant number 1828910. The funders had no role in the design of the study; in the collection, analyzes, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Acknowledgments

The authors would like to acknowledge the UMD Global STEWARDS fellows for their participation in all the evaluation elements of the program, and the University of Maryland School of Public Health for their support of the UMD Global STEWARDS program. Additionally, we would like to thank Maurice Rocque for his invaluable assistance with program administration.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2023.1114529/full#supplementary-material

Footnotes

1. ^During and following the first iteration of the courses, the evaluation team individually interviewed six students in addition to the end-of-semester focus group interviews to provide broad feedback to the instructors on the new sequence of courses. From the second iteration onwards, the evaluation team decided to continue only with focus group interviews.

2. ^“GIS” refers to ArcGIS, a geographic information system software used for creating maps.

References

Aboelela, S. W., Larson, E., Bakken, S., Carrasquillo, O., Formicola, A., Glied, S. A., et al. (2007). Defining interdisciplinary research: conclusions from a critical review of the literature. Health Serv. Res. 42, 329–346. doi: 10.1111/j.1475-6773.2006.00621.x

PubMed Abstract | CrossRef Full Text | Google Scholar

American Educational Research Association (2014). Report and recommendations for the reauthorization of the institute of education sciences. Washington, D.C: American Educational Research Association.

Google Scholar

Begg, M. D., Bennett, L. M., Cicutto, L., Gadlin, H., Moss, M., Tentler, J., et al. (2015). Graduate education for the future: new models and methods for the clinical and translational workforce. Clin. Transl. Sci. 8, 787–792. doi: 10.1111/cts.12359

PubMed Abstract | CrossRef Full Text | Google Scholar

Bosch, G., and Casadevall, A. (2017). Graduate biomedical science education needs a new philosophy. MBio :8:e01539-17. doi: 10.1128/mBio.01539-17

CrossRef Full Text | Google Scholar

Bosque-Pérez, N. A., Klos, P. Z., Force, J. E., Waits, L. P., Cleary, K., Rhoades, P., et al. (2016). A pedagogical model for team-based, problem-focused interdisciplinary doctoral education. Bioscience 66, 477–488. doi: 10.1093/biosci/biw042

CrossRef Full Text | Google Scholar

Calder, R. S. D., Grady, C., Jeuland, M., Kirchhoff, C. J., Hale, R. L., and Muenich, R. L. (2021). COVID-19 reveals vulnerabilities of the food–energy–water Nexus to viral pandemics. Environ. Sci. Technol. Lett. 8, 606–615. doi: 10.1021/acs.estlett.1c00291

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheruvelil, K. S., Soranno, P. A., Weathers, K. C., Hanson, P. C., Goring, S. J., Filstrup, C. T., et al. (2014). Creating and maintaining high-performing collaborative research teams: the importance of diversity and interpersonal skills. Front. Ecol. Environ. 12, 31–38. doi: 10.1890/130001

CrossRef Full Text | Google Scholar

De Laurentiis, V., Hunt, D. V. L., and Rogers, C. D. F. (2016). Overcoming food security challenges within an energy/water/food Nexus (EWFN) approach. Sustainability 8:95. doi: 10.3390/su8010095

CrossRef Full Text | Google Scholar

Denecke, D., Feaster, K., and Stone, K. (2017). Professional development: Shaping effective programs for STEM graduate students. Washington, DC: Council of Graduate Schools.

Google Scholar

Dodds, F., and Bartram, J. (2016). The water, food, energy and climate Nexus: Challenges and an agenda for action. Florence, United Kingdom: Taylor & Francis Group

Google Scholar

D'Odorico, P., Davis, K. F., Rosa, L., Carr, J. A., Chiarelli, D., Dell'Angelo, J., et al. (2018). The global food-energy-water Nexus. Rev. Geophys. 56, 456–531. doi: 10.1029/2017RG000591

CrossRef Full Text | Google Scholar

Esler, K., Downsborough, L., Roux, D., Blignaut, J., Milton, S., le Maitre, D., et al. (2016). Interdisciplinary and multi-institutional higher learning: reflecting on a south African case study investigating complex and dynamic environmental challenges. Curr. Opin. Environ. Sustain. 19, 76–86. doi: 10.1016/j.cosust.2015.12.002

CrossRef Full Text | Google Scholar

Food and Agriculture Organization of the United Nations (FAO) (2014). The water-energy-food Nexus: A new approach in support of food security and sustainable agriculture. Rome, Italy: Food and Agriculture Organization of the United Nations.

Google Scholar

Hoff, H. (2011). “Understanding the Nexus. Background paper for the Bonn2011” in Nexus conference: The water, energy and food security Nexus (Stockholm: Stockholm Environment Institute)

Google Scholar

Klein, J. T. (2014). Interdisciplinarity and Transdisciplinarity: keyword meanings for collaboration science and translational medicine. J. Transl. Med. 2:1024.

Google Scholar

Mandinach, E. B., and Cline, H. F. (1993). Systems, science, and schools. Syst. Dyn. Rev. 9, 195–206. doi: 10.1002/sdr.4260090208

CrossRef Full Text | Google Scholar

Mayer, V. J., and Kumano, Y. (1999). The role of system science in future school science curricula. Stud. Sci. Educ. 34, 71–91. doi: 10.1080/03057269908560149

CrossRef Full Text | Google Scholar

McKee, K. E., Serrano, D., Girvan, M., and Marbach-Ad, G. (2021). An integrated model for interdisciplinary graduate education: computation and mathematics for biological networks. PLoS One 16:e0257872. doi: 10.1371/journal.pone.0257872

CrossRef Full Text | Google Scholar

Meadows, D. H. (2008). Thinking in systems: A primer. Vermont: Chelsea Green Publishing.

Google Scholar

Murray, R. T., Marbach-Ad, G., McKee, K., and Sapkota, A. R. (2021). Experiential graduate course prepares transdisciplinary future leaders to innovate at the food-energy-water Nexus. Sustainability 13:1438. doi: 10.3390/su13031438

CrossRef Full Text | Google Scholar

National Research Council (2015) in Enhancing the effectiveness of team science. eds. N. J. Cooke and M. L. Hilton (Washington, DC: National Academies Press).

Google Scholar

O’Meara, K., and Culpepper, D. (2018). University of Maryland Language Science NRT Program: Internal evaluation, third year report. Maryland: University of Maryland, College Park.

Google Scholar

Orgill, M., York, S., and MacKellar, J. (2019). Introduction to systems thinking for the chemistry education community. J. Chem. Educ. 96, 2720–2729. doi: 10.1021/acs.jchemed.9b00169

CrossRef Full Text | Google Scholar

Qualtrics Software (2016). Available at: http://www.qualtrics.com

Google Scholar

Reeves, T. D., and Marbach-Ad, G. (2016). Contemporary test validity in theory and practice: a primer for discipline-based education researchers. CBE life. Sci. Educ. 15:rm1. doi: 10.1187/cbe.15-08-0183

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodríguez, L. F., Marshall, A.-M., Cotton, D., Koelsch, R., Koziel, J., Meyer, D., et al. (2019). The development of the INFEWS-ER: a virtual resource Center for Transdisciplinary Graduate Student Training at the Nexus of food, energy, and water. Front. Environ. Sci. 7:38. doi: 10.3389/fenvs.2019.00038

CrossRef Full Text | Google Scholar

Saldaña, J. (2015). The coding manual for qualitative researchers. 3rd. Los Angeles; London: SAGE Publications Ltd.

Google Scholar

World Economic Forum (WEF) Water Initiative (2011). Water security: Water food energy climate Nexus. Washington, DC: Island Press.

Google Scholar