Defining a Flexible Notion of “Good” STEM Writing Across Contexts: Lessons Learned From a Cross-Institutional Conversation

We respond to a surging interest in science communication training for graduate scientists by advocating for a focus on rhetorically informed approaches to STEM writing and its assessment. We argue that STEM communication initiatives would benefit by shifting from a strategic focus on products to a flexible understanding of writing as a practice worthy of attention and study. To do that, we use our experience across two universities and two distinct programmatic contexts to train STEM graduate students in writing and communication. We draw from cross-disciplinary conversations to identify four facets of “good” STEM writing: (1) connecting to the big picture; (2) explaining science; (3) adhering to genre conventions; and (4) choosing context-appropriate language. We then describe our ongoing conversations across contexts to develop and implement flexible rubrics that capture and foster conversations around “good” writing. In doing so, we argue for a notion of writing rubrics as boundary objects, capable of fostering cross-disciplinary, integrative conversations and collaborations that strengthen student writing, shift STEM students toward a rhetorically informed sense of “good” writing, and offer that kinds of assessment data that make for persuasive evidence of the power of writing-centric approaches for STEM administrators and funders.


INTRODUCTION
Scientists and educators increasingly recognize the demand for improved STEM (science, technology, engineering, and math) training in communication, including writing, and its importance for facilitating wider dissemination of research results, improved policy outcomes, and richer engagement with public audiences (Fischhoff, 2013;Kuehne and Olden, 2015). This paper discusses two separate but complementary programs at Northwestern University and the University of Rhode Island that responded to that call. Each developed focused training programs and related tools, including the rubrics discussed here, to equip STEM graduate students to communicate their science to broad audiences. Central to the philosophy of each program is to situate STEM writing and its assessment as a social, contextual, iterative, and public practice.
This authorship team-faculty and staff collaborating across different institutional, disciplinary, and programmatic contexts-had to grapple with defining a flexible notion of "good" writing applicable across a variety of STEM disciplines, taught to STEM faculty, practiced by STEM students, and ultimately supported by STEM administrators and funding agencies. Because we were in communication throughout that process, we share our experiences in this collaborative piece to build on continued calls for the development of STEM writing and communication skills as part of the education and professionalization of STEM undergraduate and graduate students (Fuhrmann et al., 2011;Denecke et al., 2017). We use the terms writing and communication here to encompass all modes of building, sharing, and reinforcing knowledge. We use rhetoric, a term often politically loaded, in reference to the ancient tradition of communication with purpose for an audience within a specific set of circumstances. Rhetorical moves refer to the intentional decisions a writer or speaker makes in order to meet the needs of those circumstances most effectively.
Here we draw from interdisciplinary literatures in science, science communication, and writing studies. We define four facets of effective communication that we argue constitute a flexible and capacious definition of "good" STEM writing across a range of genres and audiences: (1) connecting to the big picture; (2) explaining science; (3) adhering to genre conventions; and (4) choosing context-appropriate language. We then describe our work to capture these facets of "good" STEM writing in the development of two rubrics that support different contextually situated training programs designed to support STEM writers. In doing so, we build from a flexible understanding of writing rubrics (Henningsen et al., 2010;Nolen et al., 2011), conceiving of writing rubrics that formalize the expectations and definitions of good STEM writing as boundary objects: "a rhetorical construct that can foster cooperation and communication among the diverse members of heterogeneous working groups" (Wilson and Herndl, 2007). Here, writing rubrics that articulate teaching and learning goals for STEM students are an opportunity to span communities and build bridges between diverse stakeholders interested and invested in science communication outcomes. We argue that the development and implementation of writing rubrics can facilitate conversations across disciplines about good STEM writing. This process can foster collective investment in and understanding of STEM writing practices, while offering a valuable opportunity to generate data on the impacts of programs in increasingly competitive funding environments in higher education.
We deploy rubrics as rhetorical boundary objects (Wilson and Herndl, 2007) to connect knowledge-making in science with good STEM writing practice and pedagogy to develop locally situated thinking at our two institutions. This approach helped us leverage outside perspectives and empirical evidence to create resilient and flexible resources and instruments to meet local needs as part of a recursive assessment loop (Rutz and Lauer-Glebov, 2005). Our focus on writing as a practice not a product and on rubrics as a shared articulation of learning goals and essential rhetorical moves allowed us to accommodate the broader shift from a deficit model to a contextual model (Gross, 1994;Perrault, n.d). It also allowed us to emphasize rhetoric as a critical component in science communication (Gross, 1994;Druschke and McGreavy, 2016) and the importance of a user-centered paradigm for designing effective communication artifacts (Rothwell and Cloud, 2017).
We began working together several years ago as crossinstitutional collaborators looking for tools to facilitate shared approaches to the training and assessment of STEM writing. While our processes and products have converged and diverged through the years, the shared development of these rubrics enabled nuanced conversations about what defines good STEM writing across our many disciplines, encouraging us to clarify to ourselves and each other which rhetorical approaches and goals were specific to our individual program aims and which were broader, more universal element of good practice. We found that developing these tools was a profoundly helpful opportunity to open cross-disciplinary dialogue on the key ingredients of "good" writing and how those ingredients might be taught, explicated, and assessed. This is especially important in light of recent research highlighting the lack of consensus on what constitutes good science communication and the ability of current training programs to improve students' capacity in these areas (Rubega et al., 2021).
Of course, once a rubric is created, there are next steps to test its reliability and validity in the field, particularly as an instrument to assess skill-gain among students. We acknowledge this process is not yet complete for our tools. However, we are not advocating here for the broad adoption of our specific instruments. Rather, we want to shed light on their development, including discussions about the diverse but often siloed literatures that informed them, and their deployment for assessment as important conceptual steps in developing a shared understanding across faculty and students of good STEM writing, its best practices, and eventually its meaningful assessment. In particular, we hope to contribute to the conversation facilitating a shift toward science communication as a messy, iterative practice, bringing the insights of writing studies and rhetorical studies to bear on broad science communication initiatives and training in ways that can inform guiding principles implemented at the local level.

OUR PROGRAMMATIC CONTEXTS
Northwestern University's program, Skills and Careers in Science Writing, is a partnership between two academic units: Science in Society, a community-engaged research center, and Medill, a world-renowned journalism school. This semesterlong graduate-level course is for STEM doctoral students across all disciplines including microbiology, materials science, environmental engineering and developmental psychology. The course is led by journalism faculty and practicing writing professionals to cover best practices in writing, public science communication, and science reporting including principles of structure, narrative, and voice. Students produce an original magazine-style article about their own research. Critically discussing lay audience-friendly science stories also enables students to recognize and grapple with the immense shift of moving from traditional academic writing to an accessible style (Crossley et al., 2014). The course also focuses on science writing career pathways, and provides exposure to science communication and journalism professionals given the likelihood many STEM PhDs will pursue non-academic careers (Cyranoski et al., 2011;Powell, 2012).
University of Rhode Island's (URI) program, SciWrite, focuses on equipping science graduate students to move between academic and public-facing writing in two ways: (1) layering rhetorical training into graduate student curricula and (2) training faculty to support writing pedagogy in classrooms and laboratories. SciWrite is a cross-disciplinary training program funded by the National Science Foundation for STEM graduate students and faculty at URI and was collaboratively developed by faculty from Writing and Rhetoric, Nutrition and Food Sciences, and Natural Resources Science. The 2-year program includes internships and workshops alongside a four-course sequence where students gain a rhetorical foundation for writing through a series of academic and public writing projects. Full programmatic and assessment details are offered elsewhere (Druschke et al., 2018, n.d;Harrington et al., 2021).

INTERDISCIPLINARY AND INTER-INSTITUTIONAL COLLABORATION
Our programs initially developed independently. But our joint discussions about assessment helped us realize that rubrics were productive mechanisms for helping us push back against the widespread notion of writing (and communication more broadly) as strategic endpoint and for reframing the idea of writing as an intentional, situated, and messy practice. Particularly when integrated into multi-modal assessment portfolios, we argue that rubrics can serve three separate but interrelated purposes: (1) assessing STEM writing with flexible and locally-informed instruments; (2) empowering STEM faculty to engage more heartily with a rhetorical approach to writing training; and (3) communicating with students about important aspects of rhetorically savvy writing. Rather than treating rubrics-and the good writing they are meant to assess-as static, stringent structures, both programs deployed rubrics as unique opportunities for dialogue and collaboration with diverse faculty tasked with teaching (and grading) trainee writing.
During rubric development, we considered interdisciplinary sources such as impact measures in science communication and engagement (Coppola, 1999;Bucchi, 2013;Fischhoff, 2013;Denecke et al., 2017;of Sciences, Engineering, and Medicine et al., 2017), specialist assessment work being done in engineering undergraduate writing (Boettger, 2010), researcher oral presentations (Dunbar et al., 2006), and public science communication rubrics (Mercer-Mapstone and Kuchel, 2017; Murdock, n.d) as well as best practices in writing assessment (Rutz and Lauer-Glebov, 2005;Huot and O'Neill, 2009;Adler-Kassner and O'Neill, 2010). This diverse list of sources points to the disjointed and siloed nature of discussions taking place in science writing, science communication, rhetoric, and teaching and learning practices more broadly. Drawing from these various disciplines allowed us to map their commonalities and begin to stitch together a shared framework with four distinct, but overlapping features.

Connecting to the Big Picture
Good writers and communicators position themselves in the wider discourse; draw from existing understandings; make a compelling, structured articulation of their goals, purpose or main point; and vary their deployment of these elements depending on purpose and intended audience. This facet builds from perspectives present in writing studies since at least John Swales' Create a Research Space (CARS) model (Swales, 1981(Swales, , 1984(Swales, , 1990) with its emphasis on establishing a territory. This contextualizing is picked up in popular scicomm trainings like the Compass message box (Compass Science Communication Inc., 2017), and the SciWrite@URI program relied on it extensively in their training program (Druschke et al., 2018, n.d;Harrington et al., 2021).

Explaining Science
Good writers and communicators understand the highly academic ways scientists conventionally describe their research to peers, and identify how these are likely to be difficult or unfamiliar for novice readers. This facet includes understanding how the organization and technical detail provided in an explanation are critical components for effective science communication. Understanding these hurdles requires that communicators grapple with the specific challenges for communicating to novices (Wolfe and Mienko, 2007;Rottman et al., 2012) and the subject-specific vocabulary, or jargon, which impedes communication between science to non-scientists (Bullock et al., 2019). Bullock et al. found the presence of jargon impairs people's ability to process scientific information, and suggests that the use of jargon undermines efforts to inform and persuade the public (Bullock et al., 2019). At the same time, jargon serves an important function within specific discourse communities discourse communities (Porter, 1986), peer groups accustomed to specific ways of exchanging information. It is essential that good STEM writers recognize jargon as a community-specific vocabulary and make conscious choices about when and how to include it to explain complex scientific concepts to a variety of audiences with accuracy and clarity.

Adhering to Genre Conventions
Good writers and communicators understand and can appropriately navigate genre-specific expectations, which vary community to community and piece to piece. Both programs emphasize the importance of genre, but teach different genres to students, and the two program's rubrics reflect these genre-specific differences.

Choosing Context-Appropriate Language
Good writers and communicators have a solid grasp of the rhetorical moves at their disposal, such as style, tone, and register, as well as grammar, semantic and linguistic complexity, and scientific conventions such as hedging and citations. Importantly, this facet includes but moves well beyond word choice. This facet is most directly aligned with other quantifications of contextually good writing (Crossley et al., 2014) and broader discourse around stylistics and language (Pinker, 2015;Zinsser, 2016).

IMPLEMENTING RUBRICS AS CONTEXTUALLY SITUATED TOOLS
While our collective conversations coalesced around these shared facets of good writing, the rubrics we developed to articulate them were structured to our unique programmatic goals and needs. For example, the "genre conventions" our programs were designed to address were vastly different. So, while our shared goal was to articulate and teach these conventions, the ways in which our rubrics could reflect that would differ substantially.
Northwestern's program focuses specifically on lay-friendly magazine writing and science storytelling approaches (Leslie et al., 2013;Dahlstrom, 2014), and therefore this rubric deliberately defines some narrative conventions (Zinsser, 2016;Hart, n.d) which connect with research on recall and processing of narrative elements (Speer et al., 2009;Zak, 2015), as well as metaphors and analogies (Wolff and Gentner, 2011).
For example, the Science in Society rubric defined "Relevance (shows how this work is connected to real world experience in meaningful ways and why it matters)" as "Clearly defines the context and/or application of this work"; Reader perspective and real world connections meaningfully articulate the purpose/promise of this work. "Order and Structure (builds scaffolded scientific explanations)" was articulated as, "Effectively connect to reader's context and prior knowledge; Well structured and scaffolded explanations building bridges from existing understanding; Clearly walks through steps of processes and explains phenomena in a logical and coherent order; Consistently and clearly builds bridges from existing knowledge." (See Supplementary Material for more information).
URI's SciWrite, on the other hand, reinforces the idea of STEM writing as a rhetorical act in and among specific discourse communities (Penrose and Katz, 2010;Kuhn, 2012), and encompasses a range of formats including visual representation. Perhaps uniquely, this rubric is intended to span both academicand public-facing artifacts in order to reinforce the public as a valuable partner in larger conversations about science (Collins and Evans, 2002;Rowe and Frewer, 2005) and citizen science (Druschke and Seltzer, 2012;Shirk et al., 2012;Bonney et al., 2016). This rubric is therefore made up of 12 categories divided into subsections, some of which apply to all artifacts, and some of which are specific to certain modalities and formats. In both cases, the role of genre conventions is central, but how this is articulated is in conversation with broader programmatic goals and models.
In the SciWrite rubric, the category "Is the text appropriate for the target audience?" is articulated as, "The text consistently incorporates appropriate definitions and explanations of all key terms and concepts that makes the research/text fully comprehensible, accessible, and engaging to the primary intended audience." For the category, "Is there an appropriate depth of content given genre and subject matter?" "The text includes a sufficient depth of content about the subject matter for the genre and primary intended audience." And the category, "Does the text demonstrate its significance in a wider context, and build on the existing knowledge base by using literary elements appropriate to the genre (e.g., analogies, metaphors, similes, visual examples, case studies, etc.) to support deeper levels of understanding of complex ideas and phenomena?" was defined as, "The text explicitly demonstrates its significance in a wider context, and consistently builds on the existing knowledge base by using highly effective literary elements appropriate to the genre to support deeper levels of understanding of complex ideas and phenomena." (See Supplementary Material for more information).
As we mentioned above, this paper is not intended to report a validated instrument, but to call out how our processes and ultimate products converge and diverge in important ways. This transparency is intended to contribute to wider conversations about how science communication and writing programs should be developed, delivered and evaluated. We are certainly not done, and hope that sharing our process of developing rubrics as boundary objects within our own programs-and with each other across programs-helps others see how to incorporate rhetoric into STEM communication training conversations going forward.

MOVING FORWARD TOWARD "GOOD" STEM WRITING
As we well know, assessment is essential to STEM writing training and teaching. Well-structured, meaningful assessment also offers datasets and analyses that can be used to argue for funding and build a sustainable enterprise for this vital professional training. Such metrics are increasingly necessary to support and advocate for sustainable, rhetorically-informed and writing-focused practice within higher education (Rutz and Lauer-Glebov, 2005;Adler-Kassner and O'Neill, 2010).
In particular, embedded, rhetorically grounded frameworks provide a unique opportunity to create deeper interdisciplinary conversations about the values and definitions of good writingand they make disciplinary and genre conventions and practices visible. Including colleagues from a range of fields in this process is one step toward making those nebulous, frustrating guidelines for science writing more explicit.
We believe that conversations about the practice and pedagogy of good STEM writing vitally contribute to conversations about science and scientist training. A metaanalysis of over 700,000 biomedical journal abstracts over the past 150 years clearly demonstrates the readability of scientific abstracts is decreasing over time, and Rubega et al. (2021) recently demonstrated that current science communication training programs provide little evidence of improved practice (Pontus et al., 2017). Even further, the need for scientists to communicate across genres and audiences seems particularly apparent in a cultural moment of political division and policymaking challenges where cynicism and science-skepticism (Charney, 2003) inform highly-motivated interpretations of science and research (Washburn and Skitka, 2018). The need for cross-disciplinary conversations about good and great science writing, dissemination, and public engagement-and how to convey and assess these goals-has never been more obvious or more necessary.

AUTHOR CONTRIBUTIONS
SG, JM-A, CD, and IL organized the data and results. SG and