Geoffrey A. Coalson
Courtney T. Byrd- 1Arthur M. Blank Center for Stuttering Education and Research, Austin, TX, United States
- 2Speech, Language, and Hearing Sciences, The University of Texas at Austin, Austin, TX, United States
Purpose: Social validation provides external validity of treatment outcomes from the perspective of untrained observers. To date, clinical efficacy studies of the Blank Center CARE™ Model indicate post-treatment gains in communication competence from the perspective of participants and clinicians. A preliminary social validation study corroborated these positive treatment outcomes with ratings from the general public for a single participant in a single context. The present study was designed to replicate and extend these findings by assessing clinical outcomes from the perspective of untrained observers across multiple participants and contexts.
Method: Ten adults who stutter provided communication samples one week before and after completing the Blank Center CARE Model treatment. A total of 1,110 untrained observers were recruited. Each untrained observer rated only one participant at one timepoint (pre-treatment or post-treatment) in one context (dyad or presentation), and each participant was asked to provide only one rating (communication competence or stuttering severity).
Results: A significant interaction indicated that post-treatment gains were observed for communication competence, but not stuttering severity, for both contexts.
Conclusions: Present findings provide further social validation of the Blank Center CARE Model of treatment. Untrained observers confirmed that participation in this strengths-based approach significantly enhances communication competence. Notably, these changes were observed regardless of pre- to post-treatment stuttering severity, lending additional support to the premise that changes in communication and fluency are independent constructs.
Introduction
A core component of the Blank Center CARE™ Model (Communication, Advocacy, Resiliency, Education, hereinafter the CARE Model) is improving the communication competence of individuals who stutter while excluding any clinical goals designed to change stuttering frequency or severity. Training in communication competence for individuals who stutter, as conceived within the CARE Model, includes explicit training in core communication skills and, importantly, considers communication to be effective only when participants stutter openly. Positive post-treatment gains in communication competence for adults who stutter have been reported from the perspective of clinicians (1) and the participants themselves (2). Social validation studies provide external confirmation of treatment outcomes observed by individuals in the general public with no vested interest or knowledge of the expected treatment outcomes (3–5). A recent social validation study by Byrd et al. (6) asked untrained observers to rate the communication competence of an adult who stutters based on one video sample recorded either pre-treatment or post-treatment. The preliminary findings found that significant clinician-rated and self-rated gains in communication competence reported in previous studies (1, 2) were also observed from the perspective of the general public.
As noted in Byrd et al. (6), significant post-treatment gains were limited to a single participant in a single speaking context (i.e., dyadic mock interview). Further, although stuttering severity ratings of the selected pre- and post-treatment videos were carefully controlled and nearly identical, comparison of untrained observers' ratings of communication competence vs. ratings of stuttering severity were not included during analysis. The purpose of this study, therefore, was to replicate and extend findings in Byrd et al. (6) by (a) increasing the number of participants (N = 10, as opposed to N = 1) and raters (N = 1,110, as opposed to N = 81), (b) including ratings of communication competence and stuttering severity (as opposed to communication competence alone), and (c) rating video samples from two different speaking contexts—dyad and presentation (as opposed to dyad alone).
Stuttering treatment for adults
Clinical studies examining outcomes for adults who stutter have traditionally focused on fluency-focused treatments [see systematic reviews by (7, 8)]. Across these studies, post-treatment gains in fluency have been observed, but tempered by individual variability, relapse, and participant dissatisfaction [e.g., (9–15)]. Additional research has found that treatments that target fluency did not yield improved communication competence [speaker-perspective: (11, 14, 16, 17); listener-perspective: (18–20)] or self-reported quality of life [e.g., (21, 22)]. Notably, the present authors have reported (e.g., (1, 2, 23, 24)—based on the perspective of people who stutter, clinicians and untrained observers—that changes in fluency are not required for significant changes in the communication effectiveness and/or quality of life of individuals who stutter. These preliminary findings provide support for the CARE Model of treatment, an alternative approach to stuttering treatment that is consistent with the recent focus on mitigating ableism (i.e., fixing or curing a disabling condition) in national health care policies held by the National Institutes of Health National Advisory Board on Medical Rehabilitation Research (25) and the NIH Advisory Committee Subgroup on Individuals with Disabilities (25), as well as the revised policies of the American Psychological Association that move away from ableist practices during clinical design and practice (26).
Specifically, the CARE Model (27, 28) provides a manualized, strengths-based approach to stuttering treatment that targets communication competence, advocacy, resiliency, and education—and does not aim to fix or cure stuttering through the targeting of goals designed to directly or indirectly increase fluency and/or modify stuttered speech (for summary of CARE Model treatment, see Supplementary Appendix A). Clinical studies [children: (23, 24); adults: (1, 2)] indicate significant post-treatment improvements in communication competence that are independent from stuttering severity.
Clinician ratings
For example, Byrd et al. (23) examined 37 children who stutter before and after participation in CARE Model treatment. A certified and licensed speech-language pathologist (SLP) who was not trained in this approach and was not knowledgeable of its intended outcomes, rated nine specific communication competences (i.e., language use, language organization, speech rate, intonation, volume, gestures, body position, eye contact, and facial affect) during impromptu presentations provided pre- and post-treatment. In addition to replicating the cognitive and affective gains in previous studies completed with children who stutter (29–31), significant post-treatment gains were observed for eight of these nine competencies. The gains in communication competence observed by clinicians were replicated in a larger cohort (24; N = 61), with significant gains observed for seven of these nine communication competencies. Importantly, post-treatment gains were not significantly associated with pre-treatment stuttering frequency (23, 24). Similar to findings with children, Byrd et al. (1) found that an unfamiliar clinician rated adults who stutter significantly higher in eight of the nine communication competences following CARE Model treatment, and these changes were observed by the clinician independent of stuttering severity, suggesting that pre-treatment stuttering severity does not predict post-treatment communication competence. Taken together, these findings suggest that CARE Model treatment is effective in increasing perceived communication competence irrespective of stuttering severity, as rated by clinicians who are not trained in expected outcomes of this approach.
Self ratings
Clinician-rated gains in communication competence following CARE Model treatment have also been reported from the perspective of the participant. For example, Byrd et al. (1) examined self-rated communication competence of adults who stutter following CARE Model treatment using the Self-Perceived Communication Competence scale (SPCC; 32), which measures participants’ self-rated communication abilities across seven different speaking scenarios (e.g., dyad, small group, large presentation). Post-treatment SPCC scores improved but did not reach significance (p = .31) in this small sample. A larger follow-up study by Coalson et al. (2), however, found significant post-treatment gains in communication competence, as measured by the SPCC. As was the case with the clinician ratings, these gains were not accompanied by significant changes in stuttering severity post-treatment.
Collectively, both clinician- and self-rating of communication competence following CARE Model treatment indicate that improvements in communication competence are not dependent on changes in stuttering behaviors. Although these self- and clinician-rated outcomes are critical to our understanding of the clinical efficacy of this treatment program, ratings of unfamiliar observers, or rather the general public, are needed to provide corroborating evidence that the reported gains are not limited to the perspective of those involved in clinical activities.
Unfamiliar observer ratings
Social validation studies provide a third-party evaluation of treatment measures from the perspective of unfamiliar individuals who are not invested nor aware of the intended clinical outcomes (3, 33). For example, Schloss et al. (4) conducted a study to assess whether assertiveness training provided to three adults who stutter resulted in changes that were observable not only to the clinicians, but to a small cohort of untrained observers (N = 10 graduate students). Each student randomly viewed only one pre- or post-treatment video sample of an adult who stutters and rated their overall assertiveness during a mock interview. Because findings indicated an increase in assertiveness in videos recorded after treatment, relative to pre-treatment videos, the researchers concluded that findings observed within the clinic generalized to the general public. A second social validation study conducted by Schloss et al. (5) replicated these findings, again instructing untrained observers (N = 11) to rate video samples of three adults who stutter pre- and post-treatment in a dyadic interview. Notably, post-treatment gains in assertiveness ratings were not consistently associated with stuttering severity across these studies.
Werle and Byrd (34, 35) conducted a experimental study wherein untrained observers, in this case university professors (N = 158, 238), rated the communication competence of an adult actor simulating stuttering and also demonstrating high or low communication competence behaviors (e.g., facial affect, vocal variety, gestures, body positioning). Each untrained observer rated only one video, and each rated the presenter using a standardized rubric of “delivery” (i.e., communication behaviors including vocal variety, eye contact, engagement, gestures). Although stuttering was present and controlled across videos (i.e., 15% frequency, identical type and secondary behaviors), professors rated the presenter with high communication competence significantly higher than low communication competence. Although the presenter was an actor simulating stuttering, rather than an individual who stutters who had undergone treatment, findings provide some external validation of the importance of communication competence to untrained observers relative the stuttering frequency or severity.
Byrd et al. (6) conducted a social validation study to assess whether untrained observers corroborated the clinician-rated and self-rated gains in communication competence of an actual adult who stutters following CARE Model treatment. In this study, 81 untrained observers viewed one video—either a pre-treatment or post-treatment video of an adult who stutters during a one-on-one mock interview. After viewing a video, untrained observers were asked to rate the communication competence of the interviewee. Findings indicated that untrained observers rated the adult who stutters as a significantly stronger communicator in post-treatment videos. Notably, stuttering of the participant in video stimuli were rated by clinicians as similar in terms of frequency [pre-treatment—7%, post-treatment—6%, as rated by (36) guidelines] and severity (pre-treatment—Moderate [score = 27], post-treatment [score = 25], as rated by the Stuttering Severity Instrument-Fourth Edition (37). Stuttering severity was also rated as similar by a separate group of untrained observers using a 100-point visual analog scale (N = 120, pre-treatment: M = 64.71; post-treatment: M = 64.95; p = .95). Although post-treatment differences in stuttering were nominal from the perspective of clinicians and untrained observers, a supplemental replication analyses was nevertheless completed within the same study (Appendix S3, N = 96 untrained observers) based on pre- and post-treatment videos from a second adult who stutters with an inverse stuttering profile—that is, increased stuttering post-treatment—as rated by clinicians (pre-treatment stuttering frequency and severity: 5%, Mild [Total SSI-4 score = 10]; post-treatment frequency and severity: 9%, Mild [Total SSI-4 score = 14) and untrained observers (N = 85; pre-treatment: 27.19; post-treatment: 29.07, p = .65). Similar to the primary analyses, untrained observers rated communication competence in the post-treatment video sample to be significantly stronger than the pre-treatment video sample. Further, post-treatment gains in communication competence in the primary and supplementary analysis remained significant when observer-based factors (e.g., demographics, familiarity with a person who stutters) were statistically controlled.
Purpose of the present study
The design of Byrd et al. (6), wherein many untrained observers viewed and rated video samples from a single adult who stutters pre- and post-treatment, allowed researchers to capture the variance of responses amongst the general public. This provided social validation of one of the primary clinical outcomes of the CARE Model—increased communication competence with no attempts to modify stuttered speech—for one participant. Byrd et al. (6) noted that the gains reported from the perspective of untrained observers viewing only one participant may or may not be observed when evaluating a group of participants before and after treatment. Byrd et al. (6) also noted that by restricting the context of the video sample to dyadic interaction (i.e., mock interview format), the communication gains could not be generalized to additional challenging speaking contexts for adults who stutter. Replication of these findings for one participant in a larger cohort of adults who stutter would provide evidence that treatment effects, as rated by the untrained observer, generalize across a variety of participants. In addition, although stuttering severity was nearly identical in pre- and post-treatment video samples in Byrd et al. (6), and both supplemental replication data [in (6), Appendix S3] and qualitative data [in (6), Appendix S2] supported the notion that communication competence and stuttering severity were described as separate constructs by untrained observers, one might argue that ratings of communication competence, nevertheless, will reflect ratings of stuttering severity in a larger cohort. The purpose of the present study was to directly address these three factors by replicating and extending the design of Byrd et al. (6).
First, to capture the variance of treatment effects across participants, untrained observers in the present study rated a larger number of adults who stutter before and after treatment (Timepoint). If the perception of communication competence by untrained observers from a larger cohort of clients replicates post-treatment gains observed for one client in Byrd et al. (6), the findings will provide broader social validation of the intended treatment effects across multiple participants. Second, to examine whether ratings of communication competence and stuttering severity of a larger participant cohort were perceived as separate constructs by the general public, the video samples were rated for stuttering severity as well as communication competence (Condition). If untrained observers provide dissimilar ratings for stuttering and communication across multiple participants, such findings would suggest that stuttering and communication are considered distinct constructs by the general public. Third, to capture the variance of treatment effects across contexts, untrained observers viewed pre- and post-treatment videos of each participant in two different speaking situations: dyadic interaction and oral presentations (Context). If untrained observers report that changes in post-treatment communication competence extend across contexts, such outcomes will indicate the treatment effects of the CARE Model can be generalized beyond a single interaction style.
In sum, the present study will serve as a large-scale social validation study of the CARE Model treatment program described in Byrd et al. (6), wherein changes in communication competence are targeted in the absence of fluency goals and achieved while stuttering openly. Specifically, we are examining the following research question: Does communication competence and stuttering severity differ pre- vs. post-treatment from the perspective of untrained observers?
Method
The following study was approved by the Institutional Review Board at the University of Texas at Austin (IRB: 2015-05-0044; see also (38). All participants depicted in the video stimuli provided written consent prior to enrollment in treatment. None of the participants had previously received CARE Model treatment. Table 1 provides demographic and treatment history details for each participant.
Table 1. Demographics of participants.
Forty video samples (10 participants × 2 contexts × 2 timepoints) were embedded in a Qualtrics-based survey and distributed to untrained observers via the Prolific© online data collection platform. Untrained observers were recruited from over 83,000 adults in the United States, United Kingdom, Australia, and Canada. All participants were paid for their participation in accordance with standards of distribution for Prolific© ($1.80 per participant, Mdn = 9 min per survey; calculated on $12/hour base rate). Participants who elected to participate in the study were first presented with an informed consent cover letter that described the purpose of the study, compensation procedures, the right to discontinue participation, and IRB approval information. Consent to participate was acknowledged by clicking the advance button.
Each participant was randomly assigned one of 40 videos (10 participants × 2 timepoints × 2 contexts). Participants were excluded from participation if they had completed a survey associated with the other 39 video samples. Each survey including a dyadic interaction began with the following instructions: You are about to watch a video of an interview. Immediately following the video, you will be asked questions about the interviewee. The interview will be approximately 5 to 7 min in length. You will only be able to move forward in the survey after you have watched the video in its entirety. Participants were unaware of specific questions that would be asked during impromptu interviews. Interviewers were provided with a list of sample interview prompts (e.g., What do you consider strengths and weaknesses? Describe a work-related issue from previous experiences and how you addressed it.) but were also instructed to ask novel questions and follow-up questions to simulate naturalistic interview settings. Each survey including an oral presentation began with the following instructions: You are about to watch a video of a presentation. Immediately following the video, you will be asked questions about the presenter. The presenter was instructed to give an impromptu Zoom-based presentation to a small audience on one of five topics (i.e., new inventions, best advice, best purchase, favorite month, favorite historical figure). The presenter was given 1 min to prepare. The interview will be approximately 2 to 3 min in length. You will only be able to move forward in the survey after you have watched the video in its entirety. Following the video, respondents were provided only one of the two following instructions accompanied by a 0–100 visual analog rating scale (VAS): (1) Using the scale below, please rate the communication competence of the person. 0 = Communication skills not at all effective, 100 = Communication skills extremely effective, or (2) Using the scale below, please rate the stuttering severity of the person. 0 = No stuttering, 100 = Extremely severe stuttering.
As in Byrd et al. (6), participants were then asked to provide demographic information (e.g., age, race, ethnicity, gender, education, occupation, primary language). Participants also provided information about their personal relationship with stuttering, persons who stutter, or other communication disorders (e.g., Do you personally know a person who stutters? If so, please describe your relationship and how long you have known this person. Have you had previous speech, language, and/or hearing evaluation or therapy?) as well as any visible and/or nonvisible diagnoses unrelated to communication difficulties. Supplementary Appendix B provides a detailed demographic summary of untrained observers.
Consistent with Prolific© policy, and to improve response fidelity (39), each survey included two comprehension checks and two attention checks (40). Multiple-choice comprehension check screening items were included to ensure that participants understand critical elements of the survey (e.g., understanding of compensation terms, understanding of instructions). Multiple-choice attention check items included instructional manipulation checks (e.g., When asked to select your favorite color, select green. This is an attention check. What is your favorite color?) and nonsensical questions (e.g., Rate your agreement with the following statement. I swim across the Atlantic Ocean to get to work everyday). Participants must respond to both comprehension check and attention check questions correctly to proceed. The survey was initiated by 1,141 respondents. Of these 1,141 respondents, 31 (2.72%) did not pass the attention or comprehension check questions during the survey and were excluded from final analysis. The final corpus included 1,110 untrained observers as respondents (see Supplementary Appendix B for demographic summary of untrained observers).
Procedures
Three primary variables were examined in the present study: Timepoint, Context, and Condition. Timepoint was defined as whether the video sample was recorded: (1) pre-treatment (i.e., one week prior to the first treatment session) or (2) post-treatment (i.e., one week following the final treatment session). Context was defined as the sampling context of the video: (1) a dyadic interaction (i.e., virtual mock interview with an unfamiliar interviewer), or (2) an oral presentation (i.e., virtual impromptu oral presentation on one of five topics to two additional audience members). Condition was defined as which perceptual rating the untrained observers were asked to provide following the video: (1) a rating of communication competence (using a 100-point VAS), or (2) a rating of stuttering severity (using a 100-point VAS). Similar to Byrd et al. (6), each untrained observer viewed and rated only one video, thereby eliminating order effects, and were blinded to the time point of the video they observed. Table 2 provides a summary of the number of participants who rated each video stimuli used in the present study.
Table 2. Number of Non-overlapping untrained observers serving as raters for each video sample (condition × timepoint × context) across participants.
Analysis
A three-factor ANOVA was conducted to compare untrained observers' ratings, with Timepoint (Pre-Treatment, Post-Treatment), Condition (Communication Competence, Stuttering Severity) and Context (Dyad, Presentation), as the three independent categorical factors, and ratings on the 100-point VAS as the continuous dependent variable. The three-way ANOVA was two-tailed (α = .05) and Bonferroni-adjusted p-values were applied during planned comparisons. Effect sizes were calculated and interpreted using ηρ2 (41; .01 = small, .06 = medium, .14 = large).
To account for the potential influence of listener-based variables, similar to Byrd et al. (6), nine additional observer-based factors were included as covariates: (1) age, (2) race identification, (3) ethnicity identification, (4) gender identification, (5) years of education, (6) primary spoken language, (7) knowing a person who stutters, (8) number of years the observer has known a person who stutters, and (9) visible and/or mixed disability. Certain race and gender categories had an insufficient number of respondents for valid statistical analysis across groups. Rather than exclude demographic categories with a limited number of respondents, and similar to Byrd et al. (6), groups were either combined or redistributed. Race categories with ≤10 (Native American or Alaskan Native, Native Hawaiian or Other Pacific Islander, Race Not Described) were re-grouped into a single category. Survey respondents who identified as non-binary were randomly assigned and evenly redistributed to male or female.
Results
A significant main effect was detected for Timepoint F(1,1109) = 5.04, p = .025, ηρ2 = .005 [small effect size], 95% confidence interval (95% CI of ΔM) of the mean difference [.44, 6.56], Condition F(1,1109) = 92.66, p < .001, ηρ2 = .078 [medium effect size], 95% CI of ΔM [11.98, 18.11], and Context F(1,1109) = 5.08, p = .024, ηρ2 = .005 [small effect size], 95% CI of ΔM [.46, 6.55].
A summary of descriptive data for two- and three-way interactions are provided in Table 3. No significant two-way interaction was detected for Timepoint × Context F(1,1109) = 2.78, p = .097 nor Condition × Context F(1, 1109) = .09, p = .765. A significant two-way interaction was detected between Timepoint and Condition F(1, 1109) = 7.19, p = .007, ηρ2 = .007 [small effect size]. Planned comparisons indicated that a significant gain was observed for communication competence from pre-treatment to post-treatment (p < .001, ηρ2 = .011 [small effect size], 95% CI of ΔM [3.39, 11.97]), but no significant change was observed in stuttering severity (p = .757). Planned comparisons also indicated that significant differences were observed between ratings of communication competence and stuttering severity at pre-treatment (p < .001, ηρ2 = .022 [small effect size], 95% CI of ΔM [6.52, 15.20]) as well as post-treatment (p < .001; ηρ2 = .065 [medium effect size], 95% CI of ΔM [14.90, 23.55]), suggesting that untrained observers perceived communication competence and stuttering severity to be distinct variables.
Table 3. Untrained observers’ ratings of each condition (communication competence, stuttering severity) at each timepoint (pre-treatment, post-treatment) and context (dyad, presentation).
No significant three-way interaction was detected F(1, 1108) = .71, p = .401, indicating that Context did not influence the Timepoint × Condition interaction. That is, post-treatment gains were observed by untrained observers for communication competence regardless of context (dyad: p = .004, ηρ2 = .007 [small effect size], 95% CI of ΔM [2.78, 14.99]; presentation: p = .035, ηρ2 = .004 [small effect size], 95% CI of ΔM [.45, 12.50]), whereas no significant post-treatment changes in stuttering severity were observed in either context (dyad: p = .298; presentation: p = .137; see Figure 1). Further, significant differences between communication competence and stuttering severity were observed by untrained observers, regardless of context, at pre-treatment (dyad: p < .001, ηρ2 = .013 [small effect size], 95% CI of ΔM [5.64, 17.92]; presentation: p = .001, ηρ2 = .009 [small effect size], 95% CI of ΔM [3.83, 16.05]) and post-treatment (dyad: p < .001, ηρ2 = .027 [small effect size], 95% CI of ΔM [11.24, 23.52]; presentation: p < .001, ηρ2 = .041 [small effect size], 95% CI of ΔM [15.00, 27.14]).
Figure 1. Ratings of communication competence and stuttering severity for adults who stutter at before and after treatment (pre-tx, post-tx) across speaking contexts (dyad, presentation). Vertical bars reflect mean values. Error bars reflect standard error of the mean (SEM).
Of the nine observer-based factors included as covariates, three were identified as significant: age F(1, 1109) = 6.30, p = .012, ηρ2 = .006 [small effect size]; ethnicity F(1, 1109) = 13.45, p < .001, ηρ2 = .012 [small effect size]; personally knowing an adult who stutters F(1, 1109) = 5.68, p = .017, ηρ2 = .005 [small effect size].
Discussion
The purpose of this social validation study was to assess whether gains observed following CARE Model treatment by clinicians and participants were also observed by untrained observers. This study replicates and extends a previous social validation study (6) in terms of number of participants (N = 10, as opposed to N = 1), raters (N = 1,110, as opposed to N = 81), context (dyadic exchanges and oral presentations, as opposed to dyad alone), and perceptual ratings provided by untrained observers (communication competence and stuttering severity, as opposed to communication competence alone). Findings indicate significant gains in communication competence pre- to post-treatment across contexts with no significant changes in stuttering severity.
Communication competence
Untrained observers rated the communication competence of adults who stutter as significantly higher for videos recorded after CARE Model treatment compared to videos recorded before treatment. These findings replicate ratings from untrained observers in Byrd et al. (6, N = 81 raters, N = 1 adult who stutters) on a larger scale (N = 1,110 raters, N = 10 adults who stutter). Significant gains post-treatment also corroborated post-treatment gains in previous studies observed by participants (2, N = 33) and clinicians (1, N = 11). Together, outcomes suggest that the communication competence of adults who stutter following CARE Model treatment are perceived as significantly improved from multiple perspectives.
As noted, social validation can provide evidence that self-perceived or clinician-perceived gains following treatment are not based on potentially biased perspectives. It is important to note that although the general public in the present study provide corroborating evidence of post-treatment gains, the perspective of the general public should never take priority over participant self-perception during clinical intervention. That is, these data should be considered supporting evidence for the treatment, but not used as a criterion for clinical success. Failure to make this distinction risks a scenario wherein the general public, or perhaps clinicians, promote treatment outcomes they consider to be successful even when the participant does not [e.g. (11, 14, 42)].
Stuttering severity
The present study also asked untrained observers to rate videos based on stuttering severity. As noted, no significant changes in untrained observers' ratings of stuttering severity were observed. This finding replicates Byrd et al. (6) who reported significant changes in communication competence of videos despite no significant difference in stuttering severity, as rated untrained observer (primary analysis, N = 81 raters, p = .95; supplemental replication analysis, N = 96 raters; p = .65). Findings also replicate data from observers in Werle and Byrd (34, 35) and self-ratings in Coalson et al. (2), in that gains in communication competence for adults who stutter can be observed with no changes in stuttering severity.
Unlike the present study, videos in Byrd et al. (6) were selected specifically to control for pre- and post-treatment variance in stuttering severity when assessing untrained observers' ratings of communication competence. In the present study, pre- and post-treatment stuttering severity were allowed to freely vary across participants. Inclusion of these participants during analyses provided the opportunity to contrast observer ratings of stuttering severity with communication competence. At both timepoints and irrespective of context, untrained observers provided significantly different ratings for communication competence and stuttering severity, suggesting that these were considered to be independent by the general public. Untrained observers' perception of communication and fluency as distinct supports a fundamental theoretical assumption of the CARE Model (24). Specifically, Byrd et al. (24) posit that, contrary to fluency-focused treatments, communication effectiveness and fluency are independent constructs [Assumption 1, p. 5]. Together, findings provide large-scale social validation for a core tenet and clinical goal of the CARE Model—improved communication competence without changes in stuttering severity.
Speaking context
An extension of Byrd et al. (6) was the assessment of speakers in two different contexts—dyadic interaction (i.e., mock interview) and oral presentation (i.e., impromptu presentation to a small group). Untrained observers rated participants similarly—that is, significant gains in communication competence with no significant changes in stuttering severity—in both contexts. Findings from dyadic interactions replicate findings based on mock interviews in Byrd et al. (6), and extend these gains to a second, dissimilar speaking context (i.e., oral presentation). Findings suggest that changes in communication are not restricted to the communication demands of one-on-one dyadic interactions and can be generalized across these contexts.
Findings inform clinical outcomes reported by adults who stutter, as measured by the SPCC, in Coalson et al. (2). As noted, the SPCC provides an index of self-perceived communication competence across a variety of speaking contexts (i.e., public presentation, large meeting, group interaction, dyadic interaction) and audience types (i.e, stranger, acquaintance, friend). Although a priori power analysis in Coalson et al. (2) restricted pre- to post-treatment comparison to the Total SPCC score (i.e., averaged across seven context-dependent subscales), descriptive data indicated post-treatment gains communication competence for speaking contexts similar to those rated by untrained observers in the present study. Specifically, a greater percentage of adults rated themselves as having “high” communication competence (>87 Total SPCC Score) during dyadic interactions (pre-treatment: 21%, post-treatment: 36%) and presentations (pre-treatment: 30%, post-treatment: 48%) and with strangers (pre-treatment: 24%; post-treatment: 61%). Significant post-treatment gains found in the present study from the perspective of untrained observers, along with the descriptive gains from the perspective of the participant in Coalson et al. (2), suggest that treatment effects may generalize across contexts.
Observer-based factors
Three observer-based factors were found to influence perceptual ratings: age, ethnicity, and whether the untrained observer personally knows a person who stutters. In general, younger participants and participants who self-identified as non-Hispanic or Latino provided higher ratings of communication competence than older and/or Hispanic or Latino observers. These ratings, however, were not specific to either time point or either speaking context. Comparison across studies provide some insight into the relative consistency of the factors when evaluating individuals who stutter. For example, two of the three demographic factors—age and ethnicity—were not identified as influential factors on ratings of communication competence in Byrd et al. (6) during primary analysis or supplemental replication analysis. Previous research, however, has identified age as a significant factor in the evaluation of negative beliefs or attitudes towards stuttering [e.g., (43, 44)]. Recent studies have also explored the beliefs of Hispanic/Latino communities towards stuttering [e.g., (45, 46)]. A recent study by Young and Byrd (47), however, found that age and ethnicity did not significantly influence employers' beliefs about individuals who stutter or their perception of the communication competence after watching video samples of a job candidate who stutters. The exact reasons for these cross-study disparities are speculative at this point, and the purpose of including these factors as covariates was to control for observer-based factors of significance, should they be observed. It should be noted that findings across studies may be inconsistent due to the overall lack of cultural diversity in stuttering research. Nevertheless, as the range of potentially influential demographic factors continues to grow, researchers should continue to collect detailed demographic data in studies of clinical outcomes. Importantly, although a variety of demographic variables may influence ratings in studies of treatment efficacy from the perspective of the general public, the positive gains of treatment remained in the present study significant after controlling for these factors.
Familiarity with an adult who stutters, however, has been a reliably influential factor when examining perceptual ratings of individuals who stutter, as seen in this study and previous studies [e.g., (43, 47–49)] and Byrd et al.'s (6) supplemental replication study. Across studies, untrained observers who are personally familiar with an adult who stutters rate videos more favorably than observers who reported they did not know a person who stutters. This finding is consistent with the well-documented contact theory wherein individuals with first-hand knowledge of a person from a specific identity or culture view these individuals more positively [e.g., (50, 51)]. Findings suggest that rater familiarity should continue to be assessed, and controlled, when examining perceptions of the general public towards people who stutter.
Limitations and future studies
The present study is not without limitations. First, although findings were replicated in two different speaking contexts, communication samples were collected in virtual space via Zoom. Although virtual interviews and presentations are increasingly commonplace, it is possible that ratings of communication competence would be higher (or lower) if these interactions were observed in-person. Second, it was necessary to limit the number of videos included in the study due to the increased number of contexts and participants. As such, untrained observers rated only one post-treatment sample collected approximately one week following the final session. As such, outcomes of present study can only be generalized to the general public's perception of communication competence gains immediately following CARE Model treatment. To examine the long-term effects, additional social validation studies are necessary to examine whether untrained observers maintain these perceptions at subsequent timepoints. Findings would further benefit from the inclusion of control conditions (e.g., non-stuttering adults, waitlist control) also not present in this study. Finally, we acknowledge that a variety of additional factors influence rater evaluation of any speaker (e.g., enthusiasm about topic, attire, perceived attractiveness, demographics of participant vs. demographics of rater). The present study was large enough to let these factors freely vary across participants and untrained observers (N = 106 to 117 per participant), and minimize the likelihood that these factors drove the outcomes in a systematic manner. Nevertheless, additional participant- and listener-based factors not measured in the present study cannot be ruled out as influential and should remain an area of examination in future studies.
Conclusion
The purpose of this study was to replicate and extend a previous social validation study examining the treatment outcomes of the CARE Model. Findings replicated previous findings that positive gains in communication competence were observed by untrained observers in the absence of changes in stuttering severity. Findings also suggest that these effects were not limited to dyadic interactions but extend to oral presentations. Combined, outcomes of this large-scale study serve as social validation of a non-ableist treatment approach designed to improve communication while explicitly excluding fluency goals and, instead, emphasizing stuttering openly as a fundamental step in effective communication.
Data availability statement
Data will be made available upon reasonable request to the authors.
Ethics statement
The studies involving humans were approved by University of Texas at Austin—Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
GC: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. CB: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that no financial support was received for the research and/or publication of this article.
Acknowledgments
This project was supported by the foundational grant support funded to the Arthur M. Blank Center for Stuttering Education and Research and endowed support provided through the Michael and Tami Lang Stuttering Institute, the Dr. Jennifer and Emanuel Bodner Developmental Stuttering Laboratory, and the Dealey Family Foundation Stuttering Clinic awarded to the second author. The authors thank Michael Mahometa for his assistance with statistical analyses. We would also like to thank the adult participants who stutter, as well as their families, who continue to participate in our ongoing clinical research.
Conflict of interest
GC is a salaried employee of the Arthur M. Blank Center for Stuttering Education and Research. CB is a salaried employee of the University of Texas at Austin. CB also serves in a non-salaried position as the Founding and Executive Director of the Arthur M. Blank Center for Stuttering Education and Research and the President of the World Stuttering and Cluttering Organization.
Generative AI statement
The author(s) declare that no Generative AI was used in the creation of this manuscript.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fresc.2025.1541059/full#supplementary-material
References
1. Byrd CT, Coalson GA, Young MM. Targeting communication effectiveness in adults who stutter: a preliminary study. Top Lang Disord. (2022) 42(1):76–93. doi: 10.1097/TLD.0000000000000270
2. Coalson GA, Byrd CT, Werle D, Croft R, Mahometa M. Self-perceived communication competence of adults who stutter following communication-centered treatment. Am J Speech Lang Pathol. (2024) 33(4):1965–85. doi: 10.1044/2024_AJSLP-23-00234
3. Kazdin AE. Assessing the clinical or applied importance of behavior change through social validation. Behav Modif. (1977) 1(4):427–52. doi: 10.1177/014544557714001
4. Schloss PJ, Espin CA, Smith MA, Suffolk DR. Developing assertiveness during employment interviews with young adults who stutter. J Speech Hear Disord. (1987) 52(1):30–6. doi: 10.1044/jshd.5201.30
5. Schloss PJ, Freeman CA, Smith MA, Espin CA. Influence of assertiveness training on the stuttering rates exhibited by three young adults. J Fluen Disord. (1987) 12(5):333–53. doi: 10.1016/0094-730X(87)90031-3
6. Byrd CT, Coalson GA, Werle D. Observer-rated outcomes of communication-centered treatment for adults who stutter: a social validation study. PLoS One. (2024) 19(5):e0303024. doi: 10.1371/journal.pone.0303024
7. Baxter S, Johnson M, Blank L, Cantrell A, Brumfitt S, Enderby P, et al. Non-pharmacological treatments for stuttering in children and adults: a systematic review and evaluation of clinical effectiveness, and exploration of barriers to successful outcomes. Health Technol Assess. (2016) 20(2):1–vi. doi: 10.3310/hta20020
8. Johnson M, Baxter S, Blank L, Cantrell A, Brumfitt S, Enderby P, et al. The state of the art in non-pharmacological interventions for developmental stuttering. Part 2: qualitative evidence synthesis of views and experiences. Int J Lang Commun Disord. (2016) 51(1):3–17. doi: 10.1111/1460-6984.12182
9. Arya P, Geetha YV. Factors related to recovery and relapse in persons with stuttering following treatment: a preliminary study. Disabil CBR Incl Devel. (2013) 24:82–98. doi: 10.5463/dcid.v24i1.189
10. Craig AR, Hancock K. Self-reported factors related to relapse following treatment for stuttering. Australian J Hum Commun Disord. (1995) 23:48–60. doi: 10.3109/asl2.1995.23.issue-1.04
11. Cream A, Onslow M, Packman A, Llewellyn G. Protection from harm: the experience of adults after therapy with prolonged-speech. Int J Lang Commun Disord. (2003) 38(4):379–95. doi: 10.1080/13682820310001598166
12. Irani F, Gabel R, Daniels D, Hughes S. The long term effectiveness of intensive stuttering therapy: a mixed methods study. J Fluen Disord. (2012) 37(3):164–78. doi: 10.1016/j.jfludis.2012.04.002
13. National Stuttering Association. (2009). The experience of people who stutter: A survey by the National Stuttering Association. Executive summary. Retrieved from http://www.westutter.org/assets/NSAsurveyMay09.pdf
14. Stewart T, Richardson G. A qualitative study of the therapeutic effect from a user’s perspective. J Fluen Disord. (2004) 29:95–108. doi: 10.1016/j.jfludis.2003.11.001
15. Yaruss JS, Quesal RW, Reeves L, Molt LF, Kluetz B, Caruso AJ, et al. Speech treatment and support group experiences of people who participate in the national stuttering association. J Fluen Disord. (2002) 27(2):115–33. doi: 10.1016/s0094-730x(02)00114-6
16. Constantino CD, Eichorn N, Buder EH, Beck JG, Manning WH. The speaker’s experience of stuttering: measuring spontaneity. J Speech Lang Hear Disord. (2020) 63:983–1001. doi: 10.1044/2019_JSLHR-19-00068
17. Corcocan JA, Stewart M. Stories of stuttering: a qualitative analysis of interview narratives. J Fluen Disord. (1995) 23:247–64. doi: 10.1016/S0094-730X(98)00020-5
18. De Nardo T, Tetnowski JA, Coalson GA. Listener perceptions of stuttering and stuttering modification techniques. J Fluen Disord. (2023) 75:105960. doi: 10.1016/j.jfludis.2023.105960
19. Manning WH, Burlison AE, Thaxton D. Listener response to stuttering modification techniques. J Fluen Disord. (1999) 24:267–80. doi: 10.1016/S0094-730X(99)00022-4
20. Von Tiling J. Listener perceptions of stuttering, prolonged speech, and verbal avoidance behaviors. J Commun Disord. (2011) 44(2):161–72. doi: 10.1016/j.jcomdis.2010.09.002
21. Boyle MP. Identifying correlates of self-stigma in adults who stutter: further establishing the construct validity of the self-stigma of stuttering scale (4S). J Fluen Disord. (2015) 43:17–27. doi: 10.1016/j.jfludis.2014.12.002
22. Byrd CT. How to Navigate Fluency Conformity and Reduce Stuttering Stigma [Keynote Presentation]. Austin, TX: National Stuttering Association Convention (2021). Available at: https://www.youtube.com/watch?v=azDrm_NloyE (Accessed January 23, 2023).
23. Byrd CT, Winters KL, Young M, Werle D, Croft RL, Hampton E, et al. The communication benefits of participation in camp dream. Speak. Live.: an extension and replication. Semin Speech Lang. (2021) 42(2):117–35. doi: 10.1055/s-0041-1723843
24. Byrd CT, Coalson GA, Werle D, Croft R, Winters KL, Young MM. (2024). Camp Dream. Speak. Live: A virtual adaptation. MedRxIv. doi: 10.1101/2024.08.22.24311294
25. National Institutes of Health, NIH Advisory Committee to the Director Working Group on Diversity. (2022). Subgroup on Individuals with Disabilities Report. Available at: https://www.nichd.nih.gov/about/advisory/nabmrr/2022May# (Accessed December 23, 2022).
26. American Psychological Association, APA Task Force on Guidelines for Assessment and Intervention with Persons with Disabilities. (2022). Guidelines for Assessment and Intervention with Persons with Disabilities. Available at: https://www.apa.org/about/policy/guidelines-assessment-intervention-disabilities.pdf (Accessed December 23, 2022).
27. Byrd CT. The Blank Center CARE™ Model: Application for School-Age Children who Stutter. [Manual]. Austin, TX: Arthur M. Blank Center for Stuttering Education and Research (2023). Available at: https://blankcenterforstuttering.org/
28. Byrd CT, Coalson GA, Conture EG. CARE Model of treatment for stuttering: theory, assumptions, and preliminary findings. Front Psychol. (2024) 15:1488328. doi: 10.3389/fpsyg.2024.1488328
29. Byrd CT, Chmela K, Coleman C, Kelly E, Reichhardt R, Irani F. An introduction to camps for children who stutter: what they are and how they help. Perspect ASHA Spec Interest Groups. (2016) 1:55–69. doi: 10.1044/persp1.SIG4.55
30. Byrd CT, Hampton E, McGill M, Gkalitsiou Z. Participation in camp dream. Speak. Live.: affective and cognitive outcomes for children who stutter. J Speech Pathol Therapy. (2016) 1(3):1000116. doi: 10.4172/2472-5005.1000116
31. Byrd CT, Gkalitsiou Z, Werle D, Coalson GA. Exploring the effectiveness of an intensive treatment program for school-age children who stutter camp dream. Speak. Live.: a follow-up study. Semin Speech Lang. (2018) 39(5):458–68. doi: 10.1055/s-0038-1670669
32. McCroskey JC, McCroskey LL. Self-report as an approach to measuring communication competence. Commun Res Rep. (1988) 5(2):108–13. doi: 10.1080/08824098809359810
33. Finn CA, Sladeczek IE. Assessing the social validity of behavioral interventions: a review of treatment acceptability measures. Sch Psychol Q. (2001) 16(2):176. doi: 10.1521/scpq.16.2.176.18703
34. Werle D, Byrd CT. Professors’ perceptions and evaluations of students who do and do not stutter following oral presentations. Lang Speech Hear Serv Sch. (2022) 53:133–49. doi: 10.1044/2021_LSHSS-21-00069
35. Werle D, Byrd CT. The impact of self-disclosure and strategies for communication competence on professors’ perceptions and evaluations of students who do and do not stutter. J Speech Lang Hear Res. (2022) 65(9):3405–19. doi: 10.1044/2022_JSLHR-22-00118
36. Yairi E, Ambrose N. Early Childhood Stuttering: For Clinicians by Clinicians. Austin, TX: Pro-Ed (2005).
38. Byrd CT. (2023). The Blank Center CARE™ Model: A non-ableist approach to treatment. ClinicalTrials.gov identifier: NCT05908123. Available at: https://www.clinicaltrials.gov/ct2/show/NCT05908123 (Accessed June 12, 2024).
39. Peer E, Rothschild D, Gordon A, Evernden Z, Damer E. Data quality of platforms and panels for online behavioral research. Behav Res Methods. (2021) 54:1643–62. doi: 10.3758/s13428-021-01694-3
40. Oppenheimer DM, Meyvis T, Davidenko N. Instructional manipulation checks: detecting satisficing to increase statistical power. J Exp Soc Psychol. (2009) 45(4):867–72. doi: 10.1016/j.jesp.2009.03.009
41. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Erlbaum (1988).
42. Constantino CD, Manning WH, Manning SN. Rethinking covert stuttering. J Fluen Disord. (2017) 53:26–40. doi: 10.1016/j.jfludis.2017.06.001
43. Arnold HS, Li J, Goltl K. Beliefs of teachers versus non-teachers about people who stutter. J Fluen Disord. (2015) 43:28–39. doi: 10.1016/j.jfludis.2014.12.001
44. Valente ARS, Louis KOS, Leahy M, Hall A, Jesus LM. A country-wide probability sample of public attitudes toward stuttering in Portugal. J Fluen Disord. (2017) 52:37–52. doi: 10.1016/j.jfludis.2017.03.001
45. Dean L, Medina AM. Stigma and the Hispanic stuttering experience: a qualitative study. J Commun Disord. (2021) 89:106056. doi: 10.1016/j.jcomdis.2020.106056
46. Medina AM, Mead JS, Moore S. Perceptions of and beliefs about stuttering in the Hispanic/Latino community. J Commun Disord. (2024) 111:106456. doi: 10.1080/02699206.2018.1481145
47. Young MM, Byrd CT. Stuttering in the workplace: Employers’ attitudes, employability ratings, and hiring decisions. Under review (2024).
48. Hughes S, Gabel R, Irani F, Schlagheck A. University students’ perceptions of the life effects of stuttering. J Commun Disord. (2010) 43(1):45–60. doi: 10.1016/j.jcomdis.2009.09.002
49. Hughes CD, Gabel R, Palasik ST. Examining the relationship between perceptions of a known person who stutters and attitudes toward stuttering. Can J Speech Lang Pathol Audiol. (2017) 41:237–52.
Keywords: stuttering, stammering, adults, treatment, social validation, non-ableist
Citation: Coalson GA and Byrd CT (2025) Social validation of post-treatment outcomes of adults who stutter who participated in CARE Model treatment: replication and extension. Front. Rehabil. Sci. 6:1541059. doi: 10.3389/fresc.2025.1541059
Received: 6 December 2024; Accepted: 2 April 2025;
Published: 21 May 2025.
Edited by:
Tamer Abou-Elsaad, Mansoura University, EgyptReviewed by:
Horabail Venkatagiri, Iowa State University, United StatesRania Abdou, Alexandria University, Egypt
Copyright: © 2025 Coalson and Byrd. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Geoffrey A. Coalson, Z2VvZmZyZXkuY29hbHNvbkBhdXN0aW4udXRleGFzLmVkdQ==