Comparing timing of other-initiation of repair: a multimodal approach

1. Introduction

The transitions between turns at talk in conversation tend to occur quickly, and turn-taking is systematically organized (Sacks et al., 1974). Turn-taking organization strives toward minimal overlaps and minimal gaps during conversations. Stivers et al. (2009) studied video recordings of 10 languages and found additional proof of a universal system of turn-taking with only a slight gap of ~100–300 ms between turns at talk. The response peak was within 200 ms of the end of the previous turn, and turn transition was typically not withheld more than 500 ms. Levinson and Torreira (2015) have collected information concerning typical (100–200 ms) and lengthy (over 700 ms) inter-speaker gaps. However, in some cultures and certain relaxed contexts, long gaps (from 750 ms to several seconds) are common (Gardner and Mushin, 2015). Overall, in order to proceed in conversation, smooth turn-taking is well motivated.

However, not all transitions between turns at talk are minimal. Schegloff et al. (1977) reported that while recipients of talk solve various troubles during a conversation, they systematically delay the turn initiating repair. These turns of talk are known as other-initiations of repair (OIR). They are turns such as, “what” and “sorry,” which are labeled as open requests because they do not specify the trouble source in the previous turn (Drew, 1997; Dingemanse et al., 2015). On the other hand, OIRs targeting the trouble source more precisely, e.g., “where” and “you mean me,” are labeled as restricted requests and restricted offers, respectively (Dingemanse et al., 2015). These utterances reveal the recipients' need for clarification or confirmation from the speaker. A delay is claimed to be the natural positioning of the various types of OIR turns (Schegloff, 2000). Delay refers to expanded transition space between turns of talk. In other words, although an OIR occurs in the next turn, it is withheld, offering, for example, a space for the speaker of the trouble source turn to self-initiate repair (Schegloff et al., 1977). Further evidence of longer gaps before OIRs was found in Kendrick's (2015) study concerning mundane conversations. The results of 169 OIRs revealed that OIRs tend to occur after gaps of ~700 ms. A delay in turn transition is one of the main markers of dispreference, and it can be a pre-indication of social trouble (Schegloff, 2000; Pomerantz and Heritage, 2012). This means that certain actions in conversation can be seen as being preferred in comparison to the alternative. In general, the conversation is expected to continue, and thus, that is the preferred state of conversation (“preference for progressivity,” Schegloff, 2006; p. 86). In contrast, a delay halts the conversation, and for this reason alone, OIRs can be viewed as dispreferred in comparison to the alternative, such as the straightforward use of minimal responses (“yeah,” “mm,” etc.).

Individuals with hearing impairment have a high risk of encountering conversational problems due to their misperception (Scarinci et al., 2008; Barker et al., 2017), and conversational flow can be disrupted because of their frequent use of OIRs (Lind et al., 2004; Pajo, 2013). Especially, open type OIRs (“what,” “huh,” etc.) are related to hearing impairment (Pajo, 2013). Therefore, the fundamental effect of hearing impairment on conversation includes constant repetition by the speaker of the trouble source turn, which is described as tiring and annoying by the spouses of individuals with hearing impairment (Scarinci et al., 2008). However, depending on the degree of hearing impairment or the use of hearing aids, the frequency of OIR can change. Thus, a mild-to-moderate degree of hearing impairment does not necessarily increase the use of open OIRs if hearing aids are in use (Laakso et al., 2019).

The timing of OIRs among individuals with hearing impairment is not systematically approached in previous studies. Nevertheless, Pajo (2013) reported findings from 164 OIRs produced by individuals with hearing impairment and showed that particularly restricted types of OIRs were typically produced after a delay. In contrast, a case study including an individual with a profound hearing impairment reported several OIR productions in overlap with current speakers' turn (both open and restricted types of OIRs) (Lind et al., 2006). This urgent timing was related to an absolute inability to hear what was said.

In any mundane conversation, a verbal OIR turn in conversation indicating the recipient's trouble proceeding can be accompanied by visual bodily actions (Floyd et al., 2015; Oloff, 2018). Overall, the visual bodily and verbal resources inter-relate and form various multimodal sets of action. For example, gaze shifts toward the speaker, changes in facial expressions, head turns, upper body movements toward the speaker, and hand gestures are some of the typical visual bodily resources. During mundane manual activities, visual bodily actions can systematically be the first indication of shifted attentiveness and approaching verbal OIR (Kendrick, 2015; Kamunen, 2019). In addition, a visual bodily action alone can be utilized as an OIR (Seo and Koshik, 2010; Mortensen, 2016; Jokipohja and Lilja, 2022). As such, these visual bodily actions are managed with similar repair turns as verbal OIRs. The timing of these actions can occur in the turn transition space, similar to verbal OIRs (Seo and Koshik, 2010; Kendrick, 2015). Other possible points of production can be in overlap with the current speaker's turn (Jokipohja and Lilja, 2022). After the visual bodily actions are initiated, they are typically held stationary until the trouble is resolved (Floyd et al., 2015). Thus, also the timing of disengagement from the hold offers important information about the repair solution. (Streeck et al., 2011; p. 5) claim “many of the consequential actions of the hearer are performed through visible displays of the body rather than with talk.” Moreover, Stivers et al. (2009) point out that the rules for turn-taking may discourage overlap in the vocal channel, but visual bodily resources can be viewed as less intrusive upon speech. Therefore, they may be produced earlier than verbal resources and held static while the speaker is talking.

Mutual gaze between the interactants is an important perspective when considering interactions with individuals with hearing impairment. As hearing deteriorates, visual speech information available on the speaker's face becomes increasingly important (Tillberg et al., 1995). With individuals with hearing impairment, the frequency of OIRs increases in conversation when the mutual gaze is not present, and the overall progression of conversation is in jeopardy (Ekberg et al., 2017). In addition, visual bodily indications of trouble may offer an important resource and assist in potentially socially sensitive incidences of misperception (Skelt, 2012). As Pajo and Klippi (2013) point out, visual bodily actions work as a strong resource for collaboration in conversations with recipients with hearing impairment (see also Skelt, 2010). For example, the utilization of mutual gaze intensifies monitoring and assists in maintaining conversational flow. In addition, the speaker's quick response to the recipient's frown and lean toward the speaker can work as a pre-emptive repair during the construction of a turn. The response can also recycle a part of the turn in order to ensure that the recipient has perceived what was said.

The current study contributes to the previous studies dealing with OIRs, their timing, and multimodality (Schegloff et al., 1977; Floyd et al., 2015; Kendrick, 2015). However, by examining participants with hearing impairment, this study explores also the nature of hearing disability (Lind et al., 2006; Pajo, 2013; Ekberg et al., 2017; Laakso et al., 2019). Especially, this study will provide new information by comparing the timing of OIR between individuals with normal hearing and with hearing impairment. Both quantitative and qualitative results are provided. The study aims to answer to following questions:

(1) In relation to the trouble source turn, what is the timing of OIRs?

(2) What visual bodily resources does the recipient utilize during OIR sequences?

(3) Does the timing of various types of OIRs and their visual bodily style of production differ between the participants with normal hearing and with different degrees of hearing impairment?

2. Materials and methods

2.1. Data and participants

The data are video recordings of naturally occurring conversations in Finnish between friends and family members. The total duration of these recordings was 12 h and 24 min, which consisted of 14 dyads. The participants either had normal hearing or hearing impairment. Thus, the data consisted of 3 dyads who both had normal hearing (N = 6, 4 h 7 min) and 11 dyads with one participant with normal hearing (N = 11) and one participant with diagnosed hearing impairment (N = 10, total 8 h 17 min). The degree of hearing impairment varied from mild-to-severe state. The severity of hearing impairment was measured with better ear hearing level (BEHL, according to the EU expert group, Stephens, 1996). In total, four of the participants had mild, two had moderate (4 h 14 min), and four had severe hearing impairment (4 h 3 min) (see Table 1). Hearing aids were used during the recordings (either in one ear or both ears). Informed consent was obtained from all participants.

TABLE 1

Table 1. Details of the participants and other-initiation of repair data.

Due to visibility, the setting was to be a coffee table or a sofa with the participants sitting across one another at a 90-degree angle or sideways (Figures 1, 2). However, at times either of the participants was standing, moved away, and was absent from the screen for a short period of time. The content of the conversations included everyday matters, for example, providing information on recent occurrences or ongoing activity, story-telling, or stating opinions. In addition, some food and drink are often set on the table as well as other items (laptop, mobile phone, etc.).

FIGURE 1

Figure 1. Table-setting.

FIGURE 2

Figure 2. Sofa-setting.

2.2. Identification, annotation, and qualitative analysis of other-initiations of repair and multimodal actions

The study combines qualitative and quantitative methods to address the timing of OIR. For the qualitative analysis of OIR sequences, conversation analysis (CA) was utilized (Sidnell, 2010). The CA method is based on sequential analysis. Thus, the analyst examines not only what is said or done but also when these actions are produced. The analyst detects recurrent features and reveals how they are negotiated turn-by-turn in everyday interaction. The phenomenon studied in this study, OIR, was first analyzed in its local context before quantification. As a result, 191 OIRs were identified. After this, OIR sequences were annotated with ELAN 6.4 software. The Max Planck Institute for Psycholinguistics (Nijmegen, the Netherlands) has developed ELAN as a multimedia annotation tool for both audio and video recordings (Sloetjes and Seibert, 2016). A uniform ELAN annotation template was created, which was consistently applied to all participants. We annotated the other-initiated repair phenomena, according to the type of other-initiation to open requests, restricted requests, and restricted offers (cf.Dingemanse et al., 2015). From multimodal action, we annotated gaze, hand, and body movement toward the co-participant, as well as facial gestures (frowns, raised eyebrows). In addition, the duration of gaps preceding the OIRs, that is, the temporal relation between a trouble source turn and OIR was measured in milliseconds with ELAN. An OIR sequence includes a turn-by-turn negotiation of the repair. The structure of an OIR sequence, as it is analyzed in this study, is presented in the following schema:

1. A: TROUBLE SOURCE TURN
  (milliseconds)

2. B: OTHER-INITIATION

3. A: REPAIR

Other-initiation is a question-like utterance (line 2, speaker B), which retrospectively focuses on the previous turn. Thus, this previous turn is labeled as the trouble source turn (line 1, speaker A), which needs to be clarified or confirmed in some way in a repair turn (line 3, speaker A). Between the trouble source turn and the other initiation, there is potentially a transition space, which is presented in milliseconds (elsewhere silences are rounded seconds e.g., (0.6) in the transcription). The sequences analyzed are transcribed according to CA conventions. Multimodal actions are marked according to Mondada (2019) in the transcripts presented in the results section. For complete transcription keys, see Appendix A (Sidnell, 2010; Mondada, 2019). For the protection of privacy, screenshots are not provided as a part of the transcription (see examples in 2.1).

The final collection of OIRs follows a similar sequential structure. Therefore, all cases of OIR, which occurred sequentially late, i.e., not immediately after the trouble source turn (one or several turns in between), were excluded (a total of 13 cases). In addition, the focus of this study was on verbal OIRs, and thus, 11 bodily OIRs without speech were excluded. Thus, the total of OIRs included was 167 (see participant details and OIR data in Table 1).

In the analysis, a multimodal approach was considered vital. Here, “multimodal” refers to talk and visual bodily sets of action as follows: gaze, changes in facial expression, movement of the head and upper body, and pointing gestures. Although these visual bodily actions are a set of recurrently similarly occurring actions, the focus of the identification has been on their slightly different combinations. For example, a verbal OIR can be accompanied by just a gaze shift without the more overt bodily resources (e.g., changes in facial expression). Another focus point has been to annotate the durations of these various visual bodily sets of action. Overall, the annotation and qualitative analysis specified the exact timing of the multimodal resources, that is, both verbal OIRs and visual bodily actions during the OIR sequence. More specifically, the progress of the three-turn sequences of the trouble source turn, other-initiation, and repair were analyzed to detect systematic interactional patterns in the timing of OIRs. The foundation for the analysis was the ordinariness of repair in conversation. In other words, the process of analysis handled the turns of all participants equally. The focus was on local negotiation between participants during the repair sequence. After collecting the recurrent phenomena, the analysis compared the utilization of various resources between the different participant groups and how the participants oriented to these resources.

2.3. Measurements and statistics

To analyze the timing of OIRs, the durations of silent gaps between the trouble source turn and the beginning of verbal OIR were measured in milliseconds with ELAN (see Figure 3).

FIGURE 3

Figure 3. ELAN 6.4 screenshot.

If the verbal OIR was produced overlapping the ongoing trouble source turn, the duration of the overlap was measured similarly and marked negative. If the timing occurred seamlessly with no gap, the value was 0.

To analyze visual bodily actions during OIR sequences, the starting point and duration of facial expressions, hand and body movements, and eye gaze toward the speaker of the trouble source turn were also measured from ELAN annotations. In the statistical analysis, the starting point of bodily action was categorized into before verbal OIR if the visual bodily action was initiated in overlap with the trouble source turn or turn transition (=before) or during verbal OIR if bodily action was started simultaneously with the verbal OIR (=during). If no visual bodily action was utilized, it was marked as absent. In addition, the category away was used in relation to gaze direction indicating that the recipients kept their gaze averted from the speaker throughout the OIR sequence. Moreover, during some OIR sequences, the visual bodily action was not visible on the screen and thus unavailable.

The first phase of statistical analysis consisted of an initial examination of the measured variables, the timing of verbal OIRs as the function of the measured gap or overlap, duration of recipient gaze toward the speaker, and recipients' other visual bodily actions for the three groups of participants as follows: participants with normal hearing, participants with mild-to-moderate hearing impairment, and participants with severe hearing impairment. In this phase, the ranges and means of the measured variables were calculated and compared between the groups. In the next phase, SPSS statistics software (version 28) was used to reveal possible significant differences in OIR timing and the use of visual bodily actions between the groups using one-way analysis of variance (ANOVA). In statistical analysis, the groups of normally hearing participants and hearing aid users with mild-to-moderate hearing impairment were combined into one group, normal-to-moderate, as they performed similarly in other-initiation of repair according to the inspection of descriptive statistics (ranges and means). The combination of these groups was considered justified, as also a previous study had shown that hearing aids amplify hearing quite close to normal by hearing aid users with mild-to-moderate hearing impairment who then do not differ from normally hearing participants in their other-initiation of repair (Laakso et al., 2019). In addition, the number of OIRS in the group of hearing aid users with mild-to-moderate hearing impairment was very low and thus not adequate for statistical analysis. ANOVA was, then, run between the two groups, normal-to-moderate vs. participants with severe hearing impairment, to detect whether there were significant differences between these two groups in the timing of OIRs, or in their use of multimodal resources, i.e., gaze and visual bodily actions.

3. Results

The first section of the results provides a qualitative analysis of the OIR sequences. This qualitative approach reveals three patterns in the timing of OIR in the present data. The analysis is presented in three subsections according to timing as follows: turn transition with delay, turn transition without delay, and turn transition in the overlap. In addition, multimodal analysis reveals the various inter-related resources of speech, visual bodily actions, and their timing. In the second section of the results, measurements of the timing of OIRs are presented. Furthermore, distributions of gaze direction and other visual bodily actions during OIR sequences are shown. Together the qualitative analysis and descriptive statistic information of OIR timing offer a basis for comparing individuals with normal hearing and individuals with various degrees of hearing impairment.

3.1. Delayed turn transition in other-initiation of repair and gaze as a visual bodily resource

In the first two conversational extracts both the participant with normal hearing and the participant with severe hearing, both the participants with normal hearing and with severe hearing impairment other-initiate repair. In Extract 1, the participant with normal hearing produces an OIR (line 03). In Extract 2, an individual with a severe hearing impairment (ISHI) produces an OIR (line 04). The gaze of the OIR producer is marked below the turn of the current speaker.

In Extract 1, a new topic of a gold chain is introduced (line 01), and in Extract 2, the topic concerns the spouse's work and one of his colleagues (Kiuru, pseudonym) (lines 01–02). In each case, at turn transition relevance place, the conversation is halted for 844 ms (Extract 1) and 780 ms delay (Extract 2). In both of the extracts, this delay is the first indication of trouble. In other words, the recipients are not offering immediate comments but withhold the progression of the conversation. When the recipients finally produce an utterance, it is not a straightforward continuation but either a restricted request mikä kultaketju, “what gold chain” (Extract 1) or a restricted offer ei puhunu sulle mitää vai, “(you mean he) did not say anything to you” (Extract 2). At this point, by utilizing these verbal OIRs, the recipients' trouble has become overt. These requests need clarification or confirmation from the speaker before the main line of conversation can continue (see line 04 in Extract 1 and line 05 in Extract 2).

In terms of visual bodily resources, the recipient's gaze toward the speaker already during the trouble source turn is presented. At the beginning of Extract 1, the recipient of the talk shifts her gaze toward the speaker quite early during the production of the trouble source turn (line 01). She, then, holds her gaze toward the speaker throughout the extract. In Extract 2, the ISHI-recipient is gazing at the speaker already when the speaker initiates his turn (line 01). The gaze holds toward the speaker and continues to the end of the confirmation turn in line 5.

In summary, both participants, with normal hearing and severe hearing impairment, initiate repair with similarly timed patterns. The gap from 780 ms to 844 ms presented above is clearly a delay in the progression of a conversation. This delay before an OIR can be the pre-indication of trouble for all participants, which is, then, verified with a verbal OIR. Furthermore, the recipient's gaze toward the speaker already during the trouble source turn is a frequent phenomenon in the present data. Some differences in gaze behavior, however, can be seen in Extracts 1 and 2. The individual with severe hearing impairment in Extract 2 keeps a constant gaze toward the speaker while the participant with normal hearing in Extract 1 shifts their gaze toward the speaker later during the trouble source turn. In other cases, the gaze of the normally hearing recipients shifts even later during the OIR (not visible in the extracts above). These differences in gaze behavior may reveal the ISHI's recurrent need for visual compensation and speechreading, in speech perception. In addition to gaze resources, other visual bodily resources are not necessarily utilized in conjunction with verbal OIRs (see Extracts above). Nevertheless, they can be a vital part of trouble indication, which will be revealed in the next subsections.

3.2. Turn transition without delay in other-initiation of repair and a visual bodily indication of trouble

As a continuation to the previous section, the analysis in this subsection reveals another typical timing pattern of OIR production in the present data: turn transition without a delay. At the same time, this subsection shows how gaze was utilized during OIR sequences. The recipients' gaze is absent at the beginning of the trouble source turn and then shifts toward the speaker during that turn. In addition, this subsection reveals how other visual bodily resources are utilized during verbal OIRs.

In Extract 3, the recipient of talk producing the OIR is the spouse who has normal hearing ability (line 03). She is talking with her husband who has a severe hearing impairment (ISHI). The couple has been talking about a play in a theater, and whether they should go and see it. The word Gogoli in line 1 refers to the writer Nikolai Gogol. Extract 4 shows how an individual with mild hearing impairment (IMHI) also moves on to OIR without delay (line 04). The topic is an itchy nose. OIR producers' gaze, body movements, and facial expressions are marked below the current speaker's utterance.

Unlike the previous Extracts (1–2), here, the recipients move on to the OIR after only 354 ms and 313 ms gap (Extract 3, line 02 and Extract 4, line 03). In other words, instead of withholding the conversation with a delay, the recipients quite quickly announce trouble. In Extract 3, the OIR type is a restricted type OIR mikä, the nominative form of 'what' (in Finnish, the interrogative pronouns inflect for case and number), and in Extract 4, an open type request mitä, ‘what‘ (line 04).

Furthermore, in Extracts 3 and 4, the recipients' gaze is not constant throughout the OIR sequence (cf. Extract 2) but shifts toward the speaker during the trouble source turn. In addition, the shift in gaze direction is not the only inter-related resource but also other visual bodily resources are utilized. In Extract 3, the recipient slightly pokes her head toward the speaker and lifts her eyebrows. These actions are held stationary toward the end of the speaker's repair turn and released before the recipient produces confirmation nii, yeah (line 04). In Extract 4, the IMHI recipient shifts his gaze toward the speaker at the end of the trouble source turn (line 02). At the same time, he slightly turns his head so that it is directed more straightforwardly toward the speaker. The gaze is held while the OIR turn is produced. The repair turn produced by the spouse is a partial repetition of the trouble source turn (line 06). Repetition is common after an open type OIR. Here, the repetition is not specifically marked, for example, by utilizing prosodic or visual bodily resources. Therefore, the spouse is not overtly orientating toward trouble in hearing.

In summary, here, the participant with normal hearing and the participant with mild hearing impairment both produce OIRs with a minimal gap after the trouble source turn (in the extracts ~300 ms). In addition, the utilization of head and upper body movements toward the speaker and changes in facial expressions form various sets of visual bodily actions either before or during the OIR. These actions are held stationary, which displays increased attentiveness toward the conversational partner and emphasizes the indication of trouble.

3.3. Overlapping timing of other-initiation of repair using both verbal and visual bodily resources

The conversational fragment in this subsection is overtly different from the previous subsections in terms of the timing of OIR. The analysis reveals how a verbal OIR is produced in overlap with the trouble source turn. In addition, a similar timing of visual bodily resources is revealed. This timing pattern was used mostly by individuals with severe hearing impairment.

Based on the present data, the rate of OIRs increases as the ability to hear deteriorates (see Table 1). This is visible in the prolonged repair sequence in the next Extract 5. The topic is blueberry shrubs. The ISHI's friend shifts the topic by introducing a possible origin country, Germany, for the shrubs (line 01). ISHI other-initiates repair verbally two times (lines 2 and 6) and display trouble in hearing also with bodily actions. First of all, the ISHI holds her gaze toward the speaker throughout the extract. In addition, the ISHI keeps leaning toward the speaker from line 2 to line 10 until she straightens her posture.

In this extract, ISHI utilizes two OIRs, mitä, “what” (line 02) and nyt mä en kuullu, ‘now I didn't hear‘ (line 06) before she produces a turn that confirms that she has perceived what was said (line 10). Both OIRs are open type actions, which do not target anything in the previous turn. However, the second OIR offers more information on the cause of the trouble, as ISHI labels the trouble as her inability to hear. The ISHI is, therefore, explicitly struggling with her ability to perceive what was said. In addition, ISHI's friend (lines 03 and 08) is clearly repairing trouble in hearing. As in Extract 4, the repair turns are repetitions. However, here, a strong prosodic emphasis on keywords is used as a resource to assist perception.

The ISHI produces both OIRs in overlap with the speaker (first, a slight overlap of −91 ms and then a somewhat longer −319 ms). In other words, this overlapped timing is fast in the context of an otherwise prolonged OIR sequence. Moreover, timing the OIR in the immediate vicinity of the trouble source adds some targeting power to the otherwise open type OIRs. In addition, the ISHI offers an added indication of misperception by utilizing a frown (line 03). The response from the speaker of the repair turn seems to be a modification of the keyword while the repair turn is ongoing (saksalainen—Saksasta, “German—from Germany,” lines 03 and 04). Furthermore, the ISHI is leaning her body toward the speaker (lines 02–09). In addition, her friend mirrors this leaning action (line 08). These visual bodily resources are utilized as overt actions throughout the prolonged OIR sequence, displaying the ongoing trouble until the problem is solved, and ISHI nods while uttering a confirmation joo, “yes” (line 10).

In summary, Extract 5 shows that as hearing deteriorates to a severe degree, OIRs can be produced already during the trouble source turn. All ISHI participants in the present data utilized this overlapping timing style, especially with open type OIRs. Thus, the timing of OIRs can dramatically change into an urgent pattern. Similar overlapped timing can be seen in visual bodily indications of trouble, such as a frown or a lean toward the speaker. These bodily actions can hold longer than spoken utterances. Therefore, they reveal the ISHI-recipient's ongoing trouble. These embodied actions either inter-relate with the verbal OIRs or are produced as embodied-alone pre-indications of trouble during the speaker's turn. They also play a key role in indicating confirmation of when the ISHI-recipient is ready to proceed in the main line of conversation. For example, straightening the upper body posture can work as an indication of sufficient repair and the passing of trouble.

3.4. Differences between groups in the timing of other-initiations of repair and visual bodily actions

In the following section, measurements of the temporal relation between trouble source turns and OIRs are presented. In addition, the utilization and timing of gaze direction and other visual bodily actions are summarized.

3.4.1. Timing of other-initiations of repair

Table 2 and Supplementary Figure 1 present descriptive statistics of the timing of turn transition measured in milliseconds in the total amount of the data, 167 OIRs, of which the participants with normal hearing produced 60 OIRs. The timing of an OIR is the duration of the transition space between the trouble-source turn and the OIR (see conversational extracts in the previous subsection). If the timing occurred as a silent gap the value was positive, if the timing occurred seamlessly the value was 0, and finally, negative value indicated an overlapped timing with the trouble source turn. The minimum and maximum values for the total data (N = 167) were −1065–3345 ms (Supplementary Figure 1, see outlier dots).

TABLE 2

Table 2. Descriptive statistics for turn transition duration (gap and overlap) between trouble source turn and OIR (mean, standard deviation, median, and range between minimum and maximum in milliseconds) in participants with normal hearing and different degrees of hearing impairment.

As Table 2 shows, the mean values and standard deviations displayed variation. The variation in OIR timing is presented in Supplementary Figure 1, where the lower whisker lines show the variability outside the most frequent distribution. On average, the gap for participants with normal hearing was 683 ms, which indicates delay and approaches the 700 ms mean gap in the OIR-timing study by Kendrick (2015). The mean gap of 711 ms for participants with mild-to-moderate hearing impairment was quite similar to normal hearing individuals, whereas the participants with severe hearing impairment often produced OIRs without delay (mean 468 ms). When the timing distributions between the normal-to-moderate group and the group with severe hearing impairment were analyzed using one-way analysis of variance (ANOVA), there was a statistically significant difference between the two groups in the timing of OIRs (p = 0.048, F = 3.963).

A closer inspection revealed variation between the timing of open and restricted OIRs. All participants took a long time to initiate restricted OIRs than open OIRs (Table 3). The mean timing for participants with normal hearing was 585 ms for open OIRs and a somewhat longer, 703 ms for restricted OIRs. Respectively, the means for participants with mild-to-moderate hearing impairment were 610 ms vs. 793 ms, i.e., quite close to normally hearing participants. However, participants with severe hearing impairment initiated open OIRs very fast (mean 223 ms). The mean for restricted OIRs for participants with severe hearing impairment was 585 ms, which is clearly longer than the open OIRs but still faster than the timing of restricted OIRs in the conversations with participants with normal hearing or mild-to-moderate hearing impairment. The difference in the timing of open vs. restricted OIRs was statistically significant between the two hearing groups (p = 0.023, F = 3.861).

TABLE 3

Table 3. Descriptive statistics for gap durations by type of other-initiation (mean, standard deviation, and median in milliseconds) in participants with normal hearing and different degrees of hearing impairment.

Participants with severe hearing impairment initiated open type OIRs seamlessly, without any measurable gap, more often than other participants in the data (Supplementary Figure 2). Furthermore, open type OIRs in overlap with the trouble source turn were produced only by participants with severe HI (Supplementary Figure 2). However, as shown in Supplementary Figure 3, restricted OIRs were sometimes produced in overlap by all participants in the data. The distribution of restricted OIRs was overall more variable than that of open OIRs (Supplementary Figures 2, 3).

3.4.2. Timing of gaze and other visual bodily actions

Supplementary Figure 4 presents four patterns in the timing of gaze direction (see Measurements and statistics). The four patterns capture the differences across participants with various hearing statuses. The results show broadly similar patterns in the timing of gaze direction across participants with normal hearing and participants with mild-to-moderate hearing impairment. In these participant groups, the timing of gaze toward the co-participant frequently occurred while producing the OIR-turn (during OIR). Another recurrent feature was that the recipients of the talk were gazing at the speaker already during the trouble source turn (before OIR).

Some fundamental differences in the timing of gaze between the groups were also detected. Participants with normal hearing were more often directing their gaze away from the conversational partner during the OIR sequence (Supplementary Figure 4). Thus, the whole OIR sequence (trouble source, OIR, and repair) could be produced without a gaze directed toward the co-participant. Similarly, variability in gaze direction was visible in participants with mild-to-moderate hearing impairment. A robust contrast was found in connection with severe hearing impairment. An almost constant gaze toward the speaker was utilized by participants with severe hearing impairment. When the normal-to-moderate group and the group with severe hearing impairment were compared, a statistically significant difference (p = 0.039, F = 3.319) was found between the two groups in their gazing toward the speaker.

Visual indications of trouble or attempts to hear better were upper body leans and head pokes or turns toward the speaker, or if the hearing aid was only in one ear, that side of the head was tilted toward the speaker. Facial expressions such as frowns and raised eyebrows and pointing with a finger were also utilized. These visual bodily actions were often combined.

Visual bodily actions in relation to OIRs were found in all participant groups (Supplementary Figure 5). However, their frequency, timing, and quality displayed variation. The frequency of bodily actions during OIRs was 13/60 (22%) of all OIRs for participants with normal hearing. In mild-to-moderate hearing impairment, the utilization of visual bodily actions was low (3/18). Thus, in both groups, with participants with normal hearing and mild-to-moderate hearing impairment, visual bodily actions were mostly absent (Supplementary Figure 6). Visual bodily actions were mostly utilized by participants with severe hearing impairment in 48/89 (54%) of all OIRs. Supplementary Figure 6 shows the timing of visual bodily actions. Participants with normal hearing displayed bodily actions typically during verbal OIRs. Only once was the visual bodily action (frown) timed before the verbal OIR, in overlap with the trouble source turn. Similarly, the participants with mild-to-moderate hearing impairment initiated visual bodily actions twice during OIRs and only once before OIR production. Participants with severe hearing impairment also mostly utilized visual bodily actions during verbal OIRs (34/48). However, 14/48 (29%) of the visual bodily actions were initiated before the verbal OIRs, already while the trouble source turn was ongoing. At times, the set of visual bodily actions consisted of two separate actions as follows: first in overlap with the trouble source turn (e.g., a frown) and then a more emphasized action (e.g., a lean toward the speaker) during the verbal OIR (occurs 6/14 times in the severe HI-data). When the groups of normal-to-moderate and severe hearing impairment were compared, no statistically significant difference was found in the timing of visible bodily actions (p = 0.064; F = 3.482), although their frequency was clearly higher in connection with severe hearing impairment.

4. Discussion

This study presented conversation analytic qualitative evidence and quantitative results regarding repair sequences initiated by the recipient of talk in a mundane home environment (e.g., a coffee table or a sofa setting). More precisely, the focus of the study was on the timing and multimodal features of other-initiated repair sequences (OIRs). Particularly, comparisons between individuals with different hearing statuses, from normal hearing ability to severe hearing impairment, expand our knowledge of the timing of OIRs. The results revealed differences connected to hearing status, which are important to discuss further.

4.1. Timing of other-initiations of repair

The results showed that although a delay before the production of OIRs was common with all participants, the timing of OIRs displayed variation. The mean value for all OIRs for participants with normal hearing and participants with mild-to-moderate hearing impairment was ~700 ms, which is in line with the findings of Kendrick (2015). A delayed timing between trouble source turn and other-initiation was expected because, in previous studies, OIRs are regularly initiated after a gap (Schegloff et al., 1977; Schegloff, 2000; Kendrick, 2015). However, participants with severe hearing impairment were faster as their mean value for OIR was ~500 ms and for open OIRs even faster, ~200 ms. The mean value of half-second timing falls into typical turn-taking variation and is not to be considered a delay (see Stivers et al., 2009). This is exceptional because a clear delay preceding OIRs is said to offer them special interactional focus (Schegloff et al., 1977). In other words, a delay in the typically fast-proceeding main line of conversation suggests a possible threat to the shared understanding between the interactants requiring more interactive attention (Drew, 1997; Schegloff, 2000). In the current data, delayed timing displaying a special interactional focus was apparent for all participants when they took a long time to initiate restricted OIRs than open OIRs (see Table 3). These results are in clear contrast with Kendrick (2015) who found that open type OIRs, on average, took longer than restricted OIRs. The data studied by Kendrick were English conversations in a quite similar setting with friends and family as our Finnish conversations. Thus, the differences may be based on differences in cultural-linguistic conversational practices, although this should be studied further with larger conversational samples. Furthermore, the results of the current study show that the timing of different OIR types can vary, especially when hearing deteriorates severely and open type OIRs become more common.

The urgency of other-initiated repair requests was evident in our results considering severe hearing impairment. This is in line with Schegloff (2006; p. 82) who argues as follows: “doing a request early in the organization of an interaction can be a way of marking its urgency or some other feature known to be recognizable to the recipient(s).” The participants with severe hearing impairment produced OIRs faster than participants with mild-to-moderate hearing impairment whose timing was quite similar to the participants with normal hearing. Furthermore, as the OIR types were studied separately, open type OIRs by participants with severe hearing impairment were produced very fast with only a ~200 ms gap mean value (cf. restricted OIRs ~600 ms). In addition, a seamless or overlapped timing with the trouble source turn was more typical in the severe hearing impairment group than in other participant groups, which is in line with previous observations by Lind et al. (2006) and Pajo (2013). This fast and even urgent timing of OIRs suggests that participants with severe hearing impairment need to tackle hearing problems early in order to perceive what was said. The frequent utilization of open type OIRs can be interpreted to mean an absolute inability to perceive what was said. This is also in contrast with Kendrick (2015) who argued that the earlier timing of restricted OIRs in his study offers evidence for specificity preference in conversation, meaning that OIRs that specifically target the trouble source are seen as preferred over the unfocused open type OIRs. Similar findings to our study on early OIR timing were reported in a study, where immigrants who were second language learners utilized descriptive hand gestures as restricted offers (Jokipohja and Lilja, 2022). These candidate understandings were performed early in transition spaces (slight overlap, seamless production, and micropause). A delay was rare, and if it occurred, it was no more than 0.8 s. Thus, marking OIRs as urgent may be important in asymmetrical interactions where the abilities of the recipients are susceptible and can cause an internal threat to shared understanding (based on deteriorated hearing or weak second language skills). In addition, for recipients with severe hearing impairment, the fast need to halt the conversation with early OIR timing may relate to the prevention of possibly more trouble if the speaker continues. Another interpretation is that individuals with a severe hearing impairment attentively identify possible hearing problems and prepare to respond fast. Therefore, OIRs are not necessarily withheld but are more likely produced immediately after the trouble emerges. Thus, the recipients with severe hearing impairment are not acting according to the general conventions of preference, that is, offering space for the speaker of trouble source turn to self-initiate repair (Schegloff et al., 1977; Pomerantz and Heritage, 2012; Kendrick, 2015). This finding suggests that general conversational preferences may alter in asymmetric interactions.

4.2. Visual bodily resources and actions

In terms of timing of gaze, the results show broadly similar patterns across participants with normal hearing and participants with mild-to-moderate hearing impairment in that at the latest, while producing the OIR-turn and mutual eye gaze was typically established. Some fundamental differences were also detected. Participants with normal hearing and mild-to-moderate hearing impairment more often gazed away from the interlocutor during the whole OIR sequence, which is in line with Kamunen (2019) who studied typically embodied disengagements in diverse everyday settings. In contrast, individuals with severe hearing impairment kept almost a constant gaze toward the speaker. In addition, mutual gaze between the participants prevailed because the speaker was also typically gazing at the individuals with severe hearing impairment. Similarly, (Skelt, 2006; p. 320) has shown that when a conversational partner has experience in interacting with individuals with hearing impairment, they may alter their gaze utilization into a “high-gazing” state. This almost constant gaze direction toward the recipient, similar to the current study, can be seen as highly relevant because it can contribute to the prevention of perceptual problems (audio–visual speechreading, Tillberg et al., 1995). Furthermore, in general, gaze devices, such as mutual gaze, highlight the collaboration between participants (Goodwin, 1981; Rossano, 2012).

Other visual bodily resources, upper body movements toward the speaker, or changes in facial expressions (frowns or raised eyebrows) were utilized in all participant groups. This result supports the findings from previous studies on OIR sequences (Floyd et al., 2015; Kendrick, 2015; Oloff, 2018; Kamunen, 2019). However, unlike in the previous studies, the current study revealed that the hearing status of interactants can change the frequency, timing, and quality of visual bodily actions. Visual bodily actions were mostly utilized by participants with severe hearing impairment, 54% in relation to all OIRs in comparison to participants with normal hearing, and 22% of all OIRs (see Supplementary Figure 5). In severe hearing impairment data, the timing of visual bodily resources occurred typically during verbal OIRs (71%), but bodily actions were also produced before OIR in overlap with the trouble source turn (29%). In comparison, participants with normal hearing and with mild-to-moderate hearing impairment timed visual bodily actions occurring almost only during verbal OIRs. In addition, severely deteriorated hearing increased qualitatively more overt visual bodily resources. This was evident when the visual bodily resources were produced as two separate actions as follows: first in overlap with the trouble source turn and then with a more emphasized bodily action during the verbal OIR. A strong lean closer toward the speaker or the head turn so that the hearing aid side is toward the speaker was evidence of visual bodily resources, which could be seen as public displays of misperception or which displayed an increased attempt to hear better.

All the abovementioned visual bodily actions, including gaze, were put on hold and typically released toward the end of the repair turn, indicating sufficient repair (see Extract 5). These findings are similar to several other languages and interactional settings (see Floyd et al., 2015; Oloff, 2018; Kamunen, 2019). Sustained holds can also be labeled as visual bodily actions, even if the gaze or body is immobile. In any conversation, visual bodily holds offer a possibility to share attentiveness over several turns of talk (Floyd et al., 2015). However, this is vital in interactions with individuals with hearing impairment because by using resources such as a frown and lean toward the speaker, they can share their perceptual uncertainty without explicitly topicalizing it, and thus without recurrently disturbing the conversational flow more than necessary (cf. Pajo and Klippi, 2013). As shown in Extract 5, the conversational partners can mirror these visual bodily actions, which indicates intensified collaboration with the recipient with hearing impairment. For example, leaning forward toward the recipient while producing repair can be utilized to ensure understanding (Rasmussen, 2014).

4.3. Pre-indications of trouble

The discussion so far has already revealed that verbal OIRs can be produced fast to initiate the repair sequence early, even in overlap with the trouble source turn. For individuals with severe hearing impairment, this timing design seems to be especially important (cf. also Lind et al., 2006; Pajo, 2013). In addition, visual bodily pre-indications of trouble, such as, held gaze toward the speaker, and other embodied-only actions during the trouble source turn have been shown as evidence of indications of trouble preceding verbal OIRs. These findings are in line with other studies concerning visual bodily resources and verbal OIRs (Floyd et al., 2015; Kendrick, 2015; Oloff, 2018; Kamunen, 2019). Although an overlapped production is viewed as an intrusion into the speaker's turn, the visual bodily resources using gestural modality can be less intrusive upon speech than a spoken utterance. Therefore, they may be produced earlier than verbal OIRs. Similarly, Floyd et al. (2015) claim that visual bodily actions are a “parallel system,” which is allowed to occur in overlap with verbal turns of talk. However, in the current study, participants with normal hearing and with mild-to-moderate hearing impairment indicated trouble gesturally preceding verbal OIRs only once. In comparison, for individuals with severe hearing impairment, the visual bodily actions can work as a sensitive resource to target the actual trouble in the trouble source turn. For example, Extract 5 revealed how a frown in the immediate vicinity of a keyword possibly affected its repetition during the production of the repair turn.

In general, recipients' gaze toward the speaker may elicit a fast response. Support for this claim can be found in Stivers et al. (2009), who found that in connection with polar questions, gaze direction plays a role in the faster timing of responses. Similarly, in the current data, a mutual gaze seems to be a robust platform for early and fast indications of trouble. Thus, although the local context of measured response time (trouble source turn—OIR) in the current study is different from Stivers et al. (2009), we can still use it to compare the turn-taking variation (see also Levinson and Torreira, 2015). This type of gaze design was primarily utilized by participants with severe hearing impairment and their conversational partners. Therefore, from the perspective of mutual gaze, it is no wonder that also overt visual bodily indications of trouble were more often produced in overlap with the trouble source turn by participants with severe hearing impairment than in other participant groups (see also Pajo and Klippi, 2013). Thus, the severe degree of disability in the hearing must be considered an important reason for the frequent use of trouble-indicating actions preceding verbal OIRs.

The data setting, a coffee table and sofa environment allowed easy access to face (of spatial formations, see Kendon, 1990). Occasionally, however, some manual activities (eating, paging through a magazine, checking text messages from a mobile phone, etc.) caused multitasking and shifted gaze direction. Similarly, as in Kamunen (2019), in these local contexts of interaction, a recipient's gaze shifts back to the speaker during or right after the trouble source turn could be the first sign of disengagement from the manual activity toward an OIR. Nevertheless, besides gaze, the other visual bodily resources were not utilized by participants with normal hearing and mild-to-moderate hearing impairment. Thus, the public indication of trouble did not occur preceding the verbal OIR. Overall, it seems that participants in the normal-to-moderate group have more choices in their actions during possibly troublesome local conversational contexts. In contrast, participants with severe hearing impairment may have only a limited choice of action in order to perceive what was said, and that is attentive listening including the overt utilization of gaze and other visual bodily resources.

4.4. Limitations of the study

This study presented a specific sample of data. Therefore, caution is required, as the results were based on a small OIR number in each participant group. Moreover, the conversational setting may have affected the interactional designs. Therefore, even if it is safe to assume that similar conventions to initiate repair may occur in other similar conversations, generalization of the results to other interactional settings must be handled with caution.

For future research, the comparative and multimodal approach adopted in this study could be extended to, for example, group conversations. This setting is known to be challenging for individuals with hearing impairment. However, we lack interactional studies, which demonstrate and offer evidence of the residual skills of the recipients with hearing impairment to manage group conversations in collaboration with their conversational partners.

Data availability statement

The datasets presented in this article are not readily available because the data is not able to be shared. Requests to access the datasets should be directed to kati.pajo@helsinki.fi.

Ethics statement

The studies involving human participants were reviewed and approved by the Ethics Committee of obstetrics and gynaecology, otorhinolaryngology, neurology, and neurosurgery. The patients/participants provided their written informed consent to participate in this study.

Author contributions

KP: data collection, study design, data analysis, and manuscript writing. ML obtained funding and took part in study design, data collection and analysis, and manuscript writing. Both authors contributed to the article and approved the submitted version.

Funding

This research was supported by the Academy of Finland, grant number 333858 for the project: Intersubjective Understanding in Atypical Interaction.

Acknowledgments

The authors thank Jyrki Tuomainen for the advice in statistical analyses and research assistants Sara Kaskelin, Heidi Liljeblad, and Eeva-Leena Malander for Elan's annotation of the data.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomm.2023.1173179/full#supplementary-material

References

Barker, A. B., Leighton, P., and Ferguson, M. A. (2017). Coping together with hearing loss: a qualitative meta-synthesis of the psychosocial experiences of people with hearing loss and their communication partners. Int. J. Audiol. 56, 297–305. doi: 10.1080/149920171286695

PubMed Abstract | CrossRef Full Text | Google Scholar

Dingemanse, M., Roberts, S. G., Baranova, J., Blythe, J., Drew, P., Floyd, S., et al. (2015). Universal principles in the repair of communication problems. PLOS ONE 3, 100. doi: 10.1371./journal.pone.0136100

PubMed Abstract | CrossRef Full Text | Google Scholar

Drew, P. (1997). “Open” class repair initiations in response to sequential sources of troubles in conversation. J. Pragm. 28, 69–101. doi: 10.1016/S0378-2166(97)89759-7

CrossRef Full Text | Google Scholar

Ekberg, K., Hickson, L., and Grennes, C. (2017). Conversation breakdowns in the audiology clinic: the importance of mutual gaze. Int. J. Lang. Commun. Disord. 52, 346–355. doi: 10.1111/1460-6984.12277

PubMed Abstract | CrossRef Full Text | Google Scholar

Floyd, S., Manrique, E., Rossi, G., and Torreira, F. (2015). Timing of visual bodily behavior in repair sequences: Evidence from three languages. Discour. Proc. 53, 175–204. doi: 10.1080/0163853X.2014.992680

CrossRef Full Text | Google Scholar

Gardner, R., and Mushin, I. (2015). Expanded transition spaces: the case of Garrwa. Front. Psychol. 6, 251. doi: 10.3389./fpsyg.2015.00251

PubMed Abstract | CrossRef Full Text | Google Scholar

Goodwin, C. (1981). Conversational Organization: Interaction between Speakers and Hearers. New York: Academic Press.

Google Scholar

Jokipohja, A. K, and Lilja, N. (2022). Depictive hand gestures as candidate understandings. Res. Lang. Soc. Interact. 3, 1–23. doi: 10.1080/083520222067425

CrossRef Full Text | Google Scholar

Kamunen, A. (2019). How to disengage: suspension, body torque, and repair. Res. Lang. Soc. Interact. 4, 406–426. doi: 10.1080/083520191657287

CrossRef Full Text | Google Scholar

Kendon, A. (1990). Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge, England: Cambridge University Press.

Google Scholar

Kendrick, K. (2015). The intersection of turn-taking and repair: the timing of other-initiations of repair in conversation. Front. Psychol. 6, 250, 1–16. doi: 10.3389/fpsyg.2015.00250

PubMed Abstract | CrossRef Full Text | Google Scholar

Laakso, M., Salmenlinna, I., Aaltonen, T., Koskela, I., and Ruusuvuori, J. (2019). Open-class repair initiations in conversations involving middle-aged hearing aid users with mild to moderate loss. Int. J.Lang. Commun. Disord. 54, 620–633. doi: 10.1111/1460-6984.12466

PubMed Abstract | CrossRef Full Text | Google Scholar

Levinson, S. C., and Torreira, F. (2015). Timing in turn-taking and its implications for processing models of language. Front. Psychol. 6, 731, 1–17. doi: 10.3389/fpsyg.2015.00731

PubMed Abstract | CrossRef Full Text | Google Scholar

Lind, C., Hickson, L., and Erber, N. (2004). Conversation repair and acquired hearing impairment: a preliminary quantitative clinical study. Au. New Zealand J. Audiol. 26, 40–52. doi: 10.1375/audi.26.1.40.55987

CrossRef Full Text | Google Scholar

Lind, C., Hickson, L., and Erber, N. (2006). Conversation repair and adult cochlear implantation: a qualitative case study. Cochlear Implants Int. 7, 33–48. doi: 10.1179/cim.71.33

PubMed Abstract | CrossRef Full Text | Google Scholar

Mondada, L. (2019). Transcribing silent actions: a multimodal approach of sequence organization. Soc. Interact. Video-Based Stud. Human Social. 2, 3150. doi: 10.7146./si.v2i1.113150

CrossRef Full Text | Google Scholar

Mortensen, K. (2016). The body as a resource for other-initiation of repair: cupping the hand behind the ear. Res. Lang. Soc. Interact. 49, 34–57. doi: 10.1080/08351813.2016.1126450

CrossRef Full Text | Google Scholar

Oloff, F. (2018). “Sorry?”/”Como?”/”Was?”—Open class and embodied repair initiators in international workplace interactions. J. Pragmat. 126, 29–51. doi: 10.1016/j.pragma.11002

CrossRef Full Text | Google Scholar

Pajo, K. (2013). The occurrence of “what,” “where,” “what house” and other repair initiations in the home environment of hearing-impaired individuals. Int. J. Lang. Commun. Disord. 48, 66–77. doi: 10.1111/j.1460-6984.2012.00187.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Pajo, K., and Klippi, A. (2013). Hearing-impaired recipients' non-vocal action sets as a resource for collaboration in conversation. J. Pragmatics, 55, 162–179. doi: 10.1016/j.pragma.06004

CrossRef Full Text | Google Scholar

Pomerantz, A., and Heritage, J. (2012). Preference, in The Handbook of Conversation Analysis. eds J. Sidnell and T. Stivers (Oxford: Blackwell Publishing Ltd), 201–228.

Google Scholar

Rasmussen, G. (2014). Inclined to better understanding–the coordination of talk and “leaning forward” in doing repair. J. Pragmatics, 65, 30–45. doi: 10.1016/j.pragma.2013.10.001

CrossRef Full Text | Google Scholar

Rossano, F. (2012). Gaze Behavior in Face-to-Face Interaction. [dissertation thesis]. Nijmegen: Max Planck Institute for Psycholinguistics.

Google Scholar

Sacks, H., Schegloff, E. A., and Jefferson, G. (1974). A simplest systematics for the organization of turn-taking for conversation. Language 50, 696–735. doi: 10.2307/412243

CrossRef Full Text | Google Scholar

Scarinci, N., Worrall, L., and Hickson, L. (2008). The effect of hearing impairment in older people on the spouse. Int. J. Audiol. 47, 141–151. doi: 10.1080/14992020701689696

PubMed Abstract | CrossRef Full Text | Google Scholar

Schegloff, E. A. (2000). When 'others' initiate repair. Appl. Linguis. 21, 205–243. doi: 10.1093/applin/21.2.205

CrossRef Full Text | Google Scholar

Schegloff, E. A. (2006). Interaction: the infrastructure for social institutions, the natural ecological niche for language, and the arena in which culture is enacted, in Roots of Human Sociality, Culture, Cognition and Interaction, eds N. J. Enfield and S. Levinson (Oxford: Berg). 70–96.

Google Scholar

Schegloff, E. A., Jefferson, G., and Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language 53, 361–382. doi: 10.2307/413107

CrossRef Full Text | Google Scholar

Seo, M. S., and Koshik, I. (2010). A conversation analytic study on gestures that engender repair in ESL conversational tutoring. J. Pragmat. 42, 2219–2239. doi: 10.1016/j.pragma.2010.01.021

CrossRef Full Text | Google Scholar

Sidnell, J. (2010). Conversation Analysis: An Introduction. UK: Wiley-Blackwell.

Google Scholar

Skelt, L. (2006). See what I Mean: Hearing Loss, Gaze, and Repair in Conversation. [dissertation thesis]. The Australian National University: Canberra. doi: 10.1055/s-0030-1252103

CrossRef Full Text | Google Scholar

Skelt, L. (2010). “Are you looking at me?” The influence of gaze on frequent conversation partners' management of interaction with adult with acquired hearing impairment. Semin. Hear. 31, 116–126.

Google Scholar

Skelt, L. (2012). Dealing with misunderstandings: the sensitivity of repair in hearing impaired conversation, in Hearing Aids Communication (Mannheim: Verlag für geschrächsforschung). Editors M. Egbert and A. Deppermann, 56–66. Available online at: http://www.verlag-gespraechsforschung.de/2012/pdf/hearingaids.pdf

Google Scholar

Sloetjes, H., and Seibert, O. (2016). Measuring by marking; the multimedia annotation tool ELAN, in Measuring Behavior 2016, 10th. International Conference on Methods and Techniques in Behavioral Research. eds A. Spink, G. Riedel, L. Zhou, L. Teekens, R. Albatal, and C. Gurrin 492–495.

Google Scholar

Stephens, D. (1996). Study Group on Terminology, Definitions and Hearing Assessment, in European Work Group on Genetics of Hearing Impairment. Editor A. Martini. European Commission Directorate, Biomedical and Health Research Program (HEAR) Info Letter 2. EU: European Commission Directorate. Available online at: https://cordis.europa.eu/project/id/BMH4960353

Stivers, T., Enfield, N. J., Brown, P., Englert, C., Hayashi, M., Heinemann, T., et al. (2009). Universals and cultural variation in turn-taking in conversation. PNAS 30, 10626–92. doi: 10.1073/pnas.0903616106.

PubMed Abstract | CrossRef Full Text | Google Scholar

Streeck, J., Goodwin, C., and LeBaron, C. (2011). Embodied interaction in the material world: an introduction, in Embodied interaction: language and the body in the material world (Cambridge: Cambridge University Press). Editors J. Streeck,C. Goodwin, and C. LeBaron 1–26.

Google Scholar

Tillberg, I., Rönnberg, J., Svärd, I., and Ahlner, B. (1995). Audio-visual speechreading in a group of hearing aid users the effects of onset age, handicap age, and degree of hearing loss. Scandinavian Audiol. 25, 267–272. doi: 10.3109/01050399609074966

PubMed Abstract | CrossRef Full Text | Google Scholar