Common representations of non-speech interfaces mainly include auditory icons and earcons. Auditory icons are sounds used to represent their associated events or attributes in daily life (Gaver, 1989), which refer to conveying computer operations or events by imitating familiar sounds of real-world events. They are usually relatively brief and icon-like (Larsson and Niemand, 2015; Amer and Johnson, 2018). For example, the sound of a broken plate is used to represent the operation of deleting a file, and the sound of a dot-matrix printer or typewriter signifies a printing operation.

Earcons are abstract short non-verbal auditory information with musical nature used to provide information and feedback on computer operations or interactions (Blattner et al., 1989; Brewster et al., 1993; Amer et al., 2013; Larsson and Niemand, 2015). For example, the rising “login” melody and the descending “logout” melody in the Windows operating system are formed by different combinations of high and low tones. Earcons can be mapped to any object, operation, or interaction event, and are designed as a series of mappings to represent hierarchical structure by manipulating their parameters, such as timbre and pitch (Garzonis et al., 2009).

To compensate for the weaknesses of traditional non-speech auditory cues, researchers developed spearcons, a compromise between short non-speech stimuli and full speech stimuli. These signals are short, time-compressed spoken words or speech phrases that are sped up even to the point where they are no longer considered speech. Spearcons can directly and quickly convey their meaning and relevant information to the listener (Petocz et al., 2008; Walker et al., 2013; Jeon, 2015), have good learnability, and can remarkably improve the efficiency and accuracy of menu navigation search (Palladino and Walker, 2007; Dingler et al., 2008; Walker et al., 2013). Compared with earcons, spearcons can provide a direct mapping between sounds and menu items, flexibly covering more content domains, and thus having better flexibility and generation. Therefore, spearcons have been studied and applied in some fields, such as patient monitoring alarms and menu navigation (Walker et al., 2013; Li et al., 2017; Sanderson et al., 2019).

Auditory warnings have become ubiquitous in daily work environments. Although they improve the efficiency of human-computer interaction, their potential hazards must be considered. First, the environment in which auditory warnings are used may require a high degree of concentration of the operator. Despite conveying important information, these signals might be not necessarily urgent. In addition, not every warning sound is important or urgent for every operator in the same environment. Given that sound signals are omnidirectional and forced hearing, people always easily get distracted and drawn to sounds that are not relevant or meaningful at the moment, even when they try to focus on something important (Banbury et al., 2001; Watson et al., 2004). When the alarm sounds, the irrelevant operator may be engaged in a cognitively demanding task, such as driving, intensive care, or surgery. Once attracted to the sound information, the operator may not focus on the important tasks. This situation may create some potential hazards. It has been found that an auditory warning of a relatively small event may lead to errors in the input of coordinates in navigation or weapon delivery systems, resulting in potentially serious consequences (Banbury et al., 2001). Furthermore, Lacherez et al. (2016) mentioned that auditory warning identification may compete with other cognitive processes for working memory resources and result in poor performance on other tasks. Many situations relying on auditory display assistance are related to user’s defects in performing dual tasks, such as in patients with Parkinson’s disease (Ashburn et al., 2001) or head injuries (Hart et al., 2002; Hein et al., 2005). Ashburn et al. (2001) found that patients with Parkinson’s disease who are prone to fall also perform poorly in dual tasks. Hence, auditory warnings and auxiliary systems must aim to control the specific cognitive demands (Ashburn et al., 2001), to avoid additional negative impacts on users or reduce the availability of systems.

Cognitive tasks, which usually rely on a person’s working memory, have been increasingly involved in many human activities. The impact of auditory warnings on the operator’s task performance is mainly concentrated on the working memory. Working memory refers to a memory system with limited capacity for temporary processing and storage of information (Baddeley, 2003). It plays an important role in many complex cognitive activities. Many theoretical models have attempted to explain this memory system. One that is widely held is the Baddeley’s Working Memory Model (Baddeley and Hitch, 1974), which suggests that working memory consists of visuo-spatial sketchpad, phonological loop, and central executive. Later research came up with the episodic buffer, forming the four-component model of the working memory system (Baddeley, 2000b).

Auditory information may interfere with working memory in a complex task environment. For example, those with changing patterns interfere with serial recall task performance. Irrelevant sounds (e.g., the sounds do not need to be noticed) can also interfere with the current task (Banbury et al., 2001; Macken and Jones, 2003; Hughes et al., 2007; Macken et al., 2009), and this phenomenon is called “Irrelevant Sound Effect (ISE).” Using the ISE paradigm, researchers found that the accuracy of reports decreased by 30–50% when unrelated narrative statements were played during a serial recall task (Ellermeier and Zimmer, 1997). Experimental analysis on the effect of external cockpit sounds on crew performance showed that compared with quiet or ambient aircraft noise, the presence of external background sounds substantially disrupted the memory of longitude and latitude information by up to 60% (Banbury and Jones, 1999). Serial recall was also hampered by various non-speech sounds, including pure tones (e.g., Klatte et al., 1995; Neath et al., 1998) and music streams (e.g., Nittono, 1997). Moreover, the interference of sound may be stable and difficult to be habituated (Jones et al., 1997; Tremblay and Jones, 1998), even if prolonged exposure did lead to some degree of habituation, and relatively short quiet periods could drive rapid dishabituation (Banbury and Berry, 1998).

Further research revealed that the perception and identification of learned auditory warnings can also interfere with working memory. However, learned melody and rhythm auditory warnings would interfere only when the participants attempt to identify them. By contrast, learned non-word phrases would interfere even when ignored (Lacherez et al., 2016). Given their different characteristics, we speculated that various kinds of auditory warnings may interfere with working memory differently. Alarm sounds used in previous studies were either earcons (e.g., rhythm and melody) or spoken non-word phrases. The impact of auditory icons and spearcons on working memory has not been determined. Spearcon is a hybrid auditory display between speech and non-speech (Jeon, 2015), and it appears to have both verbal and non-verbal attributes. Researchers have found that concurrent verbal tasks had a negative impact on the identification of spearcons (Davidson et al., 2019), and identifying learned spearcons may interfere with speech-based working memory tasks (Wolters et al., 2012). However, the impact of ignoring spearcons and auditory icons on working memory has not been explored, and no research has compared the interference of different kinds of auditory warnings on working memory.

It is widely accepted that working memory system is divided into verbal and spatial working memory. Most previous studies have focused on the impact of auditory warnings on verbal working memory. However, the influence of auditory warnings on spatial working memory and whether they interfere differently with the two domains deserve further exploration.

Due to the forced hearing nature of the sound signal, the warning sound tends to attract people’s attention. When the alarm sounds, some operators in the workplace may need to ignore it, but it may still be distracting or interfere with working memory. In the cognitive behavioral tradition, studies on the mechanism of sound interference with working memory performance have been mainly focused on how working memory task is interfered with by unrelated sounds that change acoustically (i.e., the changing-state effect) (Jones et al., 1992; Lecompte, 1995), and the physiological and behavioral distraction effect of an auditory event that deviates in some way from the recent hearing (i.e., the deviation effect) (Cowan, 1995; Titova and Näätänen, 2001). The duplex-mechanism account holds that sound can cause unnecessary auditory distraction either by interfering specifically with the processes involved in the focal task (interference-by-process) or by diverting attention away from a focal task regardless of the type of processing involved in the task (attentional capture) (Hughes et al., 2007; Hughes, 2014). In this view, the changing-state effect can be better explained by recourse to interference-by-process, and the deviation effect may be attributed to attentional capture. In another case, and in most cases, operators may need to identify warnings and determine mentally whether they need to take corresponding actions. There may be a distraction problem both when ignoring and when identifying the warnings. Whether distraction (or switching attention) and the process of identifying warnings would affect ongoing tasks involving verbal and spatial working memory may be related to resource limitation and interference.

Multiple Resource Theory (MRT) proposes four important categorical and dichotomous dimensions that account for variance in time-sharing performance. Each dimension has two discrete “levels,” each defining a separate but limited resource. The four dimensions are processing stages (perception and cognition vs. selection and response), perceptual modalities (auditory vs. visual), visual channels (focal vs. ambient), and processing codes (spatial vs. verbal) (Wickens, 2002). MRT predicts that resource interference occurs when two tasks are performed using the same domain resources, and worsens the performance compared with that when using different domain resources. For example, the interference between two tasks both requiring verbal perception is greater than that between one task requiring spatial perception and the other requiring verbal perception. What is noteworthy is that regardless of doing one or two tasks, MRT is relevant only in the region where overload is imposed by multiple tasks but not in the residual capacity region. For example, it can predict the size of dual-task decrements once overload has been reached (Wickens, 2008).

Similarly, the multi-resource model of working memory also involves the domain-specific assumptions about limited resources: working memory consists of multiple domain-specific subsystems, and each subsystem has its own resource pool (e.g., Baddeley and Logie, 1999). The nature of resources is domain-specific, that is, specific resources support verbal or visuospatial activities. Therefore, interference occurs when the two tasks involve information belonging to the same domain, and no (or minimal) interference occurs when the tasks involve information belonging to different domains. Verbal working memory is more likely to be interfered with by verbal tasks than by spatial tasks, and spatial working memory is more susceptible to interference from spatial tasks than from verbal tasks (Vergauwe et al., 2010).

In addition, another assumption about limited resources is that a general limited resource pool supports various cognitive activities (e.g., Egeth and Kahneman, 1975; Barrouillet et al., 2004). This pool of resources is often called attention. Verbal and spatial activities are assumed to compete for a common pool of domain-general limited resources, resulting in interference between the two activities (Vergauwe et al., 2010). Banbury and Jones (1999) found that speech interfered with visuospatial task performance despite being ignored. Studies have further confirmed that verbal and spatial activities interfered with each other under dual-task conditions, indicating the existence of a domain-general resource in the mental process of verbal and spatial (Vergauwe et al., 2010; Morey et al., 2018). Mobile phone use impaired driving safety, regardless of whether the phone was hand-held or hands-free (Strayer and Johnston, 2001). This finding suggests that processing sound information interferes with spatial tasks at least to a certain extent. However, most of the concurrent tasks in previous studies were verbal tasks (e.g., speech or text). The impact of non-speech auditory displays on spatial working memory remains to be further clarified.

This study attempted to explore and explain the impact of auditory warnings on working memory and its mechanism based on the duplex-mechanism account and the related resource theories. Based on the review of relevant literature, how identifying and ignoring three types of auditory warnings (auditory icons, earcons, and spearcons) affects performance on a verbal serial recall task (i.e., spatial working memory), and whether there are differences among them have not been determined. We investigated these questions in Experiment 1. We hypothesized that warning identification would have more influence on recall task performance compared with warning ignoring, and different types of auditory warnings would worsen recall task performance differently. Furthermore, we further explored whether the performance of location recall task (i.e., spatial working memory) was similarly affected by the three types of auditory warnings in Experiment 2. According to related theories, we hypothesized that auditory warnings would worsen the performance of location recall task, and the three types of auditory warnings impact location recall differently. Overall, this study evaluated the impact of different types of auditory warnings on the performance of verbal and spatial working memory tasks. The findings may help to draw people’s attention to the potential problems of using auditory warnings in related environments, especially those that require high working memory load, and may serve as a caution against the possible existence of overuse of auditory warnings in such environments. In addition, the impact of three types of auditory warnings (auditory icons, earcons, and spearcons) on working memory was investigated to provide useful guidelines for the selection and design of auditory warning signals. Finally, the differences in the interference degree of auditory warnings for verbal and spatial working memory were also analyzed.

Mauchly’s test of sphericity was examined, and the Greenhouse-Geisser correction was used where necessary. Descriptive statistics of the mean serial recall accuracy of auditory icon, earcon, and spearcon groups in different task conditions are shown in Table 1.

One-way ANOVA was performed on the no-warning baseline scores to ensure comparability among the three groups. The result confirmed that there was no significant difference among the three warning-type groups, F (2, 69) = 0.354, p = 0.703, partial η² = 0.01.

The results from mixed-design 3 × 3 factorial ANOVA showed that both task condition and warning type significantly affected the serial recall accuracy. The task condition revealed a main effect for task performance, F(2, 138) = 34.23, p < 0.001, partial η² = 0.332, and the main effect of warning type was significant, F (2, 69) = 3.922, p = 0.024, partial η² = 0.102. In addition to the main effect, a significant two-way interaction was found between task condition and warning type (see Figure 1), F(4, 138) = 7.092, p < 0.001, partial η² = 0.171.

Further simple effect analysis was conducted by using a post-hoc test with Bonferroni correction. The results revealed that for the auditory icon group, no significant difference in mean serial recall accuracy was found among the three conditions. For the earcon and spearcon groups, the mean serial recall accuracy for the identify-warning condition was significantly lower than that for the other two conditions (p < 0.001), but no significant difference was observed between the no-warning and the ignore-warning conditions. For the identify-warning condition, the mean serial recall accuracy of the auditory icon group was significantly higher than that of the earcon and spearcon groups (p < 0.05). For the ignore-warning condition, there was no significant difference in serial recall accuracy among the three groups.

The one-way ANOVA results revealed significant differences in the mean identification accuracy of the three groups, F(2, 69) = 25.311, p < 0.001, partial η² = 0.423. The mean identification accuracy for auditory icons was significantly higher than that for earcons and spearcons (p < 0.05), and that for spearcons was significantly higher than that for earcons (p < 0.001), as shown in Figure 2.

Spearman correlation analysis was conducted on the mean serial recall accuracy and the warning identification accuracy with the learnability (related to the number of practices when reached 100% accuracy) to confirm whether the results of this experiment were related to the learnability. The fewer practices to gain 100% accuracy indicated the better learnability. No significant correlation was found between learnability and serial recall accuracy when the auditory warning required to be identified (r = 0.214, p > 0.05) or ignored (r = −0.015, p > 0.05). However, a significant correlation was observed between learnability and mean warning identification accuracy (r = 0.451, p < 0.001).

The results of Experiment 1 showed that in the identify-warning condition, earcon and spearcon identification worsened the performance on the serial recall task. These results were consistent with previous studies. Identifying spearcons may interfere with verbal working memory tasks (Wolters et al., 2012). The perception and identification of learned earcons (rhythm) interfere with working memory (Lacherez et al., 2016). However, auditory icon identification did not interfere with the performance of the serial recall task. The identification of auditory icons may require less working memory than the identification of earcons and spearcons. Participants may have been performing the identifying auditory icon warnings within their residual capacity of available resources, preserving high accuracy at serial recall tasks (Wickens, 2008).

Auditory icon warnings had the highest warning identification accuracy among the three groups. This finding may be related to the use of sounds from real, daily events in auditory icons, and the fact that these signals are strongly representative and easy to learn. Moreover, it was found in our present study that warning identification accuracy was related to the number of practices (few practices indicated high accuracy). Therefore, we speculated that warning identification accuracy may be related to its learnability. However, the impact on identification performance caused by the resource’s competition of concurrent verbal serial recall tasks could not be ruled out. Furthermore, whether the spatial working memory is similarly affected when the participants identify or ignore the three types of auditory warnings remains unclear. These issues will be addressed in Experiment 2.

Mauchly’s test of sphericity was examined, and the Greenhouse-Geisser correction was used where necessary. Descriptive statistics of the mean red square location recall accuracy of each group in different task conditions are shown in Table 2.

A mixed-design 3 × 3 factorial ANOVA was performed on the effects of task condition and auditory warning type on red square location recall accuracy. The result revealed a main effect for task condition, F (2, 138) = 11.631, p < 0.001, partial η² = 0.144. In addition to the main effect, a significant two-way interaction was found between task condition and auditory warning type (see Figure 4), F(4, 138) = 3.302, p = 0.016, partial η² = 0.087.

Further simple effect analysis was conducted by using a post hoc-test with Bonferroni correction. The results indicated that for the auditory icon and spearcon groups, the mean location recall accuracy did not differ across the three task conditions. However, for the earcon group, the mean location recall accuracy in the identify-warning condition was significantly lower than that in the other two conditions (p < 0.001). For the identify-warning condition, the mean location recall accuracy was in the order: auditory icon group > spearcon group > earcon group; however, the differences were not significant. For the ignore-warning condition, the mean location recall accuracy for the three groups was not significantly different either.

One-way ANOVA results indicated significant differences in the mean identification accuracy of the three groups, F (2, 69) = 35.701, p < 0.001, partial η² = 0.509. The identification accuracy of earcons was significantly lower than those of auditory icons and spearcons (p < 0.001), but no significant difference was observed between auditory icons and spearcons as shown in Figure 5.

Spearman correlation analysis was conducted to further confirm whether the results were related to the learnability of auditory warnings. No significant correlation was found between learnability and mean location recall accuracy when the warnings were identified (r = 0.141, p > 0.05) or ignored (r = −0.082, p > 0.05). However, the mean warning identification accuracy was found to be significantly correlated with the learnability (r = 0.559, p < 0.001), which was consistent with the finding in Experiment 1.

Experiment 2 results showed that auditory icon identification had no significant interference on location recall task and had the highest identification accuracy among the three types of warnings. By contrast, earcon identification had significant interference on location recall task and the lowest identification accuracy. These results were consistent with experiment 1. The identification of earcons may require more working memory than that of auditory icons and spearcons. Participants may not be able to identify the earcons within their residual capacity of available resources, thereby leading to a competition with the location recall tasks for limited resources and resulting in low accuracy for the latter (Wickens, 2008).

The results of warning identification accuracy in Experiment 2 were consistent with those in Experiment 1. The identification accuracy was the highest for auditory icons, followed by spearcons and earcons. The results of both experiments were combined for a rough comparison. The findings showed that the overall performance of warning identification in the concurrent verbal task was worse than that in the concurrent spatial task, especially for the spearcons (see Figure 6). In the concurrent verbal task, the identification accuracy of auditory icons was significantly higher than that of spearcons. Meanwhile, no significant difference between auditory icons and spearcons was found in the concurrent spatial task. The correlation between learnability and warning identification performance for verbal tasks was lower than that for spatial tasks. This indicated that the accuracy of warning identification was weakly affected by learnability but greatly influenced by the concurrent verbal task, compared with that in the concurrent spatial task. Recent studies found that concurrent verbal tasks would reduce the ability of participants to identify the spearcons (Davidson et al., 2019), which is consistent with the present results. Furthermore, the impact of identifying warnings on working memory was roughly analyzed by comparing the recall accuracy difference in the ignore-warning and identify-warning conditions. Although identifying auditory warnings (e.g., earcons) may interfere with spatial tasks, they consistently had a greater impact on verbal tasks, especially spearcons (see Figure 7). Therefore, we further speculated that warning identification had a greater impact on the overall performance of the verbal working memory task than that of the spatial working memory task. Meanwhile, the verbal working memory task had a greater impact on the overall performance of warning identification than the spatial working memory task. The greatest variation in spearcons might be related to their speech features. Early studies suggested that non-speech sounds did not interfere with working memory (Salamé and Baddeley, 1987), though subsequent work revealed that this depended on other factors of auditory information.

However, it is important to acknowledge that some of the comparisons and discussions here are made across two experiments. Further verification is needed, due to the possible existence of the failure of random assignment and the influence of uncontrolled changes.

The impact of different types of auditory warnings on verbal and spatial working memory was examined. Current findings showed that identifying auditory icon warnings did not interfere with verbal and spatial working memory; however, identifying earcon warnings worsened participants’ performance on both verbal and spatial recall tasks, whereas identifying spearcon warnings only affects verbal recall tasks. These results showed that identifying different types of auditory warnings has different effects on verbal and spatial working memory.

Auditory icon warnings did not interfere with verbal and spatial working memory, either when the warnings were ignored or identified. Several usability studies indicated that auditory icons have better intuitiveness, learnability, and memorability than earcons (Garzonis et al., 2009; Isherwood and McKeown, 2017; Amer and Johnson, 2018). Our current work also indicated that auditory icons had good learnability. Participants may have completed a series of actions (switching attention, analyzing the acoustical input and then mapping the sound onto the linguistic token) with fewer working memory resources. Hence, the auditory icons did not interfere with the performance of working memory tasks because the identification of auditory icons may have placed less demand on the participants’ working memory than that of earcons and spearcons. The participants may have identified the auditory icon warnings within their residual capacity of available resources, thus preserving high accuracy at recall tasks (Wickens, 2008).

Identifying earcon warnings significantly affected the performance of both verbal and spatial working memory tasks. Earcons are synthetic sounds and are not directly related to the objects, events, or concepts they represent (Bonebright and Nees, 2007; Ludovico and Presti, 2016). The abstractions caused by the lack of semantic connection between earcons and representational events increase the difficulty of users’ understanding and memory. The participants’ efforts to remember the earcon warnings or to map the earcon sounds to the warning semantics took up additional resources, resulting in interference on working memory. Furthermore, the discovered effects of earcon identification on spatial working memory tasks supported the existence of a domain common resource in the mental process of verbal and visual space (Egeth and Kahneman, 1975; Barrouillet et al., 2004). The present study verified that verbal and visual spatial activities share a common general domain resource pool to a certain extent. The generality of these results was strengthened by other findings. For example, visual recall performance (memory for colored disks) was interfered with by simultaneous non-visual activities, such as tone-pitch recognition (Stevanovski and Jolicoeur, 2007). Increasing the cognitive load of concurrent spatial processing tasks reduced the performance of verbal recall tasks (Portrat et al., 2009; Vergauwe et al., 2010).

Identifying spearcon warnings only interfered with the verbal working memory. Given the worse accuracy for spatial recall task in no-warning or ignore-warning than for verbal recall task, the overall difficulty or the resources required to complete it for the location recall task might be higher than those for the serial recall task. However, the results showed that identifying spearcon warnings only affected the performance of the serial recall task, but not that for the location recall task. This finding indicated that the general resources occupied by spearcon identification were insufficient to seriously impair spatial task performance, but spearcon identification might have caused a significant domain-specific interference on the verbal working memory. Given that spearcons are a hybrid auditory display between speech and non-speech (Jeon, 2015), this phenomenon may be related to their speech characteristics. These results are consistent with the prediction of the working memory model theory (Baddeley and Logie, 1999). When combined with verbal activities, the performance of verbal memory task is worse than that of non-verbal memory task (Logie et al., 1990; Meiser and Klauer, 1999; Bayliss et al., 2003). One possible reason is that in the pronunciation control part of the phonological loop, the participants might “convert” the digits of the text form into speech codes through subvocalization and make them access the phonological storage device during the presentation of the digital stimulus items, thus causing interference.

The above findings are also consistent with the multi-resource theory (Wickens, 2002). The perception modality of concurrent tasks differed. The spatial location recall task used the visual modality. In the serial recall task, the participants might memorize the digits by articulatory rehearsal, and the visual and auditory modalities might be employed. Thus, the serial recall task competed with the identification of auditory warnings for the same limited pool of auditory modality resources (Wickens, 2002). Furthermore, the serial recall task required an additional stage of phonological processing to convert visual text into vocalized speech, thereby increasing verbal working memory load. Meanwhile, the identification of auditory warnings which involved mentally mapping sounds to specific linguistic tags, also required a verbal working memory load. Hence, the two tasks were competing for the same resources, thus reducing the resources available for simultaneous processing of serial recall task. The domain-specific interference of identifying auditory warnings in the task of verbal working memory may also be explained by some research findings. Cowan and Morey (2007) found that the domain-specific effect of working memory was more significant in the encoding stage than in the maintenance stage. It was possible that verbal tasks interfered with both encoding and maintenance of verbal information, and visuospatial tasks interfered only with the maintenance. Additionally, verbal information was found to be maintained by two independent mechanisms: attentional refreshing and articulatory rehearsal (Hudjetz and Oberauer, 2007; Camos et al., 2009). Vergauwe et al. (2010) suggested that this phenomenon occurred because both mechanisms were interfered with by verbal processing, whereas the spatial task interfered only with attentional refreshing. Nevertheless, further research is needed for verification.

An interesting finding here was that for the three types of warnings, no interference was observed when they were ignored by participants. This finding seems to be inconsistent with the irrelevant sound effect (Macken and Jones, 2003; Hughes et al., 2007; Macken et al., 2009). However, recent research showed that verbal recall tasks were only disrupted by irrelevant speech, but not by the presence of music or noise. The findings may be explained by a functional dissociation between working memory for phonological and non-phonological auditory items (Kattner and Meinhardt, 2020). The researchers found that for melodic and rhythmic alarms, the interference on verbal tasks was observed only when the alarm was identified (Lacherez et al., 2016). This may be related to attentional capture. Participants have to divide their attention (or switch attention) between the two tasks when identifying an auditory warning, which together with the process of identifying the warning takes up limited general resources and leads to interference. In turn, there may be no such processes when the warning was ignored, and therefore the recall task performance stayed untouched. Lacherez et al. (2016) found that when the auditory warning was ignored, melodic and rhythmic warnings did not affect the recall task, while a spoken non-word phrase warning did. These findings indicated that the effect in the ignore-warning condition may be related to the warning type. Therefore, the current results cannot be completely attributed to attention capture and may need to be interpreted from a deeper resource perspective. In fact, attention may be regarded as a general-purpose pool of limited resources (Vergauwe et al., 2010). Thus, it seems that the impact of warning on working memory may ultimately be due to resource occupancy and interference; once the occupied resources reach the threshold, the performance of concurrent tasks may be affected (Wickens, 2008). Ignoring the three types of warnings did not affect working memory, which may be due to the resources occupied by the action of ignoring warnings did not reach the interference threshold. Additionally, previous studies have found that music containing many rhythms or pitch variations is more disturbing than that with many legato passages (Klatte et al., 1995). This finding was consistent with the changing state hypothesis (Jones et al., 1992; Lecompte, 1995), which indicates that speech affects the performance of working memory tasks mainly because irrelevant sound stimuli are altered before and after stimulus entities. Banbury et al. (2001) suggested that acoustical changes are the main cause of interference, which is adjusted by the sensory organization of sound (e.g., flow). Repeated sounds, tones, or speech would not cause interference (Banbury et al., 2001). The duplex-mechanism account holds that changing-state stimuli do not capture attention; rather, the pre-attentive and obligatory processing of the order of the changing stimuli (warning sounds) conflicts with the serial rehearsal of the to-be-remembered stimuli (serial recall tasks) (e.g., Jones et al., 1996; Hughes and Jones, 2005). The current results might be related to the length of the warning sound materials. In this work, the length of sounds was approximately 1 s, which was relatively short and has small acoustic variability. The cue generated by the changing-state stimuli to order that conflicts with the processing of order in the concurrent task was less. A longer sound can have more acoustic changes. Future research should use different lengths of warning sounds to determine whether the impact of irrelevant warning sounds (i.e., ignored warnings) on working memory is related to the duration of the sounds.

The results of the present study provide important preliminary evidence that the perception and identification of learned auditory warnings (earcons and spearcons) interfere with working memory, at least in the laboratory task. However, we have to recognize that this property of capturing attention of warnings and the potential to interfere with processing represents the flip-side of the property of auditory warnings, which is often held to be their greatest asset (Ljungberg and Parmentier, 2012). One might think that people want auditory warnings to break in on other tasks. Nevertheless, our present research demonstrated that there may be additional costs for individuals who need to hold information in their memory. Listeners may not realize that listening to and identifying warnings may cause them to forget or ignore details that might be important to their current work. Therefore, people in work environments that use multiple auditory warnings should consider the mental load required in the execution of duties and how these might be affected by such distractions. The findings suggest that people might need to be reminded to pay attention not only to the effectiveness of auditory warnings but also to their potential impacts when designing auditory warnings, especially given the possible overuse of auditory warnings in high workload working environments.

The three types of auditory warnings did not interfere with working memory when ignored by the participants. This news appears encouraging because it suggests that learned warning sounds are at least negligible when the listener is informed to ignore them. Only the effort of identification causes the interference. Familiarity with warnings does not lead to involuntary or compulsory processing, or the resource occupancy generated when ignoring the warning does not reach the threshold of interfering with concurrent tasks. The operators engaged in a high-priority task may be able to prioritize their work over warning identification when they are willing to disregard the auditory warnings. Alternatively, operators can set the priority of their work to be higher than that of identifying auditory warnings, thereby reducing the potential problems of auditory warnings to some extent.

The identification of auditory icon warnings did not interfere with either verbal or spatial working memory and had the highest identification accuracy among the three types of warnings. Extensive work on the development of new auditory warnings for the medical device safety standard IEC 60601-1-8 demonstrates in many different ways (audibility, learnability, localizability, etc.) that auditory icons work well as auditory warnings in simulated clinical settings (Edworthy et al., 2017, 2018; Bennett et al., 2019). They found that anesthesia providers more correctly and quickly identified auditory icon alarms than standard earcon alarms, and participants were more likely to perceive lower fatigue and task load when using auditory icon alarms (McNeer et al., 2018). Therefore, considering the potential impact of identifying auditory warnings on working memory, auditory icon warnings may be a good choice for auditory warnings.

It is worth noting that identifying earcon warnings had the largest interference on working memory and the lowest identification accuracy among the three groups. The relationship between earcon and meaning is not based on environmental experience. Users need to learn how earcons relate to events or concepts (Amer and Johnson, 2018). Studies have found that earcons are inferior to spearcons in terms of learnability and identification accuracy (Palladino and Walker, 2007; Dingler et al., 2008; Walker et al., 2013) and have worse intuitiveness, learnability, and memorability than auditory icons (Garzonis et al., 2009; Isherwood and McKeown, 2017; Amer and Johnson, 2018). Therefore, it might be necessary to avoid the use of earcons as auditory warning signals, especially in high-load environments.

Identifying spearcon warnings interfered with verbal working memory, but not with spatial working memory. Therefore, spearcons may be an appropriate choice for warning signals in environments involving spatial working memory tasks. However, given the domain-specific interference of identifying spearcons on verbal working memory tasks, it may be necessary to avoid using spearcons as warning signals in environments involving verbal tasks. Although many other factors must be considered, the current results provide useful guidelines for the selection and design of auditory warnings.

Many processes are involved in warning identification. Before identifying the presented warning, participants need to capture the entire warning sequence in their working memory and possibly need to mentally replay this warning to instill it in their memory. Some of the issues mentioned by Lacherez et al. (2016) were in agreement with the present study. The observed interference could be caused by auditory or phonological interference, or by analyzing the acoustical input (decoding the sound) and mapping the sound to linguistic tags (warning name). In the response selection, the participants were asked to identify the warning and press a specific key. This response might have affected the recall performance of the next trial (Lacherez et al., 2016). According to the theory of working memory model (Baddeley, 2000a), interference occurs at the encoding stage, that is, during item presentation rather than at the maintenance stage. Sounds affect the information storage in the phonological storage device. Visual stimuli (memorization items) are rehearsed into the form of phonemes and stored in the phonological storage device. Auditory phonemes that are automatically entered into the phonological storage device are confused with those converted from visual stimuli, thus resulting in interference. However, the object-oriented episodic record model (Jones and Macken, 1995) emphasizes that sounds weaken the performance of series recall by destroying sequential information and series rehearsal processing. It holds that sounds can cause interference in both the presentation and maintenance stages of memorization items. Therefore, many questions remain concerning the precise locus of interference. In our ongoing work, we consider the above theoretical hypothesis, and systematically manipulate the timing of warning sounds within relevant research paradigms to further elucidate the location of interference.

Novelty sounds (often called “deviant sounds”) capture attention, and the capturing of attention is a property of auditory warnings. In the present study, participants heard a set of four learned warning sounds in each condition, and the sounds were presented in random order. A learned auditory warning, once associated with a piece of information, may be more difficult to ignore than a seemingly random pattern; however, the participants’ repeated exposure to the warning sounds in the current study may reduce their elements of surprise, making the sounds easy to ignore and reducing the deviation effect (Cowan, 1995; Titova and Näätänen, 2001), and therefore performance stays unaffected. Nevertheless, whether the warning sounds in our experiments are easy to ignore might require further verification. Future research could take the role of attention capture into account and use deviant sound as an auditory warning to ensure the surprising attribute of warning sounds, and to further verify whether various types of warning sounds are ignorable.

The present research has some other limitations. First, the comparison focused on the impact of three types of auditory warnings (auditory icons, earcons, and spearcons) on working memory. In practical application, auditory warnings may consist of linguistic sounds with semantics, which may cause some high-order interference but might be easily recognized. Spearcons are a compromise between non-speech stimuli and full speech stimuli (Wolters et al., 2012). They may increase the amount of processing required compared to full speech auditory warnings that are more easily recognized. To comprehensively clarify the effects of various auditory warnings on working memory, future work should employ an identification warnings task, triggered by semantically related full speech, to determine whether speech warnings create similar interference to spearcons in terms of their effects on verbal and spatial recall tasks.

Second, it is important to acknowledge that we discuss the difference in the interference of auditory warnings for verbal and spatial working memory using data obtained across experiments. As mentioned in the discussion of Experiment 2, further verification is required due to the possible failure of random assignment and the influence of some uncontrollable factors. The current results are insufficient to conclude that auditory warnings create more interference with verbal working memory than with spatial working memory. The impact of auditory warnings on verbal vs. spatial recall tasks should be compared within one experiment in future work.

Third, the participants recruited in this study were college students. Given that working memory ability is related to age, and the age groups of operators in working environments may be various. The results should be further verified in other age groups in future studies.