Frontiers journals are at the top of citation and impact metrics

Original Research ARTICLE

Front. Behav. Neurosci., 22 August 2018 | https://doi.org/10.3389/fnbeh.2018.00179

Moral Violations and the Experience of Disgust and Anger

  • 1Menzies Health Institute Queensland, Griffith University, Southport, QLD, Australia
  • 2Department of Psychology, Macquarie University, Sydney, NSW, Australia
  • 3Department of Cognitive Science & ARC Centre for Cognition and its Disorders, Macquarie University, Sydney, NSW, Australia
  • 4Perception in Action Research Centre, Macquarie University, Sydney, NSW, Australia

Disgust is a natural defensive emotion that has evolved to protect against potential sources of contamination and has been recently linked to moral judgements in many studies. However, that people often report feelings of disgust when thinking about feces or moral transgressions alike does not necessarily mean that the same mechanisms mediate these reactions. The present study used functional magnetic resonance imaging (n = 22) to investigate whether core and moral disgusts entrain common neural systems. We provide evidence that: (i) activation of overlapping brain regions between core and moral disgust is the result of content overlap in the vignettes—core disgust elicitors—across conditions, and not from moral violations per se, and (ii) moral residue (i.e., the remaining or “residual” activation after the influence of core disgust elicitors have been taken into account) produced a pattern of activation that is more consistent with moral anger, than one of “residual disgust.” These findings run contrary to the premise that our “moral center” is connected to the area of the brain in which physical revulsion is located.

Introduction

The emotion of disgust is typically experienced as a feeling of revulsion elicited by something offensive—e.g., bodily fluids and waste, animal products, rotten food, and certain classes of sexual behavior, and is accompanied by a strong desire to withdraw from the eliciting stimulus (Rozin et al., 1984; Oaten et al., 2009). More recently, researchers have found that participants tested across a number of experimental paradigms report feelings of disgust when considering both offensive substances (e.g., core disgust—feces, vomit, blood, and rotten meat) and immoral behavior (e.g., moral disgust—incest, murder; Haidt et al., 1997; Lieberman et al., 2003), suggestive of a link between disgust and morality (Schaich Borg et al., 2008). It has been further suggested that disgust may be linked to “purity transgressions”—essentially moral violations that contain a core disgust referent—and not necessarily other types of moral violations (e.g., moral violations that do not contain a core disgust referent; Horberg et al., 2009). This is an interesting point that we return to shortly. The neural signatures produced by core disgust stimuli vs. moral disgust stimuli (i.e., involving moral infractions) are purported to entrain substantial overlap (Moll et al., 2005; Schaich Borg et al., 2008). Despite emerging consensus for the view that our “moral center” is connected to the area of the brain in which physical revulsion is mediated (Haidt, 2001; Schaich Borg et al., 2008; Schnall et al., 2008), we argue that the support for this premise is problematic. In particular, it remains unclear whether the overlap of neural activation between core and moral disgust is limited to those moral transgressions, as noted above, simply by virtue of their reference to core disgust elicitors (e.g., feces, vomit, blood, and rotten meat), or whether the presence of a moral transgression generates additional disgust, and hence greater activation, to that produced by a matched core disgust condition free of moral connotation—thereby establishing a “true” moral disgust.

Disgust is considered to be distinguishable from other emotive responses—e.g., fear (Phillips et al., 1997; Wicker et al., 2003), and anger (Sprengelmeyer et al., 1998)—and has been associated with activity in the anterior insula, basal ganglia, medial prefrontal cortex, thalamus, and visual cortex (Sprengelmeyer et al., 1998; Wicker et al., 2003; Moll et al., 2005; Williams et al., 2005; Schaich Borg et al., 2008; Schnall et al., 2008), in both clinical populations with impaired disgust processing [e.g., neurological (Hayes et al., 2007) and psychiatric (Vicaro et al., 2017) conditions] and non-clinical populations (Moll et al., 2005; Schaich Borg et al., 2008). However, investigation of the neural correlates of disgust in the context of moral behavior is limited. A widely cited fMRI study suggests that reading about certain socio-moral violations can induce patterns of neural activity that overlap with those observed when reading about more concrete disgust elicitors (Moll et al., 2005). This article is influential because of its claim that socio-moral violations cause a disgust response. However, the claim is significantly weakened by two shortcomings in the study design. First, the socio-moral elicitors contain references to core disgust elicitors (e.g., death, spiders, cockroaches, etc). This undermines the claim that it is the socio-moral violation per se that causes disgust. An equally plausible alternative is that disgust is caused by the reference to the core disgust referents contained within the stimuli (Case et al., 2012). This limitation is not restricted to the work of Moll et al. (2005). Schaich Borg et al.'s (2008) fMRI study presented men with vignettes describing various acts involving their sisters. Although the authors reported overlap in the brain regions activated for core (or in that work, “pathogen”) and socio-moral disgusts, the items used to generate these conditions incorporated vivid core disgust referents—i.e., “sipping your sister's urine” (pathogen disgust), “watching your sister masturbate” (sexual moral disgust), and “killing your sister's child (non-sexual moral disgust). That is, in addition to the moral violation, all of the vignettes would also bring to mind core disgust elicitors. For example, “killing your sister's child” would very likely bring to mind images of blood and gore, which are quintessential core disgust elicitors. Thus again, brain activation overlap might be due to the overlap in the content of stimulus items—core elicitors—across conditions, and not to moral disgust per se.

A second shortcoming in experimental design was that neither study contained a matched disgust control condition (e.g., that mentions death, spiders, cockroaches, urine, masturbation, etc); so we do not know what neural structures are routinely activated under conditions where one is simply asked to read vignettes containing disgust-related words drawn from the socio-moral stimuli but without any moral connotations. Relatedly, the absence of a moral anger control condition is noteworthy. Nabi (2002) has argued that the self-reported “disgust” which arises in response to moral offenses is only a metaphorical use of disgust to describe angering situations. Indeed, anger is a plausible response to norm violations or to deception and abuse (Gutierrez and Giner-Sorolla, 2007; Olatunji et al., 2016), and is considered to be responsive to contextual cues of harm and intentionality (Gutierrez and Giner-Sorolla, 2007). Moll et al.'s (2005) socio-moral statements were associated with both self-reported disgust and anger, yet the authors did not investigate whether socio-moral violations and anger activate common brain networks, as they did for disgust. It is therefore possible that unmeasured affective activations, separate from the induced disgust, are doing some of the work in driving participants' reactions to moral transgressions (Russell and Giner-Sorolla, 2011). The co-occurrence of anger and disgust in moral situations (Gutierrez and Giner-Sorolla, 2007; Olatunji et al., 2016) suggests that any investigation of the neural correlates of disgust and morality should be extended to also include anger.

We suggest that a strong demonstration of moral disgust—one removed of any core disgust reference—would then need six conditions: moral disgust vignettes, moral anger vignettes (i.e., to identify activation under conditions of anger-provoking stimuli), matched disgust vignettes (i.e., to control for the influence of the core disgust referent contained in the moral disgust vignettes), high disgust vignettes (i.e., to ensure a condition that exceeds the matched disgust vignettes in disgust intensity and to check for activations akin to those observed before for this emotion; Phillips et al., 1997; Sprengelmeyer et al., 1998; Wicker et al., 2003; Moll et al., 2005; Williams et al., 2005; Schaich Borg et al., 2008), neutral control vignettes (i.e., word reading controls) and scrambled vignettes (i.e., task attention controls). To convincingly establish that moral violations generate (moral) disgust, the following would need to be demonstrated: (i) Moral disgust vignettes produce activation at locations that overlap with those activations produced by the high disgust vignettes, after contrasting out the influence of the matched core referent word (i.e., moral disgust—matched disgust); and (ii) Moral disgust vignettes exceed activation at key sites (i.e., those implicated a priori as being indicative of disgust emotional processing) relative to the matched disgust vignettes. This pattern would show that moral disgust vignettes could induce disgust-like patterns of neural activity independent of core disgust references within the vignette. In addition, it would also show that moral disgust vignettes could induce greater activation relative to the effects of their core referent word.

Pilot Study

The purpose of the pilot work was to generate and evaluate a series of vignettes for the main study to ensure that they differed as expected on a range of characteristics including ratings of emotion, morality, intentionality, and wrongness. We also included ratings of purity, harm, and justice due to the suggestion that moral disgust is reserved for transgressions that are related to “purity violations” (e.g., misuse of the body, etc.), while anger is related to violations of individual rights and community norms (Haidt and Graham, 2007; Landy and Goodwin, 2015). The vignette sets were moral disgust vignettes, moral anger vignettes, matched disgust vignettes that contained the same core disgust referents as the moral disgust vignettes, high disgust vignettes that were meant to induce greater disgust than the matched disgust vignettes, neutral control vignettes and scrambled vignettes (see Table 1).

TABLE 1
www.frontiersin.org

Table 1. The vignettes evaluated in piloting and used in the main study.

Obtaining such self-report data from participants in the main study would be problematic because we wanted scanning participants simply to read the vignettes and react to them without biasing them toward particular aspects of each vignette, such as whether they were disgusting or immoral. We also did not get ratings from these participants before or after the main study because ratings under these conditions would either influence or be influenced by presentation during scanning. If administered before the task, it would have habituated the vignettes' elicitation of disgust, which is known to weaken with exposure (i.e., Rozin, 2008). If administered after the task, any ratings might have been affected by habituation from experiencing them in the scanner. Thus, we utilized an independent group of participants to establish the vignette characteristics.

We predicted the following basic differences between these groups of vignettes: (1) the moral disgust vignettes should be more disgusting and reflect greater purity violations than the moral anger vignettes, but the moral anger vignettes should reflect greater justice violations and be at least equivalent or more in terms of their ability to induce anger; (2) that the matched disgust vignettes should not be judged to reflect moral violations and that they should principally induce disgust, but that they should do so to a lesser extent than the high disgust vignettes; and (3) that the neutral and scrambled vignettes should not arouse much emotional feeling.

Materials and Methods

Participants

Twenty undergraduate participants (M age = 22.2, SD = 2.8), nine male, participated for course credit or a small cash payment. This research was approved by the Macquarie University Human Research Ethics Committee. Specifically, all participants gave their written informed consent and all procedures of the study (including the written informed consent, and the autonomy of each participant to stop at any point during the study) were kept in line with the MQ HREC regulations.

Stimuli

The vignettes are presented in Table 1.

Procedure

Participants were presented with each of the vignettes in Table 1 in a different random order. For each vignette, participants were asked to rate how angry, happy, fearful, sad, and disgusted it made them feel, in each case using seven-point category scales (1 [Not at all] to 7 [Very]). In addition, for all vignettes except the neutral and scrambled ones, participants also rated: (1) How immoral is this scenario? (1 [Not at all immoral] to 7 [Very immoral]); (2) How likely is it that a person is directly responsible for the events in this scenario? (1[Not at all likely] to 7 [Very likely]); (3) How wrong is the behavior in this scenario? (1[Not at all wrong] to 7 [Very wrong]); (4) How well does this behavior fit the following definition “Purity—Behaviors in this category are felt to be unclean, unnatural, and contaminated or polluted” (1 [Not at all] to 7 [Very well]); (5) How well does this behavior fit the following definition “Harm—Behaviors in this category are felt to cause suffering or pain to others” (1 [Not at all] to 7 [Very well]); and (6) How well does this behavior fit the following definition “Justice—Behaviors in this category are felt to create inequality, be unfair, or otherwise restrict others' rights” (1 [Not at all] to 7 [Very well]). These ratings were always presented in the same order. Once all of the vignettes had been rated (participants were allowed to stop and rest as needed) the task was complete.

Analysis

Participants' scores for each vignette were averaged and the unit of analysis was the vignette. Only the moral disgust, moral anger, matched disgust, and high disgust vignette groups were compared, as the control scores for neutral and scrambled vignettes had almost no variance with means (excepting the happiness rating) all around the lowest point on the rating scale. Each rating type was analyzed using a univariate ANOVA, with vignette group treated as a between participant factor. Post-hoc testing was conducted with Ryan-Einot-Gabriel-Welsch (REGW) Range tests. This test is recommended to describe homogenous subsets of means (Howell, 1997).

Results

All of the ratings, by vignette group, are presented in Table 2, alongside the post-hoc contrasts conducted for each of the univariate ANOVAs described below.

Disgust. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 19.40, MSE = 0.48, p < 0.001, partial-eta squared = 0.43. A post-hoc REGW Range test revealed three homogenous subsets, the matched disgust and moral anger vignettes as one, the high disgust as the next, and the moral disgust vignette group as the highest scorer on this variable.

Anger. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 33.90, MSE = 0.98, p < 0.001, partial-eta squared = 0.57. A post-hoc REGW Range test revealed two homogenous subsets, the matched disgust and high disgust vignettes as one, and the moral disgust and moral anger vignette groups as the other and highest scoring subset on this variable.

Fear. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 10.85, MSE = 0.43, p < 0.001, partial-eta squared = 0.30. A post-hoc REGW Range test revealed two homogenous subsets, the matched disgust vignettes as one, and the remaining three vignette groups as the other and highest scorer on this variable.

Sad. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 22.66, MSE = 0.41, p < 0.001, partial-eta squared = 0.47. A post-hoc REGW Range test revealed three homogenous subsets, the matched disgust group of vignettes as one, the high disgust vignette group as the next, and the moral anger and moral disgust vignettes as the third and highest scoring subset on this variable.

Happy. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 5.14, MSE = 0.14, p < 0.01, partial-eta squared = 0.17. There were no homogenous subsets.

Immoral. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 39.39, MSE = 0.89, p < 0.001, partial-eta squared = 0.61. A post-hoc REGW Range test revealed two homogenous subsets, the matched disgust and high disgust vignettes as one, and the moral disgust and moral anger vignette groups as the other and highest scorer on this variable.

Responsibility. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 20.96, MSE = 0.85, p < 0.001, partial-eta squared = 0.45. A post-hoc REGW Range test revealed two homogenous subsets, the matched disgust and high disgust vignettes as one, and the moral disgust and moral anger vignette groups as the other and highest scoring subset on this variable.

Wrong. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 30.86, MSE = 1.03, p < 0.001, partial-eta squared = 0.55. A post-hoc REGW Range test revealed two homogenous subsets, the matched disgust and high disgust vignettes as one, and the moral disgust and moral anger vignette groups as the other and highest scorer on this variable.

Purity. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 23.16, MSE = 0.72, p < 0.001, partial-eta squared = 0.48. A post-hoc REGW Range test revealed two homogenous subsets, the moral disgust vignette group as the first and highest scoring subset on this variable, with the remaining three vignette groups as the other subset.

Harm. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 26.27, MSE = 0.91, p < 0.001, partial-eta squared = 0.51. A post-hoc REGW Range test revealed two homogenous subsets, the matched disgust and high disgust vignettes as one, and the moral disgust and anger vignette groups as the other and highest scorer on this variable.

Justice. Univariate ANOVA revealed a significant effect of Vignette group, F(3, 76) = 45.44, MSE = 0.74, p < 0.001, partial-eta squared = 0.64. A post-hoc REGW Range test revealed three homogenous subsets, the moral anger vignette group as the first and highest scoring subset of this variable, the moral disgust vignette group as the second, and the matched and high disgust vignette groups as the third.

TABLE 2
www.frontiersin.org

Table 2. Characteristics (Mean and SD) of the vignette groups (Moral, Matched and High Disgust, Moral Anger, Neutral and Scrambled, respectively) evaluated in the pilot study and used in the main study.

Discussion

The results from the pilot study confirmed that the vignettes were able to induce the requisite emotions and reactions in participants that we intended to investigate in the main study. The moral disgust vignettes were more disgusting and reflected greater purity violations than the moral anger vignettes, which in turn scored higher on justice violations but equivalent in anger to the moral disgust vignettes. The matched disgust vignettes were significantly less disgusting than the high disgust vignettes, and both these sets of vignettes were judged to be less immoral and wrong compared to the moral anger and disgust vignettes. Perhaps most importantly, for our purposes, the matched disgust vignettes, which contained identical core elicitors to the moral disgust vignettes, were judged to be less disgusting, suggesting that participants regarded the addition of a degree of intentionality (i.e., responsibility, on which the moral disgust vignettes scored higher than the matched disgust vignettes) as making these scenarios more disgusting, more anger provoking, and as involving greater purity and harm violations.

That each of the vignette types generated a range of emotions is not itself a major issue. The dominant emotion in each case was consistent with our categorization of the vignettes. Indeed, most publications on this topic find that other emotions are activated beyond the target specific ones (e.g., Gutierrez and Giner-Sorolla, 2007; Russell and Giner-Sorolla, 2011; Case et al., 2012; Olatunji et al., 2016). Thus on the basis of self-report, participants felt that an immoral action involving a core disgust elicitor was more disgusting than the core elicitor without an immoral action. This suggests that, at least for self-report accounts, the presence of the moral violation generates additional disgust—in line with the term “moral disgust.” This finding is consistent with the view that our “moral center” is linked to the area of our brain in which physical revulsion—core disgust—is situated (Moll et al., 2005; Schaich Borg et al., 2008; Schnall et al., 2008). This idea was examined more directly in the main study where fMRI was used to investigate differences between moral and core disgust, rather than relying on self-report accounts.

Main Study

We used functional magnetic resonance imaging (fMRI) to measure the blood-oxygen level-dependent (BOLD) response in participants reading the various vignettes tested in the pilot. The purposes of the main study were: (1) to determine if the brain areas activated by the matched disgust vignettes were largely the same or different from those produced by the moral disgust vignettes, as both used the same set of core disgust elicitors, but the latter involved moral violations and the former did not; (2) to see if brain activation induced by moral disgust vignettes more closely resembles that of the matched or the high disgust vignettes, on the basis that self-report disgust scores were closer in the pilot study for moral and high disgust, than for moral and matched disgust; (3) to test if activation differences between the high and moral disgust conditions were similar to activation differences between the matched and high disgust condition. In both cases, the first mentioned set of vignettes groupings (i.e., high vs. moral and matched vs. high) were less disgusting than the second set, and we can use the brain activation to assess whether common neural networks are implicated; and (4) to examine if activation induced by the moral disgust vignettes more closely resembled that of the anger vignettes than that of the matched and high disgust vignettes, on the basis that self-report anger scores were closer in the pilot study for moral disgust and moral anger.

Materials and Methods

Participants

Twenty-two healthy undergraduate participants (M age = 26.3, SD = 5.6), 10 male, successfully completed the study (two were lost due to movement artifacts). They were awarded course credit or received a small cash payment. All participants provided written informed consent and were recruited on the basis of the following (self-reported) inclusion criteria: (1) healthy; (2) neurologically normal (validated by neuroradiological review); (3) non-healthcare employed; (4) between the ages of 18 and 30 years; and (5) are able to undergo fMRI scanning (i.e., no metal implants, pacemaker, claustrophobia, etc). No pilot participants were involved in the main study. All procedures of the study (including the written informed consent, and the autonomy of each participant to stop at any point during the study) were in line with the MQ HREC regulations and the study protocol was approved by this committee.

Stimuli

Participants were presented with the vignettes (validated in the pilot work and developed uniquely for this study) on a monitor viewed via a mirror mounted on the headcoil to read while being scanned. Participants were instructed to indicate (button-press) whenever a scrambled vignette appeared. The task was simply to ensure that participants were attending to the statements. The vignettes are presented in Table 1.

fMRI Procedure

We used an event-related design with six runs. Each run involved presenting 55 vignettes. Of these 55, there were 10 vignettes drawn from each of the five conditions (moral disgust, matched disgust, high disgust, moral anger, neutral). A further five scrambled vignettes were randomly interpolated amongst these trials. Vignettes could only be selected once on each run from the moral disgust, matched disgust, high disgust, and moral anger conditions (i.e., random sampling without replacement), while random sampling with replacement was used for the neutral and scrambled vignettes. Each vignette was presented for a total of 6 s, with no inter-stimulus interval. Participants were asked to press a response button whenever a scrambled vignette appeared. The presentation of stimuli was controlled using Presentation software (Neurobehavioral Systems, Albany, CA, USA; http://www.neurobs.com/). The vignettes were presented using a 15-inch Macintosh Power Book and projected onto a screen positioned ~1.2 m behind the participant's head. The participants viewed the screen via a mirror mounted ~15 cm above the eyes.

The MRI data were acquired using a Siemens Verio 3T scanner using a 32-channel head coil (Erlangen, Germany) at Macquarie Medical Imaging, Macquarie University Hospital, Sydney, Australia. Gradient echo T2*-weighted echo planar imaging (EPI) was implemented for functional imaging. One scan volume was obtained every 4 s (TR; hence stimulus presentation was jittered with respect to the scanner) and consisted of 49 slices (TE = 40 ms, FOV = 240 x 240 mm, in-plane resolution 2.0 x 2.0 mm, slice thickness 2.4 mm, interslice gap 0.5 mm, flip angle = 90°). The first four volumes in each run were automatically discarded. Each run comprised 84 recorded volumes.

For each participant a high-resolution structural image was acquired (3D-MPRAGE sequence, voxel size 0.94 isotropic, FOV: 240 x 240 mm, 176 slices, TR = 2110 ms, TE = 3.54 ms, flip angle = 9°).

fMRI Analysis

The data were analyzed with SPM8 (Wellcome Department of Cognitive Neurology, London, U.K., https://www.fil.ion.ucl.ac.uk/spm/) using standard pre-processing steps. First, all obtained volumes per participant were realigned and resliced to correct for small head-movements and slice timing correction was performed. The mean of the functional volumes was co-registered, segmented and normalized to an MNI (Montreal Neurological Institute) template within SPM. The obtained transformation parameters were then applied to the co-registered functional volumes, which were re-sampled to a 2 x 2 x 2 mm voxel size. The spatially transformed functional data were spatially smoothed using a 6 mm FWHM isotropic Gaussian kernel. A general linear model (GLM) was fitted to the data with five regressors. Each condition was modeled with a boxcar function and convolved with SPM's canonical hemodynamic response function. Furthermore, to remove low-frequency drifts, the default temporal high-pass data filter cut-off (128 s) was employed.

A general linear model (GLM) was estimated for each participant with one regressor for each of the six conditions: five conditions of interest (moral disgust, matched disgust, high disgust, moral anger, neutral) and the scrambled vignettes. The events were modeled using an event-related function lasting from the onset of each event to the onset of the following event. After fitting the GLMs, the scrambled condition was subtracted from each of the five conditions of interest (i.e., condition minus scrambled) and the resulting beta estimates were then entered into the second-level analysis.

We conducted whole brain analyses using a repeated measures design and performed with a threshold of p < 0.001 and a cluster threshold of 10 contiguous voxels (Leiberman and Cunningham, 2009), which effectively counteracts the increased Type I error rate from multiple comparisons by only taking as significant clusters of 10 or more voxels that have activation above the threshold. Anatomical labeling was done using WFU_Pickatlas on any clusters that reached significance.

Results

Disgust vs. Neutral Control Vignettes

Table 3 presents the clusters and regional maxima for the three contrasts examining high, matched, and moral disgust conditions against the neutral control vignettes (each of these first having the activation in the scrambled condition subtracted; i.e., [High Disgust-Scrambled]>[Neutral-Scrambled]; [Matched Disgust-Scrambled]>[Neutral-Scrambled]; [Moral Disgust-Scrambled]>[Neutral-Scrambled]). All three contrasts revealed activity in inferior, middle and superior frontal gyri, around similar coordinates. Similarly, activation occurred in the right mid-temporal and left fusiform gyri, in all three contrasts. Fewer activations were observed for the moral and matched disgust contrasts in the occipital gyrus and for high disgust contrast in the superior and middle temporal gyri. Unique regional activations were largely absent on the left side.

TABLE 3
www.frontiersin.org

Table 3. Exploratory analyses of disgust and control related contrasts.

In the right hemisphere, there was both less activity than on the left and more limited overlap amongst the three contrasts. Activation in the frontal inferior gyri was observed for the matched and moral disgust vignettes, lentiform nucleus activation for the high and moral disgust vignettes. There was more unique regional activity on the right side, but this was not clearly confined to one particular type of vignette. In sum, the three disgust-related vignettes, moral, matched, and high, produced similar and overlapping patterns of activation, characterized by left frontal regions and the fusiform gyrus.

Comparing the Disgust-Related Vignettes

Table 4 presents the activations following high-matched, moral-high, and moral-matched disgust contrasts. Overall, there were far fewer similarities between vignette types than observed in the disgust-neutral contrasts presented in Table 3. Figure 1 illustrates that for the moral-matched disgust contrast there were few activations, with these mainly on the right, and restricted to the uncus, culmen, and one sub-gyral location. On the left, the only activation was for the middle temporal gyrus, similar to an activation observed for moral anger (see Table 6). Thus, after controlling for matched disgust, the moral component alone was associated with little unique neural activity.

TABLE 4
www.frontiersin.org

Table 4. Exploratory analyses of disgust-related contrasts.

FIGURE 1
www.frontiersin.org

Figure 1. Contrasts from fMRI data. Moral disgust vs. matched disgust contrast (A), high disgust vs. matched disgust contrast (B), moral disgust vs. high disgust contrast (C), and overlay of the three disgust contrasts (D). Different to Legend color (i.e., yellow) in lower right panel shows the overlap of the three disgust contrasts. Legend shows T-value of the contrasts (e.g., higher T-value represents stronger activations).

Participants in the pilot study judged both the moral and high disgust vignettes as more disgusting than the matched disgust vignettes. In the neural data as depicted in Figure 1, subtracting matched from high disgust ([High-Scrambled]>[Matched-Scrambled]) revealed significant clusters in several regions including the middle occipital gyrus, cuneus, culmen, and insula, as well as sub-gyrally. Apart from the culmen, these regions were not active in the analogous contrast for moral disgust ([Moral-Scrambled]>[Matched-Scrambled])—see Table 4. Thus, although both vignette types were perceived as more disgusting than the matched vignettes by participants in the pilot, the pattern of brain activation notably differed, including significant insula activation in the high-matched contrast. This suggests a different neural basis for disgust reported for core elicitors relative to disgust reported for moral disgust elicitors.

We also contrasted the moral and high disgust vignettes directly [Moral-Scrambled]>[High-Scrambled]). Figure 1 shows significant clusters in the left inferior frontal gyrus, inferior temporal gyrus, and middle and superior temporal gyri. Interestingly, anger-related activations (see Table 6) were predominantly characterized by extensive left-middle temporal activations. Recall that the pilot data indicated that the moral disgust vignettes were more disgusting than the high disgust vignettes, and that the high disgust vignettes were more disgusting than the matched disgust vignettes—both to the same extent (see Table 2). So while these two conditions differed in disgust to the same extent in the pilot, Figure 1 shows there was no overlap in neural activations when they are contrasted in the fMRI data. In other words, the brain areas associated with differences in one were not the same as those associated with the other. This suggests again that comparable differences in reported disgust between comparisons involving core elicitors (i.e., high disgust > matched disgust) are supported by different neural structures to those involving moral disgust (i.e., moral disgust > high disgust).

Note that all the brain images follow the neurological convention (i.e., the left side of the image corresponds to the patient's left side of the brain and vice versa).

Anger and Disgust

Table 5 presents the high disgust, matched disgust and moral disgust contrasts minus anger. Two things are apparent when viewing Table 5. First, there are numerous activations for all three contrasts, indicating fairly major patterns of difference in the brain activations associated with disgust and those associated with anger. Second, right-sided differences are far more apparent here than in the various disgust-related contrasts examined so far. Predominantly, activity in two regions disambiguated moral anger and disgust-related responses. First, for both the left and right sides, there was extensive activation of inferior, and middle frontal gyri. Second, on both sides, frontal activations covered considerably larger regions for both the high disgust minus anger contrast (see Figure 2, left panel) and the matched disgust minus anger contrast (see Figure 2, middle panel), than for moral disgust anger contrast (see Figure 2, right panel). In sum, relative to moral anger, the matched, high and moral disgust vignettes produced overlapping activations (i.e., broad qualitative similarity), but the regional size of these activated regions differed between moral disgust and the two other disgust-related conditions (i.e., a broad quantitative difference).

TABLE 5
www.frontiersin.org

Table 5. Exploratory analyses of disgust-related contrasts.

FIGURE 2
www.frontiersin.org

Figure 2. Contrasts from fMRI data. High disgust vs. anger contrast (A), matched disgust vs. anger contrast (B), and moral disgust vs. anger contrast (C).

Finally, we also conducted a contrast of anger vs. the neutral vignettes, which is presented in Table 6. Here, the most predominant activation by region size was in the left middle temporal gyrus, followed by left sided activations in the inferior, medial frontal gyri. Several other activations were present, including lingual and fusiform gyri, cuneus and parahippocampal gyri.

TABLE 6
www.frontiersin.org

Table 6. Exploratory analysis of the anger-neutral contrast.

Discussion

The basic finding from the main study is that brain activations generated by moral disgust inducing vignettes closely resemble activations generated by disgust inducing vignettes. Although at first glance this might seem to confirm the existence of a “moral” disgust, our carefully matched disgust condition, which used the same set of core elicitors but without any moral violations, shows this is not the case. Contrasts between moral, matched and high disgust vignettes did not suggest that moral disgust vignettes generate “additional disgust” as reported by participants in the pilot study (i.e., moral > high > matched disgust—see Table 2). First, where the magnitude of disgust is manipulated by choosing more potent core elicitors (high disgust > matched), the resulting pattern of activation is different to that generated by the moral and matched disgust contrast (moral > matched), which also differed in magnitude of reported disgust but used the same core disgust elicitors. Thus, whatever drives higher disgust ratings for high disgust > matched is unlikely to be the same factor that drives higher disgust ratings for moral vs. matched disgust. Second, for the high disgust > matched disgust and for the moral > high disgust contrasts there were no common activations—and again where differences in the magnitude of reported disgust were evidenced in the pilot—there was no overlap in activations, suggesting once more that whatever neural structure/s drive higher disgust ratings for moral disgust elicitors these do not overlap with those driving higher disgust ratings for core disgust elicitors. Third, the comparison of the moral and high disgust conditions and the moral and matched disgust conditions both revealed activation differences that seemed to reflect a moral-anger related component to moral disgust, with common activation in the middle temporal lobe. Fourth, that after controlling for matched disgust, moral disgust is associated

with little unique neural activation. Fifth, comparisons of the disgust-related conditions with moral anger suggest that while there are qualitative similarities in activation patterns shared by all of the disgust conditions, there are quantitative differences, with the relationship to moral anger being different for moral disgust relative to matched and high disgust.

General Discussion

This study is the first investigation of whether core and moral disgusts entrain common neural systems using appropriate controls for immoral events with references to core disgust. The present results suggest that: (i) activation of overlapping brain regions between core and moral is the result of content overlap in the vignettes—core disgust elicitors—across conditions, and not from moral violations per se, and (ii) moral “residue”—the effect remaining once core disgust is taken into account—produces activation that is more consistent with moral anger than one of “additional disgust.”

These findings call attention to a number of issues. First, there is a dissociation between the self-report data and the fMRI data. On the basis of the self-report data alone, we would be led to believe that an immoral action involving a core disgust elicitor was more disgusting than a core elicitor without an immoral action. The implication of this is that the presence of the moral violation ramped up experienced disgust—establishing moral disgust. However, the objective fMRI data suggests otherwise: there is no evidence that moral disgust evokes greater activation in “disgust areas” activated by core elicitors. Consequently, we believe this delivers compelling evidence against the idea that moral violations can themselves be disgusting in the same way that core elicitors can be. Indeed, dissecting the nature of this presumed relationship is of increasing interest (Curtis, 2011). For example, one important question is whether disgust started by expanding from its origins in distaste, to serving as a pathogen avoidance mechanism, before finally entering into the social sphere (Tybur et al., 2009). A further question is whether the disgust-morality relationship is purely metaphorical (Nabi, 2002)? Or, is disgust evoked solely because moral violations include core disgust stimuli such as, for example, sexual crimes and bloody murders (Royzman and Sabini, 2001; Oaten et al., 2009). The evidence from the current study favors the latter account. To our knowledge, this is the only study to control for the influence of core disgust referents contained in moral disgust vignettes, and our findings indicated that the moral disgust vignettes did not produce a disgust-like pattern of activation after the influence of core disgust referents were accounted for (i.e., moral disgust—matched disgust). Moreover, the residual activation from the moral disgust vignettes was comparable to moral anger, rather than disgust.

This finding is especially interesting. A number of researchers have argued that verbal reports of disgust in response to moral transgressions are problematic due to the usage of the term “disgusting” in colloquial language to describe events that are angering or irritating (Chapman et al., 2009), which is why we did not ask participants to make disgust ratings in our study. Indeed, Nabi (2002) reported that the lay understanding of the word “disgust” corresponds more closely to the theoretical meaning of anger, than to that of disgust. Specifically, events recalled in response to the words “disgust” and “disgusted” tended to reflect primarily anger-related themes (e.g., demeaning offenses) rather than classic disgust-related themes (e.g., noxious elements). This claim is also supported by past studies that report the words “disgust” and “disgusted” evoke feelings associated with anger-related concepts (e.g., Roseman et al., 1994; Rozin et al., 1999). Therefore, it might be that the dissociation between the self-reported and fMRI data in the present study is an artifact of the data collection method. That is, participants may be relying on lay meanings of disgust—e.g., reflecting greater anger than repulsion—when appraising moral violations in the vignettes and are consequently misattributing the label of disgust to their experienced affect (Royzman et al., 2014), a suggestion that is consistent with our fMRI data.

Second, the data reported here challenge the view that the insula serves as the seat of disgust processing. A large area of the left frontal regions and the fusiform gyrus was identified in the analysis, but it did not extend into the insular cortex as in previous studies of disgust (Phillips et al., 1997, 1998, 2004; Calder, 2003; Krolak-Salmon et al., 2003; Wright et al., 2004)—although importantly, insula activation was present when high disgust was contrasted with matched disgust (see Table 5), suggesting that this structure may be one of those involved (along with the others identified in the contrast in Table 4) in mediating increases in emotional intensity for disgust. Nonetheless, the results from the disgust-neutral contrasts, which reveal activations unique to this emotional state (rather than say the difference from another state such as anger), are consistent with a growing number of studies showing that the insula is often, but not always involved in disgust processing (Phillips et al., 1998; Stark et al., 2003, 2005). Moreover, our findings are also consistent with a meta-analysis of imaging studies that found the anterior insula to be no more active during disgust than other emotions (Wager et al., 2007). We note in passing that the method of inducing disgust could be one variable that influences the activation profile, including the presence of insula activity, and could explain some of the variability between studies. This work involves the first fMRI investigation of whether core and moral disgusts entrain common neural systems, using immoral vignettes with appropriate controls for referents to core disgust, and our findings suggest that separate neural systems mediate these reactions.

Limitations

It is important to acknowledge that the processing of disgusting stimuli is likely influenced by individual differences (i.e., disgust sensitivity; Haidt et al., 1994), and the fact that these data were not collected is a limitation of the current study. Consequently, it is not possible to exclude potential participant differences in trait disgust across the two studies (i.e., pilot study vs. main study).

Conclusion

Our results support two clear inferences. First, that presenting moral violations that include core disgust elicitors will activate very similar brain regions to those activated by presenting the same core elicitors alone without moral violations. Thus in the absence of a matched disgust condition as used here, one could falsely conclude that a unique moral disgust existed, when in fact most of this moral disgust actually results from the presence of core disgust elicitors referred to in the vignettes, not from the moral violations per se. Second, examination of the moral residue (i.e., contrast of moral disgust with matched disgust or high disgust) provides no clear evidence that it induces a disgust-like pattern of activation in the brain. Rather, this residue seems somewhat more akin to the pattern produced by moral anger. Although the participants in the pilot study rated the moral disgust vignettes as more disgusting than both the matched and high disgust vignettes, this additional disgust does not appear to rely on the same neural correlates that generate stimulus-driven differences in disgust (e.g., seeing dog feces vs. stepping in it barefoot).

Author Contributions

MO, RS, and TC conceived of the study. MO, RS, MW, and AR designed the study. MO and RS coordinated the study. MO, RS, MW, and AR administered study. MW, MB, and RS participated in data and statistical analysis, all authors helped draft the manuscript, and gave final approval for publication.

Funding

Internal grant support for corresponding author. 9201001803 Macquarie University. www.mq.edu.au. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank Macquarie University for their financial support.

References

Calder, A. J. (2003). Disgust discussed. Ann. Neurol. 53, 427–428. doi: 10.1002/ana.10565

PubMed Abstract | CrossRef Full Text | Google Scholar

Case, T. I., Oaten, M., and Stevenson, R. J. (2012). “Disgust and moral judgment,” in Emotions, Imagination, and Moral Reasoning, eds R. Langdon and C. Mackenzie (New York, NY: Psychology Press), 195–218.

Google Scholar

Chapman, H. A., Kim, D. A., Susskind, J. M., and Anderson, A. K. (2009). In bad taste: evidence for the oral origins of moral disgust. Science 323, 1222–1226 doi: 10.1126/science.1165565

PubMed Abstract | CrossRef Full Text | Google Scholar

Curtis, V. (2011). Why disgust matters. Philos. Trans. R. Soc. Lond. B Biol. Sci. 366, 3478–3490. doi: 10.1098/rstb.2011.0165

PubMed Abstract | CrossRef Full Text | Google Scholar

Gutierrez, R., and Giner-Sorolla, R. (2007). Anger, disgust, and presumption of harm as reactions to taboo-breaking behaviors. Emotion 7, 853–868. doi: 10.1037/1528-3542.7.4.853

PubMed Abstract | CrossRef Full Text | Google Scholar

Haidt, J. (2001). The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychol. Rev. 108, 814–834. doi: 10.1037/0033-295X.108.4.814

PubMed Abstract | CrossRef Full Text | Google Scholar

Haidt, J., and Graham, J. (2007). When morality opposes justice: conservatives have moral intuitions that liberals may not recognize. Soc. Justice Res. 20, 98–116 doi: 10.1007/s11211-007-0034-z

CrossRef Full Text | Google Scholar

Haidt, J., McCauley, C., and Rozin, P. (1994). Individual differences in sensitivity to disgust: A scale sampling seven domains of disgust elicitors. Pers. Ind. Diff. 5, 701–713. doi: 10.1016/0191-8869(94)90212-7

CrossRef Full Text | Google Scholar

Haidt, J., Rozin, P., McCauley, C., and Imada, S. (1997). Body, psyche, and culture: the relationship between disgust and morality. Psychol. Dev. Soc. J. 9, 107–131. doi: 10.1177/097133369700900105

CrossRef Full Text | Google Scholar

Hayes, C., Stevenson, R., and Coltheart, M. (2007). Disgust and huntington's disease. Neuropsychologia 45, 1135–1151. doi: 10.1016/j.neuropsychologia.2006.10.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Horberg, E. J., Oveis, C., Keltner, D., and Cohen, A. B. (2009). Disgust and the moralization of purity. J. Pers. Soc. Psychol. 97, 963–976 doi: 10.1037/a0017423

PubMed Abstract | CrossRef Full Text | Google Scholar

Howell, D. C. (1997). Statistical Methods for Psychology, 4th Edn. Pacific Grove, CA; Duxbury.

Krolak-Salmon, P., Henaff, M.-A., Isnard, J., Tallon-Baudry, C., Guenot, M., Vighetto, A., et al. (2003). An attention modulated response to disgust in human ventral anterior insula. Ann. Neurol. 53, 446–453. doi: 10.1002/ana.10502

PubMed Abstract | CrossRef Full Text | Google Scholar

Landy, J. F., and Goodwin, P. G. (2015). Does incidental disgust amplify moral judgment? a meta-analytic review of experimental evidence. Perspect. Psychol. Sci. 10, 518–536. doi: 10.1177/1745691615583128

PubMed Abstract | CrossRef Full Text | Google Scholar

Leiberman, M. D., and Cunningham, W. A. (2009). Type, I., and Type II error concerns in fMRI research: re-balancing the scale. Soc. Cogn. Affect. Neurosci. 4, 423–428. doi: 10.1093/scan/nsp052

CrossRef Full Text | Google Scholar

Lieberman, D., Tooby, J., and Cosmides, L. (2003). Does morality have a biological basis? an empirical test of factors governing moral statements relating to incest. Proc. R. Soc. Lond. B 270, 819–826. doi: 10.1098/rspb.2002.2290

CrossRef Full Text | Google Scholar

Moll, J., de Oliveira-Souza, R., Moll, F. T., Ignácio, F. A., Bramati, I. E., Caparelli-Dáquer, E. M., et al. (2005). The moral affiliations of disgust: a functional MRI study. Cogn. Behav. Neurol. 18, 68–78. doi: 10.1097/01.wnn.0000152236.46475.a7

PubMed Abstract | CrossRef Full Text | Google Scholar

Nabi, R. L. (2002). The theoretical versus the lay meaning of disgust: implications for emotion research. Cogn. Emot. 16, 695–703. doi: 10.1080/02699930143000437

CrossRef Full Text | Google Scholar

Oaten, M., Stevenson, R. J., and Case, T. I. (2009). Disgust as a disease-avoidance mechanism. Psychol. Bull. 135, 303–321. doi: 10.1037/a0014823

PubMed Abstract | CrossRef Full Text | Google Scholar

Olatunji, B. O., Puncochar, B. D., and Cox, R. (2016). Effects of experienced disgust on morally-relevant judgment. PLoS ONE 11:e0160357. doi: 10.1371/journal.pone.0160357

CrossRef Full Text | Google Scholar

Phillips, M. L., Williams, L. M., Heining, M., Herba, C. M., Russell, T., Andrew, C., et al. (2004). Differential neural responses to overt and covert presentations of facial expressions of fear and disgust. Neuroimage 21, 1484–1496. doi: 10.1016/j.neuroimage.2003.12.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Phillips, M. L., Young, A. W., Scott, S. K., Calder, A. J., Andrew, C., Giampietro, V., et al. (1998). Neural responses to facial and vocal expressions of fear and disgust. Proc. R. Soc. Lond. B Biol. Sci. 265, 1809–1817. doi: 10.1098/rspb.1998.0506

PubMed Abstract | CrossRef Full Text | Google Scholar

Phillips, M. L., Young, A. W., Senior, C., Brammer, M., Andrew, C., Calder, A. J., et al. (1997). A specific neural substrate for perceiving facial expressions of disgust. Nature 389, 495–498. doi: 10.1038/39051

PubMed Abstract | CrossRef Full Text | Google Scholar

Roseman, I. J., Wiest, C., and Swartz, T. S. (1994). Phenomenology, behaviors, and goals differentiate discrete emotions. J. Pers. Soc. Psychol. 67, 206–221. doi: 10.1037/0022-3514.67.2.206

CrossRef Full Text | Google Scholar

Royzman, E., Atanasoy, P., Landy, J. F., Parks, A., and Gepty, A. (2014). CAD or MAD? Anger (not disgust) as the predominant response to pathogen-free violations of the divinity code. Emotion 14, 892–907 doi: 10.1037/a0036829

PubMed Abstract | CrossRef Full Text | Google Scholar

Royzman, E., and Sabini, J. (2001). Something it takes to be an emotion: the interesting case of disgust. J. Theor. Soc. Behav. 31, 29–59. doi: 10.1111/1468-5914.00145

CrossRef Full Text | Google Scholar

Rozin, P. (2008). Hedonic “adaptation”: specific habituation to disgust/death elicitors as a result of dissecting a cadaver. Judgm. Decis. Mak. 3, 191–194.

Google Scholar

Rozin, P., Haidt, J., and McCauley, C. R. (1999). “Disgust: the body and soul emotion,” in Handbook of Cognition and Emotion, eds T. Dalgleish and M. J. Power (New York, NY: Wiley), 429–445. doi: 10.1002/0470013494.ch21

CrossRef Full Text | Google Scholar

Rozin, R., Fallon, A. E., and Mandell, R. (1984). Family resemblance in attitudes to foods. Dev. Psychol. 20, 309–314. doi: 10.1037/0012-1649.20.2.309

CrossRef Full Text | Google Scholar

Russell, P. S., and Giner-Sorolla, R. (2011). Moral anger, but not moral disgust, responds to intentionality. Emotion 11, 233–240. doi: 10.1037/a0022598

PubMed Abstract | CrossRef Full Text | Google Scholar

Schaich Borg, J., Leiberman, D., and Kiehl, K. A. (2008). Infection, incest, and iniquity: investigating the neural correlates of disgust and morality. J. Cogn. Neurosci. 20, 1529–1546 doi: 10.1162/jocn.2008.20109

PubMed Abstract | CrossRef Full Text | Google Scholar

Schnall, S., Haidt, J., Clore, G. L., and Jordan, A. H. (2008). Disgust as embodied moral judgment. Pers. Soc. Psychol. Bull. 34, 1096–1109. doi: 10.1177/0146167208317771

PubMed Abstract | CrossRef Full Text | Google Scholar

Sprengelmeyer, R., Rausch, M., Eysel, U. T., and Przuntek, H. (1998). Neural structures associated with recognition of facial expressions of basic emotions. Proc. R. Soc. Lond. B Biol. Sci. 265, 1927–1931. doi: 10.1098/rspb.1998.0522

PubMed Abstract | CrossRef Full Text | Google Scholar

Stark, R., Schienle, A., Sarlo, M., Palomba, D., Walter, B., and Vaitl, D. (2005). Influences of disgust sensitivity on hemodynamic responses towards a disgust-inducing film clip. Int. J. Psychophysiol. 57, 61–67. doi: 10.1016/j.ijpsycho.2005.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Stark, R., Schienle, A., Walter, B., Kirsch, P., Sammer, G., Ott, U., et al. (2003). Hemodynamic responses to fear and disgust-inducing pictures: an fMRI study. Int. J. Psychophysiol. 50, 225–234. doi: 10.1016/S0167-8760(03)00169-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Tybur, J. M., Lieberman, D., and Griskevicius, V. (2009). Microbes, mating, and morality: individual differences in three functional domains of disgust. J. Pers. Soc. Psychol. 97, 103–122. doi: 10.1037/a0015474

PubMed Abstract | CrossRef Full Text | Google Scholar

Vicaro, C. M., Rafal, R. D., Martino, D., and Avenanti, A. (2017). Core, social and moral disgust are bounded: a review of behavioral and neural bases of repugnance in clinical disorders. Neurosci. Biobehav. Rev. 80, 185–200. doi: 10.1016/j.neubiorev.2017.05.008

CrossRef Full Text | Google Scholar

Wager, T. D., Lindquist, M., and Kaplan, L. (2007). Meta-analysis of functional neuroimaging data: current and future directions. Soc. Cogn. Affect. Neurosci. 2, 150–158. doi: 10.1093/scan/nsm015

PubMed Abstract | CrossRef Full Text | Google Scholar

Wicker, B., Keysers, C., Plailly, J., Royet, J. P., Gallese, V., and Rizzolatti, G. (2003). Both of us disgusted in my insula: the common neural basis of seeing and feeling disgust. Neuron 40, 655–664. doi: 10.1016/S0896-6273(03)00679-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Williams, L. M., Das, P., Liddell, B., Olivieri, G., Peduto, A., Brammer, M. J., et al. (2005). Bold, sweat, and fears: fMRI and skin conductance distinguish facial fear signals. Neuroreport 16, 49–52. doi: 10.1097/00001756-200501190-00012

PubMed Abstract | CrossRef Full Text | Google Scholar

Wright, P., He, G., Shapira, N., Goodman, W., and Liu, Y. (2004). Disgust and the insula: fMRI responses to pictures of mutilation and contamination. Neuroreport 15, 2347–2351. doi: 10.1097/00001756-200410250-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: disgust, moral disgust, moral violations, anger, fMRI

Citation: Oaten M, Stevenson RJ, Williams MA, Rich AN, Butko M and Case TI (2018) Moral Violations and the Experience of Disgust and Anger. Front. Behav. Neurosci. 12:179. doi: 10.3389/fnbeh.2018.00179

Received: 19 January 2018; Accepted: 30 July 2018;
Published: 22 August 2018.

Edited by:

Nuno Sousa, Instituto de Pesquisa em Ciências da Vida e da Saúde (ICVS), Portugal

Reviewed by:

Justin Frederick Landy, University of Chicago, United States
Carmelo Mario Vicario, Università degli Studi di Messina, Italy

Copyright © 2018 Oaten, Stevenson, Williams, Rich, Butko and Case. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Megan Oaten, m.oaten@griffith.edu.au