Development and Preliminary Validation of the Salzburg Emotional Eating Scale

Existing self-report questionnaires for the assessment of emotional eating do not differentiate between specific types of emotions and between increased or decreased food intake in response to these emotions. Therefore, we developed a new measure of emotional eating—the Salzburg Emotional Eating Scale (SEES)—for which higher scores indicate eating more than usual in response to emotions and lower scores indicate eating less than usual in response to emotions. In study 1, a pool of items describing 40 emotional states was used. Factor analysis yielded four factors, which represented both positive (happiness subscale) and negative emotions (sadness, anger, and anxiety subscales). Subsequently, the scale was reduced to 20 items (5 items for each subscale) and its four-factor structure was replicated in studies 2 and 3. In all three studies, internal consistencies of each subscale were α > 0.70 and mean subscale scores significantly differed from each other such that individuals reported the strongest tendency to eat more than usual when being sad and the strongest tendency to eat less than usual when being anxious (sadness > happiness > anger > anxiety). Higher scores on the happiness subscale related to lower scores on the negative emotions subscales, lower body mass index (BMI), and lower eating pathology. In contrast, higher scores on the negative emotions subscales related to lower scores on the happiness subscale, higher BMI, and higher eating pathology. The SEES represents a useful measure for the investigation of emotional eating by increasing both specificity (differentiation between specific emotional states) and breadth (differentiation between increase and decrease of food intake) in the assessment of the emotion–eating relationship.


INTRODUCTION
Emotional eating is often defined as an increase in food intake in order to cope with negative emotions (Macht and Simons, 2011). This type of eating style characterizes several eating-related psychopathologies. For example, negative affect precipitates binge eating episodes in individuals with bulimia nervosa and binge eating disorder (Haedt-Matt and Keel, 2011) and increases desire to eat in obese individuals (van Strien et al., 2016). Although food intake is often used as an emotion regulation strategy, it usually does not reduce negative affect effectively and, thus, the mechanisms that generate and maintain emotional eating are far from being clear (Haedt-Matt et al., 2014). Moreover, it seems that there are numerous moderators that determine the effects of emotions on eating behavior such as the arousal level that accompanies certain emotions or individual differences in eating habits (Macht, 2008).
While the majority of studies on emotional eating have focused on eating to alleviate a negative mood, it has been shown that a positive mood can also result in increased food intake (Cardi et al., 2015). Moreover, while most research has focused on increased food intake, it has been found that experiencing emotions can also result in a decrease in food intake, depending on self-reported emotional eating tendencies (van Strien et al., 2012). A recent review of the literature on self-report measures of emotional eating criticized their lack of predictive validity, pointing to the weak and inconsistent associations with actual food intake in laboratory-based and naturalistic studies (Bongers and Jansen, 2016). One reason for this might be that they only cover a restricted range of relevant aspects of emotional eating. For example, most measures solely include negative emotions and only ask about eating more in response to these emotions, leaving out the possibility of reduced food intake (Bongers and Jansen, 2016).
A range of self-report measures are available for the assessment of emotional eating tendencies such as the Dutch Eating Behavior Questionnaire (DEBQ; van Strien et al., 1986), the Emotional Eating Scale (EES; Arnow et al., 1995), the Emotional Overeating Questionnaire (EOQ; Masheb and Grilo, 2006), the Emotional Appetite Questionnaire (EMAQ; Geliebter and Aversa, 2003), and the Positive-Negative Emotional Eating Scale (PNEES; Sultson et al., 2017). To date, the only selfreport questionnaire that does differentiate between positive and negative emotions and assesses eating less or eating more in response to these emotions is the EMAQ. However, this scale was not developed via item selection based on empirical data and does not differentiate between different negative emotions (e.g., with different arousal levels; Geliebter and Aversa, 2003). Therefore, we developed a new self-report measure of emotional eating that differentiates between different emotions and between decreased or increased food consumption in response to these emotions (Salzburg Emotional Eating Scale, SEES). For this, an item pool of 40 items was created (see method section below and Table 1).
In study 1, we explored factor structure of this item pool and items with the highest factor loadings were selected for further refining the scale. We expected that participants would report eating more than usual in response to positive emotions and negative, low arousing emotions (Cardi et al., 2015). In response to negative, high arousing emotions, however, we expected that participants would report eating less than usual (Reichenberger et al., 2018). Based on the findings with the EMAQ (Nolan et al., 2010;Bourdier et al., 2017), we expected that eating more in response to positive emotions would be associated with eating less in response to negative emotions. Furthermore, men were expected to report eating more in response to positive emotions than women, and women were expected to report eating more in response to negative emotions than men (Nolan et al., 2010). Finally, we expected that eating in response to positive emotions would be associated with lower BMI while eating in response to negative emotions would be associated with higher BMI (Nolan et al., 2010;Bourdier et al., 2017).  Studies 2 and 3 aimed at replicating findings about psychometric properties (factor structure, internal consistencies), mean score differences and associations between subscales, and correlates of the scale (sex, BMI). Study 2 also examined associations with other questionnaire measures as a preliminary indication of validity. Specifically, we expected that there would be medium-to-high correlations with similar questionnaire measures on stress-and emotional eating and small correlations with other eating-related questionnaire measures (eating disorder pathology, perceived self-regulatory success in weight regulation). Finally, we expected that there would be no or only small correlations with measures that are not directly related to eating behavior (depressiveness, impulsivity).

Participants and Procedure
The study was approved by the institutional review board of the University of Salzburg. A link to the online survey at www. unipark.com was distributed via e-mail to a mailing list at the University of Salzburg addressing students and staff and via social networks. Participation was voluntary and participants did not receive any compensation. The website was visited 413 times and n = 285 participants completed the entire set of questions.

Measures
A pool of 40 items (20 positive and 20 negative emotions) was generated based on existing questionnaires of emotional eating (DEBQ, EES, EOQ, EMAQ) and the Positive and Negative Affect Schedule (Watson et al., 1988). Items are displayed in Table 1. Each item began with the stem When I am/feel. . . , followed by an adjective describing an emotional state. Response categories ranged from I eat much less than usual to I eat much more than usual (scored from 1 to 5). Items were presented in randomized order in the online survey.

Data Analyses
Sample size exceeded the minimum 5:1 subjects-to-item ratio necessary for exploratory factor analysis (Costello and Osborne, 2005). Furthermore, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO = 0.871) and Bartlett's Test of Sphericity [χ 2 (780) = 4390, p < 0.001] indicated that data were adequate for conducting an exploratory factor analysis. The number of factors was determined by both parallel analysis and the Minimum Average Partial (MAP) test using the SPSS-syntax provided by O'Connor (2000). Principal Component Analysis was chosen as extraction method and Promax (κ = 4) was selected as rotation method. Internal consistency of factors was evaluated with Cronbach's alpha. Associations between different factor scores and between factor scores and BMI were tested with Pearson correlation coefficients. Sex differences of factor scores were tested with independent samples Mann-Whitney-U-tests.

Results
Parallel analysis (Figure 1) as well as the MAP test (averaged squared partial correlation: component 1 = 0.025, component 2 = 0.010, component 3 = 0.010, component 4 = 0.009, component 5 = 0.010) suggested extraction of four factors, which explained 43.6% of variance. The first factor included items related to positive emotions. The second factor included items related to negative, but low arousing emotions. The third and fourth factor included items related to high arousing emotions such as anger and anxiety ( Table 1).
In order to reduce the number of items for the SEES, five items with the highest factor loadings were selected for further analyses (Table 1). Thus, this resulted in four subscales that we termed happiness (containing the items cheerful, happy, optimistic, proud, confident), sadness (containing the items sad, depressed, bored, lonely, frustrated), anger (containing the items angry, furious, upset, irritated, jealous), and anxiety (containing the items anxious, worried, tense, uneasy, nervous). Internal consistencies of these subscales ranged between α = 0.713-0.800 ( Table 2). Subscale scores of sadness, anger, and anxiety were positively correlated with each other while scores on happiness were negatively correlated with sadness and anxiety ( Table 2). Mean scores on all subscales significantly differed from each other (all ts > 3.37, ps ≤ 0.001) in the following descending order:  sadness > happiness > anger > anxiety (Figure 2A). Men had higher scores (M = 3.20, SD = 0.54) than women (M = 2.95, SD = 0.42) on the happiness subscale (p < 0.001) and had lower scores (M = 3.22, SD = 0.66) than women (M = 3.40, SD = 0.79) on the sadness subscale (p = 0.036). Men and women did not differ on anger and anxiety subscale scores (ps > 0.080). There were positive correlations between BMI and sadness (r = 0.256, p < 0.001), anger (r = 0.117, p = 0.05), and anxiety (r = 0.269, p < 0.001) subscale scores, but no correlation with the happiness subscale (r = 0.011, p = 0.856).

Participants and Procedure
The study was approved by the institutional review board of the University of Salzburg. A link to the online survey at www.unipark.com was distributed via e-mail to student mailing lists at several German and Austrian universities, via social networks, and via a posting on the website of the German version of Psychology Today. Three × 50 e were raffled among participants who completed the survey. The website was visited 1,396 times and n = 805 participants completed the entire set of questions. Fifteen participants were excluded from analyses because they answered questions too rapidly (total completion time less than 5 min), leaving a final sample size of n = 790. Most participants were women (82.9%, n = 655) and had German (81.3%, n = 642) or Austrian (14.2%, n = 112) citizenship. The majority of participants were students (79.6%, n = 629), employed (11.4%, n = 90), or pupils (4.70%, n = 37). Mean age was M = 24.7 years (SD = 6.79, Range: 15-65). Mean BMI was M = 22.3 kg/m 2 (SD = 3.93, Range: 15.0-50.9). Seventy-six participants (9.60%) had underweight (BMI < 18.5 kg/m 2 ), 583 participants (73.9%) had normal weight (BMI = 18.5-24.9 kg/m 2 ), 92 participants (11.7%) had overweight (BMI = 25.0-29.9 kg/m 2 ), and 38 participants (4.80%) had obesity (BMI ≥ 30.0 kg/m 2 ; BMI data missing for one participant).

Salzburg Emotional Eating Scale (SEES)
The 20-item SEES derived in study 1 with five items per subscale was used. Items were presented in randomized order in the online survey. Internal consistencies of SEES subscales ranged between α = 0.732-0.819 (Table 3).

Salzburg Stress Eating Scale (SSES)
The SSES (Meule et al., 2018a) is a ten-item questionnaire for measuring eating in response to stress. Response categories FIGURE 2 | Mean subscale scores on the Salzburg Emotional Eating Scale in study 1 (A), study 2 (B), and study 3 (C). Error bars indicate the standard error of the mean. All subscale scores differed significantly from each other at p ≤ 0.001. range from I eat much less than usual to I eat much more than usual (scored from 1 to 5). Thus, higher values represent eating more when stressed while lower values indicate eating less when stressed. Internal consistency was α = 0.899 in the current study.

Dutch Eating Behavior Questionnaire (DEBQ)-Emotional Eating Subscale
The emotional eating subscale of the DEBQ (van Strien et al., 1986;Grunert, 1989) is a ten-item questionnaire for measuring eating in response to negative emotions. Response categories range from never to very often (scored from 1 to 5). Thus, higher values indicate the frequency of eating more when experiencing negative emotions. Internal consistency was α = 0.909 in the current study.

Perceived Self-Regulatory Success in Dieting Scale (PSRS)
The PSRS (Meule et al., 2012) is a three-item questionnaire for measuring how successful individuals are in watching their weight, in losing weight, and how difficult it is for them to stay in shape. Response categories are anchored not successful/not difficult and very successful/very difficult (scored from 1 to 7). Thus, higher values indicate higher perceived self-regulatory success in weight regulation. Internal consistency was α = 0.696 in the current study.

Eating Disorder Examination-Questionnaire 8 (EDE-Q8)
The EDE-Q8 (Kliem et al., 2016) is an eight-item short form of the EDE-Q for measuring eating disorder psychopathology in the past 28 days. Response categories range from no days/never/not at all to every day/everytime/markedly (scored from 0 to 6). Thus, higher values indicate higher eating disorder psychopathology. Internal consistency was α = 0.913 in the current study.

Center for Epidemiologic Studies Depression Scale-short form (CES-D)
The short form of the CES-D (Radloff, 1977;Hautzinger et al., 2012) is a 15-item questionnaire for measuring depressive symptoms in the past seven days. Response categories range from rarely or none of the time to most or all of the time (scored from 0 to 3). Thus, higher values indicate higher depressiveness. Internal consistency was α = 0.911 in the current study.

Data Analyses
A confirmatory factor analysis was computed with Amos 24 (IBM SPSS, Chicago) to test the four-factor structure of the SEES found in study 1. Maximum likelihood estimation was used, fixing the factor loading of the first items of every subscale to 1. According to the recommendations of Hu and Bentler (1999), model fit was evaluated by two fit indices: the comparative fit index (CFI), with 0.90 ≤ CFI < 0.95 indicating acceptable fit and CFI ≥ 0.95 indicating good fit, and the root mean square error of approximation (RMSEA), with 0.05 < RMSEA ≤ 0.08 indicating acceptable fit and RMSEA ≤ 0.05 indicating good fit. In order to evaluate whether factor structure of the SEES varied between female and male participants, we tested measurement invariance at three levels: configural, factor loading and intercept invariance. Measurement invariance across sex was evaluated according to recommendations by Chen (2007). Specifically, a χ 2 difference test can be used for statistical comparison between nested models, but is almost always large and statistically significant with complex models and large samples and, thus, an impractical and unrealistic criterion for measurement invariance (Chen et al., 2005). Therefore, model fit changes were examined and decreases in CFI ≤ 0.010 or increases in RMSEA of ≤ 0.015 were considered to indicate measurement invariance (Chen, 2007). Associations between SEES subscale scores, BMI, and scores on the other questionnaires were examined with correlational analyses. Sex differences of SEES subscale scores were tested with independent samples Mann-Whitney-U-tests.

Results
The four-factor structure had acceptable model fit (CFI = 0.932, RMSEA = 0.051) and standardized estimates are depicted in Figure 3A. Subscale scores of sadness, anger, and anxiety were positively correlated with each other while scores on happiness were negatively correlated with sadness, anger, and anxiety scores ( Table 3). Similar to study 1, mean scores on all subscales significantly differed from each other (all ts > 4.81, ps < 0.001) in the following descending order: sadness > happiness > anger > anxiety ( Figure 2B). Similar to study 1, men had higher scores (M = 3.14, SD = 0.44) than women (M = 2.98, SD = 0.47) on the happiness subscale (p < 0.001) and had lower scores (M = 3.16, SD = 0.65) than women (M = 3.42, SD = 0.75) on the sadness subscale (p < 0.001). In addition, men had lower scores (M = 2.75, SD = 0.50) than women (M = 2.88, SD = 0.61) on the anger subscale (p = 0.023). Men and women did not differ on anxiety subscale scores (p = 0.709).
Scores on the happiness subscale were negatively correlated with BMI, stress eating, emotional eating, eating disorder pathology, depressiveness, and non-planning impulsivity, and positively correlated with perceived self-regulatory success in weight regulation (Table 4). In contrast, scores on the sadness, anger, and anxiety subscales were positively correlated with BMI, stress eating, emotional eating, and eating disorder pathology, and negatively correlated with perceived self-regulatory success in weight regulation (Table 4). In addition, scores on the sadness subscale were positively correlated with depressiveness and attentional impulsivity ( Table 4).

Participants and Procedure
The study was approved by the institutional review board of the University of Salzburg. A link to the online survey at www. limesurvey.org was distributed via e-mail to student mailing lists at the University of Salzburg. To broaden the age range as compared to studies 1 and 2, these included students from the university's 55-PLUS program, which is an educational opportunity for older adults at the university. Furthermore, adolescent participants were recruited at a local high school. Participation was voluntary and participants did not receive any compensation. The website was visited 623 times and n = 450 participants completed the study.

Data Analyses
Data analyses regarding factor structure and measurement invariance of the SEES, differences and associations between subscales, and relationships with sex and BMI were identical to study 2.

DISCUSSION
Progress in the research on the conceptual foundations of emotional eating, its potential mechanisms and its clinical correlates requires continuous refinement of the respective psychometric scales. The current studies document the development and preliminary validation of a new self-report measure of emotional eating-the SEES-which extends previous scales by detailing specific emotions and differentiating emotional over-and undereating. Item reduction from a pool of 40 emotional states resulted in a 20-item scale with four subscales, which were invariant across sex and had acceptable-togood internal consistencies. Scores on these subscales represent self-reported eating in response to positive emotions (happiness), negative but low arousal emotions (sadness), and negative but high arousing emotions (anger and anxiety). These four affective states are consistent with the four most often measured emotions in emotion research (Weidman et al., 2017). Higher scores represent increased food intake, medium scores represent unchanged food intake, and lower scores represent decreased food intake.
Subscale scores significantly differed from each other: there was an overall tendency to report eating more than usual when experiencing sadness, eating just as much as usual when experiencing happiness, and eating less than usual when experiencing anger or anxiety (Figure 2). These differences might be attributable to different levels of bodily arousal that accompanies these emotions (Macht, 2008) and respective neuroendocrine changes (Torres and Nowson, 2007). Therefore, results point to specific mappings of emotion type on intake type (e.g., sadness to overeating, anger/anxiety to undereating) that-if collapsed by a measure-might mask or fully occlude any relationship and, thus, lead to inconsistent or contradictory findings.
In line with findings with the EMAQ, men reported to eat more when being happy whereas women reported to  eat more when being sad. Moreover, eating in response to positive emotions was negatively correlated with eating in response to negative emotions and, similarly, correlates of these subscales diverged (Nolan et al., 2010;Bourdier et al., 2017). Although results of all three studies were not entirely consistent, an overall picture emerged such that eating more in response to happiness was associated with having a lower BMI, reporting lower eating pathology, and higher perceived success in weight regulation. Thus, it appears that such "happy overeating" represents a functional, healthy eating style that may reflect an intuitive change in eating behavior associated with appropriate perception of bodily signals (Herbert et al., 2013). Moreover, higher scores on the happiness subscale were associated with eating less in response to negative emotions, which may indicate that "happy overeating" and "unhappy undereating" might be two sides of the same coin and together associated with positive eating-and weight-related outcomes. This interpretation, however, needs to consider a person's body weight. For example, we have preliminary data available showing that inpatients with anorexia nervosa have significantly higher scores on the happiness subscale and lower scores on the sadness subscale than normal-weight control participants (Meule et al., 2018b). This suggests that the "happy overeating"-"unhappy undereating" combination can also reflect higher eating disorder severity in some individuals, particularly in those with underweight.
In contrast, eating more in response to negative emotions was associated with having a higher BMI, reporting higher eating pathology, and lower perceived success in weight regulation. Again, the flipside of this pattern was eating less in response to happiness, suggesting that the configuration of "unhappy overeating" and "happy undereating" is a more dysfunctional, unhealthy eating pattern that relates to unfavorable eatingand weight-related outcomes. Thus, these findings replicate and extend findings about correlates of eating in response to positive and negative emotions (Geliebter and Aversa, 2003;Nolan et al., 2010;Bourdier et al., 2017), providing preliminary support for validity and usefulness of the SEES.
Although the SEES allows for a more fine-grained analysis of emotional effects on eating, it still relies on self-report, which can potentially be biased. Specifically, the questionnaire requires that participants have significant insight of their dayto-day fluctuations in affect and its influence on food intake. For example, it has been suggested that individuals scoring high on self-report measures of emotional eating may overestimate how much they actually eat in response to certain emotions or may simply attribute their overeating to negative affect retrospectively (Royal and Kurtz, 2010;Adriaanse et al., 2016). Thus, validity of the SEES may be investigated in future studies that examine relationships with implicit measures (e.g., an emotional eating-related implicit association task; Bongers et al., 2013), with food intake after emotion induction in the laboratory, or with emotional eating assessed in daily life (e.g., with a combination of ecological momentary assessment and dietary assessment such as the Automated Self-Administered 24-h dietary assessment tool; Subar et al., 2012). Furthermore, our data are based on samples, which entail common limitations of online-based research (e.g., self-selection, completing the questionnaire multiple times). Thus, it is necessary to examine psychometric properties (e.g., measurement invariance across different age groups) and correlates of the SEES in more representative samples, which also include, for example, a higher proportion of individuals with lower education.
In accordance with other reports (Bongers and Jansen, 2016), our data suggest that focusing solely on increased food intake in response to negative emotions only covers a small proportion of what can be termed emotional eating. Specifically, different emotions have different effects on food intake and these have dissociable correlates. For example, "happy overeating" (or "unhappy undereating") as opposed to "happy undereating" (or "unhappy overeating") appears to be an adaptive, functional behavior in most individuals. Thus, we suggest that traditional definitions of the term emotional eating need to be opened up to all possible combinations of emotion and intake types such that emotional eating can be defined as any alteration in food intake (which can include eating less or eating more than usual) in response to affective states (which can include positive and negative emotions). By providing a fine-grained assessment of emotional eating that takes these aspects into account, the SEES opens exciting new avenues for research on the mechanisms and clinical correlates of emotional eating, not only in the psychometric domain but also in experimental and naturalistic studies.

AUTHOR CONTRIBUTIONS
AM and JB: Conceived the studies; AM and JR: Conducted the statistical analyses; AM: Drafted the manuscript; JR and JB revised the manuscript.