Because all of the papers in this special issue of Frontiers in Psychology are devoted to EI and g, we will forego an in-depth review of the literature on EI, and instead focus directly on behavioral EI. As mentioned above, EI competencies can be viewed as the behavioral level of EI (Boyatzis, 2009; Cherniss, 2010; Cherniss and Boyatzis, 2013). Competencies have been derived inductively from studies of human performance in many occupations and in many countries (Boyatzis, 2009). Because the identification of a competency and its refinement emerges from performance based criterion sampling, they are expected to be closely related to job and life outcomes. As a result, the EI competencies were discovered and measured as behaviors which were later clustered around intent and became each competency (Boyatzis, 2009).

In Boyatzis and Goleman (2007), EI includes two factors, EI and social intelligence (SI) competencies. EI includes competencies called emotional self-awareness, emotional self control, adaptability, achievement orientation, and positive outlook. In their model, SI includes: empathy, organizational awareness, influence, inspirational leadership, conflict management, coach and mentor, and teamwork. For this paper, we are treating EI and SI competencies as a single construct of EI. When universities wish to use this EI model for student development and/or outcome assessment, two cognitive competencies which have a history of predicting effective leadership, management and professional performance are added. They are: systems thinking and pattern recognition (Boyatzis, 2009).

Behavioral EI as seen and measured through others’ assessment (as compared to self-assessment) shows a consistent prediction or relationship to job and life outcomes (Boyatzis, 1982, 2006; McClelland, 1998; Nel, 2001; Cavallo and Brienza, 2002; Dulewicz et al., 2003; Law et al., 2004; Sy et al., 2006; Dreyfus, 2008; Hopkins and Bilimoria, 2008; Koman and Wolff, 2008; Williams, 2008; Boyatzis and Ratti, 2009; Ramo et al., 2009; Ryan et al., 2009, 2012; Young and Dulewicz, 2009; Boyatzis et al., 2011, 2012; Aliaga Araujo and Taylor, 2012; Gutierrez et al., 2012; Sharma, 2012; Amdurer et al., 2013; Victoroff and Boyatzis, 2013; Mahon et al., 2014; Badri, unpublished). Boyatzis et al. (2012) showed behavioral EI predicted job performance with significant unique variance, controlling for g and personality.

According to the dominant classification in Ashkanasy and Daus (2005), there are three different streams of EI research. Salovey and Mayer’s Ability EI as measured by the MSCEIT is stream 1. Although it has shown relationships with school (Brackett et al., 2004), job and life outcomes (Mayer et al., 2008), these were not of primary consideration in its development (Mayer et al., 1999). Whereas ability EI shows no relationship to personality measures, it has shown consistent prediction of g, even when controlling for personality (Webb et al., 2013).

Self-perceptions and peer-report measures based on the Ability EI model are clustered within stream 2 (Ashkanasy and Daus, 2005). These measures such as the Trait EI Questionnaire (TEIQue; Petrides and Furnham, 2000, 2001), show similar validity patterns to the MSCEIT but are not as strongly related to g, nor job and life outcomes, yet they do show a significant relationship to personality (Webb et al., 2013).

Meanwhile, stream 3 (Ashkanasy and Daus, 2005) clusters both those EI measures based on self-perception and others’ behavioral assessments (i.e., 360°, coded behavior from audiotape or videotape work samples or simulations). Consequently, there is a partition in results within this stream: some measures such as the ESCI (Boyatzis and Goleman, 2007) show a strong relationship and unique variance to life and job outcomes beyond g and personality (Byrne et al., 2007; Downey et al., 2011), while others such as the EQ-i (Bar-On, 1997) show a consistent relationship in predicting personality (Joseph and Newman, 2010; O’Boyle et al., 2011). We therefore, claim that clustering self-perception and coded or other perception measures confuses these relationships.

Instead, we support Fernández-Berrocal and Extremera’s (2006, p. 8) comprehensive view of the EI field by which all “approaches try to discover the emotional components that underlie emotionally intelligent people and the mechanisms and processes that set off the use of these abilities in our everyday life” (emphasis added). In the authors’ review of the first 15 years of EI research, behavioral EI as seen by others in 360° assessments is considered separately from self-perception approaches focused on moods and internal states, as well as personality traits such as Bar-On’s (1997, 2007; Fernández-Berrocal and Extremera, 2006). Therefore, Boyatzis (2009) extends the work of Fernández-Berrocal and Extremera (2006) to propose an organization of the literature that is framed by the three existing methodological themes: EI ability methods; EI self-perception methods; and EI behavior methods.

In sum, the relationships of EI assessed at any level or with any method are still debated with comparative arguments about its link to g and personality. In this paper, we will focus on the relationship between behavioral EI and a measure of g.

According to Carroll’s (1993) model of intelligence, the various mental abilities are structured hierarchically. General cognitive ability, located at its apex, is “the general efficacy of intellectual processes” (Ackerman et al., 2005, p. 32). Also known as general mental ability, general intelligence, or simply g, it is a well-researched construct with a large body of evidence supporting its predictive validity for such important outcomes as job performance and career success (e.g., O’Reilly III and Chatman, 1994; Schmidt and Hunter, 1998; Ferris et al., 2001). As a global ability, g can be thought of as the underlying common factor to all types of cognitive processing (i.e., verbal, mathematical, spatial, logical, musical, and emotional). From this perspective, g cannot be observed nor measured directly, it must be inferred from the positive correlations among distinct ability measures (Spearman, 1904; Jensen, 1998). As such, g subsumes different sets of abilities, each corresponding to a specialization of general intelligence.

General cognitive ability can be assessed through a variety of measures, such as IQ tests (Jensen, 1992; i.e., Ravens Progressive Matrices, Wechsler, Stanford Binet; Nisbett et al., 2012). Similarly, standardized admissions tests have been shown to “fit the general requisites of a measure of general cognitive ability” (O’Reilly III and Chatman, 1994). They also measure verbal and mathematical or quantitative reasoning skills separately. These tests such as the SAT, GRE, GMAT, MCAT, LSAT, and DAT are usually found to have strong correlations with the more direct measures of g, (Detterman and Daniel, 1989).

The GMAT is a standardized test that assesses a person’s analytical, writing, quantitative, verbal and reading skills for admission into graduate management programs worldwide. Although the GMAT is not formally validated as a measure of general cognitive ability, it is strongly correlated with the Scholastic Aptitude Test (SAT; e.g., Gottesman and Morey, 2006), which is shown to be a valid measure of g (Frey and Detterman, 2004). Considering the structural similarity of these tests (both consist of multiple choice questions that measure verbal and quantitative skills) and the general consensus that the g-factor can be measured by obtaining factorial scores across tests of different specific aptitudes, usually verbal and quantitative (O’Reilly III and Chatman, 1994), Hedlund et al. (2006, p. 102) concluded that “like the SAT, the GMAT can be characterized as a traditional measure of intelligence, or a test of general cognitive ability (g).” Indeed numerous studies have already used the GMAT as a measure of g (e.g., O’Reilly III and Chatman, 1994; Kumari and Corr, 1996; Mueller and Curhan, 2006), the latest of which is a study published in Intelligence (Piffer et al., 2014).

We suggest that the EI competencies may show a small, if any relationship to g. In fact, correlations between behavioral EI competencies coded from audiotapes of critical incident interviews about work samples and GMAT were not significant (r = -0.015, n = 200, p = ns; Boyatzis et al., 2002). In assessing predictors of sales leadership effectiveness in the financial services industry, Boyatzis et al. (2012) reported that EI as assessed by others showed a non-significant correlation with Ravens Progressive Matrices (r = 0.04, n = 60, p = ns).

In the inductive competency studies, two cognitive competencies repeatedly appeared to differentiate effective performance of managers, executives and professionals (Boyatzis, 1982, 2009; Spencer and Spencer, 1993). They were systems thinking and pattern recognition. The former is defined as seeing phenomenon as a series of causal relationships affecting each other. The latter is defined as perceiving themes or patterns in seemingly random information. As competencies, they are assessed both with a self-assessment and with observations of others as to how often a person demonstrates these behaviors. They are not defined or assessed as an intelligence measure but an indication of how often a person appears to be using these thought processes. As such, we expect them to be related to g more than EI competencies even though they are not a measure of g.

This leads us to the first two hypotheses for this study:

Differences in raters or sources of assessment are likely to play an important role in the findings. Self-perception and multi-rater assessment are different approaches to perceiving and collecting observations of a person’s behavior (Luthans et al., 1988; Church, 1997; Furnham and Stringfield, 1998; Antonioni and Park, 2001; Taylor and Hood, 2010).

Self-assessment measures generally address how individuals respond to questions pertaining to their own emotions, perceptions or thoughts. These measures are easier and faster to administer than others, allowing for low costs of administration (Saris and Gallhofer, 2007). Social desirability is often an issue in self-reported measures (Paulhus and Reid, 1991). That is, respondents may base their answers on a desired state that often leads to inflated views of themselves. The validity of these measures can be improved by including questions that help control for social desirability (e.g., Paulhus and Reid, 1991; Steenkamp et al., 2010).

Used as a stand-alone measure, self-assessment of personality traits, attitudes or behavioral tendencies show acceptable validity (e.g., Furnham et al., 1999; Petrides and Furnham, 2000; Furnham, 2001; Petrides et al., 2006; Bar-On, 2007). Similarly, self-assessed measures of EI show acceptable validity (Bar-On, 1997; Petrides and Furnham, 2000, 2001). However, with regard to EI, self-assessments are also used in combination with others’ ratings. Notably, the difference between self and others’ perceptions is known as the self-other-agreement. This difference is a highly reliable measure of self-awareness (Yammarino and Atwater, 1997).

Multi-rater or multi-source assessments involve different raters from work such as a person’s peers, collaborators, subordinates or bosses, and possibly raters from one’s personal environment. Raters provide observations of a person’s behavior (i.e., what they have seen the person do). Research on social cognition reveals that people give more weight to their own thoughts and feelings than to their behavior when forming self-perceptions, but this effect is reversed when forming perceptions of others (Vazire, 2010). Different types of raters may offer unique information about the person being assessed (Borman, 1997). People may behave differently depending on the situation (e.g., at home vs. work; Lawler, 1967).

Other behavioral assessments such as coding from audio or videotapes of critical incidents or simulations may be considered “pure” behavioral measures, but even these measures require people to code them. In the coding, observers are engaged in subjective perceptions and labeling. In such qualitative research, the scholars increase confidence in the data reported by assessing inter-rater reliability. In 360° assessments, greater confidence in the data is developed from a consensual perception of multiple raters. In EI studies, both types of measures attempt to assess how a person has been acting as seen by others (i.e., a behavioral approach to measurement of EI).

A number of studies show that there are differences among boss’s, peers’ and subordinates’ views, and sometimes even others like consultants, customers or clients. Atkins and Wood (2002) claimed specific types of raters were best positioned to observe and evaluate certain types of competencies depending on the personal and working relationships they had with the person being evaluated. For example, subordinates were found to be the best evaluators of competencies such as coaching and developing people, when compared to bosses or peers (Luthans et al., 1988). Similarly Gralewski and Karwowski (2013) showed how, even though teachers are often accurate at assessing the intelligence and academic achievement of their students (Südkamp et al., 2012), they lack the ability to assess less conventional skill areas, such as students’ creativity. Different sources of raters might interpret the same observed behavior in different ways (Tsui and Ohlott, 1988). At the same time each rater source may have idiosyncratic tendencies leading to different observations and measurement error, like errors of leniency, central tendency, and range restriction (Saal et al., 1980). These are likely to be moderated by cultural assumptions (Ng et al., 2012). The research in assessing performance as well as skills and behavior with 360° assessments is summarized in Bracken et al. (2001). Social identity theory would contend that people find more legitimacy in assessing themselves with regard to those of higher status rather than merely more power (Taylor and Hood, 2010), suggesting that raters from work will be more potent than those from home.

Outside of family business, consulting or family therapy, the sources or raters that have been studied do not include family or friends (Bracken et al., 2001), with the exception of Rivera-Cruz (2004). She reported that female managers showed more EI competencies (as seen by others) at home versus work. In a desire to be comprehensive in assessments, data was collected in this study from a wide range of a person’s relations – those from work and from their personal life (Boyatzis, 2009).

With regard to intelligence, it is expected that professional sources (i.e., sources from work) will have more of an opportunity to see and label behavior related to cognitive ability rather than those at home or in one’s personal life.

This leads us to the third hypothesis for this study:

In self-assessment, an extensive body of literature validated by a recent meta-analysis showed strong evidence of male hubris and female humility: the tendency of males to have inflated views of their abilities, opposite to females’ propensity to under-estimate their worth (Furnham, 2001; Szymanowicz and Furnham, 2011). At the same time, there may be a gender bias in the type of g measures themselves as Furnham (2001) proposes that results may be based on the fact that most of these measures are “male normative”. That is, they include specific tasks, such as spatial processing or mathematical reasoning at which males have been shown to do better than females.

As to others’ ratings of EI competencies, stereotyping will likely affect peers perceptions of males versus females, even in the same setting (Taylor and Hood, 2010). Social identity theory, along with social comparison theory and self-categorization theory are expected to result in attributions made to females differently than those made to males even if their behavior was the same (Sturm et al., 2014). For example, Taylor and Hood (2010) reports that even though female MBAs appear to be more assertive and self-confident than other female samples, sexist bias in perception results in males being seen as more assertive and confident than females. However they did find that predicted ratings of others showed a gender difference: “women leaders believed that others would rate them lower than the actual ratings they received” (p. 542).

In light of these findings, we propose females may be subject to sexist discrimination in their multi-source assessments, particularly those from raters at work. This suggests there may be an interaction of both gender and rater in the relationship between EI and g.

This leads us to the fourth hypothesis for this study:

We chose to analyze the data and test our hypotheses by specifying a Bayesian hierarchical model. The choice to work with a Bayesian model was due to two main factors: (1) the sample was an entire population in and by itself; and (2) it was not a random sample. These issues pose problems in many statistical analyses because traditional frequentist methods are based upon the assumption that the data are created by a repeatable stochastic mechanism. While mainstream statistics treat the observable data as random and the unknown parameters of the population are assumed fixed and unchanging, in the Bayesian view, it is the observed variables that are seen as fixed whereas the unknown parameters are assumed to vary randomly according to a probability distribution. Therefore, in Bayesian models, the parameters of the population are no longer treated as fixed and unchanging as a frequentist approach would assume⁴.

In sum, the main advantages of the Bayesian approach are twofold: (1) it enables highly flexible model specifications (as the one needed to account for the hierarchical structure of our data); and (2) is more appropriate for settings where the data is not a random sample, but the entire population. In addition, it offers a clear and intuitive way to present results. For example, it appears more intuitive by generating probability statements about the findings (for more readings on the advantages of Bayesian inference, check the introductory chapters of Gill, 2002; Gelman et al., 2003; Jackman, 2009).

To best accommodate the structure of our data, we used a multilevel or hierarchical model non–nested structure (by competency and rater group). Equation 1 below represents our model specification, which assumes a linear association between GMAT and ESCI-U scores.

The i subscript refers to the individual, the c subscript refers to the competency and the r subscript refers to the rater group (self, personal or professional). The intercept, α_c,r, varies by competency and rater group. The parameters that account for the ESCI-U effect, θ_c,r, have a hyper-parameter⁵, Θ_r,t, that varies by rater group and by type of competency (i.e., cognitive or emotional).

Additionally, the model includes gender as a source of variation, with coefficient β_r varying by group of raters. The moderator effect of gender on the association between ESCI-U and GMAT is also specified, an interaction that is parameterized as δ_c,r – varying by competency category and rater group, with hyper-prior specification that depends on the type of competency.

In total, there are six main parameters of interest to be estimated, which are compared regarding the type of competency (cognitive or emotional) and the rater group. Estimating a model like the one above is not possible using “canned” procedures from mainstream statistical packages. This confounds the other seemingly inappropriate assumptions from frequentist approaches based on maximum likelihood. One technical solution is to use Bayesian simulation techniques, which allow for highly flexible model specifications⁶.

As mentioned earlier, the main parameters of interest, Θ_r,t, are those that describe the association between GMAT and ESCI-U competencies depending on which type of competency, cognitive or EI, and which of the three groups of raters are considered. A caterpillar plot is shown in Figure 2 with the median of the posterior distribution of each parameter and the 90 and 95 percent credible intervals. The parameters can be interpreted as follows: (a) if the distribution crosses the zero point, there is no consistent relationship of significance; and (b) if the line is to the right or the left of the zero point, then it tells us about the relative impact. For example, in Figure 2, the cognitive competencies assessed by professional sources have a positive relationship to g. The distribution can be said to show that an increase of one unit in the cognitive competencies, as scored by professional raters, is expected to produce an on average increase of around 8.5 units in the GMAT scores. EI and cognitive competencies show no relationship to g with observations from personal sources. Observations from professional sources show a positive relationship between EI and g. Observations from self-assessment show a negative relationship between EI and g. In all three groups of raters the association between GMAT scores and the raters’ evaluation of the cognitive competencies is considerably higher than with the raters’ evaluation of EI competencies. This clearly indicates that GMAT scores are associated in a different way with the ESCI-U scores produced by the three groups of raters. Adding to the main effects mentioned, these results show that the rater group has a moderator effect on the association between ESCI-U and GMAT scores. Therefore we find support for hypothesis 1, strong support for hypothesis 2, and clarity as to the different sources for hypothesis 3.

Figure 2 also shows that others’ ratings of behavior agree more with each other than they do with self-perceptions. This is a well-established result (Atwater and Yammarino, 1992; Carless et al., 1998) that brings further support to our claim that clustering self-report with others’ ratings or 360° based approaches confuses the relationships of EI to different constructs.

Another way to examine these results is by using probability statements, which is one of the advantages of using Bayesian inference. In this sense, the probability that cognitive competencies are more strongly associated with GMAT scores than the EI competencies ranges between 81.5 percent for professional raters, 92.7 for personal raters and 97.8 for self-evaluations. Therefore, the data offers strong evidence for hypotheses 3.

To provide deeper insight into the consistency of the distributions, Figure 3 shows the caterpillar plot of all the 52 θ_c,r parameters, one per each of the 14 ESCI-U competencies, and the three rater groups. As can be seen, the parameters’ distributions are quite consistent within the EI and cognitive types of competencies results shown in Figure 2. The figure can be read as follows, taking as an example the first element of Figure 3: an increase of 1 unit in the competency score of pattern recognition by professional raters is expected to generate an on average increase of about 7.5 in the GMAT score. Yet, regardless of which rater perceptions are considered, cognitive competencies always show higher association with GMAT scores than EI competencies.

Regarding the moderator effects of gender, females showed substantially lower associations between EI and g than males, as shown in Figure 4. In fact, it is negative for observations from each of the self and professional observers and non-significant for personal observers for females. Meanwhile, there is a positive relationship between EI and g for males as viewed from professional observers. Although varying in intensity, for all sources for both EI and cognitive competencies, males show a stronger relationship to g than females. Regarding cognitive competencies, the relationship to g is stronger for males than females from all sources. This provides further support for hypotheses 3 and clarifies why hypothesis 4 is important.

The results suggest that research on EI should examine at more than one level within studies, the ability, trait, self-perception or behavioral levels. It may help in understanding the relevance of EI to life and work outcomes, as well as other constructs in psychology. They also suggest that research on EI should include measures of g to show the unique variance contributed by each concept and show the relative power of each. When collecting behavioral EI data, these results suggest that analyses should examine the sources of the observations as a possible moderator or mediator on the dependent variables. For example in this research, it is likely that the professional environment provides more opportunities for the raters to assess g-related competencies than the personal environment. It is also crucial to analyze data for gender effects that may not be apparent in more direct, statistical analysis.

Professionals using 360° assessments to coach or develop EI should be prepared to identify systemic differences across gender and rater types. Otherwise, individuals may leave their coaching session thinking they have an actual “problem” with certain raters, when in reality it is a systematic bias shared across the population.

One of the limitations of this study emerges because the data came from a single school with diverse nationalities. As such, it threatens external validity. The study should be replicated in other schools to insure that a specific school’s selection and admissions criteria have not biased results.

By focusing on MBA students, we also threatened construct validity. Social desirability is one of the most common validity threats associated with the use of questionnaires in this postgraduate population. Raters provided by the individual rated might create a halo effect, an overall positive feeling leading to inflate their perception of how often desirable behaviors are present. Specially, self-assessment is often misguided for this overall positive feeling about oneself, or because being competent is desirable, thus increased positive self-assessment tends to occur. Future research should address this issue as well.