BRIEF RESEARCH REPORT article
Factorial Validity and Measurement Invariance of the Slovene Version of the Cultural Intelligence Scale
- 1Department of Psychology, Faculty of Arts, University of Ljubljana, Ljubljana, Slovenia
- 2Department of Leadership Studies, Indiana Wesleyan University, Marion, OH, United States
This study examined the factorial validity of the Slovene version of the cultural intelligence scale (CQS) in a representative sample of 1,000 Slovenian participants (49% were female). The results of confirmatory factor analysis supported the factorial validity of the Slovene CQS and the existence of a general (second-order) cultural intelligence factor. The four scales and the overall (general) CQS scale showed satisfactory internal consistency. The results of multiple-group confirmatory factor analyses supported the hypotheses of partial measurement invariance across gender, and full measurement invariance across type of settlement (urban vs. rural).
Cultural intelligence (CQ) refers to the capability to adapt effectively to new cultural contexts (Earley and Ang, 2003). It relates to effectiveness in situations of cultural plurality (Ang and Van Dyne, 2008) in which individuals with high cultural intelligence are culturally competent and have a broad repertoire of (meta)cognitive, behavioral and motivational abilities that enable them to work effectively with members of different cultures and adapt to foreign environments.
The cultural intelligence scale (CQS) (Ang et al., 2007) is the most widely used self-report instrument for measuring the four dimensions of cultural intelligence (i.e., metacognitive, cognitive, motivational, and behavioral; the CQS items in English are provided in Supplementary Table 1). Van Dyne et al. (2008) showed that the structure of the CQS is stable over time, across multiple samples, and various countries. Although Ang et al. discussed an overall CQ as an aggregate multidimensional construct, none of the later validation studies that reported moderately high to high correlations between the scales (e.g., Ward et al., 2009; Imai and Gelfand, 2010) examined a second-order model with a general CQ factor. However, this second-order model was tested in a study on cross-border leadership effectiveness by Rockstuhl et al. (2011), but they only reported the (adequate) fit statistics of the second-order model and did not provide any information about the relationships between this (general) second-order and the four first-order factors. Regarding the criterion (concurrent and predictive) validity of the CQS, numerous and diverse studies have consistently showed that the CQS scale-scores represent important individual- and intercultural team-level predictors of various psychological, behavioral, and performance outcomes (for an overview see Leung et al., 2014; Ott and Michailova, 2018).
The CQS has already been used in Slovenia in the context of the links between knowledge and cultural intelligence (Gotnik Urnault, 2014), the role of cultural intelligence for creativity in a culturally diverse environment (Bogilović and Škerlavaj, 2016), and associations between cultural intelligence of Slovene students and their international experiences (Jakovljević et al., 2016). Because previous attempts at Slovene adaptation of the CQS included only (improper) translation and did not examine the psychometric properties of the CQS, the goals of this study were to conduct a more thorough translation, and to assess the factorial validity of the CQS in a representative Slovene sample. We also examined measurement invariance of the CQS across gender and type of settlement (urban vs. rural).
Materials and Methods
The sample included 1,000 participants from Slovenia; 49% were female (Mage = 52.2 years, SDage = 14.7), and 51% were male (Mage = 44.7 years, SDage = 15.9). Their mean age was 48.7 years (SD = 15.8). Approximately half of the participants (50.7%) had an elementary or vocational qualification, 35.4% had a secondary or higher vocational qualification, and 13.8% had a bachelor’s degree or higher. One third (33.3%) of participants came from urban settlements in Slovenia (i.e., densely populated settlements with at least 2,000 inhabitants). Mean age of urban and rural participants was 49.6 years (SD = 15.7) and 48.2 years (SD = 15.7), respectively.
The cultural intelligence scale (Ang et al., 2007) is a 20-item self-report scale measuring metacognitive (α = 0.76; the reported alphas are from Ang et al.), cognitive (α = 0.80), motivational (α = 0.79), and behavioral (α = 0.82) aspects of cultural intelligence on a 7-point Likert scale (1 = strongly disagree, 7 = strongly agree). The items of the CQS in English are provided in Supplementary Table 1.
We used the established systematic approach of a familiarity- and recognisability-driven adaptation of a questionnaire (Malda et al., 2008) when translating the CQS to Slovene. The translation process was done according to expert recommendations (e.g., Hambleton and De Jong, 2003) and thus included several rounds of translation and back translation using two experts in organizational psychology and two in sociology, as well as a bilingual language expert. The translated items were then subjected to focus groups discussions pertaining to the comprehensibility of items and specific characteristics of Slovene citizens’ perceptions regarding cross-cultural interaction. Based on the opinions of 19 participants, we reworded some items to make them more suitable for the Slovene cultural and linguistic environment.
Data was collected by a marketing research agency Ipsos. They used computer-assisted telephone interviewing and employed stratified sampling technique to ensure representativeness of the sample for the Slovene general population.
This study was carried out in accordance with the recommendations of ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics (ESOMAR, 2016). According to these ICC/ESOMAR guidelines, this type of study represents market research and does not require the approval of an ethics committee, as this survey was conducted on participants who are considered healthy and not in the medical system. At the beginning of the telephone calls, the participants were provided with a detailed description of the study. All participants gave spoken informed consent in accordance with the Declaration of Helsinki.
All confirmatory factor analyses (CFAs) were done in Mplus 7.2 (Muthén and Muthén, 1998–2012). We used the robust MLM estimator, because the majority of items had moderately non-normal distributions; the skewness of the items ranged from –0.74 to 0.29, and the kurtosis values ranged from –0.65 to 0.10. For assessing the goodness of fit of the tested models to the data, we used the CFI, TLI, RMSEA, and SRMR fit indexes. The following cut-off values were considered as indicating adequate fit (Hu and Bentler, 1999): CFI and TLI > 0.95, RMSEA < 0.05, and SRMR < 0.08. We carried out a multiple-group CFA to test the measurement invariance of the CQS. We tested the hypotheses regarding equality of factor loadings (i.e., metric invariance) and equality of indicator intercepts (i.e., scalar invariance). Because using large samples can lead to excessively restrictive test of measurement invariance (via the likelihood ratio test), we used a p-value of 0.01 as a criterion for statistical significance, and additionally observed the differences in the CFI between the nested models (ΔCFI larger than 0.002 as indicating lack of measurement invariance; Meade et al., 2008).
Factorial Validity of the Slovene CQS
The fit of the 4-factor model to the data was adequate: χ2(164) = 409.4, p < 0.01, CFI = 0.97, TLI = 0.96, RMSEA = 0.039, SRMR = 0.035. The standardized loadings across all factors ranged from 0.61 to 0.83. Table 1 shows the average standardized loadings, Cronbach’s α coefficients for each of the four factors, and correlations between the CQS factors according to the CFA. Detailed information about the standardized loadings, items’ means and standard deviations can be found in Supplementary Table 1. In sum, the fit indexes and relatively high standardized loadings indicate satisfactory construct (factorial) validity of the Slovene version of the CQS. The CQS scales also showed satisfactory internal consistency.
TABLE 1. Structure of the CQS (average standardized loadings per factor), Cronbach’s alpha coefficients, and correlations between the CQS factors according to the confirmatory factor analysis (N = 1,000).
As can be seen from Table 1, the correlations between the four factors were positive and quite high, which is in accordance with the notion of an overall/general factor of CQ (Ang et al., 2007; Thomas et al., 2008; Rockstuhl et al., 2011). Therefore, we also tested the fit of the second-order factor model by specifying the four first-order factors as indicators of a (general) second-order cultural intelligence factor. The fit of the second-order model was adequate: χ2(166) = 411.2, p < 0.01, CFI = 0.97, TLI = 0.96, RMSEA = 0.038, SRMR = 0.035. The standardized weights (M[λ] = 0.80; Supplementary Table 1) for the general CQS factor were high, and the reliability of the total CQS score was excellent (α = 0.93).
Measurement Invariance of the Slovene CQS Across Gender and Type of Settlement
To test the measurement invariance of the CQS across gender and type of settlement we tested the hypotheses regarding the equality of factor loadings and equality of item intercepts. Regarding gender, the configural models were adequate (Table 2). Imposing the constraints of the metric model (equal loadings) did not statistically significantly change the fit of the 4-factor model, while imposing the constraints of the scalar model (equal item intercepts) resulted in a statistically significant change in overall model fit. To achieve partial scalar invariance, we reviewed the modification indexes pertaining to the item intercepts. We consecutively freed the item intercepts associated with the largest modification indexes (MI) until the change in overall model fit was not statistically significant any more.
TABLE 2. Measurement invariance statistics for gender and type of settlement groups, and factor means (structural) invariance model.
We first freed the intercept of the Item 8 (MI = 9.9; see Supplementary Table 1 for item contents), but the modified scalar model still had a significantly worse fit than the full metric invariance model, Δχ2= 38.8, p = 0.001 (however, ΔCFI was –0.002 after freeing the intercept of Item 8). The second largest MI was associated with Item 5 (MI = 7.3); freeing this intercept still resulted in a significant change of the model fit, Δχ2= 31.5, p = 0.005. After freeing the intercept of Item 17 (MI = 6.0), the change in model fit was no longer statistically significant, Δχ2= 23.0, p = 0.04. To achieve partial scalar invariance, we had to free two item intercepts (out of six) in the cognitive domain [Item 8: (unstandardized) intercept was higher for females (3.56) than males (3.32); Item 5: the intercept was lower for females (3.30) than males (3.52)], and one item intercept (out of five) in the behavioral domain [Item 17: the intercept was higher for females (4.74) than males (4.52)]. In the last step, we also addressed the question whether female and male participants differ in the average level of the CQS factors. This was done by imposing an additional constraint of equal factor means across both gender groups. The change in overall model fit was statistically significant (Table 2), which indicated that females and males differ on at least one CQS factor. The tests of factor mean differences revealed that the females exhibited a slightly higher level of the metacognitive aspect of cultural intelligence (ΔM = 0.23, p < 0.01).
Concerning measurement invariance across type of settlement, imposing the constraints of the metric and scalar models did not result in statistically significant changes in the fit of the 4-factor model (Table 2). Imposing the constraints of the factor means invariance model did not lead to a significant change in overall model fit, which means that the participants form urban and rural type of settlements do not differ on any of the CQS factors.
The results of confirmatory factor analysis supported the factorial validity of the Slovene version of the CQS, and the reliabilities of the four scales were satisfactory. Our results and the results of previous validation studies (e.g., Ang et al., 2007; Van Dyne et al., 2008; Imai and Gelfand, 2010) that also found an adequate fit of the 4-factor model to the data indicate that the factorial structure of the CQS is quite robust. Since the correlations between the factors were moderately high, we tested the fit of the second-order model by adding one higher-order (general) CQ factor; the fit of this model was again adequate. The standardized loadings of the four first-order factors indicate that the metacognitive and cognitive factors of CQ are, in relative terms, the least important indicators of the general CQ, while the motivational factor is the most important indicator of the general CQ factor. We suggest that future research concerning the structure of the CQS (and CQ nomological network in general) should also always test this second-order model and further examine the added value of using the general CQ factor for predicting relevant outcome variables.
By using multiple-group CFAs we found partial measurement invariance across gender and full measurement invariance across type of settlement. To achieve partial scalar invariance across gender, we had to free three item intercepts. Partial scalar invariance means that for the majority of CQS items the observed differences in their means between gender groups were due to factor mean differences only. The most affected was the Cognitive scale (two non-invariant item intercepts out of six), so we suggest caution when comparing the means of women and men on Cognitive CQS scale.
Overall, the results of the present study confirmed the robustness of the four-factor CQS structure and provided additional evidence for a general CQ factor. Further validation studies of the Slovene CQS should especially focus on the predictive validity of the CQS against most relevant criterion variables on individual- and intercultural dyad- or team-level.
EB and RJ conceived the idea for the study. EB carried out all of the tasks for the translation of the CQS to Slovene [coordinating the (back) translations, conducting focus groups] and gathering the data. LK conducted all of the statistical analyses and wrote the manuscript with the support of EB and RJ.
This research was supported by consulting company BRIO Svetovalni Center, Ljubljana, Slovenia.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2018.01499/full#supplementary-material
Ang, S., Van Dyne, L., Koh, C., Ng, K. Y., Templer, K. J., Tay, C., et al. (2007). Cultural intelligence: its measurement and effects on cultural judgment and decision making, cultural adaptation and task performance. Manag. Organ. Rev. 3, 335–371. doi: 10.1111/j.1740-8784.2007.00082.x
Ang, S., and Van Dyne, L. (2008). ”Conceptualization of cultural intelligence: definition, distinctiveness, and nomological network,” in Handbook of Cultural Intelligence: Theory, Measurement, and Application, eds S. Ang, and L. Van Dyne (Armonk, NY: M. E. Sharpe,Inc), 3–15.
ESOMAR (2016). ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics. Available at: https://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ICCESOMAR_Code_English_.pdf
Hu, L., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. 6, 1–55. doi: 10.1080/10705519909540118
Imai, L., and Gelfand, M. J. (2010). The culturally intelligent negotiator: the impact of cultural intelligence (CQ) on negotiation sequences and outcomes. Organ. Behav. Hum. Decis. Process. 112, 83–98. doi: 10.1016/j.obhdp.2010.02.001
Jakovljević, I., Kodrič, A., Prelog, N., and Tumpej, V. (2016). “Cultural intelligence: results of the first study on Slovenian students,” in Poster at the conference Working abroad – an opportunity or a risk? Department of Psychology, Ljubljana.
Malda, M., van de Vijver, F. J., Srinivasan, K., Transler, C., Sukumar, P., and Rao, K. (2008). Adapting a cognitive test for a different culture: an illustration of qualitative procedures. Psychol. Sci. 50, 451–468.
Meade, A. W., Johnson, E. C., and Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. J. Appl. Psychol. 93, 568–592. doi: 10.1037/0021-9010.93.3.568
Rockstuhl, T., Seiler, S., Ang, S., Van Dyne, L., and Annen, H. (2011). Beyond general intelligence (IQ) and emotional intelligence (EQ): the role of cultural intelligence (CQ) on cross-border leadership effectiveness in a globalized world. J. Soc. Issues 67, 825–840. doi: 10.1111/j.1540-4560.2011.01730.x
Thomas, D. C., Elron, E., Stahl, G., Ekelund, B. Z., Ravlin, E. C., Cerdin, J. L., et al. (2008). Cultural intelligence: domain and assessment. Int. J. Cross Cult. Manag. 8, 123–143. doi: 10.1177/1470595808091787
Van Dyne, L., Ang, S., and Koh, C. (2008). “Development and validation of the CQS,” in Handbook of Cultural Intelligence: Theory, Measurement, and Applications, eds S. Ang, and L. Van Dyne (New York, NY: Sharpe), 16–38.
Keywords: cultural intelligence scale, cultural intelligence, factorial validity, measurement invariance, general cultural intelligence factor
Citation: Boštjančič E, Komidar L and Johnson RB (2018) Factorial Validity and Measurement Invariance of the Slovene Version of the Cultural Intelligence Scale. Front. Psychol. 9:1499. doi: 10.3389/fpsyg.2018.01499
Received: 05 February 2018; Accepted: 30 July 2018;
Published: 21 August 2018.
Edited by:Maicon Rodrigues Albuquerque, Universidade Federal de Minas Gerais, Brazil
Reviewed by:Hugo Carretero-Dios, Universidad de Granada, Spain
Gabriele Giorgi, Università Europea di Roma, Italy
Copyright © 2018 Boštjančič, Komidar and Johnson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Luka Komidar, firstname.lastname@example.org