Original Research ARTICLE
Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches' interpersonal styles
- 1Department of Psychology, Umeå University, Umeå, Sweden
- 2Center of Research on Welfare, Health and Sport (CVHI), Halmstad University, Halmstad, Sweden
- 3Faculty of Health, Research Institute for Sport and Exercise, University of Canberra, Canberra, ACT, Australia
- 4Department of Psychology, University of Gothenburg, Gothenburg, Sweden
- 5Department of Food and Nutrition, and Sport Science, University of Gothenburg, Gothenburg, Sweden
In the present work we investigated distinct sources of construct-relevant psychometric multidimensionality in two sport-specific measures of coaches' need-supportive (ISS-C) and controlling interpersonal (CCBS) styles. A recently proposed bifactor exploratory structural equation modeling (ESEM) framework was employed to achieve this aim. In Study 1, using a sample of floorball players, the results indicated that the ISS-C can be considered as a unidimensional measure, with one global factor explaining most of the variance in the items. In Study 2, using a sample of male ice hockey players, the results indicated that the items in the CCBS are represented by both a general factor and specific factors, but the subscales differ with regard to the amount of variance in the items accounted for by the general and specific factors. These results add further insight into the psychometric properties of these two measures and the dimensionality of these two constructs.
Coaches' interpersonal styles strongly influence athletes' need satisfaction and motivation in competitive sports (Mageau and Vallerand, 2003). Whereas a need-supportive interpersonal style generally has a positive influence on athletes' motivation, well-being, and performance (Hagger and Chatzisarantis, 2007), a controlling interpersonal style has instead been related to maladaptive outcomes, such as burnout, depression, and disordered eating (Bartholomew et al., 2011). These interpersonal styles are multidimensional constructs, each consisting of theoretically distinguishable subdimensions. Specifically, a need-supportive interpersonal style reflects a global construct as well as three specific subdimensions: autonomy support, structure, and involvement. Similarly, a controlling interpersonal style reflects a global construct and four specific subdimensions: controlling use of rewards, negative conditional regard, intimidation, and excessive personal control. Hence, two distinct sources of psychometric multidimensionality exist: a global factor and specific subdimensions. A comprehensive test of the multidimensional structure thereby requires consideration of both sources (Morin et al., 2015).
The subdimensions of need-support measures (e.g., related to parents, coaches, or exercise practitioners) are suggested to be highly interrelated, and these measures are often treated as unidimensional (e.g., Ryan, 1991; Niemiec et al., 2006; Markland and Tobin, 2010). Given the multidimensional nature of a need-supportive interpersonal style (Deci and Ryan, 2000; Mageau and Vallerand, 2003), as well as recent suggestions that more attention should be given to these various need-supportive interpersonal styles within physical activity settings (e.g., Pope and Wilson, 2012; Standage, 2012), an investigation of the multidimensionality in measures of this construct within the sports context seems warranted. The interpersonal supportiveness scale-coach (ISS-C; Wilson et al., 2009) represents one of the few attempts to develop a multidimensional—autonomy support, structure, and involvement—sport-specific measure in line with the tenets of the self-determination theory (SDT; Deci and Ryan, 2000; Mageau and Vallerand, 2003). One concern has been, however, the relatively high degree of overlap (i.e., common variance) between the three dimensions (Wilson et al., 2009). This concern in turn has raised questions related to the instrument's dimensionality and whether global and/or specific factors are captured by the ISS-C.
When examining the factorial structure of the controlling coach behaviors scale (CCBS), which assesses athletes' perceptions of their coach's controlling interpersonal style from the perspective of SDT (Deci and Ryan, 1985, 2000), Bartholomew et al. (2010) concluded “that a controlling interpersonal style is a multidimensional construct represented by a number of separate, but related, controlling coaching strategies” (p. 205). This conclusion was reached after inspecting and comparing model fit between a four-factor first-order independent cluster model (ICM) confirmatory factor analysis (CFA), a second-order ICM-CFA with four first-order factors, and a one-factor first-order model. This approach to some extent examined multidimensionality; however, it did not identify distinct sources of construct-relevant psychometric multidimensionality in terms of simultaneous estimation of global and specific constructs, as proposed by Morin et al. (2015). Although other studies have examined the psychometric properties of CCBS (e.g., Castillo et al., 2014), a comprehensive test of the structure of the multidimensionality, taking both the global construct and specific subdimensions into consideration, has not yet been performed.
The present studies contribute to the existing literature by investing distinct sources of construct-relevant psychometric multidimensionality in these two sport-specific measures of coaches' need-supportive (ISS-C; Wilson et al., 2009) and controlling interpersonal (CCBS; Bartholomew et al., 2010) styles.
Need-supportive Interpersonal Style
Previous research examining coaches' interpersonal styles in competitive sports contexts (and other contexts—see Ng et al., 2012) have primarily focused on the effects of perceived autonomy support (e.g., Adie et al., 2012; Stenling et al., 2015) and the perceived motivational climate (e.g., Sarrazin et al., 2002; Reinboth and Duda, 2006). Mageau and Vallerand (2003), however, argued that, in addition to autonomy support, coaching behaviors that provide structure and involvement are also important determinants for the satisfaction of athletes' psychological needs and their behavioral regulations. Coaches who provide autonomy support try to understand an athlete's perspective, acknowledge the athlete's feelings, encourage exploration, and curiosity, provide a meaningful rationale, and provide opportunities for choice (Mageau and Vallerand, 2003). Structure involves providing clear and understandable guidelines and expectations, instilling a sense of competence in the athletes, and providing relevant feedback to the athletes (Reeve and Su, 2014). Involvement is displayed when coaches show a genuine interest in their athletes and their well-being and spend a considerable amount of time, energy, and resources on them (Grolnick and Ryan, 1989). Need-supportive environments providing autonomy support, structure, and involvement have been examined in physical education settings (Standage et al., 2005; Haerens et al., 2013) and exercise settings (e.g., Markland and Tobin, 2010), but the potential role of perceived structure and involvement from coaches in sports is still largely unexplored (Mageau and Vallerand, 2003; Pope and Wilson, 2012). Ryan (1991) and others (e.g., Niemiec et al., 2006; Markland and Tobin, 2010) have argued the three support dimensions are highly interrelated; they are therefore often combined into a broader category labeled need support.
Controlling Interpersonal Style
Controlling coaches may actively thwart athletes' basic psychological need satisfaction, and this has been linked to ill-being and stress responses (Bartholomew et al., 2011; Taylor et al., 2015). Based on an extensive literature review, Bartholomew et al. (2009, 2010) identified a number of controlling motivational strategies. The most prominent was controlling use of rewards, which refers to the use of extrinsic rewards and praise to ensure athlete compliance, engagement, and persistence in certain behaviors. This controlling strategy is closely related to the undermining effect of rewards on intrinsic motivation (Deci et al., 1999; Vansteenkiste and Deci, 2003), which refers to the negative effect of tangible rewards (task and performance contingent) on intrinsic motivation, particularly when the reward is expected. Similar effects can also be produced by verbal rewards and praise (Henderlong and Lepper, 2002).
When athletes do not display the desired attributes or behaviors, coaches may withhold attention and affection, labeled negative conditional regard (Assor et al., 2004). Conditional regard from coaches is displayed when their attention and acceptance is highly contingent upon athletes showing appropriate thoughts and behaviors, which oftentimes forces the athletes to give up their autonomy to maintain a satisfactory relationship with their coach (Bartholomew et al., 2010).
An abusive power-based controlling motivational strategy is intimidation, used to belittle and humiliate through verbal abuse, threats, and yelling to control athletes' behaviors and promote external regulation (Bartholomew et al., 2010). Such controlling strategies create pressure from the outside, which promotes athletes to engage in certain behaviors to avoid external punishment (Deci and Ryan, 1987).
Finally, excessive personal control is displayed when coaches engage in intrusive monitoring of athletes' free time and impose strict limits (Bartholomew et al., 2010). Examples of such coach behaviors involve restricting athletes' free time (e.g., setting curfews) or engagement in other sports. This type of controlling interpersonal behavior promotes a sense of pressure from the coach to prioritize one's sports involvement over other important aspects of the athlete's life.
The Present Studies
The two instruments under scrutiny here, the ISS-C (Wilson et al., 2009) and the CCBS (Bartholomew et al., 2010), contain three and four subscales, respectively, and can both be described in terms of a broader construct as well as more specific dimensions or subdomains within that broader construct. Hence, both these instruments are suited for a comprehensive examination of these two sources of construct-relevant variance. One particularly useful approach for this aim, which recently has been rediscovered within psychology research, is the bifactor measurement model (Reise, 2012). The bifactor measurement model originates from the early work by Holzinger and Swineford (1937) but has for a long time been overshadowed by Thurstone's (1947) correlated-factor model. The bifactor model has not only been rediscovered but also extended within an ESEM framework (Jennrich and Bentler, 2011, 2012; Myers et al., 2014; Morin et al., 2015).
Theory-based multidimensional scales, such as the ISS-C and CCBS, often correspond to a bifactor structure with a general latent construct alongside several latent subdimensions that are more narrowly defined (Myers et al., 2014). Applications of bifactor measurement models, however, are rare in sport and exercise psychology research, despite the fact that a bifactor model often provides researchers with an opportunity to match the theory behind the instrument development with the model imposed on the data when evaluating multidimensional scales. This match between theory and model may be lacking when the commonly used correlated first-order model or a second-order factor model is specified because neither of these two models takes the general latent constructs' direct influence on items into account (Myers et al., 2014).
In addition to the rediscovered bifactor measurement model (Reise, 2012), the recently developed ESEM allows researchers to deal with a common problem with ICM-CFA, namely the fallible nature of indicators (Morin et al., 2015). Items incorporate a part of random measurement error, also known as item uniqueness, but items also tend to have some degree of systematic association with other constructs. Such systematic association is typically expressed as cross-loadings in exploratory factor analysis but is constrained to zero in ICM-CFA. In the context of theory-driven multidimensional scales, this assumption of zero cross-loadings might be unrealistically restrictive and, thus, lead to extensive bias in factor correlations and poor model fit (e.g., Asparouhov and Muthén, 2009; Marsh et al., 2014). Multidimensional measures typically include cross-loadings that can be justified by substantive theory or item content (Asparouhov and Muthén, 2009); most items are likely to be imperfect to some degree and have some systematic association with other constructs (Morin et al., 2015).
The recent incorporation of bifactor models and ESEM provides researchers with an opportunity to investigate two sources of “construct-relevant psychometric multidimensionality related to: (a) the hierarchical nature of the constructs being assessed (i.e., the co-existence of global and specific components within the same measurement model) and (b) the fallible nature of indicators which tend to include at least some degree of association with non-target constructs” (Morin et al., 2015, p. 30). According to Morin and colleagues, bifactor models are needed to investigate the first source, whereas the second source calls for ESEM rather than CFA. Furthermore, by estimating ESEM with target rotation, it is possible to specify a priori hypotheses about the factor structure and use ESEM for confirmatory purposes (Asparouhov and Muthén, 2009).
The purpose of the two studies outlined in this report was to apply the recently proposed bifactor ESEM framework (Morin et al., 2015) and investigate distinct sources of construct-relevant psychometric multidimensionality in two sport-specific measures of coaches need-supportive and controlling behaviors: the ISS-C (Wilson et al., 2009) and the CCBS (Bartholomew et al., 2010). Because these two instruments are expected to consist of a general latent factor alongside several narrowly defined subdimensions, we hypothesized that the first-order ESEM would provide a better fit to the data compared to the first-order ICM-CFA and that the bifactor ESEM would provide a better fit to the data compared to first-order ESEM.
Study 1—Interpersonal Supportiveness Scale–Coach
Materials and Methods
The sample comprised 277 (142 female, 135 male) floorball players competing for clubs in northern Sweden. The athletes' ages ranged from 15 to 22 years (M = 16.8, SD = 1.1) and their competitive levels from regional to international. On average, they practiced floorball 6.8 h (SD = 3.4) per week and had been competing in their sport for 8.4 years (SD = 2.8).
We used a Swedish version of the Interpersonal Supportiveness Scale–Coach (ISS–C; Wilson et al., 2009) to capture athletes' perceptions of their coach's autonomy support (six items, e.g., “My coach provides me with choices and options”), provision of structure (six items, e.g., “My coach provides clear feedback about my progress”), and involvement (six items, e.g., “My coach puts time and energy into helping me”). Responses to the 18 items were given on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). Internal consistency (omega coefficient; McDonald, 1999) of the three subscales was as follows: autonomy support = 0.860, structure = 0.902, involvement = 0.781.
Data were analyzed with Mplus version 7.3 (Muthén and Muthén, 1998–2012) and the robust maximum likelihood estimator (MLR). MLR provides standard errors and fit indexes that are robust to the Likert nature of the items and non-normality. The items were treated as continuous variables. In a recent simulation study Rhemtulla et al. (2012) showed that the robust maximum likelihood estimator performs equally well or better compared to robust categorical estimators, particularly with seven response categories. A small percentage of missing data was present in the items (< 2.2%). We choose to include all available information and used the full information robust maximum likelihood (FIML) estimation to handle the missing data (Enders, 2010).
We used a model testing procedure proposed by Morin et al. (2015). This procedure allowed us to investigate two sources of construct-relevant psychometric multidimensionality related to the co-existence of global and specific components within the same measurement model and the fallible nature of indicators which tend to include at least some degree of association with non-target constructs. We started by specifying and comparing first-order ICM–CFA with first-order ESEM models to examine the presence of cross-loadings of conceptually related or overlapping constructs. Based on the results in the first step (ICM–CFA vs. ESEM), the second step aimed to identify the presence of construct-relevant multidimensionality due to the presence of hierarchically superior constructs using bifactor models. The ICM–CFA, ESEM, and bifactor ESEM models are graphically depicted in Figure 11.
Figure 1. Graphical representation of the alternative models tested in these two studies. The top three are the ICM-CFA, first-order ESEM, and bifactor ESEM of the ISS-C. Bottom three are the ICM-CFA, first-order ESEM, and bifactor ESEM of the CCBS. Dotted lines represent non-target loadings. CUR, controlling use of rewards; NCR, negative conditional regard; INT, intimidation; and ECP, excessive personal control.
Conventional fit indices were used to evaluate the model fit in the ICM-CFA and ESEM models, such as the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the standardized root mean residual (SRMR), and the root mean square error of approximation (RMSEA). Traditional cutoff criteria with CFI and TLI values around 0.90 and SRMR and RMSEA values around 0.08 were used to indicate acceptable fit (Marsh, 2007). Target rotation was used in the ESEM models, which allows for the specification of target and non-target factor loadings in a confirmatory manner (Browne, 2001; Asparouhov and Muthén, 2009). All cross-loadings were specified to be close to zero, while all the main loadings were freely estimated (Morin et al., 2015). In line with bifactor assumptions that the specific factors explains item variance not explained by the general factor and that the general factor explains variance that is shared across all items, the general and specific factors were orthogonal in order to ensure interpretability (Chen et al., 2006; Reise, 2012). The first-order ESEM models were estimated with an oblique target rotation (Browne, 2001; Asparouhov and Muthén, 2009).
We followed the guidelines for nested model comparisons in small samples (< 300) by Chen (2007), who suggested that a change in CFI (ΔCFI) ≥ −0.005 accompanied by a change in RMSEA (ΔRMSEA) of ≥ 0.010 would support the simpler model over the more complex model. In addition, Marsh et al. (2010) suggested that some indices (e.g., TLI and RMSEA) penalize for parsimony so that a more parsimonious model can fit the data better than a less parsimonious model. Therefore, a more conservative approach is to consider the more parsimonious model as supported, if the TLI or RMSEA is as good or better compared to the more complex model. We also used several information criteria when comparing alternative models: the Akaike Information Criterion (AIC; Akaike, 1987), the Bayesian Information Criterion (BIC; Schwartz, 1978), and the sample-size adjusted BIC (ABIC; Sclove, 1987). These information criteria do not in themselves describe model fit, but a model with a lower value indicates a better fitting model compared to a model with a higher value when alternative models are compared. Note, however, that all of these guidelines for model fit have been established for CFA, and more work is needed regarding their generalizability outside of the CFA framework. Regardless of the framework these guidelines are applied within (e.g., CFA, ESEM), they should not be taken as “golden rules” but rather as rough guidelines used in combination with parameter estimates, statistical conformity, and theoretical adequacy (Marsh et al., 2004; Morin et al., 2015).
The initial contact was made with the head coach of each team, and when permission was granted to approach the athletes, a time, and place were scheduled for an informational meeting with them. During this meeting, a research assistant invited the athletes to participate in the study. Upon agreement to participate, the athletes provided written informed consent and responded to a multi-section questionnaire that took approximately 20 min to complete. Data was collected at midseason to ensure that the athletes had had enough time to establish a perception of their coach's interpersonal style. Prior to the data collection, ethical approval was obtained from the Regional Ethical Review Board at the first author's university.
Item correlations, means, standard deviations, skewness, and kurtosis are displayed in Table 1. As seen in Table 2, the first-order ICM–CFA model displayed an acceptable fit to the data. However, the first-order ESEM model displayed a better representation of the data as indicated by the ΔCFI = 0.049, ΔRMSEA = 0.018, higher TLI and lower RMSEA values, and lower AIC, BIC, and ABIC values. The ICM–CFA model displayed large factor correlations between the three factors (0.956–0.992), but these factor correlations were lower in the first-order ESEM model (0.473–0.708; see Table 3). The first-order ESEM model also revealed several cross-loading items, with relatively high loadings on non-target factors. Taken together, the ESEM model provided a better representation of the data compared to the ICM–CFA and was therefore retained in the second step when estimating the bifactor model. As displayed in Table 2, the bifactor ESEM model provided an excellent fit to the data and had lower AIC and ABIC compared to the first-order ESEM. The factor loading pattern from the bifactor ESEM model is displayed in Table 4, and almost all items had a strong standardized factor loading on the general factor and a weak loading on the specific factor. These results indicate that most of the variance in the items in the ISS-C was accounted for by the general factor.
The comparison between the first-order ESEM and the bifactor ESEM indicated that the latter provided a better fit to the data. However, the first-order ESEM indicated a substantial degree of cross-loading items, and the bifactor ESEM displayed that most of the variance in the ISS–C items was accounted for by the general factor. This indicates that the ISS–C items were captured by the general construct need support and not by the specific subdimensions autonomy support, structure, and involvement. These results are similar to previous multidimensional leadership scales, for example, within transformational leadership research (e.g., van Knippenberg and Sitkin, 2013), where the subdimensions often are highly interrelated. A high degree of overlap between subdimensions within multidimensional SDT need-support scales is quite common (e.g., Niemiec et al., 2006; Wilson et al., 2009) and the three need-support dimensions are often suggested to be highly interrelated (Ryan, 1991; Markland and Tobin, 2010). Much SDT research has therefore treated need support as a unidimensional variable (e.g., only using autonomy support), and have not acknowledged the multidimensional nature of need support as consisting of autonomy support, structure, and involvement. The high correlations between the need-support dimensions could be explained by the fact that coaches who are need supportive with regard to one dimension (e.g., autonomy support) also are need supportive with regard to the other dimensions. Such an explanation has previously been suggested, for example, within transformational leadership research, where high factor correlations are commonly observed in research with self-report measures (Barling et al., 2011). Another explanation is related to the use of self-report measures and that the respondents may not be able to distinguish between items intended to capture these various dimensions of leadership. A remedy for this type of common method bias could be to collect other types of data (Podsakoff et al., 2012), such as observational data of coach behaviors in various situations. SDT-based observations (e.g., Haerens et al., 2013; Smith et al., 2015) could be a useful source for data collection to avoid common method bias when combined with other data sources (e.g., self-report).
Having measures that can discriminate between subdimensions in multidimensional constructs is important because this would allow researchers to explore whether these leadership dimensions are additive (i.e., the more, the better), which is assumed when sum scores of need support are used or if their relationships have other forms (van Knippenberg and Sitkin, 2013). For example, instead of assuming an additive effect, we could specify a minimum value on each of these dimensions that determines when someone can be characterized as need supportive. The need-support dimensions could also be interactive, indicating that the engagement in one type of need support would make the other types more effective. This type of moderating effect has been found in educational research (Jang et al., 2010) where autonomy support and structure were two interacting engagement-fostering interpersonal styles among teachers associated with students' classroom engagement. A third possibility could be that one need-support dimension can compensate for the lack of another (van Knippenberg and Sitkin, 2013). The evidence so far suggests that the three need-support dimensions are interactive, in that structure, and involvement will be enhanced when provided in an autonomy-supportive way (Reeve and Su, 2014). In order to investigate these various hypotheses, it is imperative that the three need-support dimensions are distinguishable. The multidimensionality in need-support instruments is an important avenue of future research.
Study 2—Coaches Controlling Behaviors Scale
Materials and Methods
This sample comprised 233 male ice hockey players competing for clubs in northern Sweden. Their ages ranged from 15 to 20 years (M = 17.1, SD = 1.4) and their competitive levels from regional to international. On average, they had been competing in their sport for 10.6 years (SD = 2.1).
A Swedish version of the Controlling Coach Behaviors Scale (CCBS; Bartholomew et al., 2010) was used to measure the athletes' perceptions of their coach's controlling interpersonal style. The CCBS consists of four subscales capturing athletes' perceptions of coaches' controlling use of rewards (four items, e.g., My coach tries to motivate me by promising to reward me if I do well), negative conditional regard (four items, e.g., My coach is less supportive of me when I am not training and competing well), intimidation (four items, e.g., My coach intimidates me into doing the things that he/she wants me to do), and excessive personal control (three items, e.g., My coach tries to interfere in aspects of my life outside of my sport). Responses to the 15 items were given on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). Internal consistency (omega coefficient; McDonald, 1999) of the four subscales was as follows: controlling use of rewards = 0.723, negative conditional regard = 0.865, intimidation = 0.719, and excessive personal control = 0.706.
The same statistical analyses as in Study 1 were conducted in Study 2.
The same procedure as in Study 1 was used in Study 2.
Item correlations, means, standard deviations, skewness, and kurtosis are displayed in Table 5. The first-order ICM–CFA displayed an acceptable fit to the data (Table 2), but also for the CCBS the first-order ESEM model provided a better representation of the data as indicated by the ΔCFI = 0.033, higher TLI value, and lower AIC and ABIC values. The ΔRMSEA did not reach Chen's (2007) recommendation of 0.010 but was lower for the ESEM model, and taken together, we concluded that the ESEM model provided a better representation of the data. As seen in Table 6, the factor correlations were slightly higher in the ICM–CFA model (0.149–0.597) compared to the first-order ESEM model (0.131–0.476). No substantive cross-loadings were observed in the first-order ESEM, the largest cross-loading was −0.26 (standardized). As seen in Table 2, the bifactor ESEM displayed an excellent fit to the data and had lower AIC and ABIC compared to the first-order ESEM. The factor loading pattern from the bifactor ESEM model is displayed in Table 7. The factor loadings of items in two of the specific factors, negative conditional regard and excessive personal control, indicate that the items in both these factors are explained by the general factor as well as their specific factors. Items on the subscale controlling use of rewards displayed relatively weak loadings on the general factor and strong loadings on the specific factor, whereas items on the subscale intimidation displayed an opposite pattern with stronger loadings in the general factor and weak loadings on the specific factor. These results indicate that the items in the CCBS are represented by a general factor and specific factors but that the subscales differ with regard to the amount of variance accounted for by the general and specific factors.
Also for the CCBS, the bifactor ESEM provided a better fit to the data compared to the first-order ESEM. The bifactor ESEM provided an interesting pattern of multidimensionality, indicating that the CCBS consists of a general factor as well as specific factors but that the subdimensions differed with regard to the amount of variance accounted for by the general and specific factors. Researchers have previously concluded that the CCBS is a multidimensional instrument using first- and second-order ICM–CFAs (e.g., Bartholomew et al., 2010; Castillo et al., 2014). Although these previous studies, to some extent, examined multidimensionality, they did not examine distinct sources of multidimensionality in terms of the simultaneous estimation of global and specific factors. By using a bifactor ESEM approach (Morin et al., 2015), we extended previous research on the factor structure of the CCBS with a statistical method that matches the theory underlying the development of the CCBS with the model imposed on the data (Myers et al., 2014).
The bifactor structure for the CCBS indicated that the negative conditional regard factor and the excessive personal control factor have a bifactor pattern with items loading relatively strong onto both the general and the specific factors. These two subdimensions seem to consist of two sources of construct-relevant multidimensionality (Morin et al., 2015). Items on the factor controlling use of rewards had relatively weak loadings on the general factor compared to the other factors (range 0.181–0.278) and relatively strong loadings on its specific factor (range 0.440–0.830). This pattern was also seen in the first-order ESEM with relatively low factor correlations between the controlling use of rewards factor and the other factors (range 0.131–0.394). These results indicate that the controlling use of rewards factor may represent a slightly different aspect of coaches' controlling behaviors compared to the other three specific factors that to a larger degree seem to have a common core, as represented by the general factor. Research looking at the different CCBS factors' relationship to other variables is scarce, but controlling use of rewards has, for example, shown a weaker relationship to autonomy support compared to the other three factors in the CCBS (Bartholomew et al., 2010). Whether the factors in the CCBS predict outcomes differently is an interesting question for future research that would provide more insight into similarities and differences between the CCBS factors. Finally, the items on the intimidation factor displayed relatively strong loadings onto the general factor and weak loadings onto its specific factor. Hence, this factor is mostly explained by one source of construct-relevant multidimensionality, the general factor.
Furthermore, three items displayed meaningful cross-loadings (factor loadings > 0.20) onto the negative conditional regard factor: CUR_1, My coach tries to motivate me by promising to reward me if I do well, CUR_4, My coach only uses rewards/praise so that I complete all the tasks he/she sets in training, EPC_1, My coach expects my whole life to center on my sport participation. Future research should investigate these items further and explore whether they need to be revised or if this complexity is theoretically meaningful (cf. Myers et al., 2014). It should be noted, however, that pure items are not a requirement of a well-defined factor structure and that one can argue that it is more important to find an accurate set of items rather than a pure set of items (Asparouhov and Muthén, 2009). Hypotheses about pure factor structures with items loading solely onto their intended factors may in many cases be too restrictive due to the fallible nature of indicators, and the ESEM framework provides researchers with a tool that accounts for items' associations with non-target constructs (Asparouhov and Muthén, 2009; Myers et al., 2014; Morin et al., 2015).
General Discussion and Conclusions
Our purpose was to investigate distinct sources of construct-relevant psychometric multidimensionality in two sport-specific measures of coaches' need-supportive and controlling behaviors, the ISS-C (Wilson et al., 2009) and the CCBS (Bartholomew et al., 2010). We expected that these two multidimensional instruments would consist of a general latent factor alongside several narrowly defined subdimensions and therefore adopted a recently proposed bifactor ESEM approach (Morin et al., 2015) suitable for examining such multidimensional structures.
In the first step, we compared a traditional ICM–CFA with ESEM to examine the possibility that the items within these two instruments have systematic associations with non-target constructs that needs to be accounted for (Asparouhov and Muthén, 2009; Morin et al., 2015; Perry et al., 2015). In both cases did the ESEM model provide a better fit to the data compared to the ICM–CFA, and we also noted a decrease in factor correlations because the relations between the items have to a lesser degree been channeled through the factors. Previous research with simulated (e.g., Asparouhov and Muthén, 2009) and real data (e.g., Marsh et al., 2013) have consistently shown that factor correlations in ICM–CFA are likely to be positively biased and that ESEM more accurately estimates factor correlations (Marsh et al., 2014). A pattern of positively biased factor correlations was evident also in this study. When the assumption of zero cross-loadings does not hold, the factor correlations will be positively biased because systematic associations between items and non-target factors are not accounted for (Marsh et al., 2014). Marsh and colleagues also argued that when the assumption of zero cross-loadings does hold, allowing for cross-loadings is unlikely to result in negative bias in factor correlations. Recently there have been suggestions in the methodological literature that ICM–CFA and ESEM should routinely be performed on the same data in order to investigate these potential sources of bias inherent in many ICM–CFA models (Marsh et al., 2013, 2014).
The present study contributes to the growing body of knowledge regarding the factor structure of the ISS–C and the CCBS. However, some limitations exist. The samples in these two studies were in a quite narrow age range, only included team sport athletes, and only male athletes were included in Study 2. Although the CCBS have been found invariant across type of sport and gender (Bartholomew et al., 2010), exploring whether these results can be replicated in other samples of different ages and types of sports would be interesting in future research. Researchers could also examine if perceptions of coaches need supportive and controlling interpersonal styles vary with athletes' age and competitive level. Future research should also explore the predictive ability of the general and specific factors in these scales on various outcomes, for example, basic need thwarting and satisfaction (see Gunnell and Gaudreau, 2015, for such an approach). The results from Study 1, however, suggests that the ISS-C might best be represented as a unidimensional structure and that almost all of the variance in the items was accounted for by the general factor. The unidimensional structure of the ISS-C indicates that it might not be necessary to use the general factor from a bifactor ESEM model as an outcome or predictor; a single factor can be adequate for such purposes2.
Taken together, the present study illustrates a useful approach for examining dimensionality in multidimensional self-report scales. The results related to the ISS–C, combined with findings from previous research, indicate that the multidimensionality of the need support construct can be questioned. Future research should thoroughly examine whether this lack of multidimensionality in need-support scales can be attributed to theoretical, methodological, and/or empirical aspects. The results related to the CCBS showed an interesting pattern of psychometric multidimensionality that should be cross-validated in future research. The integration of bifactor analysis and ESEM (Myers et al., 2014; Morin et al., 2015) provide researchers with a framework uniquely suited for examinations of multidimensional constructs.
AS came up with the initial idea and design, performed the analyses and interpreted the data, and was in charge of drafting the manuscript. AI made substantial contribution to the analyses and interpretation of the data, and revised the manuscript critically for important intellectual content. PH helped out in the interpretation of data and revised the manuscript critically for important intellectual content. ML made substantial contribution to the analyses and interpretation of the data, and revised the manuscript critically for important intellectual content. All four authors (AS, AI, PH, ML) provided final approval of the version to be published, and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Grants from Umeå School of Sport Sciences and the Swedish National Centre for Research in Sports (CIF) supported this research (Grant P2014-0043). We express our sincere appreciation to John Andersson, Kristian Fagerström, Fredrik Hansson, and Pierre Hedin for helping out with the data collection.
1 ^We did not include second-order models in our model testing sequence because we wanted to examine the direct relation between the global and specific factors and the items. In a second-order model the relation between the general factor and the items is represented as an indirect effect through the first-order factors. In addition, second-order models rely on far more stringent assumptions than bifactor models as well as stringent proportionality constraints not likely to hold in real-world settings with complex instrument (Morin et al., 2015; Reise, 2012).
2 ^To further examine this issue with the ISS-C we correlated factor scores from a single-factor CFA model and factor scores from the general factor in the bifactor ESEM model. The correlation was 0.996 indicating very little impact of ignoring multidimensionality and suggests that using the ISS-C as a single factor might be adequate.
Adie, J. W., Duda, J. L., and Ntoumanis, N. (2012). Perceived coach-autonomy support, basic need satisfaction and the well-and ill-being of elite youth soccer players: a longitudinal investigation. Psychol. Sport Exerc. 13, 51–59. doi: 10.1016/j.psychsport.2011.07.008
Bartholomew, K. J., Ntoumanis, N., Ryan, R. M., Bosch, J. A., and Thøgersen-Ntoumani, C. (2011). Self-determination theory and diminished functioning: the role of interpersonal control and psychological need thwarting. Pers. Soc. Psychol. Bull. 37, 1459–1473. doi: 10.1177/0146167211413125
Bartholomew, K. J., Ntoumanis, N., and Thogersen-Ntoumani, C. (2009). A review of controlling motivational strategies from a self-determination theory perspective: implications for sports coaches. Int. Rev. Sport Exerc. Psychol. 2, 215–233. doi: 10.1080/17509840903235330
Bartholomew, K. J., Ntoumanis, N., and Thøgersen-Ntoumani, C. (2010). The controlling interpersonal style in a coaching context: development and initial validation of a psychometric scale. J. Sport Exerc. Psychol. 32, 193–216.
Castillo, I., Tomás, I., Ntoumanis, N., Bartholomew, K. J., Duda, J. L., and Balaguer, I. (2014). Psychometric properties of the Spanish version of the Controlling Coach Behaviors Scale in the sport context. Psicothema 26, 409–414. doi: 10.7334/psicothema2014.76
Deci, E. L., Koestner, R., and Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol. Bull. 125, 627–668. doi: 10.1037/0033-2909.125.6.627
Gunnell, K. E., and Gaudreau, P. (2015). Testing a bi-factor model to disentangle general and specific factors of motivation in self-determination theory. Pers. Individ. Dif. 81, 35–40. doi: 10.1016/j.paid.2014.12.059
Haerens, L., Aelterman, N., van den Berghe, L., de Meyer, J., Soemens, B., and Vansteenkiste, M. (2013). Observing physical education teachers' need-supportive interactions in classroom setting. J. Sport Exerc. Psychol. 35, 3–17.
Jang, H., Reeve, J., and Deci, E. L. (2010). Engaging students in learning activities: it is not autonomy support or structure but autonomy support and structure. J. Educ. Psychol. 102, 588. doi: 10.1037/a0019682
Markland, D., and Tobin, V. J. (2010). Need support and behavioural regulations for exercise among exercise referral scheme clients: the mediating role of psychological need satisfaction. Psychol. Sport Exerc. 11, 91–99. doi: 10.1016/j.psychsport.2009.07.001
Marsh, H. W. (2007). “Application of confirmatory factor analysis and structural equation modeling in sport/exercise psychology,” in Handbook of Sport Psychology 3rd Edn., eds G. Tenenbaum and R. C. Eklund (Hoboken, NJ: Wiley), 774–798.
Marsh, H. W., Hau, K. T., and Wen, Z. (2004). In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Struct. Equ. Modeling. 11, 320–341. doi: 10.1207/s15328007sem1103_2
Marsh, H. W., Lüdtke, O., Nagengast, B., Morin, A. J. S., and von Davier, M. (2013). Why item parcels are (almost) never appropriate: two wrongs do not make a right—Camouflaging misspecification with item parcels in CFA models. Psychol. Methods 18, 257–284. doi: 10.1037/a0032773
Marsh, H. W., Morin, A. J. S., Parker, P. D., and Kaur, G. (2014). Exploratory structural equation modelling: an integration of the best features of exploratory and confirmatory factor analyses. Annu. Rev. Clin. Psychol. 10, 85–110. doi: 10.1146/annurev-clinpsy-032813-153700
Marsh, H. W., Scalas, L. F., and Nagengast, B. (2010). Longitudinal tests of competing G-factor structures for the Rosenberg self-esteem scale: traits, ephemeral artifacts, and stable response styles. Psychol. Assess. 22, 366–381. doi: 10.1037/a0019225
Morin, A. J. S., Arens, A. K., and Marsh, H. W. (2015). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Struct. Equ. Modeling. doi: 10.1080/10705511.2014.961800. [Epub ahead of print].
Myers, N. D., Martin, J. J., Ntoumanis, N., Celimli, S., and Bartholomew, K. J. (2014). Exploratory bifactor analysis in sport, exercise, and performance psychology: a substantive-methodological synergy. Sport Exerc. Perform. Psychol. 3, 258–272. doi: 10.1037/spy0000015
Ng, J. Y. Y., Ntoumanis, N., Thøgersen-Ntoumani, C., Deci, E. L., Ryan, R. M., Duda, J. L., et al. (2012). Self-determination theory applied to health contexts: a meta-analysis. Perspect. Psychol. Sci. 7, 325–340. doi: 10.1177/1745691612447309
Niemiec, C. P., Lynch, M. F., Vansteenkiste, M., Bernstein, J., Deci, E. L., and Ryan, R. M. (2006). The antecedents and consequences of autonomous self-regulation for college: a self-determination theory perspective on socialization. J. Adolesc. 29, 761–775. doi: 10.1016/j.adolescence.2005.11.009
Perry, J. L., Nicholls, A. R., Clough, P. J., and Crust, L. (2015). Assessing model fit: caveats and recommendations for confirmatory factor analysis and exploratory structural equation modeling. Meas. Phys. Educ. Exerc. Sci. 19, 12–21. doi: 10.1080/1091367X.2014.952370
Podsakoff, P. M., MacKenzie, S. B., and Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Ann. Rev. Psychol. 63, 539–569. doi: 10.1146/annurev-psych-120710-100452
Pope, J. P., and Wilson, P. M. (2012). Understanding motivational processes in university rugby players: a preliminary test of the hierarchical model of intrinsic and extrinsic motivation at the contextual level. Int. J. Sports Sci. Coach. 7, 89–107. doi: 10.1260/1747-95126.96.36.199
Reeve, J., and Su, Y.-L. (2014). “Teacher motivation,” in The Oxford Handbook of Work Engagement, Motivation, and Self-Determination Theory, ed M. Gagné (New York, NY: Oxford University Press), 349–362.
Reinboth, M., and Duda, J. L. (2006). Perceived motivational climate, need satisfaction and indices of well-being in team sports: a longitudinal perspective. Psychol. Sport Exerc. 7, 269–286. doi: 10.1016/j.psychsport.2005.06.002
Rhemtulla, M., Brosseau-Liard, P. E., and Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychol. Methods 17, 354–373. doi: 10.1037/a0029315
Sarrazin, P., Vallerand, R., Guillet, E., Pelletier, L., and Cury, F. (2002). Motivation and dropout in female handballers: a 21-month prospective study. Eur. J. Soc. Psychol. 32, 395–418. doi: 10.1002/ejsp.98
Smith, N., Tessier, D., Tzioumakis, Y., Quested, E., Appleton, P., Sarrazin, P., et al. (2015). Development and validation of the multidimensional motivational climate observation system. J. Sport Exerc. Psychol. 37, 4–22. doi: 10.1123/jsep.2014-0059
Stenling, A., Lindwall, M., and Hassmén, P. (2015). Changes in perceived autonomy support, need satisfaction, motivation, and well-being in young elite athletes. Sport Exerc. Perform. Psychol. 1, 50–61. doi: 10.1037/spy0000027
Taylor, I. M., Turner, J. E., Gleeson, M., and Hough, J. (2015). Negative psychological experiences and saliva secretory immunoglobulin A in field hockey players. J. Appl. Sport Psychol. 27, 67–78. doi: 10.1080/10413200.2014.949907
van Knippenberg, D., and Sitkin, S. B. (2013). A critical assessment of charismatic—Transformational leadership research: back to the drawing board? Acad. Manag. Ann. 7, 1–60. doi: 10.1080/19416520.2013.759433
Keywords: controlling behaviors, dimensionality, leadership, need support, self-report scales
Citation: Stenling A, Ivarsson A, Hassmén P and Lindwall M (2015) Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches' interpersonal styles. Front. Psychol. 6:1303. doi: 10.3389/fpsyg.2015.01303
Received: 04 June 2015; Accepted: 17 August 2015;
Published: 01 September 2015.
Edited by:Pietro Cipresso, IRCCS Istituto Auxologico Italiano, Italy
Reviewed by:Bobby Naemi, Educational Testing Service, USA
Joshua Fredrick Wiley, Mary MacKillop Institute for Health Research at Australian Catholic University, Australia
Copyright © 2015 Stenling, Ivarsson, Hassmén and Lindwall. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Andreas Stenling, Department of Psychology, Umeå University, Sciences Building, SE-901 87 Umeå, Sweden, firstname.lastname@example.org