Implementing the Competencies Compound Inventory for 21st Century Competencies in Brazil: implications for research and practice

As education systems evolve to meet the demands of the 21st century, there is a growing recognition that students must develop not only academic knowledge but also a broad set of competencies, including social–emotional skills, character, and meta-learning. To support this shift, educators require reliable and scalable tools to assess and guide student development in these areas. This study examines the translation and validation of the Competencies Compound Inventory for the 21st Century (CCI-21) into Brazilian Portuguese (CCI-21-P), a tool designed to measure 12 key competencies aligned with the 4-Dimensional Education model and Brazil’s national education framework, the BNCC (Base Nacional Comum Curricular). Using data from 60,791 upper primary and secondary students across more than 450 cities in Brazil, we evaluated the psychometric properties of the CCI-21-P through analyses of item sensitivity, reliability, and internal structure. Results indicate strong internal consistency (α = 0.92) and a factor structure largely comparable to the original English version, supporting the instrument’s validity and cultural adaptability. Findings suggest that the CCI-21-P is a reliable and practical tool for assessing 21st Century Competencies in Brazilian schools. Its use enables educators to align competency development with national curriculum goals while contributing to international efforts to benchmark social–emotional learning (SEL) measurements across diverse contexts. Continued validation and adaptation will further strengthen its utility in informing policies and practices regarding holistic education.

Introduction

Increasingly, educators understand that they must not only teach students the necessary knowledge they need, but also that education must include other dimensions such as skills, character, and meta-learning (Cipriano and McCarthy, 2023; Fadel et al., 2015). These 21st Century Competencies and Social and Emotional Learning (SEL) enable students to understand and connect with themselves and others and to make responsible, positive changes in the world (Weissberg et al., 2015). As the digital age integrates the potential of Artificial Intelligence (AI) technology, SEL is becoming increasingly important to be able to learn and work with technologies (Fadel et al., 2024). However, a persistent challenge remains: while the importance of SEL and 21st Century Competencies is well established, the ability to assess these competencies in diverse, culturally relevant, and psychometrically sound ways remains limited, especially outside English-speaking contexts. Many existing tools are either too lengthy, misaligned with local educational goals, or insufficiently validated for use in diverse cultural settings. Without high-quality, scalable assessment tools, educators lack the data needed to monitor growth, tailor instruction, and evaluate the effectiveness of SEL initiatives (Bialik et al., 2016; Doss and Hamilton, 2020). As a part of this effort, researchers at Beyond Education (BE) developed the Competencies Compound Inventory for the 21st Century (CCI-21), an online English assessment tool to measure 12 competencies pertaining to SEL education (Celume and Maoulida, 2022a). This study examines the translation of the CCI-21 into Portuguese (CCI-21-P), including the psychometric comparability to the English version and a discussion of the policy context and implications of measuring 21st Century Competencies in Brazil, aligned with the BNCC (Base Nacional Comum Curricular) or the Brazilian National Common Core Curriculum.

4-dimensional model of education & CCI-21

The 4-Dimensional (4D) model of education put forward by researchers from the Center for Curriculum Redesign (CCR) at Harvard University is based on the premise that in the face of rapid technological, ecological, and economic change, the competencies needed in the future for individual and societal well-being and economic success must be consciously adapted. The strength of the 4D model by CCR is the clear organization, definition, and usability of the competencies for educational change. A detailed explanation can be found in the CCR (Fadel et al., 2015), consisting of the following dimensions:

1. Knowledge: what a person knows and understands, generally related to traditional curriculum subjects (e.g., literacy and numeracy).

2. Skills: the way a person uses what they know, “an ability or proficiency acquired through training and practice” (Vanden Bos, 2007). Skills are learned through and with the knowledge dimension.

3. Character: how people behave and engage in the world based on their set of personality traits and attributes that define their social, moral, and ethical characteristics (Bialik et al., 2016, 2015). Character education is about the acquisition and strengthening of virtues, values, and the capacity to make wise choices.

4. Meta-Learning: one’s capacity for self-reflection that a person constantly adapts as they grow and learn to pursue different goals and purposes. Meta-learning encompasses our ability to learn about learning for oneself and others, so people may have the tools to be versatile, self-directed, and self-reliant in lifelong learning.

The 4D model offers a strong theoretical foundation to orient education systems in a way that is future-oriented and adaptive to uncertain and evolving landscapes. The most recent revision of the 4D model further explored the importance of people’s motivation, agency, purpose, and identity as interconnected elements that define education (Fadel et al., 2024). The 21st Century Competencies within the 4D model are those that allow us to face our ever-changing world and provide a foundation to assess our educational practices and policies. Based on this model, Beyond Education (BE) developed the English version of the Compound Competency Inventory for the 21st Century (CCI-21) to measure 12 domains of social–emotional learning (SEL). The need for instruments such as the CCI-21 has been identified by Bialik et al. (2016) and others and includes the measurement of Skills (creativity, collaboration, communication, and critical thinking), Character (courage, leadership, curiosity, resilience, ethics, and mindfulness), and Meta-learning (metacognition and growth mindset). The psychometric properties of the CCI-21 are explained by Celume and Maoulida (2022a), which includes age calibration to account for malleability and variance across age (Chernyshenko et al., 2018). The original CCI-21 established an evidence base to measure 21st Century Competencies in support of holistic policy and practice.

SEL measurement

BE measured the SEL skills of 620 English-speaking students across 4 countries in English validation. Likewise, evidence from practice using the CCI-21 suggests that it is useful to inform pedagogy and programmatic impacts on competency development (Celume and Maoulida, 2022b; Maoulida et al., 2023). SEL skills, in which individuals need to deal with increasingly complex situations starting from childhood in areas such as physical and mental health, academic development, social relations, and citizenship, need to develop, elaborate, and integrate in life over time (Zins and Elias, 2007). Özdemir and Büyükçolpan (2021) stated that SEL programs might foster career development of individuals and hence could contribute to Positive Youth Development (PYD). According to Damon (2004), individuals may encounter many problems, such as emotional disorders, economic inadequacies, low motivation, academic failure, psychosocial crises, alcohol, drugs, or cigarette use, during the growth process. SEL provides a structure for managing this complexity, both individually and when dealing with others.

Within the growing landscape of SEL measurement, considerable attention has been paid to the multifaceted set of cognitive, emotional, and behavioral competencies that enable individuals to navigate intrapersonal and interpersonal interactions effectively (Buckley and Saarni, 2014). Given the increasing importance of SEL education, there is a growing need for assessment tools to monitor student progress in SEL competencies (Doss and Hamilton, 2020). Measurement of 21st Century Competencies, such as those in the CCI-21, is a crucial form of formative evaluation within classrooms to help guide what competencies should be strengthened as a part of planning SEL initiatives (Grant et al., 2023; Shapiro et al., 2024). These needs have already prompted the development of various assessment instruments, such as teacher-completed behavioral rating scales and questionnaires (Melnick et al., 2017). Likewise, it is known that even when schools begin to implement SEL education, there is often a lack of training and support for SEL implementation (Hon et al., 2023). While many schools approach SEL education with the pre-existing values they want to instill, it would be beneficial to first measure these competencies and then decide on an instructional approach.

Amidst the growing emphasis on SEL assessment, ensuring the validity of measurement tools becomes paramount. Gehlbach and Brinkworth (2011) conceptualized validity as an ongoing process rather than as a static attribute of a measurement scale. In other words, it is a process involving accumulating evidence to support the argument that a measure accurately assesses intended constructs within specific contexts and populations (Kane, 2006, 1992; Messick, 1995). Especially in the case of measurement translation, it is well understood that additional validation is needed (Oliveri et al., 2012; Oliveri and Ercikan, 2011). Moreover, the context in which the SEL measures are administered plays a crucial role in determining their effectiveness. Considering the importance of cultural responsiveness in education, there is a need to integrate SEL assessment practices with culturally responsive teaching methods (Cipriano and McCarthy, 2023; Heine et al., 2002). SEL skills are already being developed and used in middle-income countries such as Brazil, although their measurement is less well developed (Do Couto Fonseca, 2020).

SEL development and measurement in Brazil

In Brazil, the education system follows a highly decentralized model. As federal countries, subnational and private education systems should follow guidelines and regulations established at the national level while adapting locally their policies and programs. Brazil first stated the importance of SEL development in 2017 with the approval of the National Common Core Curriculum, or Base Nacional Comum Curricular (BNCC).

Although the importance of SEL has been debated in Brazil for decades, it was with the BNCC that this topic gained greater attention. The BNCC guides all the subnational curricula in the country, detailing for the first time the ultimate goal of achieving holistic education through the development of ten general competencies that every learner in basic education should master. The BNCC states what students should “know” but, more importantly, what they should “know how to do” considering the mobilization of knowledge (concepts and procedures), skills (practical, cognitive, and socio-emotional), attitudes, and values to solve complex demands of everyday life, the full exercise of citizenship, and the world of work (Brasil, Ministério da Educação, 2018).

Throughout the cycle of basic education, these transversal competencies should be developed by learners during different classes and projects that make up their learning journey. These competencies intend to support teaching and learning processes, but also to ensure the integral development of learners and their readiness to build a just, democratic, and inclusive society. They anchor all other learning components in BNCC and are organized into (i) Knowledge; (ii) Scientific, critical, and creative thinking; (iii) Cultural sensitivity and repertoire; (iv) Communication; (v) Fluency in information technologies; (vi) Autonomy and self-management; (vii) Argumentation; (viii) Self-knowledge and self-care to deal; (ix) Empathy and cooperation; and (x) Autonomy, responsibility (more details on more details on Supplementary material Table 1) (Brasil, Ministério da Educação, 2018).

After its launch, the process of implementing the BNCC in all Brazilian schools started in 2018, and since then, the topic of socioemotional skills development has gained attention across many levels. Governments, CSOs, and other organizations have worked together to build a task force to bring the BNCC to all Brazilian schools, both private and public. In that regard, in addition to governmental programs and guidelines, institutions such as “Movimento Pela Base” (a coalition of educational specialists to implement the BNCC) have produced several materials and courses to support pedagogical intervention and lesson plans. One of them, produced in partnership with the Center for Curriculum Redesign (CCR), which proposed the 4D model, details the dimensions and subdimensions of the ten general competencies by listing the behaviors that a learner should demonstrate in order to master the subdimensions through different milestones from elementary to high school (more details on how models correlate in Annex: Movimento Pela Base Center for Curriculum Redesign, 2020).

Even with the implementation of the BNCC, socio-emotional education programs are still incipient in Brazil (Do Couto Fonseca, 2020), although recent years are marked by some efforts to implement SEL programs in private and public schools. These programs often include teacher training and materials to support pedagogical interventions, emphasizing activities in which learners can develop competencies. They are anchored in different frameworks, sometimes produced by the educational institutions themselves. As efforts to promote these programs have become more frequent, the literature on SEL development in Brazil has also evolved, with the production of more studies and papers that report the importance of developing such programs and measuring their impacts and effects on academic outcomes and the future of the younger generation (Muto and Galvani, 2023).

As institutions are still building their programs, measurement of these efforts to develop SEL remains even more incipient. The lack of different validated instruments or flexible tools that can assess the suitability of different models or frameworks helps explain why the measurement of 21st Century Competencies is not yet prevalent in Brazil. In this regard, the development of the CCI-21-P aims to provide educational institutions with an assessment that has the required psychometric properties, and it is valid to measure the socioemotional competencies of Brazilian students aligned to the national BNCC. Other measures, such as the social and emotional nationwide assessment (SENNA) inventory, have already been used in Brazil. However, the SENNA takes 2–3 times longer than the CCI-21 and draws on other psychological frameworks less aligned to BNCC (Primi et al., 2021) in different construct domains such as focus, enthusiasm, and gratitude. Schools should consider what domains are of most value to them in developing or implementing measures.

Therefore, this study has three purposes: (1) to present psychometric evidence of the Portuguese translation of the CCI-21 (CCI-21-P), (2) to compare the Portuguese and English versions of the CCI-21, and (3) to reflect on the work of BE in Brazil and the limits and strengths of the CCI-21 to support multicultural SEL instruction.

Methods

Research design and type

This analysis has two primary purposes. First is to provide the psychometric properties of the CCI-21-P as an independent tool to be used to measure the Skills, Character, and Meta-Learning dimensions of the 4D model. Second, we compared the psychometric properties with those of the English CCI-21 to build an evidence base for cross-cultural use. As such, we aim to present evidence comparable to the analyses conducted by Celume and Maoulida (2022a). At times, we reproduce, with permission, their results for ease of comparison. The research design and type are evaluative in nature concerning the design for the assessment of translation and development.

Sample

In 2023, BE expanded to reach Portuguese-speaking students across Brazilian schools as part of a collaboration to assess SEL programming across several curricula in private schools in all Brazilian states and more than 450 cities. BE worked with a partner who already works across schools in the country by providing education solutions around SEL to implement the CCI-21-P. The provider was interested in using the CCI-21-P as a tool to help evaluate and compare their own educational solutions, which enabled BE to gain access to a large student sample. All students using their educational solution were asked to participate, and informed consent, including opt-out procedures, was provided (see Supplementary material: Informed Consent).

Altogether, the educational provider intended to administer CCI-21-P to approximately 122,000 students across the country. Inevitably, some schools had technology challenges in accessing the assessment, and other students were absent during the time of administration, resulting in approximately 100,000 students being reached. Of these, some students did not fully complete the CCI-21-P, did not provide their age, which was necessary for calibration, and others demonstrated high Social Desirability Bias (>2.5 SD from the mean), which were dropped from analysis using Barger’s version of the SDS scale simultaneously implemented with the CCI-21-p (2002) (Barger, 2002). The final analytical sample comprised 60,791 students who had complete test records (Table 1).

Table 1

Table 1. Demographics of the Brazilian sample.

The final analytical sample had a balanced sex distribution, with 52.2% female, 46.4% male, and 1.4% non-binary. Students’ ages were self-reported and ranged from 10 to 22 years, with an average of 13.4 years of age. Students were distributed across grades 5 through 12 but were mostly in grades 6 through 10, with slightly less than 20% in each grade category. Approximately 99% of the students reported Portuguese as their native language, with the remaining 1% being distributed across Spanish, Arabic, English, and other languages. Students were predominantly in private schools (96.8%), while the rest were in public schools (3.0%) or community schools (0.2%). Students were in approximately 1,000 schools that were distributed across all of Brazil. Approximately 82% of Brazilian students attended public schools in 2022, while 18% attended private schools. Although public education serves the majority of students, private schools have a significant presence, with over 40,000 institutions and 9 million students. Students who attend private schools typically come from higher socioeconomic backgrounds and often achieve better results on national standardized tests such as the Exame Nacional do Ensino Médio (ENEM). Despite these differences, the Base Nacional Comum Curricular (BNCC) is mandatory for both private and public schools, requiring them to implement the ten core competencies and integrate SEL (Social and Emotional Learning) programs focused on the development of socioemotional skills.

Technique and instrument

The first step was to develop the CCI-21-P with translation from English to Brazilian Portuguese, which was completed using backward and forward translation processes (Hambleton, 1996). The English version of the CCI-21 was independently translated by a native Brazilian Portuguese speaker who is also fluent in English. From this, the Portuguese version of the CCI-21 was then translated back into English by a native English speaker from the US, who is also fluent in Brazilian Portuguese and who had never seen the original English version. Finally, this translated version was re-translated into Brazilian Portuguese by another native Brazilian Portuguese speaker who was fluent in English. These three versions were then reconciled under the supervision of a fourth researcher experienced in test writing. When there were conflicts, they often consisted of challenges between literal and intended meanings, particularly given that the English translator had never seen the original meaning. In these moments, the researchers favored translations that were consistent with the intended meaning and not the literal meaning. For example, the item “I am always interested in discovering new things I did not know before” was originally translated as “Estou sempre interessado(a) em descobrir coisas novas que não conhecia antes.” However, this phrasing is quite redundant in Portuguese, and we find it better to refer to “unknown” things rather than things “not known” for a Brazilian Portuguese speaker. Hence, we settled on, “Estou sempre interessado em relação a assuntos que desconheço.” Another, slightly more serious cultural consideration had to do with the translation of one of the social desirability items in English referring to “Take advantage,” which directly in Portuguese is “aproveitar.” However, in Brazilian Portuguese, “aproveitar” can carry a sexual connotation when left ambiguous, so we added the clause “de alguém” to specify to take advantage “of someone” to eliminate the ambiguity while maintaining the English meaning. This is consistent with the guidance to help preserve meaning and cultural relevance. This version was then piloted in two schools, and additional feedback was used to arrive at the final version of the tool.

Data collection procedure

As a part of the delivery of the CCI-21-P, BE provided training on the purpose of SEL measurement, operational guidance to implement the CCI-21-P, assessment support during delivery, and results interpretation through a train-the-trainer model in order to provide in-service professional development that is cognizant of the current needs and limits of practicing teachers (Cipriano and McCarthy, 2023).

After the translation of the tool, the development of the CCI-21-P in the above-mentioned schools followed a sequence of steps to guarantee smooth implementation. The first step was to collect school and student data, as the CCI-21-P is an online self-reported tool in which learners enter an individual account and respond to three main questionnaires. The collected data had the sole purpose of creating student accounts, which were later shared with the responsible school staff. With the data provided, BE prepared the setup of the platform considering the translated version of the CCI-21-P and the specific context of Brazil in which the assessment was taking place. Each school had its own link through which students logged in to participate in the CCI-21-P. Information to schools was provided through shared lists containing students’ credentials in addition to their specific links.

While preparing the setup, the BE trained educators from the education provider. The purpose of the training was to provide an overview of SEL in the Brazilian context, explain the 4D model as the basis of the CCI-21-P, and give the operational details of its implementation in schools. Educators cascaded the training to other educators, which would help with the implementation, and complementary support materials were provided to prepare for the assessment sessions.

Once all setups were completed regarding training and technology, schools implemented the CCI-21-P with their students from ages 10 to 21. The assessment was conducted in different schools covering all 27 states of Brazil, considering several contexts, age groups, types of schools, grades, and socioeconomic levels. The variety of locations, school types, and contexts in which the CCI-21-P was implemented provided BE with a robust and heterogeneous sample, important to strengthen the validation of the tool. Moreover, educators had access to a live monitoring system in which they could follow schools’ progress in the assessment.

After 5 weeks of implementation in May and June 2023, the session ended, and the platform was closed. The data were then extracted for analysis. Individual data extracted were anonymized for all schools, and school reports were produced to give participating schools a glimpse of their assets and improvable competencies on the CCI-21-P, as well as average scores and statistical comparisons per gender, grades, and cross-school comparisons. To ensure the correct interpretation of results and statistical comparisons, a train-the-trainer moment with educators was set up to explain how to read and interpret school reports. In this regard, crosswalks between CCI-21-P competencies and three different models used by the educational provider were produced to guarantee an understanding of the relationship between the competencies and the dialogue between the schools’ models and the CCI-21-P.

At the end of the process, feedback considering the implementation and reports was collected, with the aim of improving the steps and collecting feedback about CCI-21-P that would require any adaptation of the tool. In this sense, feedback from educators about the CCI-21-P was positive, corroborating the methodology to adapt the current CCI-21 in Portuguese.

Type of analysis

To compare the CCI-21-P with the original CCI-21, we measured and compared the sensitivity, reliability, factorial structure, and validity. Here, consistent with the approach of Celume and Maoulida (2022b), sensitivity refers to item distributions of scores and not item discriminatory power or respondents’ sensitivity to change, as might typically be considered, but for which we do not have data. At the item level, we calculated the skewness and kurtosis values for each of the 36 items, where a negative skew suggests a distribution with the tail on the left side and where “0″ would mean perfect symmetry. For the kurtosis values, the normal distribution is “3,” where values less than three are platykurtic, meaning fewer or less extreme values as outliers. For each dimension (skills, character, and meta-learning) and for the entire tool (all 36 items), we calculated Cronbach’s alpha and split-half reliability, where an alpha of 0.7 or greater is considered good (Cronbach and Meehl, 1955). We used Spearman’s rho correlation of the split-half reliability comparing even-and odd-numbered questions, ideally with a value between 0.8 and 0.9. Finally, for each of the three dimensions, we examined the factorial structure, including item loading and goodness-of-fit, as evidence of internal validity.

Software

Data were analyzed using STATA version 16.

Results

Sensitivity

Across the 36 items, overall means ranged from 2.67 to 4.06 on a 1–5-point Likert scale, meaning that participants responded predominately on the higher end of the scale, consistent with the English version. All items except two (CR3, MN2) had negative skewness values, and the values were consistent with those reported by Celume and Maoulida (2022b) in the English version. The kurtosis values for the Portuguese-speaking students were higher than those for the English students, with values ranging from 1.94 to 3.37. Only two items (GR1 and GR3) had kurtosis values greater than 3, meaning most items had slightly higher normal distribution “peaks.” Growth Mindset appears to be slightly more centered and skewed compared to the English sample (Table 2).

Table 2

Table 2. Item sensitivity.

Reliability

Reliabilities for the entire scale and for each dimension were good. Cronbach’s alph a for the CCI-21-P was strong (α = 0.92) and was good for Skills (α = 0.78), Character (α = 0.85), and Meta-learning (α = 0.75). Alphas were slightly lower with the translated CCI-21-P when compared to the original. However, both versions have acceptable and reliable values. Similarly, the split-half reliability using Spearman’s rho comparing odd-and even-numbered items (18 each) was also good at 0.84 (Table 3).

Table 3

Table 3. Reliabilities.

Validity

We analyzed whether all 36 items converged into three dimensions using a Principal Components Analysis (PCA) factorial analysis with oblimin rotation. Then, consistent with the research by Celume and Maoulida (2022b), separately for the three dimensions of Skills, Character, and Meta-learning, we examined internal validity using PCA. To be consistent with CCI-21, we allowed rotation because we expected a correlation between the factors based on the CCR model.

For the skills dimension, our PCA consisted of 12 items and a 4-component solution, one for each intended competency (Creativity, Critical Thinking, Collaboration, and Communication). PCA with oblimin rotation (KMO = 0.86, χ² (66) = 123,834, p < 0.001) led to a four-factor solution (λ1 = 2.13, λ2 = 1.73, λ3 = 1.57, λ4 = 1.32) explaining 56.3% of the variance (Hayton et al., 2004). The overall factor structure was similar to that of the English structure. However, cross-loadings are different between the English and Portuguese versions. Both had cross-loadings between Critical Thinking and Creativity. However, whereas the English version had some cross-loadings between Critical Thinking and Communication, the Portuguese version had cross-loadings between Collaboration and Critical Thinking. In particular, two items (CM2 and CT2) had low loadings on the factor structure, one item (CR2) loaded on a different factor, and the final item (CL1) had cross-loading. Slightly more variance was accounted for in the English version than in the Portuguese version (Table 4).

Table 4

Table 4. Skills—factor structure.

Within the Character dimension, our PCA consisted of 18 items and a 6-component solution, with one item for each intended competency (Courage, Curiosity, Mindfulness, Ethics, Leadership, and Resilience). PCA with oblimin rotation (KMO = 0.91, χ² (153) = 231,386, p < 0.001) led to a six-factor solution (λ1 = 2.55, λ2 = 1.91, λ3 = 1.91, λ4 = 1.70, λ5 = 1.29, λ6 = 1.17), explaining 58.6% of the variance (Hayton et al., 2004). These results are comparable to those of the English version. Cross-loadings were similar between the English and Portuguese versions. Mindfulness was cross-loaded with Courage and Resilience in both English and Portuguese. One item (CO2) has very concerningly low loadings, and one item (RS1) loads on a different factor in Portuguese. More variance was observed in the Portuguese version than in the English version (Table 5).

Table 5

Table 5. Character—factor structure.

Within the meta-learning dimension, our PCA consisted of six items and a 2-component solution, one for each intended competency (Metacognition and Growth Mindset). PCA with oblimin rotation (KMO = 0.82, χ² (15) = 68,602, p < 0.001) led to a two-factor solution (λ1 = 1.84, λ2 = 1.70), explaining 59.0% of the variance (Hayton et al., 2004). These results are comparable to the English version. Cross-loadings between Metacognition and Growth Mindset occurred within both languages, perhaps slightly more in the Portuguese version. Item MC2 has a concerningly low loading (below 0.3), and item GR2 loads on metacognition. Slightly more variance is accounted for in the English version than in the Portuguese (Table 6).

Table 6

Table 6. Meta-learning—factor structure.

The CCI-21-P offers an important development to measure learners’ competencies in Brazil and other Lusophone countries. CCI-21-P is concise, developmentally calibrated, and directly aligned with the 4-Dimensional Education model and Brazil’s BNCC. Our results demonstrate that the Portuguese version retains strong psychometric properties, showing high reliability, a coherent factor structure across the three competency domains, and comparable validity to the original English version. Importantly, CCI-21-P allows for flexible and scalable implementation across diverse schools.

Discussion

Implications of research

As far as we know, this is the first published translation of CCI-21 in Portuguese and other languages. The overall results from the analysis of the CCI-21-P suggest that the translation of the CCI-21 from English into Portuguese was successful and that the scale was sensitive, reliable, and demonstrated good internal validity. There are some differences worth highlighting. The Portuguese-speaking sample in Brazil was roughly 100 times bigger than the English-speaking sample, and so there is greater statistical power with the Brazilian students. Although importantly, the majority (96.8%) of the sample came from private school students, although a non-significant amount of them had tuition assistance, many of them come from more economically advantaged households, given the large inequity in Brazil. Future research should include more economically diverse students. Future studies involving item linking using the item-response theory or other techniques with a larger English-speaking sample can be used to further verify compatibility. For example, the results suggested higher scores on growth mindset among Portuguese-speaking students compared to the English sample. Additional analyses and studies are needed to investigate the extent to which these results are “true,” or perhaps, artifacts of translation. The results of the sensitivity/distribution analysis suggest that respondents from the CCI-21-P tend to have greater peaks in the distribution, suggesting that more differentiation in the middle of the response scale may help. Future linking studies should investigate these differences in variances, and sensitivity analyses are needed to assess discrimination.

Internal validity based on the factor structure is comparable between the English and Portuguese versions, with equivalent factor structures and similar cross-loadings of items when they occur. However, the eight items mentioned above could benefit from either revision or analysis given the low loadings or cross-loading concerns in the Portuguese version. Overall, a slightly greater percentage of variance seems to be explained by the CCI-21 in the English version of the tool, which is likely explained by less within-sample variance, given a smaller age range than in the Brazilian context. BE’s approach to validity in connection to the CCI-21 is a pragmatic approach since we recognize the constraints that school leaders face with respect to academic research and logistical challenges. Schools cannot invest unlimited amounts of time surveying students, nor can they tolerate this. Indeed, within the field of measurement, there is an ad nauseam amount of “types” of validity that are all “unified” under construct validity (Messick, 1989). In this paper, we focus on the factor structure and internal validity of the CCI-21-P in comparison to the English version, but also seek to connect the creation and use of the CCI-21-P to a larger conversation of content validity and the benefits of implementing SEL measures. Future research should also consider other valid arguments.

Implications for practice

When implementing SEL frameworks, schools should implement multiple measures to triangulate SEL findings and to complement academic outcomes. The CCI-21-P is a valuable and useful complement to other measures, given the ease of scalability across many classrooms and as a “quick” assessment that only requires 30 min of student time. The benefit of the CCI-21-P in the context of Brazil to support the implementation of the BNCC is that the two draw upon the same theoretical foundation from the Center for Curriculum Redesign.

As Melnick et al. (2017) discuss, many schools are using teacher-rating scales. However, these rating scales are difficult to implement given the amount of time required for each teacher to use them. These Likert-style response scales are also used as teaching scales and, as such, do not offer a strong methodological advantage unless used as a complement. In their 2024 publication, ETS states:

“Rating methods, particularly self ratings, are widely used, adaptable to just about any psychological or educational construct imaginable, and relatively inexpensive to develop, administer, score, and report on, which at least partly explains their popularity. Psychometric models for rating scale methods and nomological networks of constructs based on rating scale methods are well-developed. It is likely that the world will continue to rely on rating scales for many skills and constructs for the foreseeable future.” (Kyllonen and Sevak, 2024, p. 31)

While diverse forms of assessments are on the horizon, CCI-21-P provides a quick and scalable alternative consistent with the dominant methodology in the field. However, it is important that the CCI-21-P continue to be used as a formative assessment along with other measures to support students’ SEL development from a strength-based model of education.

The extension of this work to the Brazilian context also has meaningful impacts. While it has made continuous educational improvements, Brazilian students continue to perform below the international averages from other developed countries on the program for international student assessment (PISA) (OECD, 2020). Meanwhile, Brazil has adopted and begun implementation of a national common core curriculum, which, in addition to other dimensions, seeks to support equity and holistic educational improvement (Brasil, Ministério da Educação, 2018). Educational measures such as CCI-21-P offer important and promising potential to help provide metrics of SEL efficacy to achieve the national goals of curricular reform. Indeed, Brazil has a highly decentralized education system, where one important policy goal is to empower educators through school-level professional development to nurture more positive learning contexts (OECD, 2021). The work of Beyond Education, through the implementation of the CCI-21-P, provides an important foundation to equip school-level leaders and teachers with SEL measures of educational outcomes in order to foster more inclusive and holistic learning. While this study focuses on the quantitative and psychometric properties of the CCI-21-P in the Brazilian context, future studies will include more qualitative feedback and policy-level evidence on how the use of the CCI-21-P and other SEL measures contributes to local-and national-level policy changes in Brazil.

Conclusion

This study summarizes the results of the psychometric properties of the Portuguese translation of the Competency Compound Inventory for 21^st Century Competencies (CCI-21-P) from the original English version. Using a larger sample (60,000) than the original version, Beyond Education found that the CCI-21-P has comparable psychometric properties in terms of sensitivity, reliability, and internal validity to the original CCI-21.

Limitations

Overall reliability results suggest that the CCI-21-P has strong internal consistency. Cronbach’s alpha for the Portuguese version was slightly weaker than that of the English version, as reported by Celume and Maoulida (2022b). This may be because Brazilian students represent a wider age range than English-speaking schools used in the original CCI-21. Indeed, qualitative feedback from the Brazilian sample confirmed that younger students had some difficulties with items that may have been too advanced a language. BE is working to develop “younger versions” of the assessment to accommodate a more developmental population. This study focused on the translation of the CCI-21 English version and so was limited to the assumptions of measurement embedded within the self-reflection tool. The CCI-21-P should be treated as a formative assessment tool for reflection on 21st Century Competencies and not as a summative assessment.

New lines of research

Results from the analysis of the translation of the CCI-21 from English to Portuguese provide initial evidence for the comparability of the psychometric properties between the two tools, and future work is needed to improve the use of the CCI-21-P. Future studies would benefit from a comparison of the applied context of the tools in relation to other academic outcomes and consider an analysis of students’ progression over time using the CCI-21 and CCI-21-P. Similarly, additional analyses to compare the invariance between CCI-21 and CCI-21-P can and should be conducted using methods such as multigroup confirmatory factor analyses or the alignment method (Asparouhov and Muthén, 2014; Marsh et al., 2018). However, these methods are very demanding in terms of the sample size, which would require a much larger English sample within the CCI-21. Similarly, within the context of multicultural comparisons, future research would benefit from a deeper investigation into where the CCI-21 and CCI-21-P differ as an informative, contextualized understanding of the measurement of non-cognitive domains in a socio-cultural context (Roberson and Zumbo, 2019).

Nevertheless, these results provide a firm foundation to continue to build on the measurement of SEL dimensions and to be able to make meaningful comparisons across languages and cultures (Oliveri and Ercikan, 2011). As we look toward continued improvement and integration of SEL measurement as a part of holistic education, measures such as the CCI-21-P provide important foundations for research to inform policy and practice in classrooms and schools (Cipriano and McCarthy, 2023).

The results of this study provide an important foundation for the extension of SEL measurements in Brazil in support of national curriculum reforms. Likewise, these results are an important foundation to establish an initial evidence base for future studies of cross-cultural measurement of 21st Century Competencies using the CCI-21 and its translations. While continuously and iteratively improving, these results provide an important international benchmark for SEL measurement. The CCI-21 is now widely used in English and Portuguese, and increasingly in Spanish. As such, it provides an international referent for the development and measurement of 21st Century Competencies. As the field of SEL measurement expands, the CCI-21 provides important insights to support holistic and human-centered educational experiences. In an “age of AI,” now more than ever, it is important to advance measures of uniquely human attributes. The progress of the CCI-21-P sets a foundation for formative assessment feedback to classrooms, which simultaneously helps build system-level accountability and decision-making for SEL initiatives over time (McKown, 2019).

Impact statement

The development and translation of the Competencies Compound Inventory for the 21st Century (CCI-21) into Portuguese (CCI-21-P) provide a useful instrument to help measure and monitor SEL domains to support students and teachers with data to create future-ready education. The CCI-21-P demonstrates scientifically rigorous properties that provide an evidence base for formative assessment and international benchmarking of 21st Century Competencies.

Data availability statement

The datasets presented in this article are not readily available because this data contains proprietary information and so is not publicly available. Requests to access the datasets should be directed to cmVzZWFyY2hAYmV5b25kZWR1Y2F0aW9uLnRlY2g=.

Ethics statement

Ethical approval was not required for the study involving humans in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and the institutional requirements.

Author contributions

NR: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Supervision, Validation, Writing – original draft, Writing – review & editing. TP: Data curation, Investigation, Project administration, Supervision, Writing – original draft, Writing – review & editing. MH: Funding acquisition, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2025.1543050/full#supplementary-material

References

Asparouhov, T., and Muthén, B. (2014). Multiple-group factor analysis alignment. Struct. Equ. Model. 21, 495–508. doi: 10.1080/10705511.2014.919210

Crossref Full Text | Google Scholar

Barger, S. D. (2002). The Marlowe-Crowne affair: short forms, psychometric structure, and social desirability. J. Pers. Assess. 79, 286–305. doi: 10.1207/S15327752JPA7902_11

PubMed Abstract | Crossref Full Text | Google Scholar

Bialik, M., Bogan, M., Fadel, C., and Horvathova, M.. (2015). Character education for the 21st century: what should students learn. Center for Curriculum Redesign CCR. 1–35.

Google Scholar

Bialik, M., Martin, J., Mayo, M., and Trilling, B. (2016). The Ellen Koshland Family Fund: Center for Curriculum Redesign. Evolving assessments for a 21st century education.

Google Scholar

Brasil, Ministério da Educação (2018). Brazil Ministry of Education National Common Core Curriculum]

Google Scholar

Buckley, M., and Saarni, C. (2014). “Emotion regulation: implications for positive youth development” in Handbook of positive psychology in schools. eds. M. J. Furlong, R. Gilman, and E. S. Huebner. (New York, NY: Routledge).

Google Scholar

Celume, M.-P., and Maoulida, H. (2022a). Psychometric properties of the competencies compound inventory for the twenty-first century. Front. Educ. 7, 1–12. doi: 10.3389/feduc.2022.877129

Crossref Full Text | Google Scholar

Celume, M.-P., and Maoulida, H. (2022b). Developing 21st century competencies among youth through an online learning program: BE a global citizen. Educ. Sci. 12:148. doi: 10.3390/educsci12030148

Crossref Full Text | Google Scholar

Chernyshenko, O. S., Kankaraš, M., and Drasgow, F. (2018). Social and emotional skills for student success and well-being: conceptual framework for the OECD study on social and emotional skills. Paris: OECD.

Google Scholar

Cipriano, C., and McCarthy, M. F. (2023). Towards an inclusive social and emotional learning. Soc. Emotion. Learn. 2:100008. doi: 10.1016/j.sel.2023.100008

PubMed Abstract | Crossref Full Text | Google Scholar

Cronbach, L. J., and Meehl, P. E. (1955). Construct validity in psychological tests. Psychol. Bull. 52, 281–302. doi: 10.1037/h0040957

PubMed Abstract | Crossref Full Text | Google Scholar

Damon, W. (2004). What is positive youth development? Ann. Am. Acad. Polit. Soc. Sci. 591, 13–24. doi: 10.1177/0002716203260092

Crossref Full Text | Google Scholar

Do Couto Fonseca, G. (2020). Essays on the development of socioemotional skills (dissertation). São Paulo: Escola de Economia de São Paulo.

Google Scholar

Doss, C.J., and Hamilton, L.S. (2020). Supports for social and emotional learning in American schools and classrooms RAND Corporation: Findings from the American teacher panel. RAND.

Google Scholar

Fadel, C., Black, A., Taylor, R., Slesinski, J., and Dunn, K. (2024). Education for the age of AI: Why, what and how should students learn for the age of artificial intelligence? CCR: Independently published.

Google Scholar

Fadel, C., Trilling, B., and Bialik, M. (2015). Four-dimensional education: The competencies learners need to succeed, 1st edition. CCR Create Space Independent Publishing Platform, Boston, MA.

Google Scholar

Gehlbach, H., and Brinkworth, M. E. (2011). Measure twice, cut down error: a process for enhancing the validity of survey scales. Rev. Gen. Psychol. 15, 380–387. doi: 10.1037/a0025704

Crossref Full Text | Google Scholar

Grant, N., Meyer, J. L., and Strambler, M. J. (2023). Measuring social and emotional learning implementation in a research-practice partnership. Front. Psychol. 14, 01–11. doi: 10.3389/fpsyg.2023.1052877

PubMed Abstract | Crossref Full Text | Google Scholar

Hambleton, R.K. Guidelines for adapting educational and psychological tests. ITC. (1996).

Google Scholar

Hayton, J. C., Allen, D. G., and Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organ. Res. Methods 7, 191–205. doi: 10.1177/1094428104263675

Crossref Full Text | Google Scholar

Heine, S. J., Lehman, D. R., Peng, K., and Greenholtz, J. (2002). What’s wrong with cross-cultural comparisons of subjective Likert scales?: the reference-group effect. J. Pers. Soc. Psychol. 82, 903–918. doi: 10.1037/0022-3514.82.6.903

PubMed Abstract | Crossref Full Text | Google Scholar

Hon, D., Sauve, J., Mahfouz, J., and Schonert-Reichl, K. (2023). “Cultivating SEL-ready teachers: social and emotional learning in teacher preparation” in Social and emotional learning in action: Creating systemic change in schools. eds. K. A. Schonert-Reichl and S. Rimm-Kaufman (New York, London: Guilford Press), 115–137.

Google Scholar

Kane, M. (1992). An argument-based approach to validity. Psychol. Bull. 112, 527–535. doi: 10.1037/0033-2909.112.3.527

Crossref Full Text | Google Scholar

Kane, M. (2006). Validation, in: R. Brennan (Ed.). Educational measurement. American Council on Education and Praeger, Westport, CT, pp. 17–64.

Google Scholar

Kyllonen, P.C., and Sevak, A. Charting the future of assessment. ETS. (2024).

Google Scholar

Maoulida, H., Madhukar, M., and Celume, M.-P. (2023). A case study of 21st century cognitive, social and emotional competencies using online-learning. J. Intelligence 11:116. doi: 10.3390/jintelligence11060116

PubMed Abstract | Crossref Full Text | Google Scholar

Marsh, H. W., Guo, J., Parker, P. D., Nagengast, B., Asparouhov, T., Muthén, B., et al. (2018). What to do when scalar invariance fails: the extended alignment method for multi-group factor analysis comparison of latent means across many groups. Psychol. Methods 23, 524–545. doi: 10.1037/met0000113

PubMed Abstract | Crossref Full Text | Google Scholar

McKown, C. (2019). Challenges and opportunities in the applied assessment of student social and emotional learning. Educ. Psychol. 54, 205–221. doi: 10.1080/00461520.2019.1614446

Crossref Full Text | Google Scholar

Melnick, H., Cook-Harvey, C. M., and Darling-Hammond, L. (2017). Encouraging social and emotional learning in the context of new accountability. Learning Policy Institute: Learning Policy Institute.

Google Scholar

Messick, S. (1989). “Validity” in Educational measurement. ed. R. L. Linn (New York: American Council on Education and Macmillan), 13–103.

Google Scholar

Messick, S. (1995). Standards of validity and the validity of standards in performance asessment. Educ. Meas. Issues Pract. 14, 5–8. doi: 10.1111/j.1745-3992.1995.tb00881.x

Crossref Full Text | Google Scholar

Movimento Pela Base Center for Curriculum Redesign (2020) Dimensões e Desenvolvimento das Competências Gerais da BNCC. Movimento Pela Base.

Google Scholar

Muto, J. H. D., and Galvani, M. D. (2023). O ensino das habilidades socioemocionais na escola: Uma revisão de literatura. Rev. Ibero-Am. Estud. Educ. :e023156. doi: 10.21723/riaee.v18i00.17935

Crossref Full Text | Google Scholar

OECD (2020). PISA 2018 results (volume V): Effective policies, successful schools, PISA. Paris: OECD Publishing.

Google Scholar

OECD (2021). Core considerations for Brazil: Managing change in a complex national assessment system. Paris: OECD Educational Policy Perspectives. OECD Publishing.

Google Scholar

Oliveri, M. E., and Ercikan, K. (2011). Do different approaches to examining construct comparability in multilanguage assessments lead to similar conclusions? Appl. Meas. Educ. 24, 349–366. doi: 10.1080/08957347.2011.607063

Crossref Full Text | Google Scholar

Oliveri, M. E., Olson, B. F., Ercikan, K., and Zumbo, B. D. (2012). Methodologies for investigating item-and test-level measurement equivalence in international large-scale assessments. Int. J. Test. 12, 203–223. doi: 10.1080/15305058.2011.617475

Crossref Full Text | Google Scholar

Özdemir, N. K., and Büyükçolpan, H. (2021). A scale development study: social emotional learning scale-young adult form (SELS-YF). Kastamonu Educ. J. 29, 205–218. doi: 10.24106/kefdergi.822770

Crossref Full Text | Google Scholar

Primi, R., Santos, D., John, O. P., and De Fruyt, F. (2021). SENNA inventory for the assessment of social and emotional skills in public school students in Brazil: measuring both identity and self-efficacy. Front. Psychol. 12:716639. doi: 10.3389/fpsyg.2021.716639

PubMed Abstract | Crossref Full Text | Google Scholar

Roberson, N. D., and Zumbo, B. D. (2019). Migration background in PISA’S measure of social belonging: using a diffractive lens to interpret multi-method DIF studies. Int. J. Test. 19, 363–389. doi: 10.1080/15305058.2019.1632316

Crossref Full Text | Google Scholar

Shapiro, V. B., Kim, B.-K. E., Robitaille, J. L., Mahoney, J., Lee, J., and LeBuffe, P. A. (2024). Monitoring the growth of social and emotional competence to guide practice decisions. Soc. Emotion. Learn. 3:100032. doi: 10.1016/j.sel.2024.100032

Crossref Full Text | Google Scholar

Vanden Bos, G. R. (2007). APA dictionary of psychology, APA dictionary of psychology. Washington, DC, US: American Psychological Association.

Google Scholar

Weissberg, R. P., Durlak, J. A., Domitrovich, C. E., and Gullotta, T. P. (Eds.) (2015). “Social and emotional learning: past, present, and future” in Handbook of social and emotional learning: Research and practice (New York, NY, US: The Guilford Press), 3–19.

Google Scholar

Zins, J. E., and Elias, M. J. (2007). Social and emotional learning: promoting the development of all students. J. Educ. Psychol. Consult. 17, 233–255. doi: 10.1080/10474410701413152

Crossref Full Text | Google Scholar