Psychometric Properties of the Elementary Social Behavior Assessment in Swedish Primary School: A Teacher Rated Index of Students’ Prosocial School Behaviors

Introduction

Students come to school with varying behavior. Some students already are well equipped to listen, follow instructions, do seat work, get along with peers, and behave positively at school. Other students may have substantial ground to cover to fully develop these behaviors, which makes academic achievement and school adjustment possible. Leveling this playing field is essential because, less intensive support is needed if the problems are identified early (Lane et al., 2010). The Elementary Social Behavior Assessment (ESBA, Pennefather and Smolkowski, 2015) was developed in the United States as a tool to identify prosocial school behaviors that teachers value as important to students’ social and academic success. These prosocial school behaviors have been considered as academic enablers (DiPerna and Elliott, 2002), allowing students to participate with engagement in their education (Pennefather and Smolkowski, 2015).

Prosocial School Behaviors in Swedish Children

In Sweden, there is little tradition of systematically using quantitative assessments to promote young children’s social and academic success (Allodi, 2007). Teachers subjectively assess their students’ behavior and social functioning without the benefit of much systematic or collectively agreed-upon descriptions (Allodi, 2007). In national (Swedish School Inspectorate, 2018) and international evaluations (OECD, 2015; PISA, 2018) indicators of disciplinary climate in Swedish schools are reported as non-optimal (OECD, 2015) and social competence and teacher leadership are areas of concern. A recent government initiative aims to deal with challenges posed by the occurrence of behavior problems in schools; children with disabilities, in particular, reported more bullying; one-third of teachers in compulsory school reported they spend substantial time maintaining classroom order (Department of Education in Sweden 2020). The Swedish educational system has responsibility for all children; vulnerable children should be protected at school, and positive relationships with teachers and peers are considered as supporting students’ well-being and achievement, while difficulties may be signaled through internalizing and externalizing behaviors, but also with psychosomatic symptoms, school refusal and school absenteeism (SNEA, 2021). A reliable measure of students’ prosocial school behaviors would be a useful tool, both for research, and as evaluation of interventions, which protect students who are at risk in elementary school and prevent later difficulties that might escalate without adequate monitoring. This kind of screening instrument based on teachers’ ratings has not yet been validated in Sweden. If satisfyingly validated, ESBA could fill the need for an instrument that supports teachers in their work to foster students’ prosocial behaviors.

Elementary Social Behavior Assessment

ESBA (Pennefather and Smolkowski, 2015) was developed in the United States. as a teacher-rated index of prosocial school behaviors. Approximately, 1,100 teachers across the United States. rated 56 items. The items that now make up the ESBA were rated highest by teachers (Walker et al., 2015). The items formulations are strengths-based, that is, worded with positive formulations, expressing desirable school behaviors. The measure consists of 12 items rated on a 3-point Likert scale. Teachers respond to each item: mastery 3) needs improvement 2) and cause for concern 1). An example item is, “Gets along with peers.” ESBA is designed for use with six to 11 year-olds.

To date, two United States studies investigated ESBA’s psychometric properties (Pennefather and Smolkowski, 2015). In the first study, the sample included nine teachers in kindergarten, first and second grades, and their students (N = 187). A factor analysis using structural equation modeling (SEM) indicated a one-factor structure and a pretest scale reliability of 0.95; other factor structures were tested and were not completely ruled out. In the second study, participants were 70 teachers and their students (N = 1,616) from kindergarten to third grade. The authors tested one-two- and three-factor models and concluded that the one-factor model best characterized ESBA; pretest-reliability was 0.95 (Cronbach’s alpha). The eight-week test-retest reliability was, r = 0.77 (Pennefather and Smolkowski, 2015). ESBA’s criterion validity was also tested and it was concluded that the construct measured students’ prosocial school behavior adequately, compared to other established scales: specifically, BBRS (Gresham et al., 2010) and WMS (Walker and McConnell, 1995).

ESBA has also been tested in Norwegian elementary schools. Participants were 151 teachers and their students (N = 793) from grades one to six. Intraclass correlations (ICC) tested the extent of clustering by school and classroom: low ICC estimates were found, at the school (less than 1% of variance accounted for) and classroom levels (less than 5%). SEM results supported for a one- or two-factor structure; even a three-factor model was tested. The Cronbach’s alpha ranged from 0.90 to 0.93. Eight-week test-retest reliability was r = 0.78 (Arnesen et al., 2018).

Aim

This study evaluated the psychometric properties, namely the factor structure, and internal consistency, of a Swedish edition of ESBA. Because prior ESBA psychometric studies have supported a one- and two-model factor structure, we posited that either a one or two-factor model for ESBA would be supported by the results, but we also considered a three-factor-model as an alternative to test, due to previous analyses.

Sample

The study included teacher ratings of 143 students in grade two (74 girls; M age = 8.33 years, SD = 0.34 years and 69 boys; M age = 8.31 years, SD = 0.35 years). Ratings were provided by eight second-grade-teachers in four municipal schools. Convenience sampling of schools was used. Teachers all had teaching certificates. The schools were located in two commuting municipalities and one large city¹ (Table 1). A socio-economic (SES) index for each school is provided². The SES index shows that the schools have a student composition with a slightly higher SES than the average SES of schools in Sweden.

TABLE 1

TABLE 1. Demographics school, socio-economic index, and gender.

The teachers were invited and a first meeting was arranged to explain the study. Students’ parents and students (consent form adjusted to their age) were required to provide written consent (response rate = 73%). The Swedish Ethical Review Authority reviewed the study, number of the protocol (#019-03058).

Instrument

ESBA is a 12 item, teacher-rated index of prosocial school behaviors rated on a 3-point Likert scale (ordinal) with three being better behaviors and one indicative of less than typical behavior. In the Norwegian study, ESBA items were averaged and ranged between 2.6 and 2.8 (Arnesen et al., 2018).

ESBA was translated from English into Swedish and underwent back translation (Allodi, 2007). One of the ESBA developers, Smolkowski, approved this translation process and advised that items should be formulated in Swedish in a culturally adapted way rather than in a completely literal translation of words. The translation into Swedish, however, was maintained as similar to the original as possible. Item 4 “Does seatwork assignments as directed” was culturally adapted into “Does assignments according to instructions”, because in Swedish “directed” could be perceived as too authoritarian. Item 10 “Can have “normal” conversations without becoming hostile” has been translated into “Can have regular conversations without becoming hostile” because the word “normal” in Swedish may evoke a normative impression During the data collection, the teachers received instruction material on the ESBA, which provided examples of behaviors for each item.

Design

The design was cross-sectional and represents the pre-test of a randomized controlled trial that tests a novel intervention, aimed at improving classroom climate. Randomization was conducted post recruitment. Data were collected prior to the start of the intervention and includes intervention and control schools.

Statistical Analysis

Analyses were conducted with SPSS and Mplus version 8.4 (Muthén and Muthén, 1998). At the observed level, normality (item and scale), item-total correlations, and scale reliability were examined (Cronbach’s Alpha). SEM models in Mplus were conducted with an MLR estimator. There were no missing data. Because prior evidence supported a one and two-factor structure, a series of confirmatory factor analyses (CFAs) were planned going from the most parsimonious model (i.e., one factor) to a two-factor model. Additional models would be tested if the results of the one and two-factor CFAs indicated a need to do so. Changes to all tested models were guided by theory and modification indices (Brown, 2015). Determination of model fit was based on cut-offs for several fit indices, Comparative Fit Index (CFI, value >0.95), Root Mean Square Error of Approximation (RMSEA, estimated value at or below 0.08), and Standardized Root Mean Square Residual (SRMR value at or below 0.08; Kline, 2016). Chi-square values reported providing descriptive insights into exact fit. ICC estimates indicated that less than 1% of the variance in responses was accounted for by school and 5% of the variance was accounted for by classroom. This study’s ICC estimates are consistent with those found in the Norwegian ESBA study (Arnesen et al., 2018).

Results

ESBA mean scores ranged from 2.2 to 2.9 (Table 2), indicating that most teachers rated students as having mastered prosocial school behaviors. Item correlations demonstrated that items were positively and significantly related to one another (r > 0.48) and a total average score showed excellent internal reliability (Cronbach’s α = 0.92).

TABLE 2

TABLE 2. Elementary Social Behavior Assessment items, sample means, and standard deviations.

CFA results are described in Tables 3, 4. A one-factor model with all 12 ESBA items loaded on a single factor had poor fit on CFI (e.g., 0.94), but acceptable to good fit on RMSEA and SRMR. Standardized factor loadings for the single factor model were from 0.53 to 0.84. Based on prior ESBA psychometric analyses conducted in Norway (Arnesen et al., 2018), a two factor CFA was posited and tested with the first factor called Teacher (called Academic Engagement in the Norwegian study, Arnesen et al., 2018) with seven items (items 1 – 7 see Table 2 for items) and the second factor entitled Peer with five items (items 8 – 12, see Table 2 for items; See Figure 1 for CFA). This model showed acceptable fit on CFI, RMSEA, and SRMR. Standardized factor loadings for the two-factor model were from 0.553 to 0.874 for the Teacher factor with seven items and 0.682 to 0.897 for the Peer factor with five items. The latent correlation between these two factors was significant and positive at 0.803. Cronbach’s alphas for items on the Teacher factor was 0.855 and for the Peer factor was 0.897.

TABLE 3

TABLE 3. (A) Fit indices of the three models. Confirmatory factor analysis and fit statistics.

TABLE 4

TABLE 4. (B) Factor loadings.

FIGURE 1

FIGURE 1. Shows results from a confirmatory factor analysis of a two factor model with good fit. Standardized factor loadings are acceptable for the teacher factor with loadings ranging from 0.553 to 0.874 (Cronbach’s scale alpha = 0.855) and peer ranging from 0.682 to 0.897 (Cronbach’s scale alpha = 0.897). The latent correlation between these factors is significant at 0.803.

Based on item content, we tested a three-factor model. See Figure 2. Factor 1 comprised items regarding when teachers are giving instructions to enhance student's academic engagement. Factor 2 consists of items about students´ behavior when working more independently and factor 3 items concern situations when students are working or playing with peers. This model showed acceptable fit on CFI, RMSEA, and SRMR. Standardized factor loadings for items on these three factors were largely acceptable except for item 8. This item loaded at 0.419 and 0.418 on the peer and teacher factors. The content for item eight is “Behaves appropriately outside the classroom”. Standardized factor loadings for the Teacher factor ranged from 0.419 to 0.877 (Cronbach’s scale alpha = 0.778), Work ranged from 0.599 to 0.906 (Cronbach’s scale alpha = 0.847), and for Peer ranged from 0.418 to 0.904 (Cronbach’s scale alpha = 0.897). The latent correlations between these factors ranged from 0.735 to 0.887.

FIGURE 2

FIGURE 2. Shows results from a confirmatory factor analysis of a three - factor model with good fit. Standardized factor loadings are largely acceptable except for item 8 (items loads 0.419 and 0.418 on two factors, the item is “Behaves appropriately outside the classroom”). Acceptable standardized factor loadings for the teacher factor ranges from 0.538 to 0.877 (Cronbach’s scale alpha = 0.778), work ranges from 0.599 to 0.736 (Cronbach’s scale alpha = 0.847), and for peer ranged from 0.418 to 0.904 (Cronbach’s scale alpha = 0.897). The latent correlations between these factors are positive and significant and range from 0.735 to 0.887.

Discussion

This study evaluated the psychometric properties of a Swedish adaption of ESBA. Observed scale reliability was consistently good across the different tested models. CFA results indicated that the one-factor model showed good fit except for CFI. Prior studies in the United States and Norway have shown support for a single factor model, but also two-factor models (Pennefather and Smolkowski, 2015; Arnesen et al., 2018). The present results are a first step and additional Swedish studies are needed to rule in or out the ultimate value of a single factor ESBA model.

Results indicated the clearest support for a three- and two-factor model. The three-factor structure had an instructional part (teacher), work in the classroom more independently part (work), and working and playing with peers (peers). The two-factor structure comprised an instructional part (teacher) and the part working and playing with peers (peers). Support for a two-factor model is consistent with the Norwegian ESBA study (Arnesen et al., 2018). The present study sample consisted of second graders, compared to the Norwegian study, which had a sample that ranged from grades one to six. Although the three-factor model showed good fit and largely acceptable factor loadings, item eight did evidence less than ideal factor loadings on two factors, and the model shows a high correlation between the teacher and work factors (r = 0.887). Future psychometric ESBA studies would provide more evidence in favor or against the future use of the three-factor model tested in this study.

From the perspective of practice, the one-two- and three-factor models are similar in some ways but they address different areas of prosocial school behaviors. Therefore, the scales can serve different screening purposes. Students who score inadequately on the subscale in the instructional part (teacher) can get support in how to behave in the classroom. Students who score poorly on the subscale in having difficulties working more independently (work) can get extra assistance in the classroom or students who score inadequately with peers (peers) can get support to interact with peers outside the classroom and to handle acute feelings. In the Swedish curriculum (Swedish National Agency for Education, 2011), it is stated that the teachers should stimulate, supervise and provide special support to students with difficulties. The subscales can promote this work as the difficulties are identified. This first trial of the ESBA in our context showed that the teachers accept it and find it useful. A recent review confirmed that teachers found ESBA feasible and acceptable (Brann et al., 2021). In our view, the indications of high acceptability of the ESBA are related to the strengths-based positive formulations and the construct relevance to rather universal core prosocial behaviors in the elementary classroom.

Limitations and Conclusion

The selection of schools was based on convenience sampling and therefore this limits the generalizability of these findings to other Swedish elementary school children in grade two, as well as in other grades and ages not included in this sample. While a strength of the sampling approach was that participating schools were located in different areas, future research would benefit from testing ESBA in schools with a wide range of socio-economic conditions. Another limitation was that the study involved second-grade students, which are 8-year-olds in Sweden, meanwhile, the Norwegian study involved students from grades 1–6 Further research in Sweden is needed to explore the utilization of ESBA with several ages. Another limitation is that only one aspect of validity was tested, namely, construct validity, and other Swedish psychometric tests of ESBA examining concurrent and predictive validity are important to pursue. However, despite the noted limitations, this study provided preliminary evidence in support of the psychometric properties of ESBA, when used with young Swedish children (grade 2) and this is consistent with evidence in support of ESBA’s use with Norwegian children (Arnesen et al., 2018). While there is clear support for the two-factor model, the one and three-factor models cannot be definitely ruled out for use in a Swedish primary school context and future research is needed. The use of a model (one, two, or three) could also be guided by teachers’ focus and goals: the one-factor model could be useful for screening purposes (tier 1, in multitiered models of support) and the three-factor model could be useful in planning targeted interventions (tier 3) (Hemmeter et al., 2016). ESBA has important practical value in that it can identify areas in need of improvement in regards to prosocial school behaviors. ESBA can also be used as a reliable measure of students’ prosocial school behaviors as an aid to formative work, the evaluation of interventions, and as a guide to evidence-based practices. ESBA is a positively worded scale that can also pinpoint areas in which students need extra support. It can be a basis for teachers and parents to collaborate to support students.

Data Availability Statement

The data for this study is not publicly available as this study’s ethical review does not allow for the primary data to be in a public repository. Individual data requests for de-identified (IPD) will be reviewed for qualified researchers (e.g., Ph.D.) who obtain ethical permission under Swedish ethical regulations/laws for secondary data analysis for purposes such as meta-analysis or confirmation of published study results.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

¹The definitions follow the categories defined by the Swedish Association of Local Authorities and Regions (2017). A large city is a municipality with more than 200,000 inhabitants. The commuting municipalities have more than 40% of the working population commuting to a large city.

²The socio-economic index has been calculated by Statistics Sweden on behalf of the Swedish National Agency of Education, with the purpose of distributing additional national funding to the schools, based on the SES of the students enrolled the year before. Higher school rank scores correspond to higher poverty levels and entitle schools to greater funding.

References

Allodi, M. (2007). Assessing the Quality of Learning Environments in Swedish Schools: Development and Analysis of a Theory-Based Instrument. Learn. Environments Res. 10, 157–175. doi:10.1007/s10984.007-9029-9

CrossRef Full Text | Google Scholar

Arnesen, A., Smolkowski, K., Ogden, T., and Melby-Lervåg, M. (2018). Validation of the Elementary Social Behaviour Assessment: Teacher Ratings of Students' Social Skills Adapted to Norwegian, Grades 1-6. Emotional Behav. Difficulties 23, 39–54. doi:10.1080/13632752.2017.1316473

CrossRef Full Text | Google Scholar

Brann, K. L., DanielsSandraChafouleas, B. M. S. M., and DiOrio, C. A. (2021). Usability of Social, Emotional, and Behavioral Assessments in Schools: A Systematic Review from 2009 to 2019. School Psychology Review. doi:10.9783/9780812297874

CrossRef Full Text

Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research. The Guilford Press.

Department of Education in Sweden (2020). Nationell Plan For Trygghet Och Studiero. [National Plan for Safe and Peaceful Learning Environments]. U2020/00707/S https://www.regeringen.se/rattsliga-dokument/kommittedirektiv/2020/02/u202000707s/.

DiPerna, J. C., and Elliott, S. N. (2002). Promoting Academic Enablers to Improve Student Achievement: An Introduction to the Mini-Series. Sch. Psychol. Rev. 31, 293–297. doi:10.1080/02796015.2002.12086156

CrossRef Full Text | Google Scholar

Gresham, F. M., Cook, C. R., Collins, T., Dart, E., Rasetshwane, K., Truelson, E., et al. (2010). Developing a Change-Sensitive Brief Behavior Rating Scale as a Progress Monitoring Tool for Social Behavior: An Example Using the Social Skills Rating System-Teacher Form. Sch. Psychol. Rev. 39, 364–379. doi:10.1080/02796015.2010.12087758

CrossRef Full Text | Google Scholar

Hemmeter, M. L., Fox, L., and Hardy, J. K. (2016). “Supporting the Implementation of Tiered Models of Behavior Support in Early Childhood Settings,” in Handbook of Early Childhood Special Education. Editors B. Reichow, B. Boyd, E. Barton, and S. Odom (Cham: Springer).

Google Scholar

Kline, R. B. (2016). Principles and Practice of Structural Equation Modeling. New York: The Guilford Press.

Lane, K. L., Oakes, W., and Menzies, H. (2010). Systematic Screenings to Prevent the Development of Learning and Behavior Problems: Considerations for Practitioners, Researchers, and Policy Makers. J. Disabil. Pol. Stud. 21, 160–172. doi:10.1177/1044207310379123

CrossRef Full Text | Google Scholar

Muthén, L. K., and Muthén, B. O. (1998). Mplus User’s Guide. Muthén & Muthén.

Organisation for Economic Co-operation and Development (2015). Improving schools in Sweden [Elektronisk resurs] an OECD perspective, Paris: OECD.

Pennefather, J. T., and Smolkowski, K. (2015). Validation of the Elementary Social Behavior Assessment. Assess. Eff. Intervention 40, 143–154. doi:10.1177/1534508414557562

PISA (2018). Results (Volume III): What School Life Means for Students´ Lives. Retrieved from https://www.oecd-ilibrary.org/sites/f05bb3ee-en/index.html?itemId=/content/component/f05bb3ee-en.

SNEA (2021). Skolans Ansvar För Barn Som Far Illa. [School's responsibility for children at risk] https://www.skolverket.se/skolutveckling/inspiration-och-stod-i-arbetet/stod-i-arbetet/skolans-ansvar-for-barn-som-far-illa.

Swedish Association of Local Authorities and Regions (2021). Classification of Swedish Municipalities 2017. https://skr.se/skr/tjanster/kommunerochregioner/faktakommunerochregioner/kommungruppsindelning.2051.html.

Swedish School Inspectorate (2018). Resultat Från Skolenkäten 2017-2018 [Results from the School questionnaire 2017-2018]. https://www.skolinspektionen.se/beslut-rapporter-statistik/publikationer/statistikrapporter/2019/resultatrapport-skolenkaten-2017-2018/.

Swedish National Agency for Education (2011). Curriculum for the Compulsory School System, the Pre-school Class, and the Leisure-Time centre 2011. Stockholm: Swedish National Agency for Education (Skolverket).

Walker, H. M., Marquez, B., Yeaton, P., Pennefather, J., Forness, S. R., and Vincent, C. G. (2015). Teacher Judgment in Assessing Students' Social Behavior within a Response-To-Intervention Framework: Using what Teachers Know. Education Treat. Child. 38, 363–382. doi:10.1353/etc.2015.0019

CrossRef Full Text | Google Scholar

Walker, H. M., and McConnell, S. R. (1995). Walker-McConnell Scale of Social Competence and School Adjustment: Elementary Version Technical Manual. San Diego, CA: Singular Publishing Group.