Assessing the benefits of the “Intergalactic World” social emotional learning program for 8–12-year-old children in Portugal: perspectives from teachers and caregivers

Introduction “Intergalactic World” is a new social–emotional program designed to reduce psychopathological symptoms and improve social and emotional skills in children aged 8–12. This study aims to evaluate the program’s benefits from teachers’ and caregivers’ perspectives, focusing on internalizing and externalizing behaviors. Methods The findings were obtained through self-reported measures using a pretest-posttest design with a follow-up period, but with no control group. One hundred fifty-four children (M age = 9.66, SD = 0.78) participated in this intervention study. Eleven teachers completed the Teacher’s Report Form (TRF) for these children, and 133 caregivers completed the Child Behavior Checklist (CBCL). Participants without caregivers’ reports were excluded from the analysis. Data were collected at three-time points: before the intervention (T1), immediately after (T2), and 6 months after the implementation of the program (T3). Results Results (n = 133) showed an effect of time on the Internalization scores (at T3 for teachers and T2 and T3 for caregivers) with no gender effect and a decrease in the perception of externalizing behaviors with a gender effect: Boys were perceived as exhibiting more externalizing behaviors than girls. However, these behaviors significantly decrease at T3 for teachers and at T2 and T3 for caregivers. Discussion Despite its limitations, this study highlights the benefits of employing social–emotional programs to help reduce children’s internalizing and externalizing behaviors. A multi-informant approach enables a comprehensive analysis and provides insights into the child’s significant contexts and interactions with adults.


Introduction
Social and emotional skills positively impact learning and important life outcomes, promoting positive social behavior and reducing conduct problems (Kankaraš and Suarez-Alvarez, 2019; Chatterjee Singh and Duraiappah, 2020).These skills are particularly important for children with behavioral problems, encompassing either internalizing or externalizing 10.3389/fpsyg.2023.1233335Frontiers in Psychology 02 frontiersin.orgbehaviors.Externalizing behaviors include hyperactivity, attention problems, and conduct problems (e.g., opposition).On the other hand, internalizing behaviors typically consist of self-directed difficulties (Achenbach and Edelbrock, 1978), such as anhedonia and negative moods and emotions (Schuman-Olivier et al., 2020), which are associated with various depressive and anxiety disorders (Hansen and Jordan, 2020).
The literature shows that boys usually display more externalizing behaviors than girls, whereas girls tend to exhibit more internalizing behaviors (Eme, 2016;Gutman and McMaster, 2020;Lau et al., 2021).Children and young people with behavioral problems suffer from emotional and behavioral regulation changes, leading to frequent referrals to mental health services and substantial burdens for families and organizations (Scott et al., 2001).
Despite the existence of several conceptual frameworks (e.g., Chernyshenko et al., 2018), the Collaborative for Academic, Social, and Emotional Learning (2012) defines social-emotional learning (SEL) as "the processes through which children and adults acquire and effectively apply the knowledge, attitudes, and skills necessary to understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions" (p.9).
Delivering evidence-based SEL programs worldwide is an important pathway to enhance social-emotional competencies.However, the efficacy and effectiveness of SEL programs have been mainly studied in Anglo-Saxon countries, with fewer studies conducted in Ibero-American regions (Fernández-Martín et al., 2021).

"Intergalactic World" program
The "Intergalactic World" was developed in response to the need for SEL programs for children attending primary and secondary schools in Portugal (Cristóvão et al., 2017;Antunes et al., 2022;Antunes, in press), which is considered an Ibero-American region according to some definitions.This eight-session prevention program (cf.details in Table 1) aims to promote self-regulation, selfcontrol, and attentional focus in children aged between 8 and 12 years old.
Based on a literature review (e.g., Sanders, 2008;Webster-Stratton, 2016), the program was designed as a psychoeducational and ludotherapeutic resource.Each session has an average duration of 60 min, with a weekly frequency.The sessions incorporate relaxation dynamics and cognitive-behavioral training (e.g., Ferraioli and Harris, 2013;Black and Fernando, 2014;Raveepatarakul et al., 2014;Vickery and Dorjee, 2015;Huguet et al., 2017), as there is evidence showing that cognitive and behavioral interventions are a promising avenue for promoting social-emotional competencies, and self-control, in particular (Smith et al., 2019).Typically, the program is implemented by two trained group leaders, with one having a mandatory background in psychology.
A pilot study for this program was already conducted with 95 children, although with no control group (Antunes et al., 2022).The results showed that both younger children (8-9 years old) and older children (10-12 years old), regardless of gender, reported a reduction in psychopathological symptoms (anxiety, depression, and stress) and an improvement in overall socio-emotional skills from the pretest to the posttest and follow-up.

The present study
Considering that multiple informants' evaluations provide incremental validity beyond a single type of measurement (e.g., children's self-report measures) and help capture differences in child behavior across different contexts (Alexander et al., 2017), and considering the limited number of studies conducted in Ibero-American regions on this topic, the present study aimed to evaluate the benefits of the "Intergalactic World" SEL program in Portugal from the perspectives of teachers and caregivers.
This study expected to observe a reduction in internalizing and externalizing behaviors from T1 to T2 and T3, with potential gender differences.Specifically, a significant reduction in externalizing behaviors was anticipated in the posttest and follow-up for boys, while girls were expected to exhibit lower overall levels of externalizing behaviors.For internalizing problems, based on the literature review, a significant reduction in internalizing behaviors was anticipated in the posttest and follow-up for girls, and it was expected that boys would exhibit lower overall levels of such behaviors.
By conducting research on SEL programs in diverse cultural and linguistic contexts, including Ibero-American regions, this study contributes to a better understanding of the efficacy and applicability of such programs across different populations.It also addresses the limited number of studies conducted in these regions, providing valuable insights into the potential benefits of the "Intergalactic World" program in Portugal.

Study design
The study employed a pretest-posttest design with a follow-up period and did not include a control group.The data were collected at three different time points: pre-intervention assessment (Time 1, T1), post-program evaluation (Time 2, T2), and 6-month follow-up (Time 3, T3).The T1 assessment was conducted from January to March 2019.Subsequently, the T2 assessment occurred in March and April, 1 week after the completion of the 'Intergalactic World' program.Finally, the T3 assessment was conducted in November and December, representing a follow-up evaluation conducted 6 months after the conclusion of the program, to examine the longer-term effects.

Participants
A total of 154 children, aged between 8 and 12 years (M = 9.59 years, SD = 0.86), participated in this intervention study.All children attend public schools from the urban Lisbon area.Schools from areas of diverse socio-economic status were included.Eleven teachers provided information about these children.Almost all teachers were female (n = 10), with an average age of 42.25 years (SD = 4.62) and an average of 17.29 years of professional experience (SD = 4.15; Min = 11 years; Max = 21 years).
Caregivers provided information about 133 children.The majority of the caregivers were also female (n = 125) and had an average age of 41.14 years (SD = 6.63).Regarding the caregivers' educational status, data showed that 30% of participating families were in the -Welcome and provide information.
-Reflection about personal characteristics, behaviors, and feelings associated.
-Discover and share what you consider to be your potential.

Promoting social-emotional competencies
Main SEL domains: Self-management

Social consciousness
Interpersonal relationship (1) -"An Intergalactic Day." -"School in the Galaxy of Behavior." -"My superpowers" (Super Creativity). (2) -Reflection exercises and interpersonal sharing about their daily routines vs. ideal routines (at school and at home).Sharing of tastes and interests vs. mediated by graph-expressive expression activities (e.g., drawing and free writing).
-Exercise of mime and reflection in small groups.

Promoting social-emotional competencies
Main SEL domains: Self-knowledge

Self-management
Social consciousness

Responsible decision making
(1) -"The Theater of Intergalactic Emotions." -"The Intergalactic Mirror." -"My superpowers" (Super Energy). ( -Reflection on feelings and emotions mediated by images of the four basic emotions (happiness, sadness, fear, and anger) and by dramatization and emotional venting exercises.
-Sharing in small groups and then in a large group: Exercises in groups of two.
-Face to face, define which child is the "mirror, " who copies the movements, and which one defines the movements.Then ask them to copy the movements of each other, in a large group mediated by different music, rhythms, and speeds.

Promoting social-emotional competencies
Main SEL domains: Self-knowledge

Social consciousness
Interpersonal relationship

Responsible decision making
(1) -"Intergalactic relaxation." -"An apple at the 'Intergalactic world.'" -"My superpowers" (Super Attention). (2) -Reflection and relaxation exercises mediated by imagery and dramatization (e.g., Asking them to move around the room as if they were an astronaut on the Moon) and share emotions and feelings associated.
-Emotional ventilation exercises based on the five senses (sight, hearing, smell, touch, and taste), e.g., explore an apple as if it were "the first time" and share the associated experiences.

Promoting social-emotional competencies
Main SEL domains: Self-knowledge

Social consciousness
Interpersonal relationship

Responsible decision making
And attentional focus in particular (1) -"Discover Intergalactic Objects." -The Party in Space." -"My Superpowers": What superpower would you like to receive?Design it and give it a creative name! (2) -Attention games like "find lost objects" in the "galaxy of feelings and behaviors" mediated by images and memorized exercises.
-Exercise in pairs to build a fun moment for a party (e.g., a dance, a game, and a theater) and at the end build the "best party ever" in a large group. (Continued) 10.3389/fpsyg.2023.1233335 Frontiers in Psychology 04 frontiersin.orgmedium-qualified category, 22% were in the medium-qualified, and 49% were in the low-qualified category. 1Although initially 154 children participated in this intervention study, the final sample was limited to 133 children (M age = 9.66, SD = 0.78) due to the availability of the information collected from both parents and teachers.The final sample included a balanced distribution of boys and girls (52% boys and 48% girls).
checklist administered to caregivers to detect behavioral and emotional problems in children and adolescents aged 6-18 years.Caregivers respond to the items on a three-point Likert scale (0-Not True, 1 -Somehow or Sometimes True, and 2-Very True or Often True).The time frame for item responses is the past 6 months.The CBCL includes syndrome scales combined to produce an Internalizing score (Anxious/Depressed, Withdrawn/Depressed, and Somatic Complaints) and an Externalizing score (Rule-Breaking Behavior and Aggressive Behavior).Higher scores in the CBCL indicate increased symptom severity, with T-scores above 69 indicating clinically elevated symptoms.For the Portuguese adaptation, Cronbach's alphas for the dimensions of problems ranged between 0.845 (Internalization) and 0.957 (Total Problems) in the normative sample and between 0.846 (Internalization) and 0.934 (Total Problems) in a clinical sample.In the present sample, Cronbach's alphas vary within the same range, and the average inter-item correlation for all the items on the internalization and externalization scales ranged between 0.66 and 0.72.
Teachers also completed a sociodemographic questionnaire (i.e., age, gender, and years of professional experience) and the European Portuguese version of the Teacher's Report Form (TRF; Achenbach and Rescorla, 2000;Achenbach et al., 2014).The TRF is a questionnaire

Responsible decision making
And attentional focus in particular (1) -The Intergalactic School.
-A House on Mars.
-"My Superpowers" (Super Ideas and Super Happiness). (2) -Reflection exercises about school and friendships mediated by role-plays, mimiques, or drawing "our super planet"-do the same about home and family.
-In a large group, build "the best school and the best house ever" mediated by role-plays, drawings, mimes, dramatizations, and creative writing exercises. 7 Promoting social-emotional competencies Main SEL domains: Self-knowledge Self-management

Social consciousness
Interpersonal relationship
-Work on imagery (e.g., a trip to your own planet "galaxy of feelings and behaviors"-imagine for 5 min; give emphasis to transmission to the others your sensations: smells, colors, details…).
-And after in scenario paper, add all the planets to the "galaxy of feelings and behaviors" and give it an important role.(2) -Drawing exercises and reflection on your "intergalactic friend" mediated by sharing in pairs and then in a large group about characteristics, behaviors and feelings associated with them.
-Ventilation games and emotional expression (e.g., can "Intergalactics" express emotions just like us?), through movement and the body.Role-play exercises, freezes and puppets.

Procedures
This study was conducted as a part of a broader research project, approved by the Ethics Committee of one of the universities to which the authors belong and by the Survey Monitoring System from the Portuguese Education Ministry.
The "Intergalactic's World" was presented to different schools during October and November 2018.Detailed contacts were made with the ones that expressed interest in implementing the program.Informed consent forms providing information about the study's goals, the voluntary nature of participation, and the confidentiality of collected data were provided.Written authorization from the school boards was also obtained.After signing the informed consent, teachers completed the evaluation protocol.Before the beginning of the assessments, caregivers also provided written consent for their children's participation in the study.
The intervention groups comprised approximately 20 children who met weekly for approximately 90 min per session.Each group was led by two trained group leaders who adhered to the structured manual and completed self-evaluations and checklists after each meeting to ensure program fidelity.Moreover, the group leaders received regular supervision.The participant's compliance with the program was excellent (98% attendance rate), with only occasional absences due to illness.The attrition from intervention rate in this study was 2%.
The intervention sessions were conducted in school settings for most children.However, it is important to note that the intervention for 11 children occurred within a Social Solidarity Private Institution (IPSS), even though these children were also attending the same public schools.
The TRF questionnaire was completed with face-to-face support from the research team to clarify doubts and address additional questions.On the other hand, caregivers filled out the CBCL questionnaire at home.Both teachers and caregivers took an average of 2 weeks to answer the questionnaires at the three different time points.The research team provided contacts (such as email and meeting points) to clarify doubts and address additional questions.

Data analysis
Data were analyzed using the Statistical Package for the Social Sciences (SPSS version 27).Distance-based outlier detection methods (i.e., Mahalanobis and Cook's distances) revealed that the dataset did not include any outliers or influential points.
A repeated measures ANOVA was conducted to examine the effects of time (within-subjects variable) and gender (betweensubjects variable) on Internalization and Externalization subscales of the CBCL and the TRF, as rated by caregivers and teachers.Post hoc analyses were conducted, and the effect sizes were reported using partial eta squared.Alpha was set at 0.05 for this model.
In this study, missing data were addressed using pairwise deletion.This method involves considering only the available data for each specific analysis.If a particular data point was missing, it was excluded only for that specific analysis, while the remaining existing data were used in the statistical testing.This approach helps to minimize biases and retain as much information as possible for the analysis.
As Mauchly's test of sphericity yielded significant results, the Greenhouse-Geisser corrected F values were reported.In cases where a significant overall F value was observed, pairwise comparisons using Bonferroni correction were performed to compare the individual time points.

Results
A Repeated Measures ANOVA was conducted to examine the impact of time (within-subjects) and gender (between-subjects) on the Internalization and Externalization scores obtained from the CBCL and TRF questionnaires.Table 2 provides a comprehensive overview of the means, standard deviations, and results of the repeated measures ANOVA for the Internalization and Externalization subscales.
The analysis of the CBCL data revealed a significant effect of time on the Internalization score, F (1.199, 155.886) = 369.929,p < 0.001.This indicated that time substantially influenced the Internalization score, with a moderate effect size (partial η 2 = 0.74).It is important to note that the assumption of sphericity was violated, and Greenhouse-Geisser corrections were applied.The interaction effect between time and gender was not statistically significant (F < 1), suggesting that the relationship between time and the Internalization score did not differ based on gender.To further investigate the effects of time, withinsubject contrasts were performed comparing the scores at T1, T2, and T3 on the Internalization scale.These comprehensive analyses thoroughly investigated the temporal dynamics and uncovered substantial differences across all time points, indicating a statistically significant score reduction at each assessed time point (cf.Table 2).
Regarding the Externalization score of CBCL, results also showed a significant effect of time, F (1.212, 157.580) = 356.379,p < 0.001.This finding indicated that time exerted a substantial influence on the Externalization score, yielding a moderate effect size (partial η 2 = 0.73).
It is worth noting that the assumption of sphericity was again violated, and the Greenhouse-Geisser corrections were used.Moreover, a statistically significant interaction effect between time and gender was observed, F (1, 130) = 6.986, p = 0.009, partial η 2 = 0.51, suggesting that the relationship between time and the Externalization score differed depending on gender: overall, boys showed more externalizing behaviors than girls, but caregivers reported a reduction over time (cf.Table 2).In order to further examine the effects of time, within-subject contrasts were conducted, comparing the scores at T1, T2, and T3 on the Externalization scale.This analysis showed a statistically significant score reduction at each assessed time point (cf.Table 2).
The analysis of the TRF data revealed a statistically significant effect of time on the Internalization score as well, F (1.790, 229.088) = 43.849,p < 0.001.This indicated that time had a substantial impact on the Internalization score, with a moderate effect size (partial η 2 = 0.26).The results showed that the relationship between time and the Internalization score did not differ based on gender, as the interaction effect was not statistically significant (F < 1).To further investigate the effects of time, within-subject contrasts were conducted, comparing the scores at T1, T2, and T3 on the Internalization scale.The within-subject contrasts revealed that only the scores at T3 showed a statistically significant reduction on the Internalization scale (cf.Table 2).
The analysis of the TRF data also demonstrated a significant effect of time on the Externalization score, F (1.655, 211.860) = 29.43,p < 0.001.This indicates that time substantially influenced the Externalization score, with a small effect size (partial η 2 = 0.19).It is important to note that the assumption of sphericity was violated, and Greenhouse-Geisser corrections were applied.The interaction effect between time and gender was also not statistically significant (F < 1), suggesting that the relationship between time and the Internalization score did not differ based on gender.Within-subject contrasts performed comparing the scores at T1, T2, and T3 on the Externalization scale showed that only T3 presented a statistically significant score reduction (cf.Table 2).

Discussion
Evidence-based SEL programs have been implemented worldwide.This study aimed to contribute to the evaluation of SEL programs in Portugal, specifically focusing on the "Intergalactic World" program developed for 8-12-year-old children (Cristóvão et al., 2017).Conducting research on social-emotional learning (SEL) programs in diverse cultural and linguistic contexts has enhanced our understanding of their efficacy and applicability across different populations.
The main objective of this study was to analyze the potential benefits of the "Intergalactic World" program from the perspectives of caregivers and teachers.Adopting a multi-informant approach allowed for a more comprehensive analysis and provided insights into the child's significant contexts and interactions with adults (Alexander et al., 2017).This approach was particularly important when assessing new SEL programs as it helped provide evidence-based information and a more well-rounded perspective on their effectiveness.
Our results are in line with previous findings (Tennant et al., 2017;Scafuto et al., 2022), as teachers and caregivers reported reduced internalizing and externalizing behaviors in children who participated in SEL programs.This reduction was observed immediately after the intervention (T2) and remained evident 6 months after (T3).
However, our results differ from previous studies that have reported small-magnitude improvements in terms of externalizing problems, as perceived by teachers (e.g., Aber et al., 2003;Linares et al., 2005;Hennessey, 2007;Conduct Problems Prevention Research Group, 2010;Jones et al., 2011;Miller et al., 2017).One possible explanation for these differences is that our study focused on assessing short-term effects, as studies on the effectiveness of other SEL programs, such as PATHS in Switzerland (Malti et al., 2011) and 4R in the United States (Jones et al., 2011), which also observed changes in children's aggressive behavior only in the long term (1-2 years after the end of the intervention).
Another potential factor contributing to the contrasting results could be the objectives and contents of the 'Intergalactic's World' program intervention compared to other SEL interventions where teachers perceived short-term changes in externalizing problems.Specifically, school-age SEL interventions, such as Competent Kids, Caring Communities, RBI, RCCP, and Open Circle, lasted for more than 20 sessions and aimed not only to promote self-regulation skills but also to directly address communication skills, problem-solving, and positive peer relationships.On the other hand, the "Intergalactic's World" program aimed to reduce internalizing, externalizing, and other problems (thinking, social, and attention) over eight sessions, incorporating dynamics of relaxation, mindfulness, and cognitive-behavioral training.
Despite its contribution, the study has some major limitations that should be addressed.Firstly, convenience sampling in the urban Lisbon area limits the generalizability of the findings to a broader population.Secondly, the absence of a control group raises the possibility that factors other than participation in the intervention may have influenced the observed results.Thus, the current study design only allows for partial assumptions on the program's effects on internalizing and externalizing behaviors.Moreover, the results found at T3 could be influenced by developmental trajectories, as suggested by previous research (Gutman and McMaster, 2020), and the instructions of the measures used by the present study.For instance, respondents were asked to fill in the CBCL and TRF questionnaires considering the last 2 or 6 months of the child's life, which did not specifically target the end of the intervention but rather the entire intervention period.
Additionally, it is worth noting that between T2 and T3, the children had an extended vacation period due to the summer school break.This period off from school could have impacted the results, and it should be considered a potential confounding factor in the interpretation of the findings.
Suggestions for future studies include employing a randomized control trial design with a larger and more diverse sample from various regions of the country.These studies should also consider evaluating the long-term effectiveness of the intervention by conducting follow-ups 1-2 years after the end of the program.
Additionally, future research should adopt a multi-method approach, incorporating focus groups with children and group facilitators.These focus groups can provide valuable qualitative insights into program implementation and identify variables that may contribute to the program's efficacy (Durlak et al., 2011).
Compelling evidence supports the effectiveness of interventions implemented in school contexts for reducing problem behaviors (e.g., Durlak et al., 2011;Taylor et al., 2017).Moreover, these interventions have positively impacted school achievement (Cristóvão et al., 2017).This study sheds light on the significant contributions of the "Intergalactic's World" program, as perceived by teachers and caregivers, emphasizing the need for a stronger evidence-based approach to the program.
In summary, future research should apply more robust study designs, involve larger samples, and incorporate qualitative approaches to better understand the SEL program's effectiveness and implementation.By doing so, we can further enhance the impact of SEL programs like the "Intergalactic's World" on children's well-being and development.
Superpowers": What superpower would you like to receive? -Design it and give it creative name!-Delivery the Intergalactic Program Diploma.

TABLE 1 "
Intergalactic World" program: goals and session activities(Antunes, in press).