Brief Research Report: Agreement Between Teacher and Student Reports on Students’ Acceptance and Rejection

Student-reported peer nominations are typically used to obtain information on the social status (i.e., acceptance and rejection) of students in their classrooms. However, under certain circumstances this assessment method cannot be used, for example for logistical reasons or with students with intellectual disabilities (ID) who are not always able to fill out peer nominations. In such instances, teacher reports on students’ expected nominations may serve as a proxy. As part of a larger study on students with ID, we evaluated the agreement between teachers’ and students’ peer reports regarding individual students’ acceptance and rejection using unlimited nomination procedures in Grades 4–6 mainstream classrooms. As many students with ID cannot fill out peer nominations, this evaluation study used a sample of typically developing students without ID. Teachers (n = 27, M age = 34.24 years, SD = 10.85; 85.2% female) nominated all peers from the classroom who they believed individual students would report as “liked” and “not liked”. For direct comparison, students (n = 441, M age = 11.36 years, SD = 0.89; 46.8% female) themselves were also asked to report their “liked” and “not liked” nominations. Students received both more “liked” and more “not liked” nominations from their peers than from their teachers. Students’ social status as calculated from teacher reports showed only partial agreement with social status as calculated from student reports, suggesting that in mainstream classrooms student reports cannot be easily replaced by teacher reports. Perspectives on the application of teacher-reported peer nominations in special needs settings are discussed.


INTRODUCTION
Individual social status among peers (i.e., acceptance or rejection) has important implications for students' social and academic development (e.g., Ollendick et al., 1992;Laird et al., 2001;Wentzel et al., 2021). For example, acceptance among peers contributes to the positive development of academic skills (e.g., Wentzel et al., 2021) whereas rejection increases the risk of developing more externalizing behavioral problems (e.g., Laird et al., 2001).
Obtaining information on student social status requires valid, reliable, and preferably economical survey methods. Most studies conducted in school contexts use student-reported peer nominations to assess social status (for an overview, see Cillessen and Bukowski, 2018). In these types of assessments, students report on which class-or schoolmates they like most and which ones they like least (Marks et al., 2013). However, sometimes alternative methods must be used for practical reasons: For example, in large-scale longitudinal studies it can be difficult to obtain sociometric data from all peers in schools (Cillessen and Bukowski, 2018). Moreover, when conducting research among students who have limited competence in filling out peer nomination questionnaires alternative methods are needed. One such group is students with intellectual disabilities (ID), namely students who experience considerable difficulties in intellectual and adaptive functioning (Schalock et al., 2021). While some individuals with an ID in the mild range may be able to fill out peer nominations with adequate help, many students with ID have severe cognitive, linguistic, and social difficulties that make the valid use of peer nominations impossible (see also, Finlay and Lyons, 2001). Further, the assessment of social status with peer nominations ideally includes data from all students in a group, which can pose significant challenges, especially for studies conducted in special needs schools for students with ID. Hence, alternative methods such as teacher reports must be considered (Cillessen and Marks, 2017;Cillessen and Bukowski, 2018;van den Berg, 2018).
The present study aims to evaluate teacher reports on students' acceptance and rejection as a potential alternative to peer reports, especially for use in studies on students with ID. As many of these students are not able to fill out peer nominations, this study was conducted among teachers and students in mainstream classrooms attended by typically developing children. Based on the insights gained, perspectives for using teacher instead of peer reports will be discussed.

Peer and Teacher Reports to Assess Students' Social Status
The typical approach for assessing students' acceptance and rejection among peers is to conduct student-reported peer nominations. Each student is asked to report which other students they like most and least in their class, and nominations are sometimes limited to three classmates per category in the interest of saving time (van den Berg, 2018). However, unlimited nominations are often preferred due to evidence of higher validity (Cillessen and Marks, 2017;van den Berg, 2018). After data collection, the total number of nominations a student received is counted and standardized to the size of the voter population to determine individual acceptance and rejection (Cillessen and Bukowski, 2018). These types of peer nominations are considered highly reliable and valid (van den Berg, 2018). A main advantage of this approach is that several raters provide judgments rather than a single individual. Furthermore, peer nominations provide unique insights by peers that cannot be replaced easily by other methods (Cillessen and Marks, 2017;Cillessen and Bukowski, 2018).
Given that peer nominations are typically considered the gold standard for assessing students' social status, and given the limitations for certain study populations outlined in the introduction, it is important to understand to which extent teacher reports on acceptance and rejection provide comparable results. The literature on this question is heterogeneous. The range of correlations between teacher and student reports on students' social status has varied from low to high, depending on the assessment methods used (e.g., Ledingham et al., 1982;Landau et al., 1984;Huesmann et al., 1994;Babad, 2001;Wu et al., 2001;Andrade et al., 2005;Jackson and Campbell, 2009;McKown et al., 2011;van den Berg et al., 2015;Harks and Hannover, 2017;Marucci et al., 2018;Harks and Hannover, 2019;Wilbert et al., 2020). Some studies compared teachers' ratings (e.g., McKown et al., 2011) or categorical responses (e.g., Andrade et al., 2005) with students' peer nominations. Others used nomination procedures whereby teachers nominated the students they perceived to be most or least liked (e.g., van den Berg et al., 2015). Yet other studies asked teachers to adopt the perspective of each student when nominating peers (e.g., Harks and Hannover, 2019). According to this approach, teachers fill out expected peer nominations for all students in class and individual status is derived from all these nominations (as when using classic student-reported peer nominations). In comparison to teacher's single judgments of students' status, this latter approach can be expected to have greater reliability because information on individual acceptance and rejection is derived from multiple judgments.
A few studies have shed light on the comparability between teachers' expectations of peer nomination results and actual student-reported peer nominations. Wu et al. (2001) asked 48 preschoolers to point to the pictures of the three peers they most liked to play with and the three they least liked to play with at their preschool. Their teachers (n 9) were asked to do the same, adopting the perspective of each student in turn. The authors reported low to moderate correlations between students' and teachers' nomination scores (r 0.26-0.45). In a study by Harks and Hannover (2017), teachers (n 17) were required to nominate a limited number of peers from the classroom (grades 4-10) who they assumed a given student would report liking most. Students reported the same information, but from their own perspective. On average, 30.1% (SD 11.8%) of the teachers' reported peer nominations matched the students' responses. In a similar study Harks and Hannover (2019) in primary and secondary schools reported that teachers (n 20-27) replicated 36-39% of students' individual peer nominations correctly. This percentage of agreement related to the nominations made; non-nominations were not considered. No information on the correlation between students' and teachers' nomination scores was provided in these two studies.
Overall, there is little information on the correspondence of results from teachers' and students' peer nominations. Also, the existing studies on this subject (Wu et al., 2001;Harks and Hannover, 2017;Harks and Hannover, 2019) used limited peer nominations and it is unclear whether the agreement between teachers' and students' reports is affected when asking participants to make unlimited nominations.

The Current Study
This study examined the agreement between teachers' and students' peer reports regarding individual students' acceptance and rejection using unlimited nomination procedures in Grades 4 to 6 mainstream classrooms. Teachers, who are in a position to observe students on a regular base in their classroom, filled out peer nominations from the perspective of each student. Teachers were required to nominate the peers they perceive each student to particularly like and not like. Students answered the same question as teachers to allow for direct comparison.
Our main goal was to assess how teachers and students agree on students' individual acceptance and rejection, as determined from the reported peer nominations. In a first step, we calculated descriptive statistics of the match in peer nominations between teachers' and students' reports. Second, we determined the correlation between teachers' and students' reports for individual students' acceptance and rejection status. Third, we calculated inter-rater reliability for the acceptance and rejection scores as determined from teachers' and peers' reports.

Participants
This study was part of a larger research project named "KomPeers" (Müller et al., 2020) that focused on Swiss special needs schools for students with ID. Within this project, the present study contributed to the evaluation of a teacher-based measurement tool to assess students' social status within special needs schools. As most students with ID cannot fill out peer nominations, this evaluation study used a separate sample of typically developing students without ID and their teachers in Swiss regular schools. Teachers and students from 13 Swiss primary schools were surveyed. Data assessment occurred approximately two to three months after the beginning of the school year (October/November 2018).

Teachers
The head teacher of each class was surveyed. The mean age of the 27 teachers was 34.24 (SD 10.85) years and 85.2% were women. The teachers had taught their class for an average of 22.81 (SD 3.65) lessons (45 min each) per week, had known their students for M 8.62 (SD 7.00) months, and had an average work experience as teachers of 9.35 years (SD 9.72).

Students
The 441 participating students attended Grades 4 (4 classes), 5 (9 classes), 6 (9 classes), 5-6 (3 classes with cross-grade learning), 4-6 (2 classes with cross-grade learning). They were M 11.36 (SD 0.89) years old. On average classrooms were attended by approximately 20.05 students (SD 1.96, range 14-24). The percentage of female students was 46.8%. The overall participation rate was 83.68% (i.e., total of 527). Information on the remaining students was not available due to parental decision on behalf of their child to decline participation. Two students did not fill out a questionnaire and two were absent during data collection.

Teachers' Nominations
Teachers were given a numbered list of all their students who participated in the study (i.e., those with parents' informed consent) and were asked to adopt the perspective of each participating student in nominating students from class they thought the respective student liked (i.e., acceptance: "Who does this student like especially in class?") and not liked (i.e., rejection: "Who does this student like not so much in class?"). Teachers could nominate as many students as they considered appropriate. To determine individual acceptance and rejection in class, the nominations each student received were counted and standardized within class (i.e., resulting in a percentage of nominations received from all possible nominations received).

Students' Peer Nominations
Students were given a numbered list of all classmates who participated in the study. Just as with the teachers, they were asked to nominate the classmates they liked ("Who do you like especially in class?") and not liked ("Who do you like not so much in class?"). Unlimited nominations were allowed and the same procedure of classroom standardization was used to calculate individual acceptance and rejection in class.

Match Between Teachers' and Students' Peer Nominations and Non-Nominations
In order to calculate the match between teachers' and students' nominations, we considered each nominating student in the sample. For each nominating student it was analyzed whether the teacher made the same nominations as the student. The percentage of matches of nominations by the teachers in relation to all nominations given by the students was then calculated. The same procedure was used for non-nominations. To explain this, we refer to an example for the category of acceptance: There are six students in a classroom. Student number 1 has nominated student number 2, 3, and 4 as liked, but did not nominate students 5 and 6 as liked. Taking the perspective of student 1, the teacher nominated students 2 and 3 as liked, but not students 4, 5 and 6. This means that out of three nominations by student 1 (2, 3 and 4; i.e., 100%), the teacher nominated two students (2 and 3) correctly. The teacher therefore agreed with student 1 in 66.7% of all possible peer nomination decisions. The same procedure was used for non-nominations. To assess the overall match, these calculations were made for all students in the sample and a mean score was determined for "liked" and "not liked" nominations.

Procedure
Written information about the study was provided to school headmasters of primary schools in the German-speaking part of Switzerland. If headmasters gave permission for their school to participate, teachers were informed about the study by their headmasters and were asked whether they wanted to participate. For teachers who agreed to participate, the parents of children attending those classrooms were sent a letter informing them about the nature of the study, including that anonymity was guaranteed for them and their child. Active written parental consent was obtained for all participating students. During assessments each student was given a unique number, such that all teachers and students reported student numbers and not student names. Trained Master's students introduced the questionnaires in classrooms in detail, and emphasized that no individual-level data would be given to teachers or parents. Students were given the opportunity to ask questions and they completed the questionnaires using privacy shields to allow for optimally independent answering. Teachers filled in the teacher reports at the same time (without students' knowledge).

Statistical Analyses
First, descriptive statistics of the key variables were determined. In order to report general differences between teacher and student reports paired sample t-tests were performed to test whether the number of nominations received from the teacher's perspective differed from the number of nominations received from peers. Moreover, paired sample t-tests were used to assess whether the number of "liked" nominations received differed from the number of "not liked" nominations received when comparing within teacherbased and student-based nomination reports. To examine our main research question regarding the agreement between teachers' and students' reports of acceptance and rejection, the match between teachers' and students' peer nominations and non-nominations was first determined (see Measures section). Second, correlations between the percentage of all possible nominations in class received from teachers and the percentage of all possible nominations in class received from students were calculated for both acceptance and rejection. These analyses were conducted using Mplus version 8 (Muthén and Muthén, 2017), accounting for nonindependence of students within the same classroom (i.e., nested data; Level 1: students; Level 2: classrooms). Third, to assess interrater reliability Cohen's Kappa that corrects for chance overlap was calculated for teachers' and peers' reported levels of social acceptance and rejection.

RESULTS
Starting with descriptive information, it became evident that within the student-reported data students received M 18.63% (SD 12.64%, range 0-68.75%) of the possible "liked" nominations in their classrooms and M 16.38% (SD 16.54%, range 0-100%) of the "not liked" nominations. By contrast, within the teacher-reported data, students received M 11.56% (SD 10.74%, range 0-63.64%) of the possible "liked" nominations and M 5.72% (SD 12.06%, range 0-94.12%) of the "not liked" nominations. A paired sample t-test revealed that students received significantly more "liked" nominations (p < 0.001; Cohen's d 0.6, medium effect size) and more "not liked" nominations (p < 0.001; Cohen's d 0.718, medium effect size) from their peers than from their teachers. Within the nomination formats, paired sample t-tests further showed that students received significantly more "liked" nominations than "not liked" nominations when assessed by teachers (p < 0.001; Cohen's d 0.512, medium effect size). In contrast, the difference between the number of "liked" and "not liked" nominations received from peers was not significant. It must be noted that the t-tests conducted for these explorative analyses do not take into account the multilevel structure of the data.
In order to answer our research question on the extent to which teacher and student reports agreed on students' acceptance and rejection, we first considered the degree of matching nominations and non-nominations between teachers and students. Teachers matched students' "liked" nominations in M 42.73% (SD 37.1%, range 0-100%) of the cases (see Figure 1). The remaining 57.27% were nominations by the students not replicated by the teachers. Regarding students' non-nominations as "liked," teachers matched in M 94.71% (SD 9.15%, range 0-100%) of the cases. The remaining 5.29% were non-nominations by the students that were nominated by the teachers.
Regarding "not liked," teachers had a match with their students' nominations in M 15.83% (SD 29.15%, range 0-100%) of the cases (see Figure 2). The remaining 84.17% were student nominations not replicated by teachers. In M 96.16% (SD 8.1%, range 0-100%) of the non-nominations of the students, teachers agreed with their students. The remaining 3.84% of the cases were non-nominations by the students, which were nominated by the teachers.
Second, correlations were run between the social status of students as determined by teachers' and students' reports. Thus, these analyses no longer related to the match of single nominations received (as considered above), but to the percentage of nominations individuals received relative to all possible nominations in their respective classes. For these analyses it must be noted that the same social status of a student as determined by teachers and students may be due to differing underlying nominations (e.g., student 1 may have received nominations by student 2 and 3 but the teacher may have reported nominations by students 4 and 5, both resulting in the same social status score of student 1). Individual acceptance calculated from the teacher reports was significantly positively correlated with acceptance calculated from the student reports (r 0.422, p <.001, medium effect size). This indicates that the more accepted students were among their peers from the teachers' perspective, the more they were also accepted from their peers' perspective. The teachers' and students' ratings of rejection were significantly positively correlated as well (r 0.566, p <.001, large effect size). That is, more peer rejection as estimated by teachers was related to more rejection as assessed by peers.
Third, to determine inter-rater reliability regarding students' status correcting for chance overlap Cohen's Kappa was calculated. For acceptance, Cohen's Kappa was 0.21, indicating a fair agreement between teachers and peers (Landis and Koch, 1977). For rejection Cohens' Kappa was 0.12, indicating a slight agreement.

DISCUSSION
The aim of this study was to increase our knowledge on the agreement between teacher and student reports on students' acceptance and rejection. Our results found only partial agreement between teacher and student reports using unlimited peer nominations, with regard to individual students' acceptance and rejection. We will discuss this finding in terms of whether teacher reports can generally substitute peer reports, and separately for the case of students with ID who often cannot provide peer nominations.
Frontiers in Education | www.frontiersin.org October 2021 | Volume 6 | Article 726854 On average, students were nominated as "liked" by about 19% of their peers, which is similar to other studies that used the same standardization method of received peer nominations (e.g., 17%; Chang, 2004). Students were nominated as "not liked" by approximately 16% of peers, which is lower than in other studies (e.g., around 25%; García-Bacete et al., 2019). Students received both more "liked" and more "not liked" nominations by their peers than by their teachers. That is, teachers apparently identified fewer social relations among students relative to those identified by students themselves. One explanation could be that certain student relationships are more prominent outside of the classroom and are thus less observed by teachers (e.g., breaktime, way to school). Furthermore, our results show that students received more positive than negative nominations from the teachers' perspective while no such difference was seen in the student reports. Possibly, teachers are less aware of negative relationships between students because these may not always be displayed in public and in front of teachers.
Regarding agreement between teachers and students, teachers' nominations had a match of about 43% with students' "liked"nominations, which is slightly above from what was reported in earlier studies (e.g., "liked" nominations: 36-39%, Harks and Hannover, 2019). Less agreement was found for "not liked" nominations, where there was a match on only 16% of all nominations. As mentioned above, lower agreement regarding "not liked" nominations may be related to lower visibility of negative in contrast to positive peer relations for teachers. This issue may have been specifically relevant given that assessments FIGURE 1 | Teachers' agreement with students' "liked" nominations and "liked" non-nominations.
FIGURE 2 | Teachers' agreement with students' "not liked" nominations and "not liked" non-nominations.
Frontiers in Education | www.frontiersin.org October 2021 | Volume 6 | Article 726854 were conducted only two to three months after the beginning of the school year. Results from Pittinsky and Carolan (2008) suggest that teachers' correct estimations of peer relations in class increase over the school year as teachers get to know students better. Teachers agreed with their students for about 95% of the non-nominations. While this suggests that teachers and students agree, more often than not, on who does not "like" or "not like" each other, this result should not be overestimated as it is strongly related to the overall low numbers of nominations in the classrooms. While the just reported findings indicate that teachers in regular classrooms experience difficulty in correctly predicting students' individual peer nominations, a main goal of conducting peer nomination procedures is to determine students' individual social status in class. When individual scores of acceptance and rejection within the classroom were calculated, teacher and student reports correlated moderately to largely in the expected directions. However, inter-rater reliability correcting for chance overlap resulted in only slight to fair levels of agreement between teachers and students. This picture parallels findings from studies that used a similar approach to ours with limited number of nominations, resulting in low to moderate agreement between teacher and student reports (Wu et al., 2001;Harks and Hannover, 2017;Harks and Hannover, 2019). Hence, unlimited nominations, as used in the present study, apparently did not much enhance agreement between teachers and students.
Taken together, it can be concluded that the teacher report method in mainstream classroom settings cannot easily replace students' peer nominations (an exception may be when the teachers' and not the students' view of peer status is the construct of interest in a study). Although showing a certain amount of agreement, there is less argument against considering students' peer nominations as the gold standard, measuring unique insights by peers themselves and having proved high validity and reliability (e.g., Cillessen and Bukowski, 2018). Limitations of teacher reports may not only relate to agreement with the students' view, but reporting on all relationships in large classrooms or even among all students from a grade-level may be tiring for teachers and could negatively affect reliability.
However, our results should also be considered in terms of potential chances that teacher reports may provide for use in special needs classrooms for students with ID, where researchers are faced with less assessment options. Some students with milder forms of ID may use pointing techniques or be interviewed to assess their peer preferences (e.g., Male, 2002). In case of students with profound ID, who have very low verbal abilities and sometimes severe behavior problems, this is often not possible. Still, these individuals in many European countries represent a major group in special needs schools for students with ID (e.g., Dworschak et al., 2012;Müller et al., 2020). Therefore, besides extensive direct behavior observation of individual students there is often less alternative than to assess what teachers perceive in their classroom. For this school setting, the present results of a certain agreement between teachers and students are more encouraging than when considering mainstream classrooms where alternative peer nominations can be relatively easily applied.
Still, the present study conducted in mainstream classrooms has several limitations in terms of concluding on the usability of teacher reports in special needs settings. The diverging characteristics of special needs classrooms for students with ID compared to mainstream classrooms may affect validity and reliability of teacher reports in different ways. Generally, special needs teachers are specifically trained and mandated to assess and foster students' social relationships, which may serve reliability of their reports. Increased agreement between teachers and students in special needs classrooms can be also expected as these classrooms are small (usually only 5-10 students), serving more accurate teacher observations of peer relations. Furthermore, special needs teachers spend the whole schoolday (including meals) with their students, providing for more opportunities for observation than in Swiss mainstream classrooms. These expectations are supported by findings from Harks and Hannover (2019), showing that the smaller the classroom size and the more lessons a teacher taught in a classroom, the greater the agreement between student and teacher responses. Another factor is that many students with ID have problems in impulsiveness and emotion regulation (e.g., Dykens, 2000). It may be therefore more easy to detect the peer preferences by these students compared to those of typically developing students, especially in terms of determining who a student dislikes. Nevertheless, the difficulties of teachers in replicating the students' view in mainstream classrooms sound an important note of caution also for the interpretation of assessment results in special needs settings.
In conclusion, teachers appear to be able to recognize only part of students' social relations in primary school classrooms, suggesting that this approach cannot substitute peer nominations for the assessment of social status. However, due to a lack of alternatives in certain special needs settings, asking teachers to report on expected peer nominations may serve as a starting point to advance research on the social status of students with ID. Future studies should aim to better understand which measurement features, individual student and teacher characteristics, as well as contextual factors contribute to increased levels of agreement between teachers and students.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent to Frontiers in Education | www.frontiersin.org October 2021 | Volume 6 | Article 726854 participate in this study was provided by the participants' legal guardian/next of kin.

AUTHOR CONTRIBUTIONS
NS-K organised data collection, conducted all analyses, and wrote this paper. CM is the principal investigator of the study which this project is part of. He contributed to conception and design of the study and to manuscript revision. Both authors read and approved the submitted version.

FUNDING
This study is part of a larger research project funded by the Swiss National Science Foundation (grant 172773).