Some scholars have made efforts to distinguish the difference between “teaching effectiveness,” “teacher’s quality,” and “teaching quality” (Al Ansari et al., 2020; Cai and Wang, 2022; Cherng et al., 2022). Teaching quality refers to a teacher’s innate qualities, skills, and competencies (Chan, 2002), while when considering practices and instruction, emphasis is placed on the quality of teaching (Bradney, 1996; Hansen, 2023). Teaching effectiveness is analyzed in light of student outcomes, ensuring students learn and achieve specific outcomes (Sofyan et al., 2021). The latest studies on this topic have been used to offer new knowledge and concepts about performance evaluation, effectiveness (Al Kuwaiti et al., 2021), and the pedagogical skills of teachers in higher education (Hansen, 2023; Monzalve-Macaya et al., 2023; Pham et al., 2023). On the other hand, the review of the literature shows that various theoretical models measure this construct (Sánchez-Cabrero et al., 2021; Sofyan et al., 2021; Matosas-López, 2023); such as the one proposed by Rocha (2013) that evaluates teaching effectiveness from 4 approaches: teacher-student relationship, teacher’s personality, student evaluation performed by the teacher, and his teaching method. His proposal is close to that of other specialists who in turn analyze the variable from 4 perspectives (Calaguas, 2012; Shahzad and Mehmood, 2019; Ayaneh et al., 2021). However, Aleamoni and Hexner (1980) suggest a 3-component theoretical model: Instructor evaluation, course-specific teaching procedure, and student perceptions of learning outcomes. Resembling other studies that propose the study of this construct from 3 factors (Marshall et al., 2016).

However, for this study, the four dimensions proposed by the Ministry of Science and Higher Education of Ethiopia (MOE) and the latest scientific contributions of Ayaneh et al. (2021), who consider a better theoretical model of four dimensions: subject matter knowledge (SK), professional competence (PC), ethical competence (EC), and time management (TM). Subject matter knowledge (SK) is the ability of the teacher to meet the content of the course, demonstrating preparation and using common examples to achieve the objectives of each session. Professional competence (PC) refers to the degree to which the teacher uses his or her knowledge, skills and good judgment related to professional skills to perform his or her main task with acceptable quality. Ethical competence (EC) is understood as the teaching competence that reconsiders promoting ethical education in classroom activities, conferences and through extracurricular activities. And time management (TM), known as the ability to make effective use of teaching time to efficiently fulfill their academic responsibilities.

Most of what has been reported on this construct has been of great contribution to improving the quality of education at all educational levels. In this sense, a review of previous research confirms the importance of providing valid instruments that can measure teaching efficacy in the context of higher education. These measurement instruments must comply with valid psychometric properties in order to be used in different realities. However, it is important to note that so far, teaching efficacy remains one of the most difficult constructs to measure. Various researchers have carried out studies on teaching effectiveness using different instruments (Adekanmbi and Ukpere, 2021; Anwar et al., 2021; Tadesse et al., 2021; Monzalve-Macaya et al., 2023). These studies can be separated into two groups. The first group of studies reviewed present reliability and validity.

In Ethiopia, a previous study presents the validity of the Student Evaluation of Teachers’ Effectiveness (SETE) scale that was applied to a sample of university students, which presents a 2-factor model, 18 items and α = 0.79, and a second 4-factor model, 20 items and α = 0.80 (Ayaneh et al., 2021). In 2019, Shahzad and Mehmood designed the Teaching Effectiveness Scale (TES), which was applied to university students in Pakistan; it has 32 items and 4 dimensions (α = 0.71 to 0.87). Marshall et al. (2016) developed and validated the Teacher Intentionality of Practice Scale (TIPS) in secondary school teachers in the United States; the scale confirmed 22 items and 3 dimensions (α = 0.96). Kyrgiridis et al. (2014) developed the Self-Evaluation of Teacher Effectiveness in Physical Education (SETEQ-PE) questionnaire and applied it to Greek physical education teachers. This questionnaire has 25 items and 6 dimensions (α = 0.87). Moreover, in Rocha (2013) study, he designed a Student Opinion about Teacher Effectiveness (SOTES) questionnaire and applied it to Mexican undergraduate students. It has 17 items and 4 dimensions (α = 0.947). Calaguas (2012) developed and evaluated the psychometric properties of the Teacher Effectiveness Scale in Higher Education (TESHE) in university students in the Philippines; it has 67 items and 4 dimensions (α = 0.972). In addition, the Student Evaluation of Educational Quality (SEEQ) presented 35 items discriminated in 8 dimensions (α = between 0.88 and 0.97): (1) Learning, (2) Enthusiasm, (3) Organization, (4) Interaction with the group, (5) Updated presentation of the subject, (6) Interaction of the teacher with the students individually, (7) Evaluation, (8) Feedback (Marsh, 1983; Marsh and Roche, 1997).

A second group was identified, which was made up of some studies that did not describe the validity or reliability of the instrument. This is the case with Mohebbi et al. (2022), who investigated the factors contributing to the effectiveness of language teachers in Iran and designed an instrument (EFL Language Teachers’ Effectiveness) with 18 items and 6 dimensions. On the other hand, in the study by Nema et al. (2023) they used the Student Evaluations of Teachers (SET) as a measurement tool, which has been widely used for students in colleges and universities in India (Aleamoni and Hexner, 1980). The metric consists of 33 items subdivided into 3 factors: instructor evaluation, teaching procedure in a specific course, and students’ perceptions of learning outcomes. Based on the above, the purpose of this research is to translate, adapt and evaluate the validity and reliability of the SETE scale in a sample of Latin American university students. Several specialists argue that the SETE scale captures multiple aspects of a university teacher’s good practices and that its periodic revision in different cultural contexts could help educational leaders to improve their educational and teaching management policies and strategies.

The work responds to psychometric research, defined as the science of evaluating the characteristics of tests designed to measure psychological attributes (Price, 2017). The study population was composed of university students from four South American countries, Peru, Bolivia, Chile, and Colombia, belonging to a private educational network. A condition to be part of the study was that the university students were studying the academic semester in person at the time of the survey. It should be noted that this study was approved by the Ethics Committee of the Postgraduate School of a private university (2023-CE-EPG-00071) and was conducted under the ethical standards of the Declaration of Helsinki (Manzini, 2000; Puri et al., 2009). The study was applied from September to December 2023. Non-probabilistic convenience sampling was applied (Otzen and Manterola, 2017), and the survey was carried out through a virtual link; the questionnaire was hosted on a Google form, and a Likert-type response format was utilized, ranging from (1) never to (5) very frequently. The questionnaire was self-administered, and informed consent was obtained from each participant before its administration (‘I acknowledge that by completing this questionnaire, I am giving my consent to participate in the study’). The questionnaire was shared virtually (via Email, WhatsApp, Messenger, Instagram) and in a personalized way. The total sample was 1,000 university students, who provided their answers anonymously and voluntarily. Table 1 shows the frequencies and percentages by categories (sex, age range, marital status, country of origin, university campus, year of study, and religious inclination).

The SETE scale is a harmonized instrument used to measure teacher effectiveness. Highly qualified experts originally developed it by the Ministry of Science and Higher Education (previously, Ministry of Education) of Ethiopia (MOE, 2018). Ayaneh et al. (2021) later evaluated its psychometric properties analyzing 2 models (4 and 2 factors), managing to confirm 2 factors (CFI = 0.999, TLI = 0.999, SRMR = 0.056; RMSEA = 0.008). The scale demonstrated good internal consistency (α = between 0.87 and 0.93) in a sample of Ethiopian university students.

The original version of the SETE scale required translation from its original English language to Spanish using a bilingual trial back-translation method. Three bilingual (Spanish-English) Spanish-speaking individuals completed English-to-Spanish translations of the SETE scale individually. The translations were compared, discussed, and reviewed in a focus group of six university students (Peru, Bolivia, Chile, and Colombia) who met the study’s inclusion criteria to obtain the first complete version in Spanish of the scale and its contextual application to the four selected South American countries. The English and Spanish versions of the SETE scale were tested on a target group of bilingual individuals before some final changes were made and distributed to the study sample.

Two statistical software were used to analyze the data: (1) SPSS software version 25 for descriptive analysis (sociodemographic profile of the participants, among others.) and Exploratory Factor Analysis (EFA), and (2) then Structural Equation Modeling of covariance (CB-SEM) to perform Confirmatory Factor Analysis (CFA), evaluate convergent and discriminant validity, and adjust the measurement model. This required AMOS version 24 software. This method is highly recommended to evaluate the psychometric properties of measurement models (Fornell and Larcker, 1981). Likewise, reliability was evaluated using Cronbach’s Alpha coefficient and composite reliability.

From the validation of Ayaneh et al. (2021), the proposal of 20 items was taken as a basis, where a group of educational specialists evaluated the saturated items. The scale was organized according to the content validity (Table 2) process by expert judgment (30 items), structured in 4 dimensions (SK = subject knowledge, PC = professional competence, EC = ethical competence, and TM = time management). For this analysis, six experts in university higher education with a minimum of 10 years of experience were recruited to serve as judges. At the expert committee’s suggestion, 2 items were removed (16 and 19). To analyze the results, Aiken’s V Coefficient (V > 0.50; 95% CI) was used, considering the criterion value for deciding which items should be eliminated, revised, or withdrawn. Finally, the scale was left with 28 items to be included in the virtual questionnaire. All the items were evaluated regarding clarity, relevance, and relevance to the construct (Table 2).

On the other hand, indicators such as mean, standard deviation, skewness, and kurtosis were obtained for each of the items applied in this study. The skewness and kurtosis results are nearly zero, meaning the distributions are symmetrical. Furthermore, variability is denoted in the mean, which shows the diversity of responses regarding perceptions among the study participants, as shown in Table 3.

To identify the factorial condition of the scale, an Exploratory Factor Analysis (EFA) was carried out on each element, observing that the items were distributed into four factors according to the construct analyzed (Table 4). The difference is quite clear between the four factors. The KMO and Bartlett test (Kaiser-Meyer-Olkin correlation coefficient = 0.975) has a value greater than 0.7 and the Bartlett test (Sig = 0.000) is very significant for performing factor analysis. The total variance explained in the model is 73.66%, which is greater than 50%, with Subject Knowledge (SK) = 57.15%, Professional Competence (PC) = 7.59%, Ethical Competence (EC) = 5.21%, and Time Management (TM) = 3.71%. All items have been grouped according to their original dimensions. Next, Confirmatory Factor Analysis (CFA) was performed.

The validation of the final measurement model is shown in Table 5 along with the convergent reliability and validity. Cronbach’s Alpha (α) values range between 0.927 and 0.961, considered satisfactory values since all levels of this coefficient must be above 0.70 for the model to be valid (Agbo, 2010). Furthermore, the reliability values (CR) were found between 0.927 and 0.962, which is favorable because this value must be greater than 0.70 to be considered a perfect model (Bagozzi and Yi, 1988). Likewise, the AVE values are between 0.646 and 0.799, which are considered acceptable since this index must be equal to or greater than 0.50 (Hair et al., 2014). In that sense, these values translate as an acceptable measurement model that meets favorable levels of reliability and convergent validity.

Table 6 shows each indicator of model fit that measures teaching effectiveness, showing acceptable and excellent measures.

Regarding discriminant validity (Table 7), the results show that the confidence intervals, in none of the cases, reach unity. Additionally, the quantile covariances do not exceed the AVE; therefore, there is clear evidence of discrimination between the constructs subjected to evaluation (Fornell and Larcker, 1981).

To provide further strength in evaluating discriminant validity as a requirement for analyzing the relationships between the latent variables, Table 8 presents the results regarding discriminant validity using the heterotoit-monotrait relationship criterion (HTMT); in this case, the values are less than 0.90. Therefore, it is stated that the discriminant validity between two reflective constructs has been established (Henseler et al., 2015). Additionally, Figure 1 shows the factor structure of the SETE scale in a sample of university students from Latin America.

Finally, the final version of the instrument, which underwent rigorous content validity, EFA, and CFA processes to ensure reliable psychometric properties for use, is described (Table 9). It is made up of four factors: 06 items for Subject matter Knowledge (SK), 14 items for Professional Competency (PC), 04 items for Ethical Competence (EC), and 04 items for Time Management (TM).

The objective of the present study was to evaluate the validity and reliability of the SETE scale (Ayaneh et al., 2021) in the Latin American context. This is the first study in which the evidence and reliability of this scale are published in this context. Other validations of the same construct have been carried out in other latitudes (Calaguas, 2012; Rocha, 2013; Kyrgiridis et al., 2014; Marshall et al., 2016; Shahzad and Mehmood, 2019; Adekanmbi and Ukpere, 2021; Mohebbi et al., 2022). The scales review found a diversity of factors associated with this construct, ranging from single-factor scales to scales with 10 factors (Shahzad and Mehmood, 2019; Li et al., 2024). The dimensions presented in these scales share characteristics with others that measure the same construct, although to date there is no consensus on their dimensionality.

Originally the SETE scale was validated in university students from Ethiopia with 20 items, however, in this new version for Latin America the items went through a rigorous adaptation process, resulting in 30 items. This is because the original scale items were found to address more than one specific topic, saturating them. To take an example, the factor “Professional Competence” (PC), originally had the item “Follows the continuous evaluation approach and gives feedback on continuous evaluations on time.” This item, when going through the process of semantic validation by expert judgment, it was considered that the item structure should be reformulated to make it clearer and more specific (PC9: Permanently evaluates students, PC10: Resolves tests when an evaluation is concluded). Based on the opinion of psychometric specialists who argue that by analyzing, eliminating or reformulating saturated items, the validity and reliability of the scale can be improved, ensuring that the instrument measures what it is intended to measure accurately and consistently (Matas, 2018). This explains why the 4-factor model lacked discriminant validity and probably better explains why the CFA was deficient. The adaptation carried out in the present study showed that the factors SK = Subject knowledge, PC = Professional competence, EC = Ethical competence, and TM = Time management, fit the model. Although two items (items 16 and 19) had to be eliminated from the initial 30, the results demonstrated good internal consistency (α = between 0.927 and 0.961; CR = between 0.927 and 0.962; AVE = between 0.646 and 0.799). The final version of the scale consists of 28 items.

The SETE scale is now ready to be used as a valid and reliable tool to evaluate the effectiveness of teachers in the university teaching process. Teachers can use this measurement tool to understand what students perceive as their strengths and weaknesses. College students can also use this information to make more informed course selections. This can be applied in future research to find correlations, predictors, and moderators of this construct. Institutions can also use it to make decisions about their teachers. Although the SETE scale has good psychometric properties, it is recommended that its revision be reconsidered in the future. It needs to be revised and updated periodically to meet the rapidly changing needs of university students, teachers, and educational institutions. In addition, these results can be the platform for the creation of innovative policies, strategies and educational programs that increase university students’ perception of teaching effectiveness, thus contributing to an increase in the quality of education in Latin American countries.

The results of this study should be considered taking into account the following limitations: One limitation is that, although the scale was adapted and harmonized to be applied to all private universities in the Latin American countries of South America (whose languages derive from Latin, mainly Spanish and Portuguese), this analysis only used data from some countries (4/10), which may not be generalizable to other private universities in South America. In this sense, this study highlights the need to obtain a large amount of data from multiple universities and representative samples from each South American country to strengthen existing findings further. On the other hand, the questionnaire assumed that students’ evaluations of teachers were free of prejudices or stereotypes. However, university participants with high grades in their subjects are likely to give higher scores on the questionnaire. In contrast, participants who obtained low grades in their subjects consider this evaluation a form of retaliation against their teachers.

In addition, the physical attractiveness of the teacher, the time taken to complete the survey, the proximity to a teacher, the difficulty of the course, and the teacher’s age, performance, and personality can become determining factors in students’ evaluations (Calaguas, 2012; Kim et al., 2019; Shahzad and Mehmood, 2019; Sofyan et al., 2021). It is also recommended to consider the semester for the application of the questionnaire, given that taking it in the first (students are not entirely familiar with the teacher’s methods) or last (students with failing grades, in a spirit of retaliation, among others) weeks of class can create study bias. In addition, in future studies, it is recommended that some indication or filter be included in the questionnaire to determine whether the subjects considered are all the subjects of the academic period or only the compulsory ones, since this could influence the evaluation of the students. Finally, the “Professional Competence” (PC) factor was validated with 14 items, the broadest dimension of the scale, which may be another limitation of this study. However, despite these limitations, the study’s findings will significantly help university education professionals address teacher effectiveness problems.