Teachers’ technological (pedagogical) knowledge–predictors for students’ ICT literacy?

1 Introduction

Since information and communication technologies (ICT) literacy is integral to the curriculum, teachers are responsible for promoting ICT literacy among secondary students (Guggemos and Seufert, 2021). Accordingly, addressing teacher-level factors that may impact secondary students’ ICT literacy is essential. Research has already shown that teachers’ personal factors, such as their ICT self-efficacy (Gerick et al., 2017) and perceived usefulness of ICT (Scherer and Siddiq, 2015), are important determinants of students’ ICT literacy. However, despite their scientific relevance, these factors represent teachers’ subjective self-assessments or attitudes. Therefore, it is increasingly believed that more objective factors, such as professional knowledge, should be investigated about teachers’ competent ICT use and their impact on secondary students’ ICT literacy (Schmid et al., 2021).

Studies have already shown that teachers’ objectively assessed, subject-related knowledge—also called content knowledge (CK)—influences students’ performance in the respective subject (Baumert et al., 2010). More specifically, scholars have concluded that teachers with high CK are more consistent with the content that they teach during instruction and may, therefore, have students who achieve more in the subject. Furthermore, Baumert et al. concluded that in addition to the relevance of CK, teachers’ pedagogical content knowledge (PCK) is also a key determinant of student achievement. PCK refers to instructional quality—that is, a teacher’s ability to teach subject-related content in a way that improves student achievement (Shulman, 1986).

However, apart from subject-specific competencies, there still needs to be more knowledge about how teachers’ cross-curricular competencies, such as technological knowledge (TK) or technological-pedagogical knowledge (TPK), affect students’ cross-curricular competencies, such as ICT literacy. Considering the importance of instructional quality to technology integration, TPK must be addressed as a link between teachers’ TK and students’ ICT literacy, as TPK is a prerequisite for the successful integration of ICT into instruction across all content and domains (Lachner et al., 2019). Therefore, teachers need TK and TPK to incorporate technology in the classroom, which could explain the differences in the performance of ICT literacy among secondary students. Furthermore, according to Koehler and Mishra (2009), TPK plays a central role, as some programs, related to technology, such as spreadsheets, were not initially designed for educational purposes but are now embedded in curricula [Standing Conference of the Ministers of Education and Cultural Affairs (Kultusministerkonferenz, 2019)].

In summary, previous studies have shown that teachers’ professional knowledge is indispensable for students’ subject-specific achievements. This is based on the assumption that teachers who are more proficient with subject content can better convey it to students in the classroom. Although the importance of teachers’ subject-related professional knowledge to student achievement has been demonstrated, little attention has been paid to the relationship between teachers’ cross-curricular professional knowledge and student achievement. Thus, this study extends previous research by examining the extent to which teachers’ objectively measured TK and TPK can explain achievement differences in secondary students’ ICT literacy.

1.1 Secondary students’ ICT literacy

With ICT literacy as a cross-curricular subject, the competent use of ICT has gained considerable attention in educational research. Due to the increasing relevance of competent ICT use in educational research, Siddiq et al. (2016) identified a myriad (p. 60) of terms that describe the competent use of ICT, which is also consistent with current publications (see Gnambs, 2021). According to Gnambs (2021), many concepts related to ICT literacy overlap and cannot be distinguished on the basis of clearly defined theories. However, a generally accepted theoretical framework on competent ICT use was developed by Educational Testing Service (2002, p. 2) and later used in the National Educational Panel Study (NEPS, see Senkbeil et al., 2013). Within this framework, ICT literacy is defined as the use of digital technology, communication tools, or networks to access, manage, integrate, evaluate, and create information to function in a knowledge society. According to this definition, and in light of the integration of ICT literacy into curricula, ICT literacy is adopted as a cross-curricular competency for secondary school students to acquire essential competencies and skills that are relevant in educational and professional environments to successfully participate in society (Senkbeil et al., 2013; Fraillon et al., 2020).

The widely used ETS framework regarding ICT literacy consists of two overarching facets: technological literacy and information literacy (Educational Testing Service, 2002). Technological literacy refers to competence in the operational use of technology, while information literacy describes the information skills required to use the technology to exchange and evaluate information accordingly (Senkbeil et al., 2013). In addition, ICT literacy is divided into process components (e.g., access to retrieve information) and five software applications for finding, processing, presenting, and communicating information (e.g., Internet-based search engines and databases). The ICT literacy framework has already been empirically tested (see Senkbeil et al., 2013).

The literature exhibits a high degree of variability in the terms and definitions used to describe the competent use of ICT. Therefore, there is also a high degree of variability in the measurement instruments used to examine competent ICT use. Two prominent large-scale studies that measure secondary students’ ICT literacy are the NEPS and the International Computer and Information Literacy Study (ICILS). Both aim to measure secondary students’ competent ICT use. The ICILS study uses the term computer and information literacy (CIL), which refers to a student’s ability to use computer technologies to gather and manage information and to produce and share information (Fraillon et al., 2020, p. 18). Thus, both ICT literacy and CIL refer to the extent to which students can use ICT to manage, evaluate, and communicate information, and thus, we argue that both terms can be used interchangeably, which is also supported by empirical evidence (for an overview, see Senkbeil and Ihme, 2020).

With the large-scale ICILS studies (Fraillon et al., 2014, 2020), researchers were able to provide an international overview of secondary students’ ICT literacy. In the ICILS study, the ICT literacy of secondary students was divided into four competence levels, with level one being the lowest and level four the highest. Level one describes the fact that secondary students can, for example, open a link in a new browser tab or identify who is receiving an email by looking at carbon copies (Fraillon et al., 2020, p. 57). The fourth level of proficiency, on the other hand, describes secondary students’ ability to evaluate and assess internet sources, for example, when searching for or creating information (Fraillon et al., 2020, p. 60). The most recent results of the ICILS study (2018) show that most secondary students worldwide only reach the second proficiency level. This means that secondary students in eighth grade can use computers to perform basic and explicit information retrieval and management tasks (Fraillon et al., 2020, p. 57), which is not sufficient for successful participation in society given the growing need for the proficient use of ICT (Fraillon et al., 2020).

1.2 Personal factors and ICT use in relation to secondary students’ ICT literacy

Essentially, the ICT literacy of secondary school students worldwide does not yet meet the requirements contained in the school curricula. The reasons for this can be manifold, but studies have already shown that the socioeconomic status of secondary school students plays an important role when it comes to performance differences in ICT literacy (Scherer and Siddiq, 2019). Both the 2013 ICILS results and the latest ICILS (2018) results (Wendt et al., 2014; Senkbeil et al., 2019) suggest that secondary students from privileged families with high socioeconomic status and high cultural capital have an achievement advantage in ICT literacy. Initial attempts to interpret these results have assumed that less privileged families may not be able to raise the necessary capital to provide students with access to ICT and, therefore, perform worse in ICT literacy, which can be aligned with the “digital divide” (OECD, 2019), excluding less privileged students from the process of digitalization.

Moreover, numerous studies have investigated the extent to which ICT use in instruction is related to secondary students’ ICT literacy (Bundsgaard and Gerick, 2017; Senkbeil, 2017; Senkbeil and Ihme, 2017; Lorenz et al., 2019). While it is plausible to assume that those students who often use ICT both privately and in instruction also have higher learning outcomes, this could not always be proven in empirical studies. For example, Petko et al. (2017) used data from the Programme for International Student Assessment (PISA) to show that private ICT use tended to be positively related to student achievement in science, mathematics, and reading, but ICT use in instruction was not positively related to student achievement in these subjects, which, according to the authors, is consistent with previous findings. Similar findings were also found in the relationship between students’ ICT use and ICT literacy. For example, using ICILS data, Bundsgaard and Gerick (2017) found that students who reported moderately frequent ICT use had higher ICT literacy than those who reported frequent or infrequent ICT use. The authors explain the counter-intuitive nature of their results by stressing that the purpose of ICT use (e.g., for ICT use in instruction and private ICT use) must be examined more closely to make more precise statements about the extent to which students’ ICT use affects ICT literacy.

A correspondingly more precise differentiation of ICT use in relation to ICT literacy was pursued by Senkbeil (2017) and Senkbeil and Ihme (2017). The authors were able to show that certain ICT activities, such as the target-oriented use of ICT for information search or for study-related purposes, are positively related to students’ ICT literacy. Students’ private ICT use, such as for social communication, however, is not positively related to the ICT literacy of students. Thus, according to the authors, it can be concluded that target-oriented ICT use, such as for study-related purposes, is more suitable for acquiring ICT literacy than the use of ICT for private, hedonistic purposes. Overall, it can be concluded that students’ ICT use is highly relevant for ICT literacy and should be taken into account when examining students’ ICT literacy.

1.3 Teachers’ technological knowledge

Again, teachers are responsible for fostering students’ ICT literacy (Guggemos and Seufert, 2021). Hence, teachers must be able to use ICT in instruction proficiently to be able to teach students appropriate competencies (Guggemos and Seufert, 2021). Concerning teachers’ proficient ICT use, Koehler and Mishra (2009) had a lasting impact on educational research with the TPACK, which stands for technological pedagogical content knowledge and represents a framework on how to integrate technology into instruction successfully. The TPACK framework was established based on Shulman’s (1986) PCK and represents the extension of PCK with technology (T). Hence, the TPACK framework depicts the complexity of three main components regarding teacher knowledge: content, pedagogy, and technology. According to Koehler and Mishra (2009), TK is a part of the TPACK framework and is an indispensable component of integrating ICT in instruction. According to the authors, teachers’ TK “requires a person to understand information technology broadly enough to apply it productively at work and in their everyday lives to recognize when information technology can assist or impede the achievement of a goal and to continually adapt to changes in information technology” (p.64). Therefore, the description of TK by Koehler and Mishra (2009) reveals similarities to general ICT literacy definitions (see Educational Testing Service, 2002; Lachner et al., 2019). We, therefore, conclude that the term TK is an accurate term to describe the ICT literacy of teachers.

1.4 Teachers’ technological-pedagogical knowledge

It might be plausible that if teachers want to successfully integrate ICT into teaching and learning, they must have not only sound TK but also high TPK (Lachner et al., 2019). Referring to the TPACK framework, Koehler and Mishra (2009) described TPK as an understanding of how teaching and learning can change when particular technologies are used in particular ways (p. 65). As a result, teachers with a high TPK should be able to sufficiently assess which ICT resources are appropriate and sufficient for teaching. According to Koehler and Mishra (2009), TPK is essential for teachers because many programs, such as Word or Excel, are commonly used in classrooms, but these programs are designed for something other than pedagogical purposes (p. 65). However, these programs are important for students’ ICT literacy to successfully participate in social and professional life (Fraillon et al., 2020). Based on the assumptions of previous studies (see Baumert et al., 2010; Baier and Kunter, 2020), TPK can be described as the link between teachers’ TK and instruction.

With regard to TPK, several studies have examined the frequency of teachers’ ICT use in the classroom as a mediator variable that might affect students’ ICT literacy (e.g., Gerick et al., 2017; Gerick, 2018) or assessed teachers’ TPK via self-assessment, which may not be sufficient to provide a clear picture of the still poorly understood interaction between teachers’ competencies and students’ achievement (Lachner et al., 2019). However, assessing only the frequency of teachers’ ICT use in the classroom might not represent sufficient information about teachers’ TPK (Sailer et al., 2021), leaving educational science with a knowledge gap regarding the actual TPK of teachers. Scholars in the field have considered the lack of knowledge about teachers’ actual use of ICT in the classroom a significant weakness (Lorenz et al., 2019, p. 914) and have called for investigating how teachers use ICT in the classroom (e.g., Lucas et al., 2021) and, consequently, teachers’ actual, objective assessment of TPK (Lachner et al., 2019). Overall, educational research could benefit from contributions to teachers’ actual TPK to gain a clearer picture of how differences in teachers’ TPK might lead to differences in student achievement.

2 The present study

Despite the integration of ICT literacy into the curricula, international studies have indicated that secondary school students still need to meet the requirements in the curricula that are necessary to participate successfully in society and professional life. With the integration of ICT literacy into curricula, teachers became responsible for teaching ICT literacy to secondary school students. Previous studies have shown that teacher-level factors, such as the frequency of media use in class, play a role in students’ ICT literacy. However, the measurement of teacher-level factors has been limited to self-reporting and frequency measures, which may affect the validity of the findings. Accordingly, the focus has shifted to teachers’ actual professional knowledge, such as TK and TPK, and the extent to which they influence students’ ICT literacy. This study aims to fill the research gap on how the objectively measured TK and TPK of teachers predict the ICT literacy of students. While Baumert et al. (2010) succeeded at providing evidence that the professional mathematical knowledge of teachers is important to secondary students’ achievement in mathematics, it remains to be identified how cross-curricular competencies of teachers, such as TK and TPK, affect the cross-curricular ICT literacy of secondary students (see Figure 1). This leads to the following research questions for this study:

Figure 1

Figure 1. Hypothesized multilevel model of the teacher level (Level 2) and the students’ level (Level 1); RQ, research question.

RQ1: To what extent does the objectively measured technological knowledge (TK) of teachers predict the ICT literacy of secondary students?

RQ2: To what extent does the objectively measured technological knowledge (TK) of teachers, mediated by technological pedagogical knowledge (TPK), predict the ICT literacy of secondary students?

Based on previous research findings, we assume that secondary students who have high-performing teachers in TK, and TPK themselves perform highly regarding ICT literacy.

3 Method

3.1 Research design and participants

Participating schools for this study were selected randomly stratified. Schools were stratified by counties and independent cities in Bavaria. The probability of drawing a county or county-free city depends on the number of schools in that county or county-free city, which was determined through publicly available statistics. The selected schools were recruited to participate in the study via official invitation emails from the universities and the ministry. If a school’s principal agreed to participate in the data collection, teachers and students were invited. Data collection was conducted at the schools in October 2021 during a regular school day. Students were surveyed in the morning and teachers in the afternoon. Both NEPS tests, i.e., the students’ ICT literacy test and the teachers’ TK test-were administered on paper. The teachers’ TPK test was administered online via Unipark (Questback GmbH, 2019). Trained test administrators administered the tests to both students and teachers. A total of 2,421 8th grade students participated in the cross-sectional study as part of the DigitUS project, attending a total of N = 39 schools, with N = 134 classes taught by N = 220 teachers. The majority of the students visited grammar school (n = 1,131), while n = 853 students visited secondary schools and n = 437 lower secondary schools. All students were in eighth grade, so they were all approximately 14 years old. Furthermore, 46% (n = 1,117) indicated themselves as male, and n = 1,143 (47%) indicated themselves as female; 1% of the students indicated themselves as diverse, while 6% did not specify their gender. From the N = 2,421 students, N = 1,620 students completed the ICT literacy test due to planned missing data design, thus representing the sample for this study. Most of the teachers taught biology and mathematics (69%). The other teachers taught mainly languages and social sciences. Finally, the teachers were between 30 and 59 years old. An official ethics committee approved the study.

3.2 Measurement instruments

3.2.1 Dependent variable – students’ ICT literacy

To assess secondary students’ ICT literacy, the paper-based test of the NEPS for secondary students was applied (further information can be derived by Senkbeil et al., 2013). The test was provided by the Leibniz Institute for Educational Trajectories. The ICT literacy test of students consists of realistic problems embedded in a range of authentic situations where students were exposed to screenshots of electronic databases or spreadsheets (see Senkbeil et al., 2013, p. 145). The test construction is based on seven process components. Four of the process components (Define, Access, Manage, Create) refer to the facet of technology literacy, whereas the remaining three (Integrate, Evaluate, Communicate) refer to the facet of information literacy. In addition to the process components, the test is guided by a categorization of software applications that are used to locate, process, present, and communicate information (Senkbeil et al., 2013, p. 143). The answer format of the test is based on multiple-choice items.

The NEPS test was assigned to the participants based on their school type. Since the NEPS developed various test booklets for measuring ICT literacy for the ninth grade with the difficulty levels low, medium, and high, we examined in advance using power analysis which levels of difficulty were suitable for eighth grade in the different schools. After empirical examination, the medium test booklet was used for grammar schools (Gymnasiums), and the low-difficulty test booklet was used for the secondary schools (Realschule) and lower secondary schools (Mittelschule). The NEPS ICT literacy test consists of 36 items. Both difficulty levels, low and medium, included items that were identical but with different levels of difficulty, which allowed for mean-level comparisons (see Fischer et al., 2016). The students had 28 min to complete the ICT literacy test.

3.2.2 Control variables

3.2.2.1 Cultural capital

The cultural capital was surveyed via the number of books in the parental home as an indicator of the socioeconomic status of the students (Senkbeil et al., 2019; Fraillon et al., 2020). Students had five categories to choose from: 0 to 10 books (1), 11 to 25 books (2), 26 to 100 books (3), 101 to 200 books (4), and more than 200 books (5). We assessed students’ ICT use (for further information on the measurement of students’ ICT use, see Fraillon et al., 2020) with two subscales that relate to learning processes with ICT: ICT use for study-purposes and for class activities The scale for study- purposes included items regarding school-related purposes, such as preparing reports or essays or to complete exercises and tasks. Secondary students could answer from “less than once a month” to “every school day.” The scale ICT use for class activities included items regarding the use of ICT for learning of ICT (coding) tasks at school and the use of specialist and general applications in class. Secondary students could answer how often they use ICT for specific purposes, from “never” to “in every or almost every lesson.”

3.2.2.2 Independent variable—teachers’ TK

To measure teachers’ TK, the paper-based Test of Technological and Information Literacy (TILT) of the NEPS for adults was used (for further details, see Senkbeil et al., 2013; Senkbeil and Ihme, 2015). The ICT literacy framework can be applied to constructing ICT literacy tests for all age cohorts (e.g., students and adults; Senkbeil and Ihme et al., 2015, p. 3). Therefore, teachers were also presented with multiple-choice items of realistic problems of authentic screenshots—namely, internet browser or spreadsheet—as prompts (see Senkbeil and Ihme et al., 2015, p. 3). The tests consisted of 29 items;. Teachers had 28 min to complete the test.

3.2.2.3 Independent variable—teachers’ TPK

To assess teachers’ TPK in the present study, the TPK test of Lachner et al. (2019, p. 16) was administered. The TPK test includes conceptual and situational TPK domains. The conceptual TPK domain includes psychological components in that teachers are instructed to indicate which educational technology is appropriate to support student learning. Furthermore, this domain includes questions regarding technology-related research. Teachers were prompted to assess the degree to which, in the context of current technology-related research, the use of technology in the classroom may have potential. Regarding the situational TPK domain, short vignettes were included in which teachers were requested to assess specific teaching situations in which technologies were used. The final test consisted of 10 items. The original test design suggested a three-dimensional structure, but our data suggested one-dimensionality of the test structure (see BLINDED for REVIEW). Therefore, raw scores were transformed into person ability scores (Weighted Likelihood Estimate; WLE) using a unidimensional Rasch model. The cut-off values (0.5–1.5) proposed by Linacre (2002) for Infit-MNSQ and Outfit-MNSQ 0.5–1.5 were supported. Test fairness across school type and gender was tested with differential item functioning (DIF). The individual item Differential Item Functioning (DIF) analyses consistently revealed equitable test outcomes across different school types, with an average DIF of 0.19. Likewise, gender-based analyses exhibited a negligible average DIF of 0.08 across all items. Notably, the DIF analyses yielded results comfortably within the recommended thresholds (0.25–0.30), as advocated by Hungi (2005) and OECD – Organisation for Economic Co-operation and Development (2009). For a more in-depth exploration of the DIF analyses based on individual items, please refer to Stegmann and Berger (2024). On average, teachers took 15 min to complete the TPK test.

3.3 Procedure

Data was collected on students and teachers on a typical school day. Students were surveyed from second to fourth period, and regular school breaks were observed. The teacher survey took place in the afternoon after school. To avoid overload, students were given various paper-based tests with different tasks through the planned missing design. The teacher survey was paper-based at TK, and the rest of the survey was online in the school’s computer lab. The teachers’ data could be matched with the students’ data by generating class-specific tokens, which were also used to pseudonymize the data. Data collection was ensured by ethical treatment of all participants, maintaining data privacy and confidentiality (see ethical statement).

3.4 Statistical analysis

All analyses in this study were performed using the statistical software R (version 4.1.3). Person ability scores for students’ ICT literacy, teachers’ TK and TPK, and students’ ICT use were analyzed using the Rasch model (1PL) with the TAM package (Robitzsch et al., 2021) to obtain comparable Rasch measurement values (see Table 1). Due to the nested data structure, the data of this study were successively implemented using multilevel analysis with a three-level structure using the package lme4 (Bates et al., 2015). Level 1 represents the data of the students. Level 2 represents the data of the teachers who teach classes in which the students are enrolled, and Level 3 represents the schools of the students and teachers. To perform the multilevel analysis, a null model (Model 0) was first defined without any additional variables to clarify the need for a multilevel model based on the data structure. In the second model (Model 1), the same model as in Model 0 was implemented, but the intercepts were allowed to vary across classes to implement random effects of the model for the contextual variable. The third model (Model 2) included the control variables from three levels—the student level, teacher level, and school level—to identify the proportion of the included variables to explain additional variance in students’ ICT literacy. To determine whether varying intercepts improved the models, the Akaike information criterion (AIC) and Bayesian information criterion (BIC) were used; lower AIC and BIC indicate a better model fit (Rost, 2004). Furthermore, the assessment of model adequacy utilized the Likelihood Ratio Test (LRT). The LRT follows a chi-square distribution, allowing us to assess whether the extra parameters in the final model significantly contribute to a better explanation of the data compared to the null model.

Table 1

Table 1. Descriptive statistics of the data.

4 Results

Table 2 shows the results of the multilevel analysis (RQ1 and 2) for both the student (Model 1) -and teacher-level predictors (Model 2) only and the overall analysis including both predictors on the teacher-and-student-level regarding secondary students ICT literacy (Model 3). Regarding the comprehensive, multilevel analysis of the personal factors, the amount of books at home significantly positively affected students’ ICT literacy (b = 0.11, t = 4.55, p < 0.05). to secondary students’ ICT use, the results show that ICT use for study-related purposes (b = 0.02, t = −0.75, p > 0.05) has a non-significant relationship with secondary students’ ICT literacy. Regarding ICT use for class activities, the relationship with students’ ICT literacy shows a slightly negative direction, but this result is not significant (b = −0.01, t = −0.75). In addition, both TK (b = 0.00, t = 0.15 0.01, p > 0.05) and TPK (b = −0.02, t = −0.75 p > 0.05) did not significantly predict secondary students’ ICT literacy (RQ1), nor did the interaction between TK and TPK (RQ2) (b = 0.02, t = −0.61, p > 0.05). The results remain constant when the multilevel model is analyzed using only student-level variables (Model 1) and teacher-level variables (Model 2).

Table 2

Table 2. Results from multilevel analysis with secondary students’ ICT literacy as dependent variable and number of books at home, ICT use for study-related purposes, and ICT use for class activities, teachers’ TK, and teachers’ TPK as independent variables.

5 Discussion

In the present study, we investigated the extent to which teachers’ TK and TPK predict secondary students’ cross-curricular ICT literacy. Previous studies (e.g., Baumert et al., 2010) have shown that teachers’ subject knowledge is essential to students’ performance in the respective subject. Despite earlier results and the high scientific relevance of teachers’ professional knowledge in relation to students’ achievement, we were unable to replicate this finding in our study for the cross-curricular subject ICT literacy. Neither teachers’ TK nor TPK as an indicator of instructional quality significantly predicted ICT literacy among secondary students.

The results of this study are surprising because teachers are highly relevant regarding the ICT implementation in instruction (Davis et al., 2013), which might be plausible to have an impact on secondary students’ ICT literacy. Although the results of this study are surprising, earlier research has partially shown similar results. Indeed, studies that examined school- and teacher-level characteristics related to secondary students’ ICT literacy have already shown that other teacher-level factors, such as teachers’ computer use in school or collaboration related to ICT in the classroom, have no or even negative effects on secondary students’ ICT literacy (e.g., Gerick et al., 2017). The same direction can be found when secondary students report using ICT frequently for class activities or study-related purposes, indicating no significant relationship with students’ ICT literacy. Interestingly, Petko et al. (2017) found similar results about ICT use in school and student achievement in specific subjects, as students who reported using ICT often in school had lower scores in the Programme for International Student Assessment Study (PISA) test scores.

In summary, despite the integration of ICT literacy into curricula and the accompanying responsibility of teachers to teach ICT literacy to secondary students, secondary students’ ICT literacy does not appear to be influenced by teachers’ professional knowledge regarding TK and TPK, nor by the ICT use in class and for study-related purposes. At the same time, the question arises as to the extent to which ICT literacy in secondary students can be enhanced when neither the professional knowledge of teachers nor the use of ICT in the classroom impacts students’ ICT literacy. Accordingly, it would be important to investigate more thoroughly how ICT is used in the classroom, as in the present study, students only reported how frequently they use ICT in the classroom. We would like this study to be seen as a kind of “starting point” by providing evidence that it is not sufficient to rely solely on the frequency of media use for study-related purposes or in class and to consider merely teachers’ professional knowledge. Instead, further factors are needed to generate evidence on how students’ ICT literacy can be promoted. For example, in addition to the quantity of ICT use in the classroom, quality may also play a role (e.g., Sailer et al., 2021). Thus, the results of this study can be interpreted in the view of situational classroom instruction as researchers have already called for ICT use in the classroom to be more closely aligned with student-centered instructional quality aspects (e.g., Lachner et al., 2019 Scheiter, 2021). With this finding, in addition to teachers’ professional knowledge of TK and TPK, classroom instruction and ICT for study-related purposes become the focus of further research, as teachers need to initiate appropriate ICT use in the classroom and for study-related purposes, such as exercises, tasks or homework with ICT. Further research could investigate in more detail how different learning activities involving ICT in the classroom impact the ICT literacy of secondary students. Therefore, a starting point can be identified here to align ICT use in class with more student-centered instructional quality features such as the cognitive activation of students (e.g., Baumert et al., 2010) with specific target interventions in the professional development of teachers, in order to promote the ICT literacy of secondary students in a positive direction. This would be promising, since schools and thus teachers are also responsible for promoting the ICT literacy of secondary students in the classroom, which does not yet seem to be sufficiently the case.

Furthermore, scholars (Kaiser et al., 2017; Backfisch et al., 2020; Depaepe et al., 2020) have postulated that focusing exclusively on teachers’ professional knowledge is not sufficient to explain differences in student achievement: Rather, situational instructional actions and aspects of instructional quality must be examined simultaneously, as teachers’ professional knowledge alone does not necessarily correspond to actual instructional actions. Consequently, in order for students to use ICT in the classroom successfully, mere professional knowledge of the teachers might not always be transformed into actual classroom instruction (Kaiser et al., 2017; Depaepe et al., 2020), whereby other facets, such as self-efficacy and motivation should be taken into account to assess whether teachers implement appropriate learning scenarios in the classroom (Backfisch et al., 2020). Accordingly, it would be profitable for further research to investigate the process and situational characteristics of instruction, as well as personal factors of teachers, such as self-efficacy and motivation. Furthermore, the results of our study show that socioeconomic status had a significant effect on students’ ICT literacy, which is in line with previous studies in which socially disadvantaged students always performed worse in terms of ICT literacy (e.g., Hatlevik et al., 2015). With the responsibility of schools to specifically promote the ICT literacy of secondary students, there is also the possibility that students with a low socioeconomic status could acquire ICT literacy in the classroom, as less household-relevant factors (e.g., technical resources) would play a role in the development of secondary students ICT literacy. However, schools must be equally equipped with ICT and use ICT appropriately in teaching and learning processes, which could be investigated in more detail in future research.

In summary, future studies investigating the impact of teachers’ cross-curricular knowledge of TK and TPK on students’ ICT literacy should take into account situational process components of instruction on the part of teachers in order to gain a more holistic picture of teaching and learning processes with ICT, which seems to be relevant regarding the genesis of secondary students ICT literacy. Moreover, educational science would benefit greatly from qualitative research approaches to further investigate teachers’ professional knowledge and factors that prevent or support teachers from using ICT in the classroom, thus promoting ICT literacy among secondary school students and their ICT use in class and for study-related purposes.

6 Limitations and future directions

A limitation of this study is that no school-level factors, such as ICT equipment, were included in the analysis. However, factors such as ICT support or participation in professional development may be responsible for differences in teachers’ ICT use (Gerick et al., 2017) and, thus, TK and TPK, which may also affect secondary students’ ICT literacy. Future studies should therefore consider other structural factors at the school level. Scholars (e.g., Lachner et al., 2019; Lorenz et al., 2019; Baier and Kunter, 2020; Schmid et al., 2021) have often demanded the increased use of objective measures in educational science regarding teachers’ TK. Even though we agree with this demand, it is important to note that previous studies have shown that self-efficacy measures are an important indicator of whether technology is actually being used in the classroom (e.g., Scherer and Siddiq, 2015; Gerick et al., 2017; Konstantinidou and Scherer, 2022). For example, teachers’ motivation and ICT self-efficacy were important predictors of whether teachers frequently use ICT in the classroom (Scherer and Siddiq, 2015; Konstantinidou and Scherer, 2022).

Accordingly, self-efficacy and motivation measures could be used alongside objective assessment measures, which were used exclusively in this study, as the mere presence of expertise related to TK and TPK is not necessarily indicative of the actual implementation of technology in the classroom and thus has implications for students’ ICT literacy. Furthermore, a limitation arises from the reliance on the TPACK model, as this has been a recurring challenge in educational science (for more details, see Petko et al., 2017). Often, studies have struggled to empirically substantiate the theoretical TPACK model e.g., (see Scherer et al., 2017), which may suggest that the TPACK model may not encompass all relevant aspects. Nevertheless, it remains widely used in educational science. Therefore, it is necessary to consider additional factors, such as critical thinking to explain differences and influences in teachers’ TK and TPK. In addition, teachers’ other personal factors, such as teaching experience and subject combination, can be included as control variables in future studies. Beyond that, this study was conducted using a cross-sectional design. Further longitudinal studies can provide more information about the relationship between teachers’ TPK and TK and students’ ICT literacy. Furthermore, although Lachner et al. (2019) suggest a three-dimensional structure for assessing TPK, we analyzed the TPK scale with a one-dimensional structure using item response theory to cope with possible differences in difficulty levels of the three dimensions. Therefore, the results related to TPK must be interpreted with caution. While our study strives to provide meaningful insights, it is crucial to transparently acknowledge the limitations associated with the accuracy of the individual scores. Future research endeavors could explore advanced modeling techniques or incorporate robust statistical methods to further address and account for measurement errors in the analysis.

7 Conclusion

While previous studies have shown that teachers’ subject-specific professional knowledge—for example, in mathematics—positively affects students’ performance in that subject, this result could not be replicated for cross-curricular competencies—namely, ICT literacy. Nonetheless, by embedding ICT literacy into the curriculum, teachers are responsible for teaching ICT literacy to secondary students and promoting their own TK and TPK in professional development programs to use ICT effectively in the classroom. Accordingly, the relationship between students’ and teachers’ cross-curricular competencies and more detailed instruction process and situational aspects should be further explored to develop targeted interventions to successfully use ICT in the classroom to promote students’ ICT literacy. Furthermore, it is essential to identify additional factors on both the students’ and teachers’ sides that contribute to enhancing students’ ICT literacy and using digital media in the classroom. There is a lack of sufficient scientific evidence in this regard.

Data availability statement

The datasets for this article are not publicly available due to concerns regarding participant anonymity. Requests to access the datasets should be directed to the corresponding author.

Ethics statement

The studies involving human participants were approved by the Bavarian State Ministry of Education and Culture. The research was conducted in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was obtained from the participants and their legal guardians.

Author contributions

TK: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing – original draft, Writing – review & editing. KS: Data curation, Funding acquisition, Methodology, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by German Federal Ministry of Research and Education (Grant number: Grant 1JD1830A).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Backfisch, I., Lachner, A., Hische, C., Loose, F., and Scheiter, K. (2020). Professional knowledge or motivation? Investigating the role of teachers’ expertise on the quality of technology-enhanced lesson plans. Learn. Instr. 66:101300. doi: 10.1016/j.learninstruc.2019.101300

Crossref Full Text | Google Scholar

Baier, F., and Kunter, M. (2020). Construction and validation of a test to assess (pre-service) teachers' technological pedagogical knowledge (TPK). Stud. Educ. Eval. 67:100936. doi: 10.1016/j.stueduc.2020.100936

Crossref Full Text | Google Scholar

Bates, D., Mächler, M., Bolker, B., and Walker, S. (2015). Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48. doi: 10.18637/jss.v067.i01 [In Citavi anzeigen].

Crossref Full Text | Google Scholar

Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., et al. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student Progress. Am. Educ. Res. J. 47, 133–180. doi: 10.3102/0002831209345157

Crossref Full Text | Google Scholar

Bundsgaard, J., and Gerick, J. (2017). Patterns of students’ computer use and relations to their computer and information literacy: results of a latent class analysis and implications for teaching and learning. Large-scale Assess. Educ. 5:16. doi: 10.1186/s40536-017-0052-8

Crossref Full Text | Google Scholar

Davis, N., Eickelmann, B., and Zaka, P. (2013). Restructuring of educational systems in the digital age from a co-evolutionary perspective. J. Comput. Assist. Learn. 29, 438–450. doi: 10.1111/jcal.12032

Crossref Full Text | Google Scholar

Depaepe, F., Verschaffel, L., and Star, J. (2020). Expertise in developing students’ expertise in mathematics: bridging teachers’ professional knowledge and instructional quality. ZDM 52, 179–192. doi: 10.1007/s11858-020-01148-8

Crossref Full Text | Google Scholar

Educational Testing Service. (2002). Digital transformation. A framework for ICT literacy. Princeton, NJ: ETS.

Google Scholar

Fischer, L., Rohm, T., Gnambs, T., and Carstensen, C. H. (2016). Linking the data of the competence tests Bamberg: Leibniz Institute for Educational Trajectories, National Educational Panel Study

Google Scholar

Fraillon, J., Ainley, J., Schulz, W., Friedman, T., and Duckworth, D. (2020). Preparing for life in a digital world Springer International Publishing. doi: 10.1007/978-3-030-38781-5

Crossref Full Text | Google Scholar

Fraillon, J., Ainley, J., Schulz, W., Friedman, T., and Gebhardt, E. (2014). Preparing for life in a digital age Springer International Publishing. doi: 10.1007/978-3-319-14222-7

Crossref Full Text | Google Scholar

Gerick, J. (2018). School level characteristics and students’ CIL in Europe – a latent class analysis approach. Comput. Educ. 120, 160–171. doi: 10.1016/j.compedu.2018.01.013

Crossref Full Text | Google Scholar

Gerick, J., Eickelmann, B., and Bos, W. (2017). School-level predictors for the use of ICT in schools and students’ CIL in international comparison. Large Scale Assess. Educ. 5, 1–13. doi: 10.1186/s40536-017-0037-7

Crossref Full Text | Google Scholar

Gnambs, T. (2021). The development of gender differences in information and communication technology (ICT) literacy in middle adolescence. Comput. Hum. Behav. 114:106533. doi: 10.1016/j.chb.2020.106533

Crossref Full Text | Google Scholar

Guggemos, J., and Seufert, S. (2021). Teaching with and teaching about technology – evidence for professional development of in-service teachers. Comput. Hum. Behav. 115:106613. doi: 10.1016/j.chb.2020.106613

Crossref Full Text | Google Scholar

Hatlevik, O. E., Guðmundsdóttir, G. B., and Loi, M. (2015). Digital diversity among upper secondary students: a multilevel analysis of the relationship between cultural capital, self-efficacy, strategic use of information and digital competence. Comput. Educ. 81, 345–353. doi: 10.1016/j.compedu.2014.10.019

Crossref Full Text | Google Scholar

Hungi, N. (2005). Employing the Rasch model to detect biased items. In S. Alagumalai and D.D. Curtis, & Hungi, N. (Eds.), Applied Rasch measurement: A book ofexemplars–papers in honour of John P. Keeves. The Netherlands: Springer. p. 139–157

Google Scholar

Kaiser, G., Blömeke, S., König, J., Busse, A., Döhrmann, M., and Hoth, J. (2017). Professional competencies of (prospective) mathematics teachers: cognitive versus situated approaches. Educ. Stud. Math. 94, 161–182. doi: 10.1007/s10649-016-9724-5

Crossref Full Text | Google Scholar

Kultusministerkonferenz. (2019). Bildung in der digitalen welt. Strategie der Kultusministerkonferenz [education in a digital world. Strategy of the conference of federal ministers of Germany]. Available at: https://www.kmk.org/themen/bildung-in-der-digitalen-welt/strategie-bildung-in-der-digitalen-welt.html

Google Scholar

Koehler, M. J., and Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary issues in technology and teacher education. What is technological pedagogical content knowledge? Available at: https://www.researchgate.net/publication/241616400_What_Is_Technological_Pedagogical_Content_Knowledge#fullTextFileContent (Accessed March 02, 2022).

Google Scholar

Konstantinidou, E., and Scherer, R. (2022). Teaching with technology: a large-scale, international, and multilevel study of the roles of teacher and school characteristics. Comput. Educ. 179:104424. doi: 10.1016/j.compedu.2021.104424

Crossref Full Text | Google Scholar

Lachner, A., Backfisch, I., and Stürmer, K. (2019). A test-based approach of modeling and measuring technological pedagogical knowledge. Comput. Educ. 142:103645. doi: 10.1016/j.compedu.2019.103645

Crossref Full Text | Google Scholar

Linacre, J. M. (2002). Optimizing rating scale category effectiveness. J. Appl. Meas. 3, 85–106.

PubMed Abstract | Google Scholar

Lorenz, R., Endberg, M., and Bos, W. (2019). Predictors of fostering students’ computer and information literacy – analysis based on a representative sample of secondary school teachers in Germany. Educ. Inf. Technol. 24, 911–928. doi: 10.1007/s10639-018-9809-0

Crossref Full Text | Google Scholar

Lucas, M., Bem-Haja, P., Siddiq, F., Moreira, A., and Redecker, C. (2021). The relation between in-service teachers' digital competence and personal and contextual factors: what matters most? Comput. Educ. 160:104052. doi: 10.1016/j.compedu.2020.104052

Crossref Full Text | Google Scholar

OECD (2019). Measuring the digital transformation. OECD. doi: 10.1787/9789264311992-en

Crossref Full Text | Google Scholar

OECD – Organisation for Economic Co-operation and Development (2009). PISA 2006 technical report. Paris, France: OECD

Google Scholar

Petko, D., Cantieni, A., and Prasse, D. (2017). Perceived quality of educational technology matters. J. Educ. Comput. Res. 54, 1070–1091. doi: 10.1177/0735633116649373

Crossref Full Text | Google Scholar

Questback GmbH. (2019). EFS survey, version EFS winter 2018 Cologne: Questback GmbH.

Google Scholar

Robitzsch, A, Kiefer, T, and Wu, M (2021). TAM: test analysis modules. R package version. Available at: https://CRAN.R-project.org/package=TAM

Google Scholar

Rost, J. (2004). Lehrbuch Testtheorie - Testkonstruktion [test theory and test construction](2., vollst. überarb. und erw. Aufl.). Aus dem Programm Huber: Psychologie Lehrbuch. Verlag Hans Huber.

Google Scholar

Sailer, M., Murböck, J., and Fischer, F. (2021). Digital learning in schools: what does it take beyond digital technology? Teach. Teach. Educ. 103:103346. doi: 10.1016/j.tate.2021.103346

Crossref Full Text | Google Scholar

Scheiter, K. (2021). Lernen und Lehren mit digitalen Medien: Eine Standortbestimmung [Technology-enhanced learning and teaching: An overview]. Zeitschrift fur Erziehungswissenschaft. 24, 1–22. doi: 10.1007/s11618-021-01047y

Crossref Full Text | Google Scholar

Scherer, R., and Siddiq, F. (2015). Revisiting teachers’ computer self-efficacy: a differentiated view on gender differences. Comput. Hum. Behav. 53, 48–57. doi: 10.1016/j.chb.2015.06.038

Crossref Full Text | Google Scholar

Scherer, R., and Siddiq, F. (2019). The relation between students’ socioeconomic status and ICT literacy: findings from a meta-analysis. Comput. Educ. 138, 13–32. doi: 10.1016/j.compedu.2019.04.011

Crossref Full Text | Google Scholar

Scherer, R., Tondeur, J., and Siddiq, F. (2017). On the quest for validity: testing the factor structure and measurement invariance of the technology-dimensions in the technological, pedagogical, and content knowledge (TPACK) model. Comput. Educ. 112, 1–17. doi: 10.1016/j.compedu.2017.04.012

Crossref Full Text | Google Scholar

Schmid, M., Brianza, E., and Petko, D. (2021). Self-reported technological pedagogical content knowledge (TPACK) of pre-service teachers in relation to digital technology use in lesson plans. Comput. Hum. Behav. 115:106586. doi: 10.1016/j.chb.2020.106586

Crossref Full Text | Google Scholar

Senkbeil, M. (2017). Empirische Arbeit: profile computerbezogener Anreizfaktoren: Zusammenhänge mit ICT literacy und sozialen Herkunftsmerkmalen. Ergebnisse aus der internationalen Schulleistungsstudie ICILS 2013. [empirical work: profiles of computer-related incentive factors: associations with ICT literacy and social origin characteristics. Results from the international school achievement study ICILS 2013]. Psychol. Erzieh. Unterr. 64:138. doi: 10.2378/peu2017.art07d

Crossref Full Text | Google Scholar

Senkbeil, M., Drossel, K., Eickelmann, B., and Vennemann, M. (2019). “Soziale Herkunft und computer- und informationsbezogene Kompetenzen von Schülerinnen und Schülern im zweiten internationalen Vergleich [social background and computer and information literacy of students in the second international comparison.]” in ICILS 2018 #Deutschland: Computer- und informationsbezogene Kompetenzen von Schülerinnen und Schülern im zweiten internationalen Vergleich und Kompetenzen im Bereich Computational Thinking. eds. B. Eickelmann, W. Bos, J. Gerick, F. Goldhammer, H. Schaumburg, and K. Schwippert, et al. (Münster, New York: Waxmann), 301–333.

Google Scholar

Senkbeil, M., and Ihme, J. M. (2015). NEPS technical report for computer literacy scaling results of starting cohort 6–adults (NEPS working paper no. 61) (Bamberg: Leibniz Institute for Educational Trajectories, National Educational Panel Study).

Google Scholar

Senkbeil, M., and Ihme, J. M. (2017). Motivational factors predicting ICT literacy: first evidence on the structure of an ICT motivation inventory. Comput. Educ. 108, 145–158. doi: 10.1016/j.compedu.2017.02.003

Crossref Full Text | Google Scholar

Senkbeil, M., and Ihme, J. M. (2020). Diagnostik von ICT literacy: Messen multiple-choice-Aufgaben und simulationsbasierte Aufgaben vergleichbare Konstrukte? [diagnosis of ICT literacy: are multiplie-choice tasks and simulation-based tasks measure comparable constructs?]. Diagnostica. 66, 147–157. doi: 10.1026/0012-1924/a000243

Crossref Full Text | Google Scholar

Senkbeil, M., Ihme, J. M., and Wittwer, J. (2013). The test of technological and information literacy (TILT) in the National Educational Panel Study: Development, empirical testing, and evidence for validity. J Educ Res. 5, 139–161. doi: 10.25656/01:8428

Crossref Full Text | Google Scholar

Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educ. Res. 15, 4–14. doi: 10.3102/0013189X015002004

Crossref Full Text | Google Scholar

Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., and Scherer, R. (2016). Taking a future perspective by learning from the past – a systematic review of assessment instruments that aim to measure primary and secondary school students' ICT literacy. Educ. Res. Rev. 19, 58–84. doi: 10.1016/j.edurev.2016.05.002

Crossref Full Text | Google Scholar

Stegmann, K., and Berger, S. (2024). Technical Report for the DigitUS-Project [manuscript in Preparation]. Department of Psychology, Ludwig-Maximilians-University Munich.

Google Scholar

Wendt, H., Vennemann, M., Schwippert, K., and Drossel, K. (2014). “Soziale Herkunft und computer-und informationsbezogene Kompetenzen von Schülerinnen und Schülern im internationalen Vergleich [social background and computer and information literacy of students in international comparison.]” in ICILS 2014: Computer- und informationsbezogene Kompetenzen von Schülerinnen und Schülern in der 8. Jahrgangsstufe im internationalen Vergleich. eds. W. Bos, B. Eickelmann, J. Gerick, F. Goldhammer, and H. Schaumburg, et al. (Münster, New York: Waxmann)

Google Scholar