Development and pilot testing of multi-stakeholder questionnaires for assessing University 4.0 transformation in higher education institutions

Abstract

Introduction:

The transition to University 4.0 requires higher education institutions to develop reliable and context-appropriate tools capable of evaluating digital, pedagogical, research, and governance transformations from the perspective of multiple stakeholders. In Morocco, the available evaluation tools remain limited, as they were developed in different institutional contexts or focus on isolated aspects of digital transformation. This study aimed to develop, test, and exploratorily refine a set of multi-stakeholder questionnaires designed to assess the transition to University 4.0 in Moroccan higher education institutions.

Methods:

Six questionnaires were developed for students, faculty members and administrative staff, covering Education 4.0, Research 4.0 and Governance 4.0. Their refinement combined pilot tests, psychometric indicators, XAI-based item importance using SHAP and importance by permutation, as well as content validation based on expertise.

Results:

Following the combined process of psychometrics, XAI, and expert review, 52 of the initial 183 items were retained, 51 required reformulation, and 80 were recommended for removal. The student questionnaires showed the strongest overall preliminary performance. Student_Education exhibited questionable internal consistency (α = 0.663) and an average corrected item-total correlation of 0.253, while reaching a relatively high S-CVI/Ave of 0.894. Student_Gov_Research demonstrated acceptable internal consistency (α = 0.794), an average corrected item-total correlation of 0.297, close to the conventional criterion of 0.30, and the highest S-CVI/Ave value (0.941).

Discussion:

The results support the use of XAI as an exploratory decision-support tool for the refinement of questionnaires when combined with psychometric evidence, theoretical considerations, and expert judgment. The refined instruments remain pilot prototypes and require further validation using larger, balanced, and multi-institutional samples.

Introduction

The Fourth Industrial Revolution has profoundly transformed the strategic role of higher education institutions, forcing them to rethink their missions of teaching, research, governance, innovation, and societal engagement. In this changing context, the concept of University 4.0 has emerged as an institutional response to the convergence of digital transformation, Industry 4.0 technologies, data-driven management, artificial intelligence, the demands of sustainable development, and new skills requirements. Unlike traditional approaches to university modernization, University 4.0 refers to a systemic transformation where pedagogical innovation, research capacity, digital governance, institutional agility, and stakeholder engagement are understood as interdependent elements of a unified transformation process (Mukul and Büyüközkan, 2023; Ülker and Otrar, 2024).

Recent studies emphasize that the transition to University 4.0 cannot be reduced to the simple adoption of digital tools, online platforms, or technological infrastructures. Rather, it involves a profound reconfiguration of institutional processes, including the implementation of Education 4.0 practices, the development of digitally-enabled research and innovation ecosystems, the strengthening of transparent governance mechanisms, the use of data-driven decision-making, and the institutionalization of continuous quality improvement (Bravo-Jaico et al., 2025a; González-Pérez et al., 2025). Consequently, the digital transformation of higher education is increasingly analyzed through maturity models and multidimensional assessment frameworks capable of capturing the complexity of technological, organizational, pedagogical, and sociocultural changes (Alfirević, Mabić and Alfirević, 2025; Bravo et al., 2025).

In Morocco, this transformation has become a major national strategic priority. Previous research has highlighted the growing importance of digital transformation in Moroccan higher education, as well as the challenges associated with the transition to University 4.0 (Tamer and Knidiri, 2023). In this context, the ESRI 2030 Pact aims to establish a renewed model for Moroccan higher education by promoting pedagogical reform, scientific excellence, effective governance, local engagement, employability, and innovation. Simultaneously, the UM4.0 program aims to support the emergence of a digital, entrepreneurial, and inclusive Moroccan university (Benhima, 2025). These reforms indicate that the central question is no longer whether Moroccan universities should engage in digital transformation, but rather how this transformation can be empirically assessed, monitored, and improved using reliable and context-appropriate evaluation tools.

The Moroccan context differs from many settings in which existing digital-maturity instruments were developed because the transformation is being implemented through a coordinated national reform architecture that simultaneously addresses academic excellence, scientific research, Governance 4.0, territorial innovation, and digital transformation. The PACTE ESRI-2030 explicitly differentiates the digital pathways of students, academic staff, and administrative and technical personnel. This policy-driven and multi-actor configuration requires instruments that reflect national governance arrangements, stakeholder roles, linguistic practices, and the uneven implementation of digital services across public institutions. Directly importing an instrument developed in another higher education system could therefore compromise content relevance and contextual validity.

Despite the abundant international literature on Higher Education 4.0, Education 4.0, digital maturity, and institutional transformation, several methodological limitations persist. First, many existing frameworks were designed in cultural, linguistic, and institutional environments very different from the context of Moroccan higher education. Second, many assessment tools focus primarily on technological readiness or pedagogical digitization, giving little attention to the transformation of governance and research. Third, most studies do not jointly integrate the perspectives of the main groups of stakeholders directly involved in university transformation, namely students, teachers, and administrative staff. Therefore, there is a clear need for contextualized, multidimensional, and multi-stakeholder instruments capable of assessing the transition to University 4.0 within Moroccan higher education institutions.

In the present study, the transformation of university 4.0 is operationalized as the level perceived by stakeholders of the current implementation and maturity of institutional practices related to Education 4.0, Research 4.0, and Governance 4.0. The proposed instruments do not directly measure future institutional readiness, understood as the prospective capacity, willingness, or degree of preparedness of an institution to undertake the transformation. Rather, they capture stakeholders’ perceptions regarding the practices, structures, resources, services, and institutional mechanisms currently in place.

Another major methodological challenge concerns the development of instruments that are both comprehensive and practically applicable. Questionnaires designed to cover the multiple dimensions of University 4.0 can be excessively long, which risks increasing the response burden, raising dropout rates, producing superficial answers, and causing respondent fatigue. Therefore, instrument development must ensure a balance between theoretical comprehensiveness and empirical feasibility. This requires the application of classic psychometric procedures, such as content validation, internal consistency analysis, and item-total correlations, while also integrating complementary methods to identify the most relevant items. In this study, XAI is presented as an exploratory decision-support approach for item reduction. SHAP-based item importance and permutation feature importance are used as complementary indicators of item contribution, alongside traditional psychometric evidence and expert judgment (Mondol and Lee, 2026). It is important to note that XAI results are not used as standalone criteria for deletion, but rather as complementary evidence to justify retaining, rewording, or removing questionnaire items.

This research paper addresses these gaps by documenting the development, pilot testing, and refinement of multi-stakeholder questionnaires to assess the university's transition to a 4.0 model in Moroccan higher education. The study was conducted at the Higher Normal School of Technical Education (ENSET Mohammedia) as a pilot site due to its technical and educational missions, its relevance to digital transformation initiatives, and its accessibility for exploratory data collection. The proposed process aims to be transparent, replicable, and applicable to other Moroccan public universities and similar higher education systems undergoing digital transformation.

The study seeks to achieve three main objectives. First, to develop a context-sensitive conceptual framework for assessing the university's transition to a 4.0 model, structured around three domains: Education 4.0, Research 4.0, and Governance 4.0. Second, to describe the development and pilot testing of six initial stakeholder questionnaires targeting students, faculty, and administrative staff. Third, it evaluates an exploratory hybrid improvement procedure that combines initial psychometric screening, response quality analysis, the significance of XAI-based items, and expert judgment in order to produce shorter, more practical tools for subsequent large-scale validation.

The main contribution of this article is threefold. Conceptually, it proposes a multidimensional operationalization of University 4.0 adapted to the context of Moroccan higher education. Methodologically, it introduces a hybrid approach, combining psychometric tests with explainable artificial intelligence (XAI) methods, which reduces the length of the questionnaire while preserving theoretical coverage and the structure of the scores. Practically, it provides improved pilot instruments that can support future empirical assessments of the perceived maturity and the implementation level of University 4.0 practices.

The remainder of this article is organized as follows: after a literature review of related works, Section 3 presents the methodological approach used to develop, test, and refine the questionnaires; Section 4 reports the results of the preliminary psychometric and XAI; and Section 5 examines the implications, limitations, and methodological contributions of the study. Finally, section 6 concludes the article and outlines future prospects for large-scale validation and comparative research.

2 Literature review and conceptual framework

2.1 A new era for universities: University 4.0

The rapid advancement of digital technologies has significantly reshaped economic systems, industrial processes, and social structures worldwide. This transformation, commonly referred to as the Fourth Industrial Revolution, is characterized by the integration of cyber-physical systems, artificial intelligence, big data analytics, and interconnected digital infrastructures (Ahmad, 2024; Giesenbauer and Müller-Christ, 2020). As a result, many sectors have undergone profound transformations, including education, where higher education institutions are increasingly required to adapt to new technological and societal challenges (Mian et al., 2020).

Digital transformation in higher education requires coordinated changes in leadership, organizational culture, communication, teaching processes, and data-driven decision making (Díaz-Garcia et al., 2023; Rosak-Szyrocka, 2024).

According to Miranda et al.(2021), the concept of University 4.0 emerged to describe the transformation of higher education institutions in response to digitalization and the demands of knowledge economies. University 4.0 represents a shift from traditional university models to more agile, technology- and innovation-driven institutions (Chernaya et al., 2023).

This model emphasizes the integration of digital technologies into teaching, research, and governance processes to improve institutional performance and societal impact. Many researchers also see University 4.0 as building upon previous stages of university development. Early university models focused primarily on teaching (University 1.0), while later models incorporated scientific research as a core function (University 2.0). The emergence of entrepreneurial universities (University 3.0) expanded the role of universities to include innovation, knowledge transfer, and collaboration with industry. University 4.0 reinforces these developments by integrating digital technologies and data-driven management practices into all institutional functions (Bravo-Jaico et al., 2025b; Chounta et al., 2024; Satish Kumar Mahariya et al., 2023). Therefore, the University 4.0 model may be envisioned in three major dimensions: Formation 4.0, Research 4.0, and Governance 4.0. These dimensions reflect the transformation of educational practices, research systems, and institutional governance structures into the digital age.

The selection of Education 4.0, Research 4.0, and Governance 4.0 is based on a functional vision of the university's essential institutional missions. Education 4.0 reflects the transformation of knowledge transmission, learning environments, curricula, and skills development (Chaengpromma and Pattanapairoj, 2022; Gallon et al., 2024). Research 4.0 concerns the transformation of knowledge production, scientific collaboration, innovation, dissemination, and valorization. Governance 4.0 encompasses the institutional mechanisms through which strategies are formulated, resources are allocated, stakeholders participate, and educational and research transformations are coordinated and supported (Nguyen et al., 2024). The cross-cutting themes, including sustainability, entrepreneurship, digital infrastructure, internationalization, and social engagement, are integrated within these three areas rather than treated as independent institutional missions. The framework therefore claims functional comprehensiveness for institutional evaluation, rather than ontological comprehensiveness of every possible characteristic of University 4.0.

Table 1 below summarizes the main conceptual dimensions of University 4.0 identified in the literature, specifying for each axis the key references and their contributions, including empirical studies conducted in the Moroccan context (Amini et al., 2023; Chahid et al., 2025).

These transversal dimensions can be grouped into three main operational axes, which constitute the core of the university's transformation in the digital age: Education 4.0, Governance 4.0, and Research 4.0.

2.2 Education 4.0

The concept of Training 4.0, also called Education 4.0, refers to the transformation of learning and training processes through the integration of digital technologies and innovative teaching practices (Bonfield et al., 2020; Zulkipli and Musa, 2022). This transformation is increasingly seen as essential to meet the changing demands of the digital economy, which requires advanced tech skills, adaptability, and multidisciplinary abilities (Masdoki et al., 2021; Shal et al., 2026).

Education 4.0 places particular importance on the use of digital learning technologies in order to create more flexible, interactive, and personalized learning environments. These technologies include learning management systems, online learning platforms, virtual laboratories, and collaborative digital tools, which allow students to access educational resources beyond the limits of the traditional classroom (Bond et al., 2018; Laciok et al., 2020; Pajpach et al., 2022). Their integration enables higher education institutions to develop student-centered learning models that are more interactive, adaptable, and tailored to the individual needs of learners (Mourtzis et al., 2023; Shal et al., 2026; Yondri et al., 2020).

Beyond using digital tools, Education 4.0 also involves adopting innovative teaching approaches aimed at boosting student engagement and strengthening skill development (Amini et al., 2023; Salah, 2020; Vrignat et al., 2025). Methods like project-based learning, flipped classrooms, and blended learning setups are widely seen as effective ways to build critical thinking, problem-solving skills, teamwork, and student independence (Venkatraman et al., 2022). These teaching innovations help shift from passive knowledge transfer to active learning, where students take a more central role in creating and applying knowledge (Masdoki et al., 2021; Vrignat et al., 2025).

Another important dimension of Education 4.0 concerns the development of technology-enhanced learning environments, often referred to as smart classrooms or smart campuses. These environments integrate advanced digital infrastructure, interactive technologies, and learning analytics tools to improve teaching quality and student performance (Dai et al., 2024; Pan et al., 2024). In particular, data analytics enables teachers to track student progress, identify learning difficulties, and adjust pedagogical strategies based on real-time evidence. The effectiveness of these environments also depends on teachers' competencies, attitudes toward Education 4.0, and preparedness to integrate emerging technologies into teaching practices (Tai et al., 2022; Zulkipli and Musa, 2022).

Furthermore, Education 4.0 reflects the growing importance of lifelong learning and continuous skills development in contemporary societies. Universities are increasingly expected to provide flexible learning pathways that allow individuals to acquire, update, and certify skills throughout their professional careers. This evolution has encouraged the development of micro-credentials, online certification programs, and modular learning systems, which enable learners to respond more effectively to changing labor market needs (Miranda et al., 2021). Overall, Education 4.0 represents a fundamental transformation in the way knowledge is delivered, acquired, and applied within higher education institutions.

Table 2 summarizes the main contributions of the literature related to education 4.0, distinguishing between educational frameworks, teaching competencies, and educational technologies emblematic of this transformation.

All the dimensions presented here concretely illustrate one of the three operational pillars of university 4.0, namely education 4.0, which unfolds in close interaction with Governance 4.0 and Research 4.0.

2.3 Research 4.0: Transformation of university research systems

Alongside the transformation of teaching practices, universities are also undergoing profound changes in the organization and management of their research activities. The concept of Research 4.0 reflects the evolution of research systems in the face of digital technologies, international collaborative networks, and innovation-driven economic models (Machado et al., 2020; Rejeb et al., 2025; Sikandar et al., 2021).

Research 4.0 emphasizes the growing role of digital technologies in scientific research (Eger and Žižka, 2024). Advances in data analytics, cloud computing, and digital research infrastructures have significantly expanded the possibilities for creating and disseminating knowledge. Researchers can now collaborate across geographical boundaries, share large datasets, and conduct interdisciplinary studies that were previously difficult to achieve (Rejeb et al., 2025).

One of the defining characteristics of Research 4.0 is the increasing importance of open science practices. Open science initiatives encourage the sharing of data, publications, and research methodologies to improve transparency, reproducibility, and collaboration within the scientific community. Digital platforms and open archives have played a crucial role in facilitating these practices, enabling researchers to disseminate their results more widely and effectively. Research 4.0 is closely linked to the development of innovation ecosystems, where universities collaborate with industry, government institutions, and civil society organizations to generate technological and social innovations (Hervas-Oliver, 2022). This collaborative model is often described by the Triple Helix model, which highlights the dynamic interactions between universities, industry, and public authorities in the production of knowledge and innovation (Cai and Lattu, 2022; Hervas-Oliver, 2022).

In this context, universities are increasingly creating innovation centers, incubators, and technology transfer offices to facilitate the commercialization of research results and support entrepreneurial initiatives. These structures allow universities to contribute more directly to economic development and societal progress by translating scientific discoveries into concrete applications.

Furthermore, Research 4.0 entails an increasing internationalization of research activities. International collaborations allow researchers to access diverse expertise, cutting-edge infrastructure, and international funding. As a result, universities are increasingly participating in international research networks and joint scientific projects that enhance their global visibility and competitiveness (Secundo et al., 2020). Overall, Research 4.0 reflects a transition toward more collaborative, data-driven, and innovation-oriented research systems that leverage digital technologies to improve scientific productivity and societal impact (Coello Pisco et al., 2024; Sikandar et al., 2021; Zainal Abidin et al., 2023).

Table 3 summarizes the main works dedicated to scientific dynamics and the reconfigurations of knowledge production induced by the 4.0 paradigm, with a focus on research skills, university-industry interfaces, and the specificities of developing countries.

While this work provides a valuable mapping of international trends in Research 4.0, it remains largely anchored in the contexts of industrialized countries and still addresses little the empirical conditions of implementation in universities in emerging countries, where constraints related to funding, infrastructure, and researcher training remain decisive (Eger and Žižka, 2024; Hammad et al., 2025a). The “Research 4.0” dimension nonetheless constitutes one of the three operational pillars of University 4.0, in constant interaction with education 4.0 and Governance 4.0, and its full effectiveness requires an explicit articulation with national science policies and local industrial ecosystems (El Majjaoui et al., 2026).

2.4 Governance 4.0: digital transformation of institutional management

The transition of universities to the University 4.0 paradigm requires significant changes not only in teaching and research but also in institutional governance (Saunila et al., 2024). Governance 4.0 refers to the modernization of governance practices through the integration of digital technologies, data-driven decision-making, participatory management, and more flexible institutional structures within higher education institutions (Amrani et al., 2026; Saunila et al., 2024).

A key feature of Governance 4.0 is rolling out digital administration systems that help make managing institutions more efficient, transparent, and coordinated. Universities are relying more and more on integrated information systems to handle admin tasks like student services, finances, HR, quality checks, and research management (Chounta et al., 2024; Zorrilla and Yebenes, 2022). These systems make routine admin work easier to automate, cut down on organizational silos, and improve coordination between the institution's different departments.

Another important aspect of Governance 4.0 has to do with using data analytics, institutional dashboards, and performance indicators to boost strategic decision-making. By relying on institutional data, university leaders can keep an eye on key performance metrics, evaluate academic programs, spot organizational weaknesses, and allocate resources more effectively. This data-driven approach helps universities react more quickly and accurately to changing educational, technological, and societal needs. In this sense, digital governance is increasingly tied to how well institutions can manage change through evidence-based decision-making and well-structured digital policies (König, 2022; Zorrilla and Yebenes, 2022).

Governance 4.0 also emphasizes the importance of involving stakeholders in institutional decision-making. Increasingly, universities are realizing that it is necessary to include students, teachers, administrative staff, external partners, and public actors in governance processes to strengthen transparency, accountability, and responsiveness of the institution (Alkaraan et al., 2023a). Participatory governance can help higher education institutions identify new challenges, make decisions that align with stakeholders’ expectations, and create policies that are more inclusive and context-sensitive. This idea aligns with recent studies showing that digital transformation in education requires governance models capable of managing technology adoption, changes in teaching methods, and institutional accountability (Tømte and Smedsrud, 2023; Zhu et al., 2024).

Furthermore, Governance 4.0 supports the development of Collaborative networks linking Universities with industry, government agencies, civil society, and international partners. These networks facilitate the exchange of knowledge, expertise, data, and resources, thereby enabling Universities to address complex societal and technological challenges through interdisciplinary and Interinstitutional collaboration. In addition, the rapid development of artificial intelligence, particularly generative AI, introduces new governance challenges related to ethics, transparency, accountability, data protection, and the responsible use of AI in educational and research contexts (König, 2022; Saxena et al., 2023). Recent work on generative AI governance in educational research highlights the need for structured governance models that can guide institutions in managing AI-related risks while supporting innovation and scientific development (Pinho, Costa and Pinho, 2025).

Accordingly, the Transformation of university governance structures is essential for enabling higher education institutions to manage the technological, organizational, ethical, and cultural changes associated with the University 4.0 paradigm (Hammad et al., 2025b). Governance 4.0 therefore represents a strategic pillar of university transformation, as it provides the institutional conditions required to ensure transparency, agility, accountability, innovation, and sustainable digital development (Amrani et al., 2026; Saxena et al., 2023).

Table 4 brings together recent contributions on Governance 4.0, highlighting digital regulation mechanisms, the integration of ESG criteria, and performance management systems applied or transferable to the university context.

The literature on Governance 4.0 remains strongly rooted in the industrial sector and is still in its infancy when it comes to its empirical application to the management of higher education institutions, particularly in emerging countries. The adaptation of ESG models, performance measurement systems, and data governance frameworks to universities therefore constitutes a research project in its own right. Nevertheless, Governance 4.0 forms the third operational pillar of university 4.0, alongside Education 4.0 and Research 4.0, ensuring the coherence, transparency, and sustainability of the entire institutional transformation.

It is precisely to meet this need for a global articulation between these three pillars that the following section proposes an integrative conceptual framework, synthesizing the three constitutive dimensions of the University 4.0 transformation validated by the international literature.

2.5 Research gap and research contribution

Despite the increasing attention given to the concepts of Formation 4.0, Research 4.0, and Governance 4.0 in the literature, empirical studies examining these transformations from a multi-stakeholder perspective remain relatively limited. Many existing studies focus on a single dimension of university transformation, such as digital learning environments or research collaboration, without considering the broader institutional context.

Existing instruments provide important foundations for assessing digital transformation in higher education, but they remain limited in their scope. González-Pérez et al. primarily focused on the technological maturity of academic actors, whereas Valdivia-Salazar et al. examined several dimensions of digital transformation without explicitly structuring them around Education 4.0, Research 4.0, and Governance 4.0. In the Moroccan context, Amini et al. studied university changes only from the students’ perspective, without adopting a multi-stakeholder approach. The present study addresses these limitations by developing instruments for three groups of actors and integrating the three complementary dimensions of University 4.0. However, as the current phase remains exploratory, a large-scale definitive validation is still necessary.

Moreover, there is a lack of structured survey instruments capable of simultaneously capturing the perceptions of different university actors regarding the transformation of higher education institutions across these three dimensions. The perspectives of students, academic staff, and administrative personnel are rarely examined together, even though these groups play complementary roles in shaping the functioning of universities.

In the context of Moroccan Higher Education, research on digital transformation and institutional governance remains relatively scarce. While several initiatives have been implemented to modernize universities and promote innovation, there is limited empirical evidence on how these transformations are perceived by the different actors involved in the university ecosystem.

3 Methodology for developing instruments

3.1 Research design

This study adopted a methodological and exploratory research design aimed at developing, pilot testing, and refining multi-stakeholder questionnaires for assessing University 4.0 transformation in Moroccan higher education.

More specifically, the instruments were designed to measure stakeholders’ perceptions of the current implementation and maturity of university 4.0 practices, rather than future institutional readiness.

The methodological process combined classical instrument development procedures, pilot survey analysis, preliminary psychometric screening, and XAI-assisted item reduction.

The objective was not to establish definitive construct validity at the pilot stage, but rather to evaluate the feasibility, clarity, response burden, preliminary internal consistency, and empirical relevance of the initial item pool before large-scale validation. Accordingly, the pilot study was treated as an exploratory refinement phase, while full psychometric validation is planned for a subsequent multi-institutional deployment. The overall process was structured around seven successive steps, as shown in Figure 1.

Figure 1

3.2 Conceptual framework development

The conceptual framework of this study is grounded in the University 4.0 paradigm, which describes the transformation of higher education institutions in response to digitalization, the Fourth Industrial Revolution, and the evolving requirements of the knowledge-based economy. In this perspective, University 4.0 is not limited to the adoption of digital tools, but refers to a systemic institutional transformation involving education, research, and governance.

Based on the literature review, a three-dimensional conceptual framework was developed to structure the assessment of university 4.0 transformation in Moroccan higher education. The framework integrates three major domains: Education 4.0, Research 4.0, and Governance 4.0. These domains have been retained because they correspond to the university's three main functional missions: the transmission of knowledge, the production and enhancement of knowledge, as well as institutional coordination and governance.

To ensure methodological transparency, the conceptual framework was documented in Excel matrices structured into sections, components, sub-components, stakeholder perspectives, theoretical sources, and associated questionnaire items (see Supplementary Appendix 1). Each sub-dimension was linked to relevant literature and translated into measurable elements adapted to the Moroccan higher education context. This organization made it possible to move from a theoretical model of university 4.0; towards an operational assessment instrument designed for empirical validation.

Table 5 presents the conceptual framework of university 4.0 adopted in this study, showing the integration of Education 4.0, Research 4.0, and Governance 4.0, as well as their corresponding sub-dimensions and stakeholder perspectives.

3.3 Initial item generation

Based on the conceptual framework, an initial pool of questionnaire items was generated. The items were formulated in French using a five-point Likert scale, ranging from low agreement or low perceived maturity to high agreement or high perceived maturity.

Initially, six questionnaires were developed, corresponding to two instruments per stakeholder group, this initial structure allowed each stakeholder group to evaluate the dimensions most relevant to its institutional role.

The initial version comprised 183 items distributed across six questionnaires specific to different stakeholders and was not administered as a single instrument encompassing all these items. Students completed the “Student_Education” and “Student_Governance_Research” questionnaires, containing 22 and 34 items respectively. Academic staff completed the “Teacher_Education” and “Teacher_ Governance _Research” questionnaires, each containing 33 items. Administrative staff completed the “Administrative_Education” and “Administrative_ Governance _Research” questionnaires, containing 15 and 46 items respectively. Thus, the initial maximum number of items was 56 for students, 66 for academic staff, and 61 for administrative staff. This intentionally extensive item set was created to ensure sufficient conceptual coverage before item selection and reduction, based on a pilot test in accordance with scale development recommendations, which stipulate that the initial number of items should exceed the expected final number to allow for rigorous selection, reformulation, and deletion.

3.4 Pilot test administration

The pilot test was conducted at ENSET Mohammedia using the six preliminary questionnaires. The purpose of the pilot was to evaluate the practical feasibility of the instruments before large-scale deployment. The pilot phase examined response completion, item comprehension, missing responses, response consistency, and the empirical usefulness of the items.

Figure 2

Each dataset was structured as a matrix in which rows represented respondents and columns represented questionnaire items. Items were coded numerically on a five-point Likert scale. Metadata such as stakeholder group, questionnaire type, respondent identifier, and completion status were also included when available.

Because some stakeholder sub-samples were small, the pilot was interpreted primarily as a feasibility and refinement phase rather than as a definitive psychometric validation study.

3.5 Data preprocessing and response quality analysis

Straight-lining was interpreted as an exploratory indicator of potential satisfaction. A high proportion of repeated identical responses may indicate cognitive overload, low motivation, or insufficient understanding of the items. This indicator was particularly important because the initial questionnaires were relatively long and required a considerable response time.

3.6 Psychometric and XAI-assisted item screening

3.6.1 Preliminary psychometric screening

A preliminary psychometric screening was conducted to evaluate the empirical behavior of the items (Rice et al., 2020) before producing the refined versions of the questionnaires. This stage was not intended to establish definitive construct validity, but rather to identify weak, redundant, unstable, or poorly discriminating items during the pilot phase.

First, item-level descriptive indicators were examined, including missing-response rates, means, standard deviations, response variance, neutral-response frequency, and potential floor or ceiling effects. Items with high missingness or very low variance were considered problematic because they provided limited empirical information and could indicate ambiguity, non-applicability, or insufficient discrimination among respondents.

Second, internal consistency was assessed using Cronbach's alpha for each questionnaire. Although a threshold of α ≥ 0.70 is commonly considered acceptable in exploratory research, alpha values were interpreted cautiously because the questionnaires addressed multidimensional constructs related to Education 4.0, Research 4.0, and Governance 4.0. Therefore, high alpha values were not automatically interpreted as evidence of validity, since they may also reflect item redundancy. Conversely, low alpha values were interpreted as possible indicators of conceptual heterogeneity, poor item alignment, insufficient sample size, or the presence of items requiring reformulation.

Third, corrected item-total correlations were calculated to assess the degree to which each item was aligned with the remaining items of the questionnaire. Items with corrected item-total correlations below 0.30 were considered candidates for further review. In addition, Cronbach's alpha if item deleted was computed to examine whether removing a given item improved the internal consistency of the scale.

Finally, full-scale and reduced-scale scores were computed by averaging the Likert-scale responses across the initial item set and the retained item set, respectively. The concordance between both scores was assessed using Pearson correlation. A high full reduced score correlation was interpreted as evidence that the shortened version preserved the overall scoring structure of the original questionnaire.

As summarized in Table 7, all psychometric indicators were used as exploratory evidence and integrated into a broader multi-criteria decision-making process combining item statistics, qualitative feedback, expert judgement, and XAI-based item importance. The detailed results of these indicators are provided in Supplementary Appendix 2.

3.6.2 Explainable AI-assisted item reduction

To complement classical psychometric indicators, an Explainable Artificial Intelligence-assisted item reduction approach was introduced. Each questionnaire item was treated as a predictive feature, while the total questionnaire score was used as the target variable. The objective was to estimate the relative contribution of each item to the prediction of the overall University 4.0 maturity score within each questionnaire.

We used an ExtraTrees regression model as the underlying machine learning model, because it is well-suited for tabular data and can capture nonlinear relationships and interactions among the questionnaire items. Thus, this model comprised 500 randomized decision trees, with a minimum of two observations per terminal leaf (min_samples_leaf = 2), all available predictors considered at each split (max_features = 1.0), without bootstrap resampling, and a fixed random seed of 42.

A preliminary step was carried out prior to model fitting. Items with fewer than three non-missing responses, or showing no variation in responses, were excluded from the machine learning component. However, these items were retained during the psychometric analysis and expert review stages. So, missing responses to items, were handled using median imputation. To avoid any leakage during preprocessing, the median was estimated separately from the training observations, within each cross-validation fold and then applied to the corresponding validation observations.

In addition, we used repeated cross-validation to estimate the predictive performance and stability of item importance. To this end, for the two student questionnaires (each including 60 observations) a five-fold cross validation procedure (repeated ten times) was applied, resulting in 50 evaluations per questionnaire on a test sample. For the questionnaires intended for teachers and administrative staff (each including 10 observations) a two-fold cross validation repeated ten times was used.

Predictive performance was evaluated using the mean absolute error (MAE), the root mean square error (RMSE), and the mean and median coefficients of determination (R²). It should be noted that the item importance rankings derived from models with negative R² values after cross-validation were not considered sufficiently reliable to justify the removal of items.

We also implemented two complementary XAI methods. The TreeSHAP method was applied to the set-aside observations from each cross-validation partition. For each item, the explanatory importance was summarized using the mean absolute SHAP value. This value was calculated across all repeated set-aside partitions. Thus, higher mean absolute SHAP values indicated a greater average contribution to the prediction of the questionnaire's internal score. The stability of each item's position was assessed using the standard deviation of its SHAP rank across all repeated folds, with lower values indicating greater stability in the ranking.

Second, the importance by permutation was calculated using the same set of observations that had been set aside. Each variable was randomly permuted 20 times and its importance was quantified based on the resulting change in the MAE. Thus, a variable was considered more important when its permutation led to a greater deterioration in model performance.

To this end, the adopted rule stipulated that a low XAI result was considered convergent only if the item simultaneously fell within the lower 20% of both the SHAP absolute mean validated by cross-validation and the significance distribution by permutation. Thus, a low ranking according to only one XAI method was deemed insufficient to justify removing the item.

3.7 Expert validation and construction of the reduced questionnaires

The initial group of items was subjected to an expert review to assess content relevance, clarity of wording, conceptual scope, and contextual appropriateness. The expert review involved academic and institutional stakeholders familiar with digital transformation, higher education governance, pedagogy, and research management (Lynn, 1986; Polit and Beck, 2006).

In this section, following psychometric analyzes and those related to XAI, we employed an expert evaluation process to determine whether the items identified by the statistical analyzes should be retained, revised, or removed. This step ensured that the data-driven recommendations did not compromise the theoretical coverage of the concepts “Education 4.0,” “Research 4.0,” and “Governance 4.0.”

A panel of experts including three specialists in higher education, university governance, questionnaire design, and psychometrics independently evaluated each item. The items were assessed based on their relevance, clarity, conceptual consistency, contextual appropriateness, theoretical necessity, and redundancy.

Subsequently, the experts reached a final consensus on whether to retain, revise, or delete each item, weighing statistical evidence against theoretical relevance. This combined approach strengthened the content validity and conceptual robustness of the final questionnaire.

To do so, the experts evaluated each item based on its relevance, clarity, conceptual consistency, and suitability for the context, using a four-level rating scale, namely:

1 = irrelevant;

2 = requires extensive revision;

3 = relevant with minor revisions;

4 = highly relevant).

The experts were asked to propose a revised wording and to provide qualitative comments to justify their recommendations.

After assigning a score to the questionnaire, the item-level content validity index (I-CVI) was calculated. It represents the proportion of experts who assigned a score of 3 or 4 to an item.

The calculation formula is given in Equation 1:where n represents the number of experts who assigned a score of 3 or 4 to item i, and N_e represents the total number of experts.

After expert validation, for each questionnaire, the retained items were grouped according to their corresponding stakeholder category and conceptual domain. The objective was to obtain shorter instruments while preserving the theoretical coverage of the initial framework, particularly across the three main University 4.0 dimensions: Education 4.0, Research 4.0, and Governance 4.0.

To evaluate the adequacy of the reduced versions, a comparative scoring procedure was defined. First, a full-scale score was computed for each respondent by averaging the responses across all initial items of the corresponding questionnaire.

4 Results

4.1 Classical psychometric results

In this section, we will begin by presenting the psychometric results obtained. To this end, Table 9 presents the initial psychometric properties of the six questionnaires prior to the integration of XAI data and expert opinions. To this end, among the instruments designed for students, the “Student_Education” questionnaire showed questionable internal consistency (Cronbach's α = 0.663), with a relatively low mean corrected item-total correlation (M CITC = 0.253) and an average missing response rate of 13.6%. Furthermore, the “Student_Gov_Research” questionnaire showed acceptable reliability (α = 0.794), although its mean corrected item-total correlation was slightly below the conventional threshold of 0.30 (MCITC = 0.297), with 11.3% of responses missing.

In addition, the questionnaires for teachers yielded reliability coefficients ranging from acceptable to good: α = 0.774 for Teacher_Education and α = 0.806 for Teacher_Gov_Research. Their mean item-total correlations were 0.361 and 0.317, respectively. However, these results should be interpreted with caution, as both analyzes concerning teachers were based on only 10 respondents, and the rate of missing data was particularly high for “Teacher_Education” (35.5%) and remained significant for “Teacher_Gov_Research” (19.4%). Moreover, the questionnaires for administrators exhibited the weakest psychometric properties. “Admin_Education” showed unacceptable internal consistency (α = 0.018) and a very low corrected mean item-total correlation (MCITC = 0.046), while “Admin_Gov_Research” also demonstrated unacceptable.

4.2 Explainable AI results

4.2.1 Item importance based on explainable AI

Explainable artificial intelligence analysis provided further evidence regarding the relative contribution of each item to predicting the overall questionnaire score. SHAP values were used to rank the items according to their mean absolute contribution.

Table 10 presents the results of the internal consistency metrics, the predictive performance validated by cross-validation, the stability of the SHAP ranking and the concordance between the SHAP rankings and importance by permutation.

The results Table 10 show that for Student Education, the model demonstrated the best predictive performance, with an average R² of 0.441, a median R² of 0.493, an MAE of 0.188, and an RMSE of 0.239. Also, the agreement between the rankings established by SHAP and those obtained by permutation was high (ρ = 0.880), indicating that the two methods produced generally consistent item rankings.

For this category, Student_Education, according to Figure 3, questions Q07, Q12, Q16, Q03, and Q08 had the highest absolute average SHAP values after cross-validation. Their respective SHAP values were 0.0767, 0.0517, 0.0390, 0.0383, and 0.0286. Questions Q07 and Q12 also ranked first and second in terms of permutation importance, confirming their contribution to the questionnaire's internal score.

Figure 3

Regarding Student_Gov_Research, the model yielded a MAE of 0.211, a RMSE of 0.290, and an R² of 0.085. Although the agreement between the SHAP rankings and those based on permutation impotance was high (ρ = 0.874), the low R² value and the relatively high mean standard deviation of the SHAP rankings (6.33) indicated that the model had poor predictive generalization ability or that the item rankings varied across validation folds.

Thus, questions Q22, Q24, Q06, Q30, and Q32 had the highest SHAP values for Student_Gov_Research. However, these results were interpreted as exploratory rankings rather than as definitive evidence of the items’ importance.

Meanwhile, the four models related to teachers and administrative staff yielded negative mean R² values. Their XAI rankings were therefore reported for the sake of computational transparency, but were not considered sufficiently robust evidence to justify the removal of items. This proved particularly important for the Teacher_Gov_Research questionnaire and the two administrative questionnaires, for which the agreement between SHAP and permutation significance was weak or nonexistent.

4.2.2 Item retention, reformulation, and removal decisions

The XAI results were combined with other parameters related to corrected item-total correlations, missing response rates, item variance, comprehensibility data, theoretical coverage, and expert opinion. For this reason, a low contribution from the XAI was considered convergent only when an item simultaneously appeared in the bottom 20% of the SHAP importance distributions and permutation distributions.

Table 11 summarizes the provisional decisions generated by the multi-criteria procedure.

It should be noted that the notation “Retain with monitoring” indicates that an item presented a psychometric problem or weak convergent evidence of XAI, without sufficient justification for reformulation or deletion. The notation “Reformulate and submit to expert review” indicates multiple psychometric problems or a combination of weak statistical evidence and content-related issues. No items were deleted solely on the basis of XAI.

Analysis of Table 11 shows that no item met the full multi-criteria suppression rule. The Student_Education and Student_Gov_Research questionnaires each contained three items requiring reformulation or further expert review. The questionnaires for teachers and administrators contained more flagged items, but their negative R² values, validated by cross-validation, prevented the corresponding XAI rankings from being considered reliable evidence of suppression.

4.3 Interpretation by stakeholder group

4.3.1 Student questionnaires

The results of the analysis of Student_Education are as follows: the questionnaire initially consisted of 22 items, 20 of which were selected for the XAI analysis. It showed moderate internal consistency (Cronbach's α = 0.663) and achieved the best predictive performance among all questionnaires (mean R² = 0.441), with strong agreement between SHAP and permutation significance (ρ = 0.880), indicating robust feature importance. The most influential items were Q07, Q12, Q16, Q03, and Q08, while Q15, Q21, and Q22 were identified for expert review and possible reformulation. No items met the criteria for deletion.

For the “Student_Gov_Research” questionnaire consisted of 34 items, of which 32 were included in the analysis. It exhibited good internal consistency (Cronbach's α = 0.794) but limited predictive performance (mean R² = 0.085). Despite the strong agreement between SHAP and permutation significance (ρ = 0.874), the stability of the ranking of characteristics across rounds was lower, suggesting that results should be interpreted with caution. It was recommended that questions Q16, Q20, and Q34 be reformulated, while no questions were deemed suitable for deletion. Any further refinement should be based on expert opinion and validation using a larger sample.

4.3.2 Teacher questionnaires

For the results of Teacher_Education, the questionnaire initially consisted of 33 items, of which 25 were selected for the XAI analysis. It exhibited acceptable internal consistency (Cronbach's α = 0.774), but its predictive performance was low (mean R² = −0.283), indicating insufficient generalizability of the model. The moderate agreement between SHAP and permutation significance (ρ = 0.474) was not sufficient to allow for reliable item selection based on XAI. For this reason, twelve items were recommended for review by experts and possible reformulation, while no items met the criteria for deletion.

Regarding the Teacher_Gov_Research, the questionnaire contained 33 items, 31 of which were included in the analysis. It exhibited good internal consistency (Cronbach's α = 0.806), but the model showed very low predictive power (mean R² = −0.908) and no significant agreement between SHAP and permutation significance (ρ = −0.038), indicating unstable feature importance. Consequently, 15 items were recommended for reformulation or expert review, while no items were deemed suitable for deletion.

4.3.3 Administrative staff questionnaires

The results of the administrative staff questionnaires revealed the greatest psychometric and predictive weaknesses. In fact, as shown in Table 11, the “Admin_Education” questionnaire consisted of 15 items, all of which were technically usable for XAI. However, Cronbach's alpha was only 0.018, indicating a lack of coherent internal structure in the current responses. The model produced an average R² of −1.009, while the agreement between SHAP and permutation significance was very low (ρ = 0.076).

Consequently, items Q03, Q04, Q07, Q08, Q11, and Q13 were returned for reformulation and expert review.

Furthermore, the Admin_Gov_Research scale initially included 46 items, but only 29 exhibited sufficient response variation and non-missing observations for the XAI model. Cronbach's alpha was 0.274, indicating low internal consistency. The model produced a negative average R² of −0.354 and low agreement between SHAP and permutation significance (ρ = 0.148).

Thus, twenty-six items were returned for rewording and review by experts. This high number should not be interpreted as a recommendation to delete all the flagged items. Rather, it indicates possible conceptual heterogeneity, wording issues, limited applicability of the responses, insufficient variance, or a disconnect from the experiences of administrative staff.

As summarized of the XAI-Assisted Refinement Results, the Student_Education questionnaire yielded the most reliable and interpretable results, while the “Student_Gov_Research” questionnaire showed strong agreement between SHAP and permutation importance measures, despite limited predictive performance. In contrast, the questionnaires for teachers and administrative staff yielded negative R² values during cross-validation, indicating that their XAI results should be interpreted with caution. Consequently, no questionnaire items were removed; instead, XAI was used to identify items requiring expert review or reformulation.

4.4 Expert review of the flagged items

Building on the results of psychometrics and XAI, which will be used as a decision-making tool.The expert panel evaluated the 183 items based on their relevance, clarity, consistency, and appropriateness to the context. The I-CVI scores ranged from 0.000 to 1.000. The S-CVI/Ave values were 0.894, 0.941, 0.505, 0.384, 0.422, and 0.428, as shown in Table 12, for the Student_Education, Student_Gov_Research, Teacher_Education,Teacher_Gov_Research,Admin_Education, and, Admin_Gov_Research questionnaires, respectively. Among the 65 items initially flagged by the psychometric procedure and XAI, 03 were retained without modification, 14 were recommended for rewording, and 48 were deleted following expert consensus.

Consequently, and based on the results in Table 13, out of the total of 183 items, 52 (28.4%) were retained without modification, 51 (27.9%) were recommended for rewording, and 80 (43.7%) were recommended for deletion. The questionnaires for students had the highest proportions of retained items, while those for teachers and administrative staff required more extensive revisions. In particular, no items from the “Teacher_Education,” “Teacher_Gov_Research,” or “Admin_Education” questionnaires were retained as is. These results indicate that the questionnaires for students demonstrated comparatively stronger content validity, while those for teachers and administrative staff require significant improvements before large-scale validation.

5 Discussion

Our study aimed to develop and refine multi-part questionnaires designed to assess the transition to university 4.0 within Moroccan higher education institutions. Preliminary results indicate that a hybrid methodological approach, combining classical psychometric analysis and explainable artificial intelligence, allowed us to reduce the number of items while preserving the overall scoring structure of the original instruments. This contribution is particularly relevant in a context where the assessment of university 4.0 requires both broad conceptual coverage and practical feasibility.

In fact, the findings indicate the questionnaires designed for students achieved the best psychometric performance. The “Student_Gov_Research” questionnaire demonstrated acceptable internal consistency (Cronbach's α = 0.794) and excellent content validity (S-CVI/Ave = 0.941), while the “Student_Education” questionnaire showed slightly lower reliability (α = 0.663) but achieved satisfactory content validity (S-CVI/Ave = 0.894). Consequently, these results suggest that only a limited number of items required revision, so as a result most of the items intended for students were retained.

However, while the teacher questionnaires showed acceptable reliability coefficients (α = 0.774–0.806), their low content validity indices (S-CVI/Ave = 0.384–0.505) revealed a significant discrepancy between statistical reliability and expert opinion. This finding indicates that internal consistency alone cannot guarantee adequate content validity and highlights the importance of expert validation during questionnaire development.

The results further suggest that Explainable Artificial Intelligence can serve as a useful exploratory tool for questionnaire refinement, particularly by identifying items with high or low contributions to the predicted overall score. Nevertheless, the study also confirms that XAI should not replace psychometric theory or expert judgment. Item reduction must remain grounded in theoretical relevance, content validity, stakeholder-specific interpretation, and empirical performance.

On the other hand, the administrative questionnaires achieved the lowest overall performance. Both questionnaires exhibited low reliability (α = 0.018 and 0.274), poor item discrimination, and low inter-expert consensus (S-CVI/Ave ≈ 0.42), indicating that thorough conceptual and linguistic revisions are needed before any further validation.

XAI analysis provided useful additional evidence. The results showed that only the Student_Education questionnaire displayed satisfactory predictive accuracy (R² = 0.441), thus allowing for the interpretation of SHAP and importance by permutation with reasonable confidence. However, the other questionnaires produced low or negative cross-validation R² values which indicate limited generalizability. The integration of XAI into the item reduction process represents a methodological contribution of this study. Therefore, the results of the XAI analysis were considered exploratory rather than definitive and were never used as the sole criterion for item removal. Instead, they were combined with corrected item-total correlations, missing response rates, reliability statistics, theoretical considerations, and expert assessments to facilitate decision-making.

The overall results are as follows: 52 items (28.4%) were retained, 51 (27.9%) were reformulated, and 80 (43.7%) were deleted. The higher retention rate observed for the student questionnaires suggests that these questionnaires better reflected respondents’ perceptions and expert assessments.

In summary, the results support the use of a hybrid, multi-criteria framework for questionnaire improvement. The complementary use of psychometric indicators, explainable AI, and expert opinion enabled more informed decision-making while reducing the risk of eliminating theoretically important elements based solely on statistical criteria or machine learning. However, the results also confirm that satisfactory reliability does not necessarily imply adequate content validity.

6 Conclusion

Our study presented the development, pilot testing, and exploratory refinement of multi-stakeholder questionnaires designed to assess the transition toward “University 4.0” in Moroccan higher education institutions. The proposed instruments were structured around three core dimensions, namely Education 4.0, Research 4.0, and Governance 4.0, and were adapted to three main stakeholder groups: students, faculty members, and administrative staff.

The main methodological contribution of our work lies in the integration of classical psychometric analysis with Explainable Artificial Intelligence-assisted item reduction. The combined use of corrected item-total correlations, Cronbach's alpha coefficient, missing-response analysis, variance analysis, full-versus-reduced score comparison, SHAP values, permutation importance, and expert judgment enabled the development of a transparent, multi-criteria procedure for refining the initial pool of items.

The findings showed stronger psychometric and content-validity evidence for the student questionnaires. However, the teacher and administrative instruments required more substantial revision. Across the 183 items, 52 were retained, 51 were recommended for reformulation, and 80 were recommended for removal. XAI provided complementary evidence but was interpreted cautiously when cross-validated predictive performance was weak or negative.

The results further suggest that Explainable Artificial Intelligence can serve as a useful exploratory tool for questionnaire refinement, particularly by identifying items with high or low contributions to the predicted overall score. Nevertheless, the study also confirms that XAI should not replace psychometric theory or expert judgment. Item reduction must remain grounded in theoretical relevance, content validity, stakeholder-specific interpretation, and empirical performance.

Several limitations should be acknowledged. First, the pilot sample was limited in size and unevenly distributed across stakeholder groups. Second, some teacher and administrative staff subsamples were too small to support robust conclusions regarding the psychometric and XAI-based results. Third, factor validation could not be rigorously conducted at this exploratory stage. Fourth, the response-time variable was not available in the analyzed dataset, although respondent burden remains a key justification for reducing questionnaire length.

Future research will focus on validating and applying refined tools to assess the current state of University 4.0 and digital transformation within Moroccan higher education institutions. A larger, more balanced, and multi-institutional sample will be used to examine the factorial structure, reliability, construct validity, and measurement invariance of the questionnaires. The validated instruments will then be used to identify institutional strengths and weaknesses, disparities, and levels of transformation in the areas of Education 4.0, Research 4.0, and Governance 4.0. In addition, empirical studies using a mixed-methods approach will combine quantitative statistical analyses with interviews, focus groups, and institutional case studies to provide a deeper understanding of stakeholders’ experiences and the contextual factors influencing the transformation towards University 4.0 in Morocco.

In conclusion, this study proposes a reproducible methodological approach for developing and refining University 4.0 assessment instruments. By combining psychometric evaluation, expert knowledge, and Explainable Artificial Intelligence, it provides a rigorous and transparent framework for reducing questionnaire length while preserving conceptual relevance, measurement consistency, and practical usability.

References

• 1

  AhmadE. (2024). The convergence of education 4.0 and industry 4.0: a twin peaks model. J. Innov. Digit. Transform.1 (1), 68–83. 10.1108/JIDT-10-2023-0029

  • CrossRef
  • Google Scholar

• 2

  AlfirevićA. M.MabićM.AlfirevićN. (2025). Evaluating digital maturity in higher education institutions: a preliminary empirical study in the western balkans. World6 (4), 130. 10.3390/world6040130

  • CrossRef
  • Google Scholar

• 3

  AlkaraanF.ElmarzoukyM.HussaineyK.VenkateshV. G. (2023). Sustainable strategic investment decision-making practices in UK companies: the influence of governance mechanisms on synergy between industry 4.0 and circular economy. Technol. Forecast. Soc. Change.187, 122187. 10.1016/j.techfore.2022.122187

  • CrossRef
  • Google Scholar

• 4

  AminiN.SefriY.ChakliA.MahiriF.AassoulA.RadidM. (2023). University changes in the 4.0 educational era: a study into Moroccan Students’ interests. J. Curric. Teach.12, 100. 10.5430/jct.v12n3p100

  • CrossRef
  • Google Scholar

• 5

  AmraniH.ChaiboubM.AftissA. (2026). Performance indicators and management control tools in Moroccan public universities: a quantitative survey of managers. Probl. Perspect. Manag.24 (1), 838–856. 10.21511/ppm.24(1).2026.54

  • CrossRef
  • Google Scholar

• 6

  BenhimaM. (2025). The national plan to accelerate the transformation of the Moroccan higher education, scientific research, and innovation system (Pacte ESRI 2030): trends and future orientations. J. Qual. Educ.15 (26), 162–181. 10.37870/joqie.v15i26.465

  • CrossRef
  • Google Scholar

• 7

  BondM.MarínV. I.DolchC.BedenlierS.Zawacki-RichterO. (2018). Digital transformation in German higher education: student and teacher perceptions and usage of digital media. Int. J. Educ. Technol. High. Educ.15 (1), 48. 10.1186/s41239-018-0130-1

  • CrossRef
  • Google Scholar

• 8

  BonfieldC. A.SalterM.LongmuirA.BensonM.AdachiC. (2020). Transformation or evolution? Education 4.0, teaching and learning in the digital age. Higher Educ. Pedagog.5, 223–246. 10.1080/23752696.2020.1816847

  • CrossRef
  • Google Scholar

• 9

  Bravo-JaicoJ.AlarcónR.ValdiviaC.GermánN.AquinoJ.SerquénO.et al (2025a). Model for assessing the maturity level of digital transformation in higher education institutions: a theoretical-methodological approach. Front. Educ.10, 1581648. 10.3389/feduc.2025.1581648

  • CrossRef
  • Google Scholar

• 10

  Bravo-JaicoJ.Maquen-NiñoG. L. E.GermánN.ValdiviaC.AlarcónR.AquinoJ.et al (2025b). Assessing digital transformation maturity in higher education institutions: a correlational analysis by actors and dimensions. Front. Comput. Sci.7, 1549262. 10.3389/fcomp.2025.1549262

  • CrossRef
  • Google Scholar

• 11

  CaiY.LattuA. (2022). Triple Helix or quadruple Helix: which model of innovation to choose for empirical studies?Minerva60, 257–280. 10.1007/s11024-021-09453-6

  • CrossRef
  • Google Scholar

• 12

  ChaengprommaN.PattanapairojS. (2022). A gap study between industry expectations and current competencies of bachelor’s degree graduates in industrial engineering in Thailand 4.0 era: a case study of industrial engineering graduates of Khon Kaen university. Cogent Educ.9, 2093491. 10.1080/2331186X.2022.2093491

  • CrossRef
  • Google Scholar

• 13

  ChahidA.AhrizS.El GuemmatK.MansouriK. (2025). Developing digital capabilities through IT governance: a PLS-SEM analysis in Moroccan higher education institutions. Bull. Electric. Eng. Inform.14 (2), 1543–1552. 10.11591/eei.v14i2.8182

  • CrossRef
  • Google Scholar

• 14

  ChernayaI. P.MasyukN. N.ProsalovaV. S.BodunkovaA. G.BushuevaM. A. (2023). University 4.0 concept: educational and scientific policies, innovative development of vocational education and training. Front. Educ.8, 1125361. 10.3389/feduc.2023.1125361

  • CrossRef
  • Google Scholar

• 15

  ChountaI.-A.Ortega-ArranzA.DaskalakiS.DimitriadisY.AvourisN. (2024). Toward a data-informed framework for the assessment of digital readiness of higher education institutions. Int. J. Educ. Technol. High. Educ.21, 59. 10.1186/s41239-024-00491-0

  • CrossRef
  • Google Scholar

• 16

  Coello PiscoS. M.RodríguezB.BangueraL.BaidalE. (2024). Research skills R+ D+ I and industry 4.0, STEM and TRIZ and their application in the professional skills of applied physics students. Revista Mexicana de Física E21 (1), 1–12. 10.31349/RevMexFisE.21.010212

  • CrossRef
  • Google Scholar

• 17

  DaiZ.WangL.PengX.ZhaoL.XiongJ. (2024). A model for assessing student satisfaction with smart classroom environment in higher education. J. Comput. Assist. Learn.40 (6), 2901–2916. 10.1111/jcal.13045

  • CrossRef
  • Google Scholar

• 18

  DeVellisR. F.ThorpeC. T. (2021). Scale Development: Theory and Applications. 5th Edn. Thousand Oaks, CA: Sage.

  • Google Scholar

• 19

  Díaz-GarciaV.Montero-NavarroA.Rodríguez-SánchezJ.-L.Gallego LosadaR. (2023). Managing digital transformation: a case study in a higher education institution. Electronics (Basel)12, 2522. 10.3390/electronics12112522

  • CrossRef
  • Google Scholar

• 20

  EgerL.ŽižkaM. (2024). Industry 4.0, digital transformation and human resource management: emerging themes and research trends in the context of the Visegrad countries. Oeconomia Copernicana15, 1021–1065. 10.24136/oc.3034

  • CrossRef
  • Google Scholar

• 21

  El MajjaouiA.El HaddadiA.BahriA.BouhaferF.El HaddadiA.RharrabeK.et al (2026). Mapping Morocco’s scientific contribution to the sustainable development goals (2015–2024): a bibliometric analysis. Discov. Sustain.7, 58. 10.1007/s43621-025-02313-3

  • CrossRef
  • Google Scholar

• 22

  GallonL.SalamehK.ChbeirR.BachirS.AniortéP. (2024). Educational cyber–physical systems (ECPSs) for University 4.0. Information15 (12), 790. 10.3390/info15120790

  • CrossRef
  • Google Scholar

• 23

  GiesenbauerB.Müller-ChristG. (2020). University 4.0: promoting the transformation of higher education institutions toward sustainable development. Sustainability12 (8), 3371. 10.3390/su12083371

  • CrossRef
  • Google Scholar

• 24

  González-PérezL.-I.Enciso-GonzálezJ. A.Vicario-SolorzanoC. M.Ramírez-MontoyaM.-S. (2025). Measuring digital transformation in education 4.0 with DT-smarty: valid and reliable model. Technol. Knowl. Learn., 10.1007/s10758-025-09844-8

  • CrossRef
  • Google Scholar

• 25

  HammadM. Y.RahamaddullaS. R.FauziM. A. (2025b). Environmental and governance strategies in ESG for industry 4.0: a systematic review. AIMS Environ. Sci12, 557–575. 10.3934/environsci.2025025

  • CrossRef
  • Google Scholar

• 26

  HammadM. Y.RahamaddullaS. R.SorooshianS.TamyezP. F. M.FauziM. A. (2025a). Developing countries and the industry 4.0 movement: research mapping for practical insights. Bangladesh J. Multidiscip. Sci. Res10, 46–55. 10.46281/bjmsr.v10i6.2587

  • CrossRef
  • Google Scholar

• 27

  Hervas-OliverJ.-L. (2022). Industry 4.0 in industrial district SMEs: understanding collective knowledge transfer by research and transfer institutes. Compet.. Rev.: An Int. Bus. J.32 (5), 647–666. 10.1108/CR-06-2022-0075

  • CrossRef
  • Google Scholar

• 28

  KönigP. D. (2022). Fortress Europe 4.0? An analysis of EU data governance through the lens of the resource regime concept. Eur. Policy Anal.8 (4), 484–504. 10.1002/epa2.1160

  • CrossRef
  • Google Scholar

• 29

  LaciokV.BernatikA.LesnakM. (2020). Experimental implementation of new technology into the area of teaching occupational safety for industry 4.0. Int. J. Saf. Secur. Eng.10 (3), 403–407. 10.18280/ijsse.100313

  • CrossRef
  • Google Scholar

• 30

  LikertR. (1932). A technique for the measurement of attitudes. Arch. Psychol.140, 1–55.

  • Google Scholar

• 31

  LynnM. R. (1986). Determination and quantification of content validity. Nurs. Res.35 (6), 382–386. 10.1097/00006199-198611000-00017

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32

  MachadoC. G.WinrothM. P.Ribeiro da SilvaE. H. D. (2020). Sustainable manufacturing in industry 4.0: an emerging research agenda. Int. J. Prod. Res.58 (5), 1462–1484. 10.1080/00207543.2019.1652777

  • CrossRef
  • Google Scholar

• 33

  MasdokiM.DinR.Effendi Ewan Mohd MatoreM. (2021). Teaching 4.0 competency in higher learning institutions: a systematic mapping review. Int. J. Learn. Teach. Educ. Res.20 (10), 217–231. 10.26803/ijlter.20.10.12

  • CrossRef
  • Google Scholar

• 34

  MianS. H.SalahB.AmeenW.MoiduddinK.AlkhalefahH. (2020). Adapting universities for sustainability education in industry 4.0: channel of challenges and opportunities. Sustainability12 (15), 6100. 10.3390/su12156100

  • CrossRef
  • Google Scholar

• 35

  MirandaJ.NavarreteC.NoguezJ.Molina-EspinosaJ.-M.Ramírez-MontoyaM.-S.Navarro-TuchS. A.et al (2021). The core components of education 4.0 in higher education: three case studies in engineering education. Comput. Electr. Eng.93, 107278. 10.1016/j.compeleceng.2021.107278

  • CrossRef
  • Google Scholar

• 36

  MondolS. I. M. M. R.LeeS. (2026). Explainability guided feature selection for speech emotion recognition using SHAP and permutation importance. Biomed. Eng. Lett., 10.1007/s13534-026-00592-x

  • CrossRef
  • Google Scholar

• 37

  MourtzisD.PanopoulosN.AngelopoulosJ. (2023). A hybrid teaching factory model towards personalized education 4.0. Int. J. Comput. Integr. Manuf.36 (12), 1739–1759. 10.1080/0951192X.2022.2145025

  • CrossRef
  • Google Scholar

• 38

  MukulE.BüyüközkanG. (2023). Digital transformation in education: a systematic review of education 4.0. Technol. Forecast. Soc. Change.194, 122664. 10.1016/j.techfore.2023.122664

  • CrossRef
  • Google Scholar

• 39

  NguyenQ. M.NguyenV. P.VuT. H.DzungV. T.HoangT. S. V. (2024). Indicators for university autonomy in the context of university 4.0: a case study of Hanoi university of civil engineering from Vietnam. Int. J. Sustain. Constr. Eng. Technol.15 (4), 100–122. 10.30880/ijscet.2024.15.04.008

  • CrossRef
  • Google Scholar

• 40

  PajpachM.HaffnerO.KučeraE.DrahošP. (2022). Low-cost education kit for teaching basic skills for industry 4.0 using deep-learning in quality control tasks. Electronics (Basel)11 (2), 230. 10.3390/electronics11020230

  • CrossRef
  • Google Scholar

• 41

  PanZ.BiegleyL.TaylorA.ZhengH. (2024). A systematic review of learning analytics: incorporated instructional interventions on learning management systems. J. Learn. Anal.11 (2), 52–72. 10.18608/jla.2023.8093

  • CrossRef
  • Google Scholar

• 42

  PinhoI.CostaA. P.PinhoC. (2025). Generative AI governance model in educational research. Front. Educ.10, 1594343. 10.3389/feduc.2025.1594343

  • CrossRef
  • Google Scholar

• 43

  PolitD. F.BeckC. T. (2006). The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res. Nurs. Health.29 (5), 489–497. 10.1002/nur.20147

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44

  RejebA.RejebK.SüleE.HassounA.KeoghJ. G. (2025). Knowledge flows in industry 4.0 research: a longitudinal and dynamic analysis. J. Data Inform. Manag.7 (2), 123–145. 10.1007/s42488-025-00146-3

  • CrossRef
  • Google Scholar

• 45

  RiceS. M.ParkerA. G.MawrenD.CliftonP.HarcourtP.LloydM.et al (2020). Preliminary psychometric validation of a brief screening tool for athlete mental health among male elite athletes: the athlete psychological strain questionnaire. Int. J. Sport. Exerc. Psychol.18 (6), 850–865. 10.1080/1612197X.2019.1611900

  • CrossRef
  • Google Scholar

• 46

  Rosak-SzyrockaJ. (2024). The era of digitalization in education where do universities 4.0 go?Manag. Syst. Product. Eng.32 (1), 54–66. 10.2478/mspe-2024-0006

  • CrossRef
  • Google Scholar

• 47

  SalahB. (2020). Integrating the concept of industry 4.0 by teaching methodology in industrial engineering curriculum. Processes8 (9), 1007. 10.3390/pr8091007

  • CrossRef
  • Google Scholar

• 48

  Satish Kumar MahariyaJ.Awaneesh KumarA. G.Rajesh SinghD.Anita GehlotP.Shaik Vaseem AkramP.Bhekisipho TwalaM.et al (2023). Smart campus 4.0: digitalization of university campus with assimilation of industry 4.0 for innovation and sustainability. J. Adv. Res. Appl. Sci. Eng. Technol.32 (1), 120–138. 10.37934/araset.32.1.120138

  • CrossRef
  • Google Scholar

• 49

  SaunilaM.UkkoJ.NasiriM.GarengoP. (2024). The role of digital governance in the integration of performance measurement systems uses and industry 4.0 maturity. Int. J. Comput. Integr. Manuf.37 (7), 809–822. 10.1080/0951192X.2023.2257918

  • CrossRef
  • Google Scholar

• 50

  SaxenaA.SinghR.GehlotA.AkramS. V.TwalaB.SinghA.et al (2023). Technologies empowered environmental, social, and governance (ESG): an industry 4.0 landscape. Sustainability15 (1), 309. 10.3390/su15010309

  • CrossRef
  • Google Scholar

• 51

  SecundoG.Del VecchioP.SimeoneL.SchiumaG. (2020). Creativity and stakeholders’ engagement in open innovation: design for knowledge translation in technology-intensive enterprises. J. Bus. Res.119, 272–282. 10.1016/j.jbusres.2019.02.072

  • CrossRef
  • Google Scholar

• 52

  ShalT.GhamrawiN.Abu-TinehA.AlshaboulY. (2026). Higher education 4.0: capturing the voices of faculty members. Front. Educ.10, 1626340. 10.3389/feduc.2025.1626340

  • CrossRef
  • Google Scholar

• 53

  SikandarH.VaicondamY.KhanN.QureshiM. I.UllahA. (2021). Scientific mapping of industry 4.0 research: a bibliometric analysis. Int. J. Interact. Mob. Technol.15 (18), 129–147. 10.3991/ijim.v15i18.25535

  • CrossRef
  • Google Scholar

• 54

  TaiM. K.OmarA. K.MusaK.GhouriA. M. (2022). Leading teaching and learning in the era of education 4.0: the relationship between perceived teacher competencies and teacher attitudes toward change. Asian J. Univ. Educ.18, 65–80. 10.24191/ajue.v18i1.17171

  • CrossRef
  • Google Scholar

• 55

  TamerH.KnidiriZ. (2023). University 4.0: digital transformation of higher education evolution and stakes in Morocco. Am. J. Smart Technol. Solut.2 (1), 20–28. 10.54536/ajsts.v2i1.1300

  • CrossRef
  • Google Scholar

• 56

  TømteC. E.SmedsrudJ. H. (2023). Governance and digital transformation in schools with 1:1 tablet coverage. Front. Educ.8, 1164856. 10.3389/feduc.2023.1164856

  • CrossRef
  • Google Scholar

• 57

  ÜlkerN.OtrarM. (2025). University 4.0 essentials: influence of industry 4.0 on higher education. Industry High. Educ.39, 528–538. 10.1177/09504222241295625

  • CrossRef
  • Google Scholar

• 58

  VenkatramanS.BenliF.WeiY.WahrF. (2022). Smart classroom teaching strategy to enhance higher order thinking skills (HOTS). an agile approach for education 4.0. Fut. Internet14 (9), 255. 10.3390/fi14090255

  • CrossRef
  • Google Scholar

• 59

  VrignatP.AvilaM.DucultyF.BardetC.BegotS.MarangéP. (2025). Innovative pedagogies for industry 4.0: teaching RFID with serious games in a project-based learning environment. Educ. Sci.15 (8), 953. 10.3390/educsci15080953

  • CrossRef
  • Google Scholar

• 60

  YondriS.Ganefri-Krismadinata-JalinusN.Sukardi- (2020). A new syntax of teaching factory IR 4.0 model in vocational education. Int. J. Adv. Sci. Eng. Inf. Technol.10 (6), 2270–2275. 10.18517/ijaseit.10.6.13197

  • CrossRef
  • Google Scholar

• 61

  Zainal AbidinN. B.Mat JanN. A.OthmanA. Z.ThiruchelvamL.JinnL. S.Abdullah SaniSA’et al. (2023). A five-year bibliometric analysis of education 4.0 and direction for education 5.0 future research. Int. J. Learn. Teach. Educ. Res.22, 382–400. 10.26803/ijlter.22.9.21

  • CrossRef
  • Google Scholar

• 62

  ZhuR.AliasB. S.HamzahM. I. M.HamidM. R. A. (2024). A threefold examination of university digital leadership, teacher digital competency, and teacher technology behavior for digital transformation of education. Int. J. Learn. Teach. Educ. Res.23, 272–289. 10.26803/ijlter.23.10.13

  • CrossRef
  • Google Scholar

• 63

  ZorrillaM.YebenesJ. (2022). A reference framework for the implementation of data governance systems for industry 4.0. Comput. Stand. Interf.81, 103595. 10.1016/j.csi.2021.103595

  • CrossRef
  • Google Scholar

• 64

  ZulkipliN. H.MusaM. (2022). Education 4.0: an analysis of teachers’ attitudes towards the use of technology in teaching mathematics. Int. J. Inf. Educ. Technol.12 (7), 609–614. 10.18178/ijiet.2022.12.7.1660

  • CrossRef
  • Google Scholar