Improving engineering students’ understanding of classical physics through visuo-haptic simulations

1 Introduction

The field of education is currently experiencing a significant transformation known as Education 4.0, aligned with the ongoing Industrial Revolution World Economic Forum (2020). The current trend in education emphasizes a shift from traditional, teacher-centered approaches towards more customized, learner-driven practices (UNESCO, 2023). Simultaneously, technological advancements have shown the necessity for educational approaches that not only enrich learning experiences but also develop a deeper comprehension of complex concepts.

Modern technologies, especially those incorporating haptic feedback and embodied learning, are central to the aforementioned revolution. By integrating multi-modal experiences, these technologies have the potential to significantly enhance educational environments, thereby improving the learning process’s overall quality. According to McLinden et al., the presence of receptors spread throughout our skin and the rest of our bodies makes the sense of touch an effective tool for gathering information Mclinden et al. (2019). When engaging in active touch, a dynamic interaction exists between our kinesthetic system and the surrounding environment. This interaction fulfills three fundamental functions: a) acquiring information about the environment; b) recognizing textures, rigidity, and contours; and c) recognizing related characteristics of objects, such as shape and weight Lederman and Klatzky (2009).

While studies have explored the potential of multi-modal technologies in education, there remains a gap in understanding the full impact of these technologies on students’ cognitive processes and learning experiences (Neri et al., 2015; Shaikh et al., 2017; Yuksel et al., 2017; Neri et al., 2018; Neri et al., 2020; Walsh et al., 2020; Walsh and Magana, 2023). This underlines the importance of further research into how visuo-haptic simulations can be optimized during their development to support and implement effective learning strategies.

This work presents the design and development of three visuo-haptic simulators aimed at teaching classical physics concepts. Through interactive engagement, these simulators allow students to explore and physically experience forces, enhancing their learning experience. The objective of this study is to evaluate the usability of these simulators and understand students’ perceptions of their educational value. To this end, the following research questions were proposed:

1. How does visuo-haptic technology influence students’ engagement and perception of learning fundamental physics concepts?

2. What are students’ perceptions regarding the intuitiveness and educational potential of visuo-haptic simulators?

3. What useful information can be derived from students’ interactions and feedback when engaging with these simulations?

To address these questions, the end-user computer satisfaction survey, proposed by Doll and Torkzadeh (1988), was adapted to assess the usability and perceived educational effectiveness of the simulators. This adaptation aims not only to assist educators in designing multi-modal learning scenarios but also to establish a framework for evaluating the impact of these technologies on the learning experience.

This article is structured as follows: The related work is presented in Section 2. Section 3 provides the materials and methods employed in the study. Sections 4 and 5 present the results and discussions related to the design and implementation of the simulators, according to the feedback and outcomes of the questionnaire, respectively. Finally, Section 6 outlines the study’s conclusions, key observations, and potential directions for future research.

2 Related work

Classical physics refers to the theories and principles that describe the physical universe’s various phenomena in terms of a handful of fundamental laws and concepts. These classical concepts form a foundational framework that effectively explains a wide range of macroscopic events and phenomena society observes daily. Due to the nature of physics as a fact-based science, students need to interact in some way with the natural phenomena they wish to describe. However, analyzing these phenomena in the classroom is not always feasible.

Hence, the addition of multi-modal interactions has allowed the development of enhanced teaching and learning models. The idea of embodied cognition plays an essential role in this progress. It is a paradigm that considers all forms of human knowledge and cognition “embodied”, as they are acquired through bodily experiences (Mahon and Caramazza, 2008; Maggio et al., 2022). Recent advancements in e-learning courses and Technology Enhanced Learning (TEL) methodologies have experienced an increase in recent years within the fields of Science, Technology, Engineering, and Mathematics (STEM), serving as additional resources to traditional classroom lectures, and they aid students in developing their problem-solving abilities and critical and creative thinking (Park et al., 2010).

According to Taljaard, multi-modal technologies are rapidly gaining attention as effective and novel methods for improving educational practices Taljaard (2016). As a result, there has been a significant surge in the development of educational simulators and learning platforms centered on physics concepts. The Coriolis effect is the way a moving object seems to deviate when seen from a rotating point of reference. Hamza-Lup and Page created a visuo-haptic simulation to explain this effect Hamza-Lup and Page (2012). The simulator allowed participants to manipulate a ball from a rotating surface. Furthermore, the simulation offered participants a synchronized perspective of the rotational movement following the surface. Despite encountering difficulties in navigating the ball during their initial interactions, the majority of participants later perceived the controls to be intuitive. After becoming accustomed to the simulation, the feedback provided by the students was predominantly positive, with a notable 94% expressing enthusiasm for the positive aspects of the simulation.

Zhuoluo et al. performed a study to evaluate the potential of haptic technology for self-learning (Zhuoluo et al., 2019). The authors designed an educational application integrated with a haptic device to teach the concept of friction. The haptic feedback was provided using a 2-degree-of-freedom pantograph device named “Haply”. To ensure seamless communication with the Haply devices, the Unity3D game engine and the Haply application programming interface were used in the development of the visuo-haptic simulation ecosystem. Participants were split into two groups: a control group and an experimental group. Post-experiment evaluations encompassed both a test and a feedback questionnaire. Results showed that the integration of the haptic device not only enhanced the effectiveness of the application but also stimulated increased levels of enthusiasm and motivation among the students.

Yuksel et al.‘s study on friction forces used haptic feedback and visual cues to create the educational visuo-haptic simulation (Yuksel et al., 2019). The simulator simulates the dynamics of frictional forces between two rigid bodies, specifically focusing on phenomena such as stick-to-slip transitions and steady sliding. The study analyzed the impact of variables like object mass, size, and the contacting surface (i.e., friction coefficient) on the nature of the frictional force. Students who had previously enrolled in at least one physics course tested the visuo-haptic simulator. Results highlighted that the participants demonstrated an improved understanding of friction concepts. The authors suggested that improved comprehension resulted from a seamless integration of visual and kinesthetic techniques.

Qi et al. built a visuo-haptic physics simulator to teach the effect of buoyancy (Qi et al., 2020). Buoyancy refers to the upward force exerted on an object submerged in a fluid. The study focused on how visual and haptic feedback could enhance participants’ comprehension of this phenomenon. A 2 x 2 between-subjects design study was conducted. The experiment had four conditions based on the presence or absence of haptic and visual feedback, and participants were randomly allocated to one of these conditions. After an initial evaluation to measure participants’ knowledge of buoyancy, they proceeded to use the simulator and subsequently answered a post-interaction questionnaire. Results suggested that participants in the combined haptic and visual condition exhibited notable progress in their learning outcomes.

In the work of Hamza-Lup and Goldbach, the authors developed a visuo-haptic training tool designed as a gamified scenario (Hamza-Lup and Goldbach, 2021). It was designed to facilitate students’ comprehension of abstract physics concepts, particularly those related to electromagnetism and the fundamental Lorentz force. When a charged particle travels through electric and magnetic fields, this force describes the interaction it encounters. In this study, undergraduate students with non-physics backgrounds volunteered, and they were placed into three groups: the control group, the visuo-haptic group, and the visual group. The control group used only traditional teaching methods. In contrast, the visuo-haptic group used the simulator with haptic feedback features, while the visual group engaged with the simulator without this kinesthetic component. The study’s results showed that the students in the visuo-haptic group not only did better on tests but also showed how advanced visual-haptic interfaces might be able to keep students interested and help them learn more effectively.

A comprehensive review by Crandall and Karadogan emphasizes the pedagogical effectiveness of haptic systems in learning, exploring how these technologies can be designed to align with cognitive theories like Cognitive Load and Embodied Cognition (Crandall and Karadoğan, 2021). This study explores the subtle influence of haptic design on learning effectiveness and suggests optimal strategies for creating haptic simulations. Moreover, a study by DeCoito underscores the transformative potential of digital media in conveying the dynamic and historical development of science, presenting it as a product of its sociocultural context DeCoito (2020). DeCoito showed that using explicit-reflective teaching along with digital tools like scientific timelines and video games helped students understand nature of science concepts better, such as how it is tentative, durable, and self-correcting, and how creativity and sociocultural factors affect scientific inquiry.

The authors’ earlier research mostly looked at the technical progress and immediate educational effects of visuo-haptic simulators (Neri et al., 2015; Shaikh et al., 2017; Neri et al., 2018; Neri et al., 2020). These works have laid a foundational understanding of how kinesthetic and visual feedback can synergize to improve learners’ grasp of complex physics concepts. Our research builds upon these studies, not only exploring the technical and pedagogical efficacy of visuo-haptic simulators but also extending the investigation into user satisfaction and its role in the effectiveness of learning tools. This approach aligns with the emerging recognition within the field of educational technology that learner engagement and satisfaction are critical components of successful educational experiences.

Furthermore, this study employs an enhanced survey methodology to measure user satisfaction with multi-modal applications. This survey could provide researchers with a better understanding of the qualitative aspects of learning with visuo-haptic simulators. This focus on user experience and satisfaction proposes that the effectiveness of educational simulators extends beyond cognitive outcomes to include engagement and emotional responses.

3 Materials and methods

While traditional physical laboratories play a pivotal role in education, they often require significant infrastructure and resources. In contrast, visuo-haptic environments can offer a more cost-effective and accessible alternative for simulating physical experiments, thereby making advanced learning experiences more widely available. In this context, the study focused on developing three distinct visuo-haptic simulators to improve the teaching of classical physics concepts through interactive engagement. The selected scenarios were selected for their potential to benefit significantly from haptic feedback: (a) a block resting on a flat surface with friction; (b) equilibrium of forces (balance); and (c) equilibrium of forces (tug-of-war). These scenarios were created using the HaDIU (Haptic Device Integration for Unity) framework (Escobar-Castillejos et al., 2020). This plugin is accessible via the following link. The force feedback in these simulators was provided through a Novint Falcon haptic device, chosen for its affordability and its capability to deliver three degrees of freedom (DOF) for both spatial input and haptic output. The integration of the Novint Falcon with the Unity3D engine, via the HaDIU plugin, enabled a seamless experience of graphical and haptic renderings.

Each simulation incorporated Unity’s graphical user interface (GUI), which allowed users to control and visualize essential parameters for each scenario, facilitating a hands-on learning experience. The GUI was designed to be intuitive, ensuring that users could easily interact with the simulation and receive immediate haptic feedback relevant to the physics concepts being explored. This approach aimed to not only make the learning process more engaging but also to provide insights into the usability and educational value of visuo-haptic simulations in teaching complex physics concepts. In the following subsections, a detailed description of each scenario is provided.

3.1 Design of the visuo-haptic simulators

3.1.1 Block resting on a flat surface with friction

This scenario aimed to provide students with the opportunity to observe and understand the concept of frictional force f by simulating a block sliding on a flat surface when a force is applied to it (Figure 1)

Figure 1

Figure 1. Force diagram for a block resting on a flat surface with friction.

Let $\overrightarrow{F_a}$ be the force applied by the user in a direction orthogonal to the box’s surface, and F_a its magnitude. Similarly, $\vec{F}$ represents the force detected by the simulator’s sensors, and F is its magnitude. μ_s corresponds to the static friction coefficient, and μ_k to the kinetic friction coefficient, where it always holds that μ_s > μ_k. The magnitude of the friction force is considered as f, with f_s representing the static friction force magnitude and f_k the kinetic friction force magnitude. Additionally, m corresponds to the simulated mass value, and P is the magnitude of the box’s weight. Starting from the initial state of rest, the resultant force $\overrightarrow{F_R}$ can be determined as follows:

If F_a < f_s = μ_sP, then $\vec{F}=-\overrightarrow{F_a}\Rightarrow \overrightarrow{F_R}=0$.

If F_a ≥ f_s then $F=f=f_k={\mu }_{k}P\Rightarrow \overrightarrow{F_R}=\left(F_a-f_k\right)\widehat{ı}$.

Where $\widehat{ı}$ represents the unit vector on the horizontal axis.

Consequently, users could apply a force (F_a) to the block by using the haptic interaction point (HIP), which interacts with and penetrates the virtual object to simulate the applied force. On the other hand, the ideal haptic interaction point (IHIP) acts as a visual representation, depicting the handle of the haptic device and illustrating the contact point of the HIP within the virtual environment, as demonstrated in Figure 2. Users could grasp the block, allowing them to perceive the virtual weight of the block. The haptic device transmitted the F_R, whether it was in a static or kinetic state, to the user using the F_a value.

Figure 2

Figure 2. “Block resting on a flat surface with friction” Scenario: (Top) Initial scenario setup; (Bottom) Simulation Parameters.

3.1.2 Equilibrium of forces (balance)

The aim of this scenario was for students to interact with the system to understand the concept of torque in terms of the magnitude of the applied forces and their points of application. To determine the resultant force in the simulation, the following considerations, as seen in Figure 3, were taken into account:

Figure 3

Figure 3. Force diagram for the balance simulator.

The center of rotation is situated at the origin of a Cartesian plane. The variable L denotes the distance from the center of rotation to each end of the table. The vector $\overrightarrow{F_a}$ represents the force that the student applies vertically downward on the left side of the table, and F_a is the magnitude of this force. The term s indicates the distance from the center of rotation to the point on the left side where the force $\overrightarrow{F_a}$ is applied. Conversely, on the right side, the centers of mass for each of the boxes i (where i ∈ 1, 2, 3, 4, 5) are positioned at distances t_i. It is assumed that all boxes have an identical weight, denoted as P. The variable h represents the height of the center of rotation above the base, while θ is the angle that the table makes with the horizontal. The force necessary to maintain the system in equilibrium is described by the vector $\vec{F}$, with F being its magnitude. Consequently, F_R is defined as the resultant force acting on the system.

To obtain the resultant force, it is necessary to know in advance the force $\vec{F}$ that the simulator will produce. This force will be of the same magnitude as what the student would need to apply to maintain the simulation in equilibrium. Starting from the condition of rotational equilibrium, we have:

$\sum \tau =0$

$\Rightarrow Fscos\left(\theta \right)={\displaystyle \sum _{i=1}^n}P{t}_{i}cos\left(\theta \right)$

Where n is the number of boxes in the simulation. Since P and cos(θ) do not depend on i, they can be factored out of the sum. In this way, terms dependent on the angle are canceled on both sides of the equation.

$Fs=P{\displaystyle \sum _{i=1}^n}{t}_i\Rightarrow F=\frac{P}{s}{\displaystyle \sum _{i=1}^n}{t}_i$

Therefore, for a given number of boxes n, the force on the sensor will depend solely on the position s and the lengths t_i.

$F\left(s,t_1,\dots ,t_n\right)=\frac{P}{s}{\displaystyle \sum _{i=1}^n}{t}_i$

Thus, the resultant force on the systems will be:

$\overrightarrow{F_R}=\overrightarrow{F_a}+\vec{F}\left(s,t_1,\dots ,t_n\right)$

Since $\overrightarrow{F_a}$ is applied downward on the left side of the pivot point:

$\overrightarrow{F_R}=\left(\frac{P}{s}{\displaystyle \sum _{i=1}^n}{t}_i-F_a\right)\widehat{ȷ}$

Where $\widehat{ȷ}$ represents the unit vector on the vertical axis.

In this simulation scenario, participants can exert a force, represented as $\overrightarrow{F_a}$, through the HIP on the left side of the simulated table (Figure 4). As the user moves the blue ball, which represents the IHIP’s position on this plane, the HaDIU plugin computes the resultant force $\overrightarrow{F_R}$, by using the HIP’s position combined with Unity’s physics engine values to provide appropriate haptic feedback to the user. Additionally, users can move the boxes on the right side of the table freely by approaching them with the IHIP. Upon contact with a box, pressing the button on the haptic device allows the user to ‘grab’ and reposition the box. Modifying the position of the boxes changes the distribution of forces and the resultant torque in the calculation, enabling students to physically sense and understand the changes in equilibrium.

Figure 4

Figure 4. “Equilibrium of forces (balance)” Scenario: (Top) Initial scenario setup; (Bottom-Left) Simulation with three stacked boxes vertically; (Bottom-Right) Simulation with three-box pyramid formation.

3.1.3 Equilibrium of forces (tug-of-war)

This scenario was designed to help students understand the concept of equilibrium concerning the magnitude of applied forces through a simulation that represents accelerated motion on the rope, which depends on the resultant force applied to it (Figure 5).

Figure 5

Figure 5. Force diagram of equilibrium of forces (tug-of-war).

Let F_s be the force initially exerted on the rope by the simulator, and F_a be the force applied by the student. The resultant force F_R will be:

$\overrightarrow{F_R}=\overrightarrow{F_s}+\overrightarrow{F_a}$

In this environment, students were able to test another equilibrium of forces simulator (Figure 6). Through the HIP, participants could exert an external force, represented as F_a, on the virtual rope. The blue and red spheres represent the same IHIP; the blue sphere indicates the HIP’s position when it is not in contact with the rope, whereas the red sphere signifies that the HIP is engaging with the rope. The acceleration and movement of the rope are determined by the interaction between the student’s force and the simulator’s preset force F_s. It should be noted that although users can move freely within the simulation space, the HIP’s movement is constrained to the x-axis once contact is made with the rope. This design constraint was considered to simulate the feeling of the opposing force, thereby enriching the user’s haptic experience and providing a more realistic sense of equilibrium.

Figure 6

Figure 6. “Equilibrium of forces (tug-of-war)” Scenario: (Top) Initial scenario setup; (Bottom) Simulation with the user applying an opposite force.

3.2 Measures

To make it suitable for multi-modal simulations, Doll and Torkzadeh’s end-user computer satisfaction survey (Doll and Torkzadeh, 1988) was modified (Figure 7). The modified survey included questions that spanned various aspects of the simulators, addressing clarity, output value accuracy, visual and kinesthetic realism, reliability, user interface efficacy, understandability, motivation, and real-time information delivery. These questions were structured around a five-point Likert scale, with five representing the highest level of satisfaction or agreement and 1 being the lowest. However, items S1 and S2 were conceived as open-ended questions, allowing participants to provide suggestions and general comments. The survey was delivered via Google Forms, and the data collected was analyzed using Python with the aid of the Pandas and Matplotlib libraries.

Figure 7

Figure 7. Proposed end-user computer satisfaction survey. C1: Do the simulators provide the precise information you need to understand the activities? C2: Do the simulators’ configuration parameters meet your needs? C3: Do the simulators provide output values that seem to be helpful in your activity? C4: Do the simulators provide sufficient instructions to perform activities? C5: Do you consider the simulators could support you in classes? A1: Are the output values accurate? A2: Do you feel the kinesthetic perception is realistic? A3: Do you feel the visual aspects are realistic? A4: Do you find the simulators dependable? F1: Do you think the graphical user interface (GUI) presents information in a useful format? F2: Is the information clear? F3: Do you find the visualization of the visuo-haptic simulators attractive? E1: Are the simulators user-friendly? E2: Are the simulators easy to use? E3: Did you feel motivated when you were using the simulators? E4: Did you feel the information offered by the simulators was useful? T1: Do you get the simulators’ information you need in time? T2: Do the simulators provide real-time rendering? S1: What suggestions do you have to improve the simulators to better support you in your learning process? S2: General comments.

3.3 Participants

During the academic semester of August to December 2023, an experimental group of 45 engineering students from Universidad Panamericana, ranging from the 1st to the 7th semester and with differing levels of familiarity with the topics, participated in testing sessions for the three simulators developed. These sessions, spanning 1 week, aimed to assess the usability of the visuo-haptic simulators and gather students’ perceptions of their utility in enhancing the learning experience of classical physics concepts.

Each session started with a detailed introduction to the simulators, presenting their features and the tasks participants were expected to complete. Positioned in front of computers equipped with the simulators and the Novint Falcon haptic device, students were able to adjust the device for their dominant hand, ensuring comfort and ease of use. Through interaction with the simulators using the Novint Falcon, participants gained an intuitive understanding of the forces associated with the presented classical physics concepts. Through the graphical user interface (GUI), participants use the mouse to modify simulation parameters, including the number of boxes, static and dynamic friction coefficients, and opposing forces. The GUI was intentionally designed to reduce the need for users to alternate between the mouse and the Falcon haptic device, ensuring a seamless and intuitive interaction. Most parameter adjustments are completed before engaging with the Falcon, allowing for uninterrupted kinesthetic feedback during the simulation. After adjustments, participants were asked to access the ‘Results Tab’ to monitor the simulation’s outcomes. This strategy guarantees that participants can both visually and kinestheticly perceive the immediate effects of their modifications. By minimizing device switching, this approach not only preserves the ergonomic integrity of the setup but also sustains an immersive and efficient simulation environment.

The average exploration time was approximately 30 min per student, focusing on engaging with the simulations rather than a quantitative measure of learning outcomes. Following each session, feedback was collected using the adapted end-user computing satisfaction questionnaire. This feedback was used to measure the simulators’ effectiveness in providing an intuitive and engaging learning experience. This approach allowed the study to examine the educational potential of visuo-haptic technology from the perspective of user satisfaction and perceived learning enhancement.

4 Results

The satisfaction survey results, as depicted in Figure 8, provide insights into participants’ perceptions of the visuo-haptic technology’s effectiveness in enhancing the learning experience.

Figure 8

Figure 8. Obtained results in the proposed end-user computer satisfaction survey. The colors in the pie. charts correspond to values on a 5-point Likert scale: Represents the lowest value. Represents the 2nd lowest value. Indicates a midpoint or neutral response. Represents the 4th value, leaning towards the positive end of the scale. Represents the highest or most positive value.

4.1 Content (C1-C5) ratings distribution

Analysis of the content quality ratings reveals that most responses range between 4 and 5, suggesting that participants generally perceived the visuo-haptic technology’s effectiveness in providing an engaging learning experience. However, the variance in ratings for C2, concerning specific content-related aspects of the simulators, suggests room for improvement in content delivery. Specifically, feedback on C2 highlighted a need for clearer explanations of complex physics concepts and suggested incorporating more varied learning materials to cater to different learning styles. For example, participants expressed a desire for more interactive elements, such as quizzes or real-time feedback, to test their understanding throughout the simulation. This feedback is instrumental in guiding enhancements to the simulator design, ensuring content is both accurate and presented in a manner that is accessible to learners with diverse preferences.

4.2 Accuracy (A1-A4) ratings distribution

Ratings on the accuracy of the simulations validate the simulators’ potential to provide a realistic representation of physics principles, with A1 and A2 indicating strong user approval. However, the varied responses for A3 and A4 suggest opportunities for further research into how these elements influence the overall educational value of the simulations. Such insights could guide enhancements in the accuracy and realism of simulated physics phenomena.

4.3 Format (F1-F3) ratings distribution

The distribution of ratings for the presentation format reveals areas for improvement, particularly for F1, which received lower scores. This feedback points to a possible misalignment between the simulators’ educational content and its presentation. Enhancing the format could significantly impact user engagement and learning experiences. Conversely, the positive responses to F2 and F3 suggest that some aspects of the format are effective and should be retained or further developed in future developments.

4.4 Ease of use (E1-E4) ratings distribution

The ease of use ratings were consistently positive, which highlights the simulators’ user-friendly design and underscores their potential as valuable educational tools in physics learning. These results suggest that visuo-haptic simulators, with further development, could become integral for teaching complex concepts that benefit from kinesthetic learning experiences.

4.5 Timeliness (T1-T2) ratings distribution

The timeliness ratings, especially the inclination towards a score of five for T2, demonstrate participants’ satisfaction with the simulators’ responsiveness and promptness. This feedback highlights the potential of visuo-haptic environments to deliver immediate, interactive feedback, a key factor in facilitating an effective and engaging learning environment.

4.6 Box plot for distribution of responses to each question

Box plots provide a visual representation of the central tendency, variability, and presence of outliers in the distribution of responses for each question (Figure 9). For example, the interquartile range (IQR), which is defined as the difference between the first quartile (the lower boundary of the box) and the third quartile (the upper boundary of the box), provides insight into the central 50% of the data distribution for each question.

Figure 9

Figure 9. Box plot for distribution of responses to each question.

Based on the data, the box plot analysis offered a clear depiction of the user feedback, capturing both general trends and the range of responses. While the simulators were generally well-received, indicated by a predominance of ratings in the 4 to five range, some variations and outliers suggest some aspects did not fully meet all participants’ expectations or preferences. These results highlight that while features such as the simulators’ accuracy and ease of use were highly rated, the presentation and format, particularly for F1, require further attention to meet user satisfaction across all dimensions of the simulator experience.

5 Discussion

The satisfaction survey results indicated a generally positive perception of the visuo-haptic technology, with content and accuracy ratings being particularly strong. However, areas for improvement were identified in the content delivery specifics and presentation format.

5.1 Comments obtained in the suggestions category

The qualitative feedback received from participants shows the positive impact of the visuo-haptic simulators on the learning experience. Among the 33 participants who provided comments, a significant majority (63.6%) praised the simulators for providing a more immersive understanding of physics concepts than traditional methods. Common feedback included appreciations such as ‘they facilitated a deeper comprehension of classical concepts’ and ‘the simulations were engaging and instructive,’ underscoring the value of incorporating interactive, multi-modal tools in educational settings.

These results not only highlight the potential of visuo-haptic technologies to enhance conceptual understanding beyond traditional methods but also point to specific areas for enhancement. Participants also recommended expanding the simulations’ content to ‘cover a wider range of physics topics’ and to ‘improve the visual interface for more intuitive 3D navigation.’ Furthermore, the suggestion to ‘integrate learning features’, such as quizzes, underlines a strong interest in personalizing the learning experience to support diverse learning styles and speeds.

5.2 Comparison with research questions

This study contributes to understanding the potential of visuo-haptic technology in enhancing the educational experience, particularly in how it influences students’ engagement with and perceptions of physics concepts. However, it also highlights the need for a more in-depth analysis of how interactions with visuo-haptic simulators affect long-term knowledge retention and fit diverse learning preferences.

According to the results, multi-modal technology significantly influences students’ engagement and their understanding of basic physics concepts, thereby answering our initial research question. Feedback and rating distribution indicate that multi-modal technologies improve student engagement and enrich their comprehension of physics topics. Positive feedback indicates that these tools make abstract topics more understandable and clear, improving learners’ involvement and understanding.

Responses about the user-friendliness and educational value of visuo-haptic simulators provide clear answers to our second question. The consistently high scores for simplicity of use emphasize the user-friendly design of the simulators. Simultaneously, content evaluations and feedback show the substantial instructional value of the simulations. Recommendations to expand the content and incorporate diverse learning materials indicate ways to enhance their educational impact.

Regarding our third question, the diverse feedback, provided by the different metrics along with the qualitative comments, indicates specific areas for improvement, such as content delivery and presentation format. This feedback highlights the importance of interactive, multi-modal educational tools and provides direction for future improvements. Recommendations for more interactive features and expanding the variety of topics are crucial to guide the ongoing development of multi-modal educational technology.

The conducted analysis underscores the simulators’ positive reception, aligning with our research questions regarding their usability and intuitive design. Nonetheless, the presence of ratings in the 3-4 range indicates areas requiring attention. While the simulators’ accuracy and ease of use were praised, improvements in the format, especially concerning F1, are necessary to elevate the overall learning experience. This aspect suggests the importance of developing educational tools that are not only pedagogically effective but also broadly engaging and accessible.

5.3 Comparison with related research

This study extends existing research in the field by investigating not only the direct educational effects of visuo-haptic simulators but also user satisfaction and engagement. Walsh and Magana’s study of physical manipulative tools and visuo-haptic simulations for teaching statics supports this approach. They focused on how different kinds of visual and haptic feedback can help students learn about friction Walsh and Magana (2023).

This research investigates the precise effects of multi-modal simulation on student engagement and comprehension. It is based on the growing consensus in the educational technology area that active learner engagement with kinesthetic interaction is essential for improving conceptual understanding and improving outcomes in STEM education. Additionally, comparing the obtained results with related research indicates a consistent finding that interactive technologies can significantly enhance learning experiences. However, the emphasis on user feedback for specific improvements provides unique insights into optimizing visuo-haptic technologies for educational purposes. By integrating these insights with quantitative data, we gained a comprehensive understanding of user experiences and preferences. This feedback provides a solid foundation for future development in multi-modal educational technology. With a focus on user-centric design and educational effectiveness, future simulators could be refined to better meet the diverse needs of learners. This approach promises not only to enhance engagement but also to deepen the understanding of complex physics concepts.

5.4 Limitations and recommendations for further studies or analyses

Lastly, the authors acknowledge that a limitation of this study is its sole reliance on subjective feedback, which offers a limited view of the simulators’ educational impact. Incorporation of objective measures, such as engagement metrics and performance assessments, would offer a more comprehensive assessment of the simulators’ learning impact and efficacy. Future developments of new visuo-haptic environments will aim to integrate objective metrics of learning outcomes to ensure a thorough evaluation of the simulator’s qualitative and quantitative advantages. Additionally, further development should consider feedback on content delivery and presentation format to enhance the educational impact of visuo-haptic technologies.

6 Conclusion and future work

In this study, we developed three visuo-haptic simulators to investigate the potential benefits of multi-modal learning experiences within STEM education. We focused on three classical physics concepts: friction on a flat surface, balance in force equilibrium, and tug-of-war dynamics. A graphical user interface was integrated to facilitate user interaction, providing instructions, allowing for the adjustment of simulation parameters, and displaying force values as a direct feedback mechanism.

Moreover, an enhanced survey was utilized to measure end-user computing satisfaction. It was adapted to capture feedback on the multi-modal simulation experience. This study presented an alternative methodology that could potentially enable more structured evaluations in this area.

A detailed study involving students was conducted to determine the level of understanding and the insights obtained through interactions with multi-modal simulators. Results suggest that using haptic devices along with carefully planned visuo-haptic ecosystems could help participants understand the complex nature of forces in classical physics and where they come from. The feedback obtained regarding the simulations was predominantly good. Feedback collected through the enhanced survey highlighted the simulators’ positive impact on participants’ understanding and engagement with physics concepts.

Despite the overall positive reception, an analysis of open-ended responses revealed areas for further improvement. Key themes from this analysis include:

1. Participants expressed a desire for simulations covering a broader spectrum of physics phenomena, suggesting the need for expanding our simulator library to encompass more varied and complex concepts.

2. Enhancements to the visual representation, during the movement of the haptic device avatar for more intuitive navigation and interaction were frequently requested, indicating a priority for future iterations to focus on streamlining the user experience.

3. The call for integrating quizzes and additional educational resources highlights a demand for personalized learning paths within the simulators, catering to diverse educational backgrounds and learning speeds.

Future research will focus on these insights, starting with a comparative analysis aimed at quantifying the educational impact of visuo-haptic simulators against traditional teaching methods. Scheduled for January-May 2024, this study will adopt a rigorous methodology designed in collaboration with faculty members, ensuring a comprehensive evaluation of learning outcomes. Additionally, recognizing the vast potential of classical physics for educational exploration, we plan to convene focus group discussions to strategically select topics for subsequent simulator development. These discussions will prioritize areas where visuo-haptic integration offers maximal educational value, thus refining our approach to enhancing engagement and comprehension of complex physics concepts. Through this targeted future work, we aim to address the identified areas for improvement, further advancing the field of multi-modal educational technology and enriching the learning experience for students.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

GG-M: Writing–original draft. OL-F: Writing–original draft. DC: Writing–original draft. JN: Writing–review and editing. DE-C: Writing–original draft, Writing–review and editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Acknowledgments

We thank the Engineering Faculty, the Research Group of Multi-modal Mixed Reality Spaces for Education, and the Computing Department of Universidad Panamericana, Mexico City Campus. We would also like to thank the Vicerrectoría de Investigación y Posgrado, the Research Group of Advanced Artificial Intelligence, and the CyberLearning and Data Science Laboratory of Tecnologico de Monterrey. Finally, we would also like to thank Irvine Jordan Monroy Rueda de León and Angel Alberto Arista Villanueva for their help during the initial development and testing of the visuo-haptic simulations.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Crandall, R., and Karadoğan, E. (2021). Designing pedagogically effective haptic systems for learning: a review. Appl. Sci. 11, 6245. doi:10.3390/app11146245

CrossRef Full Text | Google Scholar

DeCoito, I. (2020). “The use of digital technologies to enhance learners’ conceptions of nature of science,” in Nature of science in science instruction (Cham: Springer International Publishing), 343–357. chap. 3-19. doi:10.1007/978-3-030-57239-6_19

CrossRef Full Text | Google Scholar

Doll, W. J., and Torkzadeh, G. (1988). The measurement of end-user computing satisfaction. MIS Q. 12, 259–274. doi:10.2307/248851

CrossRef Full Text | Google Scholar

Escobar-Castillejos, D., Noguez, J., Cárdenas-Ovando, R. A., Neri, L., Gonzalez-Nucamendi, A., and Robledo-Rella, V. (2020). Using game engines for visuo-haptic learning simulations. Appl. Sci. 10, 4553. doi:10.3390/app10134553

CrossRef Full Text | Google Scholar

Hamza-Lup, F. G., and Goldbach, I. R. (2021). Multimodal, visuo-haptic games for abstract theory instruction: grabbing charged particles. J. Multimodal User Interfaces 15, 1–10. doi:10.1007/s12193-020-00327-x

CrossRef Full Text | Google Scholar

Hamza-Lup, F. G., and Page, B. (2012). “Haptics-augmented physics simulation: coriolis effect,” in 7th International Conference on Virtual Learning (ICVL 2012), 34–38. doi:10.48550/arXiv.1903.11567

CrossRef Full Text | Google Scholar

Lederman, S. J., and Klatzky, R. L. (2009). Haptic perception: a tutorial. Atten. Percept. Psychophys. 71, 1439–1459. doi:10.3758/APP.71.7.1439

PubMed Abstract | CrossRef Full Text | Google Scholar

Maggio, M. G., Piazzitta, D., Andaloro, A., Latella, D., Sciarrone, F., Casella, C., et al. (2022). Embodied cognition in neurodegenerative disorders: what do we know so far? a narrative review focusing on the mirror neuron system and clinical applications. J. Clin. Neurosci. 98, 66–72. doi:10.1016/j.jocn.2022.01.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Mahon, B. Z., and Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. J. Physiology-Paris 102, 59–70. doi:10.1016/j.jphysparis.2008.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Mclinden, M., Mccall, S., and Hodges, L. (2019). Learning through touch: supporting learners with multiple disabilities and vision impairment through a bioecological systems perspective. 2 edn. London, United Kingdom: Routledge. doi:10.4324/9780429260735

CrossRef Full Text | Google Scholar

Neri, L., Noguez, J., Robledo-Rella, V., Escobar-Castillejos, D., and Gonzalez-Nucamendi, A. (2018). Teaching classical mechanics concepts using visuo-haptic simulators. J. Educ. Technol. Soc. 21, 85–97.

Google Scholar

Neri, L., Robledo-Rella, V., García-Castelán, R. M., Gonzalez-Nucamendi, A., Escobar-Castillejos, D., and Noguez, J. (2020). Visuo-haptic simulations to understand the dependence of electric forces on distance. Appl. Sci. 10, 7190. doi:10.3390/app10207190

CrossRef Full Text | Google Scholar

Neri, L., Shaikh, U. A., Escobar-Castillejos, D., Magana, A. J., Noguez, J., and Benes, B. (2015). “Improving the learning of physics concepts by using haptic devices,” in 2015 IEEE Frontiers in Education Conference (FIE), El Paso, TX, USA, 21-24 October 2015, 1–7. doi:10.1109/FIE.2015.7344069

CrossRef Full Text | Google Scholar

Park, J., Kim, K., Tan, H. Z., Reifenberger, R., Bertoline, G., Hoberman, T., et al. (2010). “An initial study of visuohaptic simulation of point-charge interactions,” in 2010 IEEE Haptics Symposium, Waltham, MA, 25-26 March 2010, 425–430. doi:10.1109/HAPTIC.2010.5444623

CrossRef Full Text | Google Scholar

Qi, K., Borland, D., Williams, N. L., Jackson, E., Minogue, J., and Peck, T. C. (2020). “Augmenting physics education with haptic and visual feedback,” in 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA, 22-26 March 2020, 439–443. doi:10.1109/VRW50115.2020.00093

CrossRef Full Text | Google Scholar

Shaikh, U. A. S., Magana, A. J., Neri, L., Escobar-Castillejos, D., Noguez, J., and Benes, B. (2017). Undergraduate students’ conceptual interpretation and perceptions of haptic-enabled learning experiences. Int. J. Educ. Technol. High. Educ. 14, 15. doi:10.1186/s41239-017-0053-2

CrossRef Full Text | Google Scholar

Taljaard, J. (2016). A review of multi-sensory technologies in a science, technology, engineering, arts and mathematics (steam) classroom. J. Learn. Des. 9, 46–55. doi:10.5204/jld.v9i2.274

CrossRef Full Text | Google Scholar

UNESCO (2023). Digital learning and the transformation of education. Available at https://www.unesco.org/en/digital-education (Accessed on August 22, 2023).

Google Scholar

Walsh, Y., and Magana, A. J. (2023). Learning statics through physical manipulative tools and visuohaptic simulations: the effect of visual and haptic feedback. Electronics 12, 1659. doi:10.3390/electronics12071659

CrossRef Full Text | Google Scholar

Walsh, Y., Magana, A. J., and Feng, S. (2020). Investigating students’ explanations about friction concepts after interacting with a visuohaptic simulation with two different sequenced approaches. J. Sci. Educ. Technol. 29, 443–458. doi:10.1007/s10956-020-09829-5

CrossRef Full Text | Google Scholar

World Economic Forum (2020). Schools of the future: defining new models of education for the fourth industrial revolution. Available at https://www3.weforum.org/docs/WEF_Schools_of_the_Future_Report_2019.pdf (Accessed December 22, 2023).

Google Scholar

Yuksel, T., Walsh, Y., Krs, V., Benes, B., Ngambeki, I. B., Berger, E. J., et al. (2017). “Exploration of affordances of visuo-haptic simulations to learn the concept of friction,” in 2017 IEEE Frontiers in Education Conference (FIE), Indianapolis, IN, USA, 18-21 October 2017, 1–9. doi:10.1109/FIE.2017.8190471

CrossRef Full Text | Google Scholar

Yuksel, T., Walsh, Y., Magana, A. J., Nova, N., Krs, V., Ngambeki, I., et al. (2019). Visuohaptic experiments: exploring the effects of visual and haptic feedback on students’ learning of friction concepts. Comput. Appl. Eng. Educ. 27, 1376–1401. doi:10.1002/cae.22157

CrossRef Full Text | Google Scholar

Zhuoluo, M., Liu, Y., and Zhao, L. (2019). Effect of haptic feedback on a virtual lab about friction. Virtual Real. Intelligent Hardw. 1, 428–434. doi:10.1016/j.vrih.2019.07.001

CrossRef Full Text | Google Scholar