- 1Artificial Intelligence Laboratory, University of Tsukuba, Tsukuba, Japan
- 2Data Science Laboratories, NEC Corporation, Tokyo, Japan
In recent years, virtual, augmented, and mixed reality (MR) technologies have gained increasing attention in sports training for their potential to improve motor skills and team coordination. However, existing systems predominantly emphasize individual skills or small-scale settings, offering limited support for realistic multi-player, full-court tactical training. To address this gap, this paper proposes a large-scale mixed reality environment for training coordinated tactical plays in basketball. In this environment, players can practice various tactics such as ball passing and screens with virtual players and visual instructions overlaid in the real environment using a see-through head-mounted display. Two evaluation experiments were conducted. In Experiment 1, expert players performed pick-and-roll plays in both MR and real environments. The results showed no significant differences in execution time or movement trajectories, suggesting that the MR environment may offer spatial and temporal consistency comparable to real play. In Experiment 2, novice players trained with the MR system and with a conventional method in real space. The results showed higher improvements in both spatial positioning and timing in the MR environment, suggesting it could support the training of coordinated tactical plays. These findings suggest the potential of MR technology for skill training involving multiplayer coordination in realistic tactical scenarios.
1 Introduction
Recently, there has been growing interest in the application of virtual reality (VR), augmented reality (AR), and mixed reality (MR) in sports. Several studies have shown that these technologies are effective in supporting motor skill acquisition and in improving learner motivation (Kickmeier-Rust et al., 2019; Oki et al., 2022; Gutierrez et al., 2022). Despite these benefits, existing research has primarily focused on relatively constrained motor tasks or individual skill training (Tsai et al., 2020; Lin et al., 2021; Ueyama and Harada, 2024). In contrast, tactical learning in team sports like basketball requires coordinated actions among multiple players across a full-size court, yet such large-scale and multi-agent learning contexts remain underexplored. Previous work has also shown that the physical size of the learning environment influences motor learning outcomes (Rodrigues et al., 2022), suggesting that athletes should train in environments that approximate the dimensions of actual competition settings.
In such large-scale and dynamically situations, conventional approaches used in prior research and in on-court training practices fail to capture several essential elements. Coordinated tactical plays, in particular, emerge from interactions among multiple players rather than from isolated actions, requiring athletes to understand the spatiotemporal relationships with teammates and opponents (Araújo et al., 2010). As a result, their acquisition involves challenges that are fundamentally distinct from those of individual skill training. The first challenge is the difficulty of individual practice. Tactical coordination requires athletes to position themselves relative to surrounding players (Kannekens et al., 2011), which necessitates the presence of multiple training partners. The second challenge concerns the difficulty of reproducing training scenarios consistently across sessions. Replicating formations and execution sequences is inherently unstable, and variations in players’ physiques (Sallet et al., 2005; Kucsa and Mačura, 2015) further complicate realistic simulation of opponents for advanced practice. The third challenge relates to limitations in learning from third-person instruction. Verbal explanations and observational learning are well established as effective for motor learning (Pollock and Lee, 1992; Paraskevaidis and Fokides, 2020; Wulf et al., 2002), yet coordinated tactical plays demand a first-person understanding of spatiotemporal dynamics that is not easily conveyed from external viewpoints.
To address these challenges, we aim to develop a large-scale MR training environment that enables athletes to repeatedly practice coordinated tactical plays in both temporally and spatially reproducible conditions, regardless of the number of players. In particular, this study focuses on the pick and roll1, which is one of the most widely used coordinated tactical plays in basketball and requires multi player coordination as well as advanced spatiotemporal decision making (Marmarinos et al., 2016; Lama et al., 2011). In our proposed system as shown in Figure 1, visual elements essential for tactical coordination such as virtual players, passing lanes, and recommended positioning are spatially aligned with the real world through a see-through head-mounted display. This design allows athletes to configure the physiques and movements of virtual players and to rehearse representative coordinated plays, including the pick and roll, from a first-person perspective. We hypothesize that visual instructions from a first-person perspective, which provide spatiotemporal information relevant to coordinated tactical plays, may facilitate the learning of coordination skills compared to conventional methods. This hypothesis will be explored in the present study.
Figure 1. Overview of Developed MR environment. (a) Demonstration of the coordinated tactical plays with virtual players. (b) Coaching from a first-person perspective. (c) Reviewing one’s own performance from various angles.
The contributions of this work are as follows:
1. The design and implementation of a large-scale MR training environment equipped with a pass evaluation system based on ball tracking.
2. Examination of the spatial and dynamic consistency between the MR environment and a conventional real-world setting through experiments with skilled players.
3. An exploratory comparison of first-person spatiotemporal visual guidance with conventional third-person observational learning in the acquisition of coordination skills.
2 Related works
This section reviews related research to explore the potential application of VR/AR/MR technology to Coordinated Tactical Plays training.
2.1 Application of immersive technologies in sports training
In recent years, the development of immersive technologies such as VR, AR, and MR has led to their rapid adoption in various fields including healthcare, education, industry, and sports (Lee et al., 2024; Wang and Li, 2024). In the field of sports in particular, these technologies are increasingly regarded as effective tools to complement and enhance traditional field-based training and video analysis (Lin et al., 2021; Lin et al., 2023; Graf et al., 2019).
VR technology enables athletes to experience fully simulated game situations and to practice cognitive or perceptual skills in controlled environments (Pastel et al., 2023; Marshall et al., 2023). For example, Tsai et al. (2020) developed a VR system that presents tactical situations from a first-person viewpoint, supporting cognitive learning with reduced physical effort. However, VR training generally lacks physical interaction, free locomotion, and realistic ball handling, which limits the reproduction of sport-specific motor behaviors (Tsai et al., 2020). Furthermore, cybersickness and decreased learning efficiency—particularly among beginners—have been frequently reported (Biswas et al., 2024; Ben Mahfoudh and Zoudji, 2024). These limitations restrict the applicability of VR when training tasks require natural movement, continuous spatial interaction, or physical coordination with teammates.
AR technology has also been applied to enhance motor learning in real environments. AR systems can overlay optimal shot trajectories or volumetric annotations onto the athlete’s field of view, thereby improving performance and enabling in-situ coaching (Ueyama and Harada, 2024; Wen et al., 2024). Despite these benefits, AR-based instruction may suffer from increased visual or cognitive load when large amounts of information are presented simultaneously (Makransky et al., 2019; Radu and Schneider, 2019). Moreover, AR studies have mostly targeted isolated motor skills—such as shooting or gesture execution—and have rarely addressed cooperative or tactical behaviors involving multiple players.
MR technology integrates virtual elements with the real environment in real time, offering greater potential for interactive and spatially grounded sports training (Kim et al., 2022; Suzuki et al., 2023). For instance, Takagi et al. (2025) demonstrated that MR-based multimodal feedback can help novices imitate expert gaze behaviors in basketball. More recently, Cheng et al. (2024) proposed an MR system that allows athletes to rehearse synchronized tactical movements with virtual teammates. While their results indicate some learning benefits, the system was not evaluated against conventional coaching methods, and critical aspects of team tactics—most notably passing, a central cooperative action in basketball (Maimón et al., 2020)—were not supported. Existing MR studies also remain limited to relatively small spaces and do not assess how interaction with virtual defenders affects behavioral adaptation or tactical decision-making.
To overcome these limitations, the present study develops a large-scale MR training environment capable of tracking both player and ball positions in real time, enabling interactive tactical training with virtual defenders across an actual half-court space. By integrating first-person visual guidance, dynamic virtual teammates, and real ball handling, this study aims to empirically evaluate how MR-based instruction influences learners’ coordination, timing, and tactical performance in comparison with conventional training methods.
2.2 Visual instruction methods in motor learning
In the field of motor learning, visual instruction for learners has been widely recognized as a core factor influencing the quality and efficiency of learning processes (Leite and Vieira, 2025; Mödinger et al., 2022). One of the most common traditional methods of visual instruction in motor learning is observational learning (Han et al., 2022), in which learners watch the movements of a model. In this approach, learners observe the model’s performance from a third-person perspective and are believed to acquire skills by comparing the observed movements with their own (Fitton et al., 2024). Recent advances in HMDs and AR/VR/MR technologies have made it possible to overlay visual instructions onto a learner’s first-person view. This allows for instruction that provides immediate and spatial visual feedback during motion execution within real-world performance contexts (Diller et al., 2025; Cheng et al., 2024). Previous research has suggested that these two instructional approaches should be selected based on the nature of the task. For example, Lee et al. (2019) conducted a comparative experiment on visual instruction in immersive VR environments and demonstrated that the effectiveness of spatial annotation-based feedback and model presentation via a virtual tutor varies depending on the task. Annotation-based feedback, which supports performance in real time, showed high effectiveness in both accuracy and speed for relatively simple tasks involving spatial targets. On the other hand, the virtual tutor, which presents model movements from a third-person perspective in advance, was found to be more effective for acquiring skills that require full-body imitation or the mastery of sequential movements. As such, selecting appropriate instructional methods according to the task characteristics and the viewpoint and modality of feedback presentation is a crucial consideration for the efficient acquisition of motor skills (Leite and Vieira, 2025). However, many of these findings are based on relatively static tasks that focus on individual motor skills, and may not fully generalize to learning situations in sports where multiple players are involved and real-time spatial decisions and tactical coordination are required (Richlan et al., 2023). In dynamic and complex environments such as sports, it is essential not only to develop individual skills but also to acquire team coordination and tactical understanding. Therefore, further empirical investigation is needed to clarify how feedback using AR/VR technologies affects learning outcomes in such practical, interactive contexts.
To address this gap, the present study focuses on tactical training in basketball. We developed a MR system that provides explicit feedback using virtual objects from a first-person perspective. By comparing this approach with traditional observational learning commonly used in practice, the study aims to clarify which modality and viewpoint of feedback are more effective for the acquisition of tactical skills in a sports context.
3 Proposed methods
The proposed method is composed of a large-scale MR stadium and mechanisms for players and ball position estimation and pass success detection. These components will be explained below:
3.1 Large-scale mixed reality stadium
This system was developed in a large-scale mixed-reality environment where visual information of virtual players, appropriate passing courses, and positioning, can be superimposed on real space with spatial consistency for a user wearing a see-through HMD. Figure 2a provides an overview of the proposed method. The system displays the offensive player and the defensive player of the opposing team to the user who is about to make a pass. When the user passes the ball within the offense displayed by the AR, the system determines whether the ball was passed within the range of the offense based on the measurement of the ball’s trajectory and provides feedback to the user. The system can also be applied to various practice scenarios, such as showing teammates and opposing players in MR, allowing a single player to practice passing and having a real person play defense for the HMD wearer.
Figure 2. Proposed method and system configuration. (a) Overview of the proposed method and system configuration. (b) Marker location. (c) Developed visual instructions using MR. Instruction Before making a Pass (Top), and Feedback after making a pass (Bottom).
3.2 Ball position estimation method
In this study, markers were attached to the user’s HMD and the ball to spatially align the MR system with the real-world environment. As shown in Figure 2b, the markers used for position tracking were arranged on the ball in the shape of an icosahedron. An icosahedron is a polyhedron composed entirely of equilateral triangles, allowing the recognition of the ball as long as at least three markers are detected—even if others are occluded due to the ball’s speed or rotation. This marker configuration provides geometric redundancy, ensuring stable tracking even when several markers become temporarily invisible due to partial occlusion.
Since the markers were placed on the surface of the ball, their detected positions were offset from the true center by approximately the ball’s radius. To correct this, we implemented an additional geometric calculation to determine the center coordinates.
Equation 1 describes the algorithm for calculating the center of the ball when a triangle ABC on the surface, as shown in Figure 2b, is detected. If the detected centroid of the triangle ABC is
Due to the symmetric nature of the icosahedral marker configuration, each detected triangle yields two possible normal vectors. Since no optical cameras were placed beneath the ball in this experiment, markers on the lower hemisphere were not visible. To resolve this ambiguity and prevent rigid-body flipping, we selected the normal vector with a stronger downward component (i.e., more aligned with the gravity direction) as the valid orientation.
After calculating the center coordinates of the ball, we applied a moving average filter to the position data to reduce jitter caused by high-speed motion or sensor noise.
To validate the feasibility and accuracy of the proposed ball center calculation, a test was conducted. A basketball with attached markers was rolled approximately 3 m along a linear rail in the x-axis direction. During this motion, the ball’s position was recorded using an optical motion capture system, and the center was estimated using the proposed geometric method. The estimated center coordinates were then compared with the true center (defined as 122.5 mm above the floor on the y-axis and 0 mm on the z-axis), and absolute errors were calculated in the y-axis, z-axis, and y-z plane. The mean absolute errors were 8.99 mm (y), 11.42 mm (z), and 15.81 mm (y-z), respectively. In addition to the mean absolute errors, the maximum errors were also obtained, yielding deviations of 25.37 mm in the y direction, 30.62 mm in the z direction, and 33.12 mm in the y z plane.
The optical motion capture system used in this study operates at 150 fps, updating position estimates at intervals of approximately 6.7 m. The center estimation process relies on a simple geometric computation based on triangle normals, and its computational cost is sufficiently small compared with the frame interval. The estimated positions are transmitted to the HMD via a wired connection, so no additional latency due to wireless communication is introduced. Under these conditions, the overall display latency of the system is expected to be very small, and no perceptible delay was observed during the experiment.
3.3 Detecting the success or failure of a pass
As depicted in Figure 2a, a transparent geometric object is positioned along the direction of the virtual ally player’s progression, with a height H, serving as the collision detection region for the ball. The volume V of this geometric object can be adjusted by varying the width w, depth a, and height h. This flexibility allows for arbitrary adjustment of the difficulty of passing exercises. Additionally, the shape of the collision detection system can be customized to any desired form. The system determines whether the pass released by the user comes into contact with the geometric object containing the collision detection and provides feedback to the user based on the results.
4 System configuration
4.1 LargeSpace
This study utilized the world’s largest virtual and mixed-reality environment, called LargeSpace (Takatori et al., 2016), at the University of Tsukuba. LargeSpace has dimensions of 25 m in width, 7.7 m in height, and 15 m in depth, providing ample space to accurately replicate a half-court of a basketball court. The court for the experiments is reproduced according to the specifications set by FIBA (International Basketball Federation) to recreate a half-court. An optical motion capture system (OptiTrack) consisting of 21 cameras is integrated into the space. This system enables the measurement of the positions of the markers placed within the space at a maximum frame rate of 150 Hz.
4.2 See-through HMD
4.2.1 Hardware
We used the Varjo XR-3, a camera pass-through HMD developed by Varjo, based in Finland. This HMD weighs 980 g and provides users with a field of view of a horizontal angle of 115° and a vertical angle of 95°. The HMD resolution is 70 PPD and it incorporates an Eye Tracker.
4.2.2 Software
Unity was used as the platform for overlaying AR information onto the real-world environment through the HMD. The success determination of the ball was achieved through the integration of OptiTrack and Unity. The user’s perspective when using the developed system and an overhead view of the user are illustrated in Figures 1a–c.
4.2.3 Virtual player projection algorithm
In this system, the recreation of a pick-and-roll is achieved by controlling the coordinates and animations of virtual players drawn using Unity. The virtual player defending the user is set to constantly follow a position 1 m away from the user on the straight line connecting the user and the goal, simulating movements similar to actual defense. On the offensive side, the virtual player initially waits near the goal. When the user reaches a pre-defined position, the offensive player moves laterally beside the defensive player to set up a screen. After the virtual player completes the screen and the user moves 0.25 m in the direction of utilizing the screen, a roll is performed, and the virtual player moves toward the goal.
4.2.4 Visual coaching using MR
The visual instruction to the learner includes the trajectory of the move, the timing of the pass, and the trajectory of the pass, as shown in Figure 2c. Virtual markers support the target location, and as the learner approaches the marker, the next target location is revealed. Upon making a pass, a marker of a different color is placed at the coordinates where the learner was when the pass occurred, utilizing collision detection. These markers signify the learner’s position at the moment that the pass was determined. The markers used to guide the learner’s movement path can be flexibly added during system design. In the implementation of the pick-and-roll scenario in this study, two markers were used: one to indicate the location for calling the screener, and another to indicate the position for making the pass.
To assist in the timing of the pass, a user interface displays the recommended time until the pass is positioned in the lower-right direction of the field of view. A countdown is initiated when the learner begins to perform the screen. Following a pass, the time until the pass is calculated, and the difference from the recommended time is displayed to provide feedback to the learner. In the timing indicator for the pass, green color represents the remaining time until the recommended pass moment, while red indicates how much time has elapsed beyond the recommended timing. This color-coded feedback helps the learner understand and adjust their timing more intuitively.
Lastly, the trajectory of the pass is conveyed to the learner by placing a translucent ball at the coordinates set in the system and connecting the ball to the recommended arrival point of the pass with a dashed line. This allows the learner to observe the variance between the actual pass and the recommended passes after making a pass. The above visual coaching is intended to support practicing both the timing and positioning in pick-and-roll.
5 Experimental design
The experimental methods and tools are first described here for clarity.
Pick-and-roll: Pick-and-roll (shown in Figure 3) is a tactic where the player with the ball utilizes a screen set by a teammate (Marmarinos et al., 2016). After setting the screen, the screener moves towards the basket in a rolling motion, aiming to receive a pass from the ball-handling player (Lama et al., 2011). Screen refers to a technique in which an offensive player without a ball uses their body to create an open position for the ball handler, acting as a barrier against the defender (Gómez et al., 2015; Remmert, 2003). After setting the screen, the screener moves toward the basket in a rolling motion, aiming to receive a pass from the ball-handling player. The crucial determinants in pick-and-roll are: the direction of dribbling, the timing of passing after using the screen, and delivering the ball to the open teammate in the correct position.
Figure 3. Scenario of pick-and-roll. Marker-L: the learner, Marker-S: the player performing on the screen, Marker-D: the defender.
5.1 Evaluation metrics
The movement trajectories of the participants during the experiment were analyzed using dynamic time warping (DTW) Berndt and Clifford (1994), a method for assessing the similarity of time-series data, used for evaluating movement trajectory similarities in sportsTsai et al. (2020). DTW can compare datasets of different lengths and timings, making it suitable for accurately evaluating movement patterns between different players or trials. A lower DTW value indicates a higher similarity between the trajectories, reflecting a closer match.
During pick-and-roll, the movement trajectory and time from the point of using the screen to making the pass were used for evaluation. In the experiment to measure learning effects, these values were compared with the movement patterns of skilled participants, allowing for the assessment of deviations and improvements in the learners’ movements and a quantitative measurement of the training effect. In addition to the movement trajectory and time taken to make the pass, a 5-point scale rating from 0 to 4 by the pass recipient was also conducted.
5.2 Ethical considerations
All experimental procedures were approved by the Ethics Review Board of the Institute of Systems and Information Engineering, University of Tsukuba (No. 2023R816). This study adhered to the Ethical Guidelines for Medical and Health Research Involving Human Subjects, as stipulated by the Ministry of Health, Labour and Welfare (MHLW) of Japan. Additionally, all participants provided written informed consent prior to inclusion in the study.
6 Experiments
6.1 MR stadium validation with skilled players
This experiment aimed to evaluate MR training by comparing the consistency of spatial dynamics with that in a conventional environment, namely, a regular gymnasium. The pick-and-roll performance by skilled players was used as a reference for evaluating deviations and improvements in the learners’ actions in subsequent experiments. Therefore, aside from the Coordinated Tactical Plays with virtual players, the environment was identical to the conventional one. This allows for evaluating the impact of the MR environment against the un-mediated behavior of skilled players.
6.1.1 Participants and collaborators
This experiment involved three skilled participants and three skilled collaborators who performed pick-and-roll. The skilled participants were males aged 21 to 23, with 13–15 years of basketball experience, including two small forwards and one power forward. The skilled collaborators were also males aged 22 to 23, with the same 13–15 years of basketball experience, including one guard, one small forward, and one power forward.
6.1.2 Procedure
The participants performed pick-and-roll under two conditions: a Real condition, in which they wore hats with markers attached, and an MR condition, in which the ball handler wore an HMD. Before the experiment began, all participants received standardized verbal instructions and a demonstration regarding the task objectives, movement positions, passing timing, and coordination with the screener. Each trial began with an auditory cue (“Start”) given by the experimenter. After the cue, the participants moved from the designated starting position to a waiting point located 1 m ahead, as shown in Figure 3, and waited for the screener to approach. The screener was instructed to initiate the screen movement along the same trajectory in every trial once the participant had reached the waiting point. The defender was instructed to perform a defensive switch after the screen, changing the offensive player they were guarding to reproduce realistic defensive behavior. In the Real condition, this sequence of actions was repeated 10 times. In the MR condition, once the participant arrived at the waiting point, the virtual screener automatically began the screen movement, and the virtual defender similarly performed a defensive switch after the screen. For fairness, no first person visual feedback or instructional cues were provided during the MR condition; participants simply carried out the pick-and-roll sequence against virtual defenders. The same scenario was repeated for 10 trials, consistent with the Real condition. Each trial followed a fixed sequence consisting of the start cue, movement to the waiting point, use of the screen, and execution of the pass. All participants performed the tasks under both conditions with identical instructions and trial structures. To avoid order effects, the presentation order of the two conditions was counterbalanced across participants. The Real condition setup is shown in Figure 4a.
Figure 4. (a) Participants performing pick-and-roll in real space. (b) Experimental results comparing MR and real space performance by skilled players. Trajectory similarity was calculated by comparing each MR and real trial to the average trajectory of real trials (left), and the average time to pass was compared between MR and real trials (right). Error bars represent 95% confidence intervals. (c) Coordinate trajectories of MR and real trials. Blue line: MR trials, gray line: real trials, dashed line: average of real trials.
6.1.3 Results
The execution times and DTW scores obtained from the experiment are presented in Figure 4b, while the actual movement trajectories of the participants are illustrated in Figure 4c. The average movement trajectory in the Real condition was used as a reference, and the similarity of each trial in the Real and MR conditions was evaluated using DTW. Additionally, the time required to execute the pick-and-roll and complete a pass was measured and compared across conditions. The mean execution time was 1.008 s (95% CI [0.930, 1.085]) in the Real condition and 1.078 s (95% CI [0.999, 1.156]) in the MR condition. The mean DTW distance was 47.65 (95% CI [38.39, 56.90]) for the Real condition and 51.98 (95% CI [44.58, 59.39]) for the MR condition. To statistically examine differences between conditions and participants, aligned rank transform analysis of variance (ART ANOVA) was conducted with condition (Real vs. MR) and subject as fixed factors. Holm correction was applied to adjust for multiple comparisons. For DTW scores, no significant effects were found for condition (
6.2 Effect of MR coaching for improving beginners’ skills
The objective of this experiment was to evaluate the learning effectiveness using the proposed system and those based on conventional training methods in the context of pick-and-roll. Additionally, the goal was to compare the learning effects between visual instructions from the first-person perspective and the third-person perspective.
6.2.1 Participants and collaborators
Nineteen beginner participants and three skilled collaborators took part in this experiment to execute the pick-and-roll task shown in Figure 3. Originally, twenty beginner participants were recruited, but one was excluded due to corrupted data. All participants were males, and their ages ranged from 20 to 24 years (M = 21.6, SD = 1.27). The beginners were defined as individuals who had never received formal basketball instruction under a coach. Among them, five had experience in ball sports such as soccer or volleyball, three had experience in track and field, and the remaining participants had no prior sports experience. The four skilled collaborators were the same individuals as in Experiment 1.
6.2.2 Procedure
Nine participants were assigned to the conventional method group (CM group). In this group, participants practiced by observing the movements of skilled players and attempting to reproduce those movements themselves. The remaining ten participants were assigned to the MR group, where they performed the same task while wearing a HMD.
As the overall procedure of the experiment, participants were first given an explanation of the experimental content. Then, a pick-and-roll movement was demonstrated once by four skilled collaborators to help participants understand the task. In this experiment, the movement rules and instructions for both the screener and the defender during the pick-and-roll were identical to those in Section 6.1. Following this, as shown in Figure 3, each participant performed the pick-and-roll three times in collaboration with three of the skilled collaborators. After this familiarization, participants were divided into two groups—CM and MR—and each group followed its designated training protocol. In the MR group, participants wore the HMD and practiced the pick-and-roll ten times with a virtual player. During this practice, visual cues were provided, including the skilled player’s pass release position, pass timing, and pass trajectory, as shown in Figure 2c. These instructions were based on the average values obtained from ten real-world trials conducted by a skilled player in a previous experiment. This same skilled player also performed the ball-handler role during all trials observed by the CM group. Specifically, the target points, x(m), z(m) = -1.78, −1.35, and the transit time, 1.044 s, were used. In the CM group’s training procedure, participants also performed ten practice trials. For each trial, they first observed a live demonstration of the pick-and-roll movement performed by the four collaborators, and then immediately attempted to reproduce the movement in coordination with them. The CM group received delayed and implicit feedback, such as model observation and introspective reflection based on proprioceptive awareness, throughout each trial. Participants relied solely on the demonstration provided before each trial to guide their performance. Finally, both the conventional and MR groups were asked to perform three pick-and-roll trials with their collaborators, followed by an open-ended interview regarding their impressions. During the experiment, the collaborator in charge of the screen rated the passes from the participants on a 5-point scale for each trial. The trajectories of the participants were measured using OptiTrack at 150 fps.
6.2.3 Results
Figure 5 shows the average results of the pass time difference compared to skilled participants (left), the trajectory similarity when comparing each trial to the average trajectory of a skilled player (middle), and the pass evaluation before and after training (right).
Figure 5. The results of time from screen use to passing, similarity of movement trajectories evaluated by DTW distance (lower values indicate higher similarity), and evaluation by the Screener before and after the practice. Statistical significance is indicated as follows: n.s = not significant, *
6.2.4 Between-group comparison: pre-post intervention differences
To examine the effectiveness of training within each group, Wilcoxon signed-rank tests were conducted to compare pre- and post-intervention outcomes. To assess differences in intervention effects between the conventional training group and the MR training group, Welch’s t-tests were applied to each evaluation metric. Between-group differences at baseline were examined using Welch’s t-tests, and no significant differences were found in any of the metrics, whether or not outliers were removed. These findings remained consistent after applying the Holm correction for multiple comparisons (Time difference:
Similarly, post-intervention differences between the groups were examined. Significant differences were found in trajectory similarity and pass evaluation, but not in pass timing difference, regardless of outlier removal. These findings remained consistent after applying the Holm correction for multiple comparisons. (Time difference:
6.2.5 Effect of intervention and training method (group time interaction)
To examine the main effects of training method (group) and intervention timing (pre/post), as well as their interaction, aligned rank transform ART ANOVA was conducted for each metric, with p-values adjusted using the Holm correction for multiple comparisons. For trajectory similarity, significant main effects were found for both group (
6.2.6 Within-group comparison: pre-post changes
To examine the effects of training within each group, Wilcoxon signed-rank tests with Holm correction were conducted to compare pre- and post-intervention values. In the MR group, significant improvements were observed in pass timing difference (
6.2.7 Effect of performance metrics on pass evaluation
To further investigate how performance metrics influence pass evaluation scores, a generalized linear mixed model (GLMM) was constructed with pass evaluation as the dependent variable. Fixed effects included training method, intervention timing (pre/post), pass timing difference, and trajectory similarity. Participant ID was treated as a random intercept to account for individual differences.
As shown in Table 1, three of the fixed effects (training method, intervention timing, and pass timing) significantly contributed to the model
6.2.8 Interview results
A review of the free-form interview responses revealed several recurring patterns regarding the strengths and weaknesses of the two methods. For the conventional method, participants noted strengths such as “it was possible to observe detailed movements such as foot movements when moving and hand movements when passing.” However, some participants also reported difficulties in identifying what to focus on, with comments such as “I did not know where to look.” For the MR method, participants frequently mentioned advantages such as “the points to learn were specifically indicated, making it easier to understand than verbal instruction from the instructor,” and “I was able to freely observe information about the plays I made, which helped in correcting the trajectory of passes.” At the same time, several drawbacks specific to the HMD were raised, including comments like “the weight of the HMD during learning was burdensome” and “feeling slightly nauseous.”
7 Discussion
7.1 Consistency and reproducibility of MR-based training
The first experiment examined the spatial and temporal consistency of skilled players performing pick-and-roll movements in both real and MR environments. No statistically significant differences were observed between the Real and MR conditions in terms of movement trajectory similarity, as measured by DTW, or in the execution time required to complete the pass. While the limited sample size should be taken into account, these results suggest that the proposed MR system could approximate the spatial and temporal dynamics involved in tactical execution. No significant effects were found for condition (Real vs. MR), participant, or their interaction. This indicates that the performance patterns of skilled players did not vary substantially across environments, however, confirming this interpretation will require further investigation with larger datasets. The average execution time in the MR condition was approximately one second, which is similar to values observed in real gameplay. This finding suggests that the system could maintain the real-time pace necessary for in-game decision-making and motor responses without introducing notable delays from the HMD or virtual overlays.
7.2 Effectiveness of MR-based coaching
The second experiment examined effectiveness of the proposed MR-based coaching method for novice players by comparing it with a conventional coaching approach. As shown in Figure 5, the MR method tended to produce greater improvements in both spatial positioning (trajectory similarity) and timing in cooperative tactics. One possible explanation is the limitation of traditional third-person observational learning, in which learners often struggle to accurately recognize or adjust their own errors. In contrast, MR allows learners to experience correct movements and timing from a first-person perspective, which may facilitate imitation and self-correction. This interpretation aligns with Lee et al. (2019), which reported that first-person spatial visualization can be more effective than observing others’ demonstrations for acquiring spatial knowledge. Furthermore, although there were no significant differences between groups in key indicators prior to the intervention, the MR group showed significant improvement in both trajectory similarity and pass evaluation afterward. This suggests an advantage for learning these aspects of coordinated tactical performance.
Regarding the timing difference of passes (Time), although no statistically significant difference was observed between the groups after the intervention, a significant improvement was found within the MR group. A similar tendency has been reported in prior MR-based tactical training research (VisCourt) (Cheng et al., 2024), which observed improvements in timing following MR instruction. However, a post hoc power analysis revealed a power of only 0.34 for the between-group comparison of the Time metric after the intervention, indicating that a potential effect may not have been detected. It is also noteworthy that potential benefits of the MR method were observed even for the relatively simple tactic of the pick-and-roll. This differs from Tsai et al. (2020), who reported VR-related improvements only for more complex tactics involving four or more players. They attributed the limited impact of VR for simpler tactics to the absence of tactile ball-handling feedback in their VR setup. In contrast, the proposed MR system allowed learners to physically interact with a real ball during training, which may have supported more realistic and transferable coordination practice. This interpretation is compatible with prior research emphasizing the importance of tactile engagement for promoting game-realistic understanding in tactical training (Vencúrik et al., 2021).
In the analysis of pass evaluations during pick-and-roll practice, both the conventional and MR methods showed improvements from pre-to post-intervention. A comparison of effect sizes indicated a tendency toward a larger effect for the MR method (conventional: r = 0.73, MR: r = 0.87). Both conditions showed large effects, with a relatively larger effect observed in the MR method. Although no significant group differences were observed prior to the intervention, a significant difference
Finally, the qualitative feedback provides additional context for interpreting the quantitative results. Participants in the MR group frequently remarked that the key learning points were visually highlighted and that they could review information about their own plays. These comments are consistent with the observed improvements in spatial positioning and timing under the MR method, and they support the possibility that visual feedback facilitated learners’ awareness and self correction. In contrast, several participants in the conventional group noted that they “did not know where to look,” which corresponds to the more limited performance gains seen in the quantitative data.
In summary, first-person visual coaching via MR appeared to support learners in developing the spatial understanding, timing, and self-correction skills required for cooperative tactics. This approach may complement and enhance conventional imitation-based learning methods. Future work should examine its applicability to more complex tactics, different player positions, and longer-term learning outcomes.
7.3 Limitations and future directions
This study has several limitations that should be addressed in future research. First, the sample sizes in both experiments were relatively small. In Experiment 1, only three expert participants were involved, resulting in low statistical power (approximately 11%–15% based on post hoc estimation). A more diverse pool of expert participants is necessary to capture a broader range of play styles. In Experiment 2, the group sizes (9 vs. 10) and power analysis indicated that the sensitivity was insufficient to detect moderate interaction effects. Moreover, all participants in both experiments were male, which may limit the generalizability of the findings across genders. Second, this study focused solely on the representative tactical scenario of pick-and-roll. Future research should expand MR training to encompass other set offenses, zone offense/defense, and fast break coordination. This would allow for evaluation of the scalability and adaptability of the MR system in more complex, real-world team contexts. Third, only short-term learning outcomes immediately after training were assessed. Longitudinal studies are needed to investigate skill retention over weeks or months and the transfer of these skills under pressure during actual games. Finally, future work should consider how the MR training environment can be dynamically personalized for advanced or elite athletes. Because highly skilled players demonstrate faster decision making and stronger anticipatory abilities, they may require scenarios in which opponent behaviors and task demands adapt in real time to the player’s actions. Dynamically adjusting ball trajectories and decision-making complexity based on individual tendencies and weaknesses could provide a more realistic and high-intensity training environment.
8 Conclusion
This paper introduced a MR environment designed for coordinated tactical training, offering potential benefits such as facilitating individual tactical practice, preserving kinesthetic sensations during movement, and supporting learning through first-person visual instruction. In the validation experiment with skilled players, no significant differences were observed between movements performed in the MR environment and those in the real world, suggesting that the system may approximate real-world spatial and temporal dynamics. The coaching experiment with beginner players indicated that the MR environment may provide advantages for motor learning and visual instruction, particularly in spatial and temporal aspects, compared with conventional methods. Furthermore, the consistency of skilled players’ movements across environments, together with the observed improvements in novice learners, suggests the possibility that skills practiced in MR may transfer to real-world performance, although further investigation is required.
Future work will examine a wider range of tactical contexts, including different player roles and positions, and will incorporate random or dynamic instructional elements to promote more robust learning. Interview responses indicated positive impressions of first-person perspective learning; however, the conventional method supported detailed movement learning not directly addressed by the proposed system. Thus, an important direction for future research is to explore ways to integrate detailed motor instruction into the MR-based approach.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by the Ethics Review Board of the Institute of Systems and Information Engineering, University of Tsukuba. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.
Author contributions
HS: Investigation, Writing – review and editing, Validation, Conceptualization, Methodology, Visualization, Software, Formal Analysis, Writing – original draft, Project administration, Data curation. MaH: Conceptualization, Writing – review and editing, Supervision, Validation, Methodology. MoH: Conceptualization, Writing – review and editing, Supervision, Validation, Methodology. KS: Conceptualization, Resources, Funding acquisition, Writing – review and editing, Validation, Methodology, Supervision.
Funding
The author(s) declared that financial support was received for this work and/or its publication. This research was supported by funding from the Japan Sports Agency.
Conflict of interest
Author MaH was employed by NEC Corporation.
The remaining author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
1Pick and roll is a cooperative offensive tactic where a screener sets a screen for the ball handler and then rolls toward the basket to receive a pass.
References
Araújo, D., Travassos, B., and Vilar, L. (2010). Tactical skills are not verbal skills: a comment on kannekens and colleagues. Percept. Motor Skills 110, 1086–1088. doi:10.2466/pms.110.c.1086-1088
Ben Mahfoudh, H., and Zoudji, B. (2024). The use of virtual reality for tactical learning: the moderation of expertise and visuospatial abilities. Int. J. Sport Exerc. Psychol. 23, 1–18doi. doi:10.1080/1612197X.2024.2380693
Berndt, D. J., and Clifford, J. (1994). “Using dynamic time warping to find patterns in time series,” in Proceedings of the 3rd international conference on knowledge discovery and data mining, 359–370.
Biswas, N., Mukherjee, A., and Bhattacharya, S. (2024). “are you feeling sick?”–A systematic literature review of cybersickness in virtual reality. ACM Comput. Surv. 56, 1–38. doi:10.1145/3670008
Cheng, L., Jia, H., Yu, L., Wu, Y., Ye, S., Deng, D., et al. (2024). “Viscourt: in-situ guidance for interactive tactic training in mixed reality,” in Proceedings of the 37th annual ACM symposium on user interface software and technology, 1–14doi. doi:10.1145/3654777.3676466
Diller, F., Henkel, N., Scheuermann, G., and Wiebel, A. (2025). Skillar: omnipresent in-situ feedback for motor skill training using ar. Virtual Real. 29, 1–11. doi:10.21203/rs.3.rs-4841183/v1
Fitton, I. S., Dark, E., Oliveira da Silva, M. M., Dalton, J., Proulx, M. J., Clarke, C., et al. (2024). “Watch this! observational learning in vr promotes better far transfer than active learning for a fine psychomotor task,” in Proceedings of the 2024 CHI conference on human factors in computing systems, 1–19. doi:10.1145/3613904.3642550
Gómez, M. A., Battaglia, O., Lorenzo, A., Lorenzo, J., Jiménez, S., and Sampaio, J. (2015). Effectiveness during ball screens in elite basketball games. J. Sports Sciences 33, 1844–1852. doi:10.1080/02640414.2015.1014829
Graf, R., Benawri, P., Whitesall, A. E., Carichner, D., Li, Z., Nebeling, M., et al. (2019). “Igym: an interactive floor projection system for inclusive exergame environments,” in Proceedings of the annual symposium on computer-human interaction in play, 31–43doi. doi:10.1145/3311350.3347161
Gutierrez, C. S., Salazar, J., and Hirata, Y. (2022). Mixed-reality human-machine-interface for motor learning of physical activities. Adv. Robot. 36, 583–599. doi:10.1080/01691864.2022.2076569
Han, Y., Syed Ali, S. K. B., and Ji, L. (2022). Use of observational learning to promote motor skill learning in physical education: a systematic review. Int. J. Environ. Res. Public Health 19, 10109. doi:10.3390/ijerph191610109
Kannekens, R., Elferink-Gemser, M. T., and Visscher, C. (2011). Positioning and deciding: key factors for talent development in soccer. Scand. Journal Medicine and Science Sports 21, 846–852. doi:10.1111/j.1600-0838.2010.01104.x
Kickmeier-Rust, M. D., Hann, P., and Leitner, M. (2019). Increasing learning motivation: anempirical study of vr effects on the vocational training of bank clerks. Entertain. Comput. Serious Games 11863, 111–118. doi:10.1007/978-3-030-34644-7_9
Kim, J., Ka, J., Lee, Y., Lee, Y., Park, S., and Kim, W. (2022). “Mixed reality-based outdoor training system to improve football player performance,” in 2022 international conference on engineering and emerging technologies (ICEET), 1–3. doi:10.1109/ICEET56468.2022.10007354
Kucsa, R., and Mačura, P. (2015). Physical characteristics of female basketball players according to playing position. Acta Fac. Educ. Phys. Univ. Comen. 55, 46–53. doi:10.1515/afepuc-2015-0006
Lama, L., Junior, D. D. R., Santana, F., Rostaiser, E., Negretti, L., and Ugrinowitsch, C. (2011). Space creation dynamics in basketball offence: validation and evaluation of elite teams. Int. J. Perform. Analysis Sport 11, 71–84. doi:10.1080/24748668.2011.11868530
Lee, H., Kim, H., Monteiro, D. V., Goh, Y., Han, D., Liang, H.-N., et al. (2019). “Annotation vs. virtual tutor: comparative analysis on the effectiveness of visual instructions in immersive virtual reality,” in 2019 IEEE international symposium on mixed and augmented reality (ISMAR), 318–327doi. doi:10.1109/ISMAR.2019.00030
Lee, L.-K., Wei, X., Chui, K. T., Cheung, S. K., Wang, F. L., Fung, Y.-C., et al. (2024). A systematic review of the design of serious games for innovative learning: augmented reality, virtual reality, or mixed reality? Electronics 13, 890. doi:10.3390/electronics13050890
Leite, H., and Vieira, L. R. (2025). The use of virtual reality in human training: trends and a research agenda. Virtual Real. 29, 1–25. doi:10.1007/s10055-024-01093-x
Lin, T., Singh, R., Yang, Y., Nobre, C., Beyer, J., Smith, M. A., et al. (2021). “Towards an understanding of situated ar visualization for basketball free-throw training,” in Proceedings of the 2021 CHI conference on human factors in computing systems, 1–13doi. doi:10.1145/3411764.3445649
Lin, T., Aouididi, A., Zhu-Tian, C., Beyer, J., Pfister, H., and Wang, J.-H. (2023). Vird: immersive match video analysis for high-performance badminton coaching. IEEE Transactions Visualization Computer Graphics 30, 458–468. doi:10.1109/TVCG.2023.3327161
Maimón, A. Q., Courel-Ibáñez, J., and Ruíz, F. J. R. (2020). The basketball pass: a systematic review. J. Human Kinetics 71, 275–284. doi:10.2478/hukin-2019-0088
Makransky, G., Terkildsen, T. S., and Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn. Instruction 60, 225–236. doi:10.1016/j.learninstruc.2017.12.007
Marmarinos, C., Apostolidis, N., Kostopoulos, N., and Apostolidis, A. (2016). Efficacy of the “pick and roll” offense in top level European basketball teams. J. Human Kinetics 51, 121–129. doi:10.1515/hukin-2015-0176
Marshall, B., Uiga, L., Parr, J., and Wood, G. (2023). A preliminary investigation into the efficacy of training soccer heading in immersive virtual reality. Virtual Real. 27, 2397–2404. doi:10.1007/s10055-023-00807-x
Mödinger, M., Woll, A., and Wagner, I. (2022). Video-based visual feedback to enhance motor learning in physical education—a systematic review. Ger. Journal Exercise Sport Research 52, 447–460. doi:10.1007/s12662-021-00782-y
Oki, M., Akizuki, S., Bourreau, B., Takahashi, I., Aoki, Y., Yamamoto, J., et al. (2022). Supporting collective physical activities by interactive floor projection in a special-needs school setting. Int. J. Child-Computer Interact. 32, 100392. doi:10.1016/j.ijcci.2021.100392
Paraskevaidis, P., and Fokides, E. (2020). Using 360 videos for teaching volleyball skills to primary school students. Open J. Inf. Technol. 3, 21–38. doi:10.32591/coas.ojit.0301.03021p
Pastel, S., Petri, K., Chen, C., Wiegand Cáceres, A. M., Stirnatis, M., Nübel, C., et al. (2023). Training in virtual reality enables learning of a complex sports movement. Virtual Real. 27, 523–540. doi:10.1007/s10055-022-00679-7
Pollock, B. J., and Lee, T. D. (1992). Effects of the model’s skill level on observational motor learning. Res. Q. Exerc Sport 63, 25–29. doi:10.1080/02701367.1992.10607553
Radu, I., and Schneider, B. (2019). “What can we learn from augmented reality (ar)? Benefits and drawbacks of ar for inquiry-based learning of physics,” in Proceedings of the 2019 CHI conference on human factors in computing systems, 1–12doi. doi:10.1145/3290605.3300774
Remmert, H. (2003). Analysis of group-tactical offensive behavior in elite basketball on the basis of a process orientated model. Eur. Journal Sport Science 3, 1–12. doi:10.1080/17461390300073311
Richlan, F., Weiß, M., Kastner, P., and Braid, J. (2023). Virtual training, real effects: a narrative review on sports performance enhancement through interventions in virtual reality. Front. Psychol. 14, 1240790. doi:10.3389/fpsyg.2023.1240790
Rodrigues, M. C. J., Rocha, A. C. R., Lira, C. A. B., Figueiredo, L. S., Lima, C. O. V., Laporta, L., et al. (2022). How small-sided games’ court dimensions affect tactical-technical behavior in beginner volleyball athletes. Int. J. Sports Sci. and Coach. 17, 1385–1395. doi:10.1177/17479541211058447
Sallet, P., Perrier, D., Jm, F., and Baverel, G. (2005). Physiological differences in professional basketball players as a function of playing position and level of play. J. Sports Medicine Physical Fitness 45, 291.
Suzuki, Y., Sakamoto, D., and Ono, T. (2023). “Gino. Aiki: mixed reality-based physical motor skill training in aikido,” in 2023 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct) (IEEE), 519–524. doi:10.1109/ISMAR-Adjunct60411.2023.00112
Takagi, S., Matsuura, K., and Takeuchi, H. (2025). “Support for learning gaze-trend in basketball using mr environment,” in 2025 19th international conference on ubiquitous information management and communication (IMCOM) (IEEE), 1–8. doi:10.1109/IMCOM64595.2025.10857503
Takatori, H., Enzaki, Y., Yano, H., and Iwata, H. (2016). Development of a large-immersive display “largespace”. Trans. Virtual Real. Soc. Jpn. 21, 493–502. doi:10.18974/tvrsj.21.3_493
Tsai, W. L., Pan, T. Y., and Hu, M. C. (2020). Feasibility study on virtual reality based basketball tactic training. IEEE Trans. Vis. Comput. Graph. 28, 2970–2982. doi:10.1109/tvcg.2020.3046326
Ueyama, Y., and Harada, M. (2024). Basketball free-throw training with augmented reality-based optimal shot trajectory for novice shooters. Sci. Rep. 14, 891. doi:10.1038/s41598-024-51190-9
Vencúrik, T., Nykodỳm, J., Bokǔvka, D., Rupčić, T., Knjaz, T., Dukarić, D., et al. (2021). Determinants of dribbling and passing skills in competitive games of women’s basketball. Int. Journal Environmental Research Public Health 18, 1165. doi:10.3390/ijerph18031165
Wang, Q., and Li, Y. (2024). How virtual reality, augmented reality and mixed reality facilitate teacher education: a systematic review. J. Comput. Assisted Learn. 40, 1276–1294. doi:10.1111/jcal.12949
Wen, J., Gold, L., Ma, Q., and LiKamWa, R. (2024). “Augmented coach: volumetric motion annotation and visualization for immersive sports coaching,” in 2024 IEEE conference virtual reality and 3D user interfaces (VR) (IEEE), 137–146. doi:10.1109/VR58804.2024.00037
Keywords: basketball, first-person guidance, mixed reality, sports training, tactical training
Citation: Saiki H, Hirokawa M, Hassan M and Suzuki K (2026) A large-scale mixed reality stadium for training coordinated tactical plays in basketball. Front. Virtual Real. 6:1722248. doi: 10.3389/frvir.2025.1722248
Received: 10 October 2025; Accepted: 29 December 2025;
Published: 30 January 2026.
Edited by:
Bruno Figueira, University of Evora, PortugalReviewed by:
Jeremias Braid, University of Salzburg, AustriaMansi Sahu, Uttaranchal University, India
Copyright © 2026 Saiki, Hirokawa, Hassan and Suzuki. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Hayato Saiki, c2Fpa2lAYWkuaWl0LnRzdWt1YmEuYWMuanA=