Reliability and validity of Polar Team Pro measurements in running at different velocities in an indoor setting

Introduction

Performing in sports at a high level during the whole year requires regular training and monitoring of the training load to avoid injuries (1). There are different ways to control the training load during each training session. One is monitoring the total distance and distances at different intensities during each training session (2, 3). These measurements are easy to monitor in athletics, where athletes run at prescribed distances at different intensities. However, in team sports like soccer and handball, these measurements are much more difficult to perform during training sessions and games. Therefore, in the last two decades, measurement systems like Catapult and ZXY have been developed to monitor these distances and intensities using the Global Positioning System (GPS) and accelerometers. These systems have been shown to measure distance and intensity accurately (1). However, they are relatively expensive and use a local position measurement system under outdoor and indoor conditions.

Polar Team Pro is a low-cost system that also uses GPS and accelerometers. It also claims to accurately measure distances at different intensities in outdoor and indoor situations. Thus, under outdoor conditions, Polar Team Pro uses GPS, a system that is not functional under indoor conditions. Two studies (4, 5) have been conducted to measure the validity and reliability of Polar Team Pro under outdoor conditions. First, Akyildiz et al. (4) compared the accuracy and reliability of two Polar Team Pro units for interunit reliability with GPSports as the reference standard to determine concurrent accuracy in the total distance and distances at different velocities. They observed acceptable interunit reliability (<5% typical error of the mean) and concluded that Polar Team Pro was suitable for tracking team sport variables. Furthermore, the Polar Team Pro units were accurate under the same conditions. However, Polar Team Pro is not advised to be used interchangeably with GPSports for quantifying distances covered at higher speeds. Second, Huggins et al. (5) assessed the validity and reliability of Polar Team Pro units in measuring the total distance and velocity under outdoor conditions, but during linear running and a team sport simulation circuit among 15 male soccer athletes. They showed that validity and reliability measures in terms of the total distance had <5% error at all velocities. Furthermore, the validity of the device in measuring velocity was significantly different (p < 0.05) at all velocities between the 40 m and the 100 m runs, with effect sizes ranging from trivial to small. Despite the trivial to large effect sizes for the validity of the total distance, the Polar Team Pro units demonstrated good reliability during linear and team sport simulation circuit movements during outdoor testing.

The validity of Polar Team Pro was tested during an indoor situation by Fox et al. (6), who investigated the sensor during continuous locomotive and change-of-direction tasks at low, medium, and high intensities. They found a low agreement between the sensor and the measured speed and distance indoors. However, it was not clearly mentioned if they calibrated the system before using it outdoors to link the GPS-measured data with the inertial measurement unit (IMU) sensor for accurate IMU measurements indoors, as the manufacturer specifies. Furthermore, the validity was measured only at three different intensities per participant, which could lead to missing information. Moreover, as the authors (6) stated, no reliability measurements were conducted, which are important to investigate if the reliability between the different units over several days is acceptable in an indoor setting. This information is very important for athletes, trainers, and embedded scientists for monitoring the training load in indoor sports, e.g., handball and futsal, to avoid overtraining for a competition. Therefore, this study aimed to test the reliability and validity of Polar Team Pro measurements of velocity and distance at different intensities in an indoor setting. Based on the findings of Fox et al. (6) on the continuous locomotive and Change of direction (COD) tasks using Polar Team Pro indoors, it was hypothesized that running speeds would be underestimated.

Methods

Participants

A total of 10 female handball players (age 15.7 ± 0.4 years, body mass 61.3 ± 5.3 kg, body height 1.69 ± 0.07 m), who are playing on the national junior league level and free from any injuries or health conditions, performed running tests. Informed consent was obtained prior to testing from all subjects and parents, with the approval of the Norwegian Center for Research Data (NSD), and conformed to the latest revision of the Declaration of Helsinki.

A repeated-measures design was used to test the reliability and validity of Polar Team Pro, in which, in two test sessions, actual 100 m runs (reference distance) at different intensities were performed on a rectangular track at an indoor facility. The total distance, distance, and velocity at different intensities were used as variables and compared with control-measured 100 m distances. Reliability was tested according to systematic bias, random error, and retest correlation.

Procedure

Before using Polar Team Pro (Polar Electro, Kempele, Finland) indoors, the system was used in two training sessions, i.e., warmup and running outdoors. The manufacturer recommends calibrating the inertial movement units with GPS signals. Each participant wore the same Polar Team Pro strap (Polar Electro, Kempele, Finland) positioned on the center of the chest at the Xiphoid process level at each session to calibrate the sensors. After calibration, the system was used during indoor handball training sessions for 4 months before the testing days. On the test and retest days, the same sensor was used by each participant. After an individualized warmup of 5 min, each of the participants started at a cone after each other, with the same distance between them on a rectangular track (20 m × 30 m) marked with cones on each 5 m. The distances were control-measured with a measuring tape to millimeter accuracy. On a signal, they started to jog at a prescribed tempo. Each participant had to hold the same distance from the other participants the whole time (Figure 1). The starting tempo was 8 km/h. Every 25 m, a signal was given to ensure that the participants had the correct pace. After 200 m, the pace was increased to 10 km/h for 200 m. It was followed with 2 m × 100 m at 12, 15, and 18 km/h with some rest between the runs. See Table 1 for full details on the running time and rest between each run. In the rest periods, the participants had to stand still to prevent the system from detecting extra meters. A total of 1,000 m had to be covered.

FIGURE 1

Figure 1. Experimental setup: a rectangle (20 m × 30 m) track marked with cones at an indoor facility.

TABLE 1

Table 1. Running protocol with the distances, intensities, and rest between runs.

For every occasion for each participant, the total distance and the velocities at different intensities (3–6.9, 7–10.9, 11–14.9, 15–18.9, >19 km/h) were automatically calculated by the accelerometers in the Polar Team Pro system, with sampling frequencies of 10 Hz together with the average velocity, which should be 7.66 m/s for this protocol. Furthermore, each 100 m from the raw data at a different intensity was calculated based on the prescribed time for each 100 m on each occasion and used in further analysis.

Statistical analysis

Validity and reliability were tested in three different ways, i.e., systematic bias, random error, and test–retest, similar to the validation studies of Balsalobre-Fernandez et al. (7) and van den Tillaar and Ball (8). The systematic bias was calculated for the total distance, average velocity during the whole session per speed category, and each 100 m between prescribed distances, velocities, and actual measured parameters. These biases were calculated by using a paired sampled t-test on each testing occasion. A 2 (runs 1 and 2) × 5 (speed) ANOVA with repeated measures was performed between the measured distances per speed and per run to investigate whether the measured distances changed between runs and velocity. A random error was estimated as the standard deviation between the sensors/mean of the sensors × 100 for the different variables using the coefficient of variance (CV), in which a CV under 10% is considered good (9). The test–retest was expressed by an intraclass correlation coefficient (ICC) for the total distances and distances per speed, measured by each sensor during the two testing days. Furthermore, a one-way ANOVA for repeated measures was performed between the first and the second tests for each variable (velocity and distance). Where the sphericity assumption was violated, the Greenhouse–Geisser adjustments of the p-values were reported. The effect size was evaluated with eta partial squared, where 0.01 < η² < 0.06 constitutes a small effect, 0.06 <η² < 0.14 a medium effect, and η² > 0.14 a large effect (9). The interpretations of ICC were that values below 0.5 indicated poor reliability, between 0.5 and 0.75 moderate reliability, between 0.75 and 0.9 good reliability, and above 0.9 excellent reliability (10). The level of significance was set at p ≤ 0.05, and all data are expressed as mean ± SD. Statistical analysis was performed in SPSS version 27.0 (SPSS, Inc., Chicago, IL, United States).

Results

The total running distances, average velocity of the whole protocol, and running distances in the different speed categories (directly measured and calculated by Polar Team Pro) were significantly different from the actual distances (t ≥ 2.45, p ≤ 0.037). A post-hoc comparison showed that the total distance was underestimated on each testing day by, on average, 50 and 75 m. Furthermore, the distance covered at 0–10.99 km/h was significantly overestimated by ≈94 m, while the distance covered at 11 km/h was underestimated by 150–170 m. The average velocity was underestimated by 0.66 m/s (Table 2), while the peak velocity reached over 18 km/h.

TABLE 2

Table 2. Average (±SD) total distance, maximal and average velocity, and distance per different velocity measured with Polar Team Pro.

When analyzing the raw data on velocity and distance with the actual distance and corresponding speeds, a significantly lower velocity than the prescribed velocity for each speed was measured with Polar Team Pro (t ≥ 2.61, p ≤ 0.028), except at 8 km/h (t ≤ 1.79, p ≥ 0.11, Figure 2). On test day 1, the covered distance per prescribed speed (100 m) was significantly affected by speed (F = 30.0; p < 0.001; η_p² = 0.76), run (F = 20.9; p = 0.001; η_p² = 0.70), and interaction (F = 18.7; p < 0.001; η_p² = 0.67). On test day 2, only a significant effect of speed was found (F = 34.5; p < 0.001; η_p² = 0.81), but no significant effect of run and interaction (F ≤ 2.69; p ≥ 0.082; η_p² ≤ 0.25). On test day 1, the post-hoc comparison showed that the covered distance decreased with increasing speed ranging from 15 to 18 km/h. Furthermore, more distance was covered in the second 100 m run, which was only significant at 15 km/h (Figure 2). On test day 2, the covered distance decreased at increasing speeds from 8 to 10 km/h and 10 to 15 km/h (Figure 2).

FIGURE 2

Figure 2. Means (±SD), velocities, and distances for each run at different intensities on each test day. * indicates a significant difference (p < 0.05) between the first and the second runs. † indicates a significant difference (p < 0.05) between the first and the second testing days. ‡ indicates a significant difference (p < 0.05) with actual velocity.

FIGURE 3

Figure 3. Relationship between body height and total distance covered during the test.

A repeated-measures design showed no significant differences in total running distances in the different speed categories and average velocity directly measured and calculated by Polar Team Pro between the two test days (t ≤ 1.47, p > 0.186, Table 2). However, when testing the raw data (distance and velocity per speed) between the two test days, a significant difference in distance and velocity between runs 1 and 2 was found only at 15 km/h, in which the running velocity was lower and measured at a shorter distance on test day 1 than that on test day 2. In run 2 at 15 km, the opposite was found (Figure 2). Day-to-day variation tested by the ICCs on the total distance and distances at different speeds between the two test days showed that the ICCs varied from 0.49 (poor) to 0.89 (good) reliability (Table 3).

TABLE 3

Table 3. ICC between test days 1 and 2 for the total distance, distance at 0–11 km/h, distance at 11–19 km/h, and distance for each 100 m run.

The random error viewed by the CV showed that CVs on the overall running distance, average velocity, and distances covered at different intensities, which were calculated by Polar Team Pro, varied from 6.7% to 31.6% (Table 2). The CVs calculated from the velocities at each speed, run, and test day were on an average of 8.5% over all runs and speeds and varied from 4.2% to 12.4% (Table 4).

TABLE 4

Table 4. Coefficient of variation between the sensors on velocity and distance for each run, speed, and test day.

Discussion

The aim of this study was to examine the reliability and validity of Polar Team Pro measurements of velocity and distance at different intensities in an indoor situation. The main finding revealed that Polar Team Pro underestimated the running distance and velocity, especially at higher speeds (15% and 6% at 15 and 18 km/h, respectively). Between test days, the measurements varied at different speeds, and coefficients of variations varied from 4.2% to 12.4%. However, a difference was found only at 15 km/h for the two runs between the two test days.

Polar Team Pro measured the total distance at 9.5%, which was shorter than it actually was. This was mainly caused by the shorter measured distances at speeds higher than 8 km/h. With increasing speed, the measured distance with the system became shorter until 15 km/h, in which only an average of 85 m was measured, which was 15% shorter than the actual distance. Thereby, the systematic error increased with increasing speed, but with no significant systematic error at 8 km/h (Figure 2). Since the distance was underestimated at higher speeds, the measured velocities at these speeds were lower, and thereby more distance was covered in the lower speed range (0–11 km/h) and less in the high range (11 to >19 km/h) than in reality (Table 2). This was probably caused by the inaccuracy of the accelerometers and algorithms in measuring and calculating the step length with increasing speed. With increasing speed, the step length increases, and an inaccuracy of approximately 0.3 m for each step can occur when sprinting at maximal velocity (11). Therefore, with increasing speed, longer steps are underestimated. This is also visible when comparing the body height with the total distance (Figure 3). Taller women generally have longer steps to cover the same distance, but with Polar Team Pro, it appears that the longer women run shorter, while they run the same distance. Thus, the validity of Polar Team Pro is high at 8 km/h and for women who are approximately 1.60 m tall. Our findings contradict those of Akyildiz et al. (4) and Huggins et al. (5), who investigated the accuracy and reliability of Polar Team Pro outdoors mainly in a straight line with acceptable reliability and validity. However, in outdoor settings, the system uses a GPS signal, which is not possible indoors, and the system then must rely on the IMU and the algorithms behind it to calculate the distances covered at different velocities during the rectangular run. Therefore, the IMU underestimates the distance in general during the indoor situation.

The variation in test–retest was apparent only at 15 km/h between runs 1 and 2. This finding could be explained by the following example. On test day 1 in the first trial, the participants did not know exactly at what pace 15 km/h was and ran too slowly (2–3 m). In the second run, they tried to compensate by running a bit too fast (reaching the finish cone 1–2 s too early). On test day 2, they became aware of the pace and thereby did not change the velocity from run 1 to run 2 (Figure 2). Our findings indicate that Polar Team Pro is accurate when measuring over different days. Although the ICC for the total distance was 0.70 (moderate reliability) and ICCs varied from 0.49 (poor) to 0.89 (good reliability) at different velocities (Table 2), the variation is probably caused by the small differences between the participants at each velocity. Even if they had to hold their distance between each other at every intensity, small changes in length between the participants (1–2 m) could have a large effect on the ICCs, as shown at 15 km between runs 1 and 2. To avoid this, each participant should have worn two Polar Team Pro belts to investigate the interunit accuracy better, as Akyildiz et al. (4) did. They showed similar ICCs for the total distance (0.63), but higher ICCs of 0.99 for the different speeds, but these were taken together.

The CVs for the total distance and average velocity were 6.7% on day 1% and 9.4% on day 2, which are acceptable, as the variation is below 10% (9). However, when specified per velocity, the CV varied from 4.2% to 12.4% (Table 3). For the speed categories 3–10.9 and 11–19 km/h, the CVs varied from 14% to 30% (Table 2). The high CVs for the speed categories can be explained by the fact that most units measured the velocities at 12 km/h and some units at 15 and under 11 km/h, and thereby the velocity was identified in another category, causing these high CVs. Furthermore, it was found that the CV between the units increased with increasing speed (Table 2), as also shown by increased systematic bias with these speeds. This indicates that the reliability decreases with higher speed in indoor situations, which was not the case in the studies by Akyildiz et al. (4) and Huggins et al. (5), who found the CV to be <5% in their outdoor testing situations.

The present study has some limitations. First, we only tested the system indoors and did not compare it directly in an outdoor setting, which could give information about how accurate the GPS is compared with the IMU algorithms. However, the manufacturer stated that one or two outdoor sessions are enough to calibrate the GPS with the IMU sensors to obtain accurate readings. Since the participants had worn the sensor for the prior 3 months during almost every indoor session, the system was expected to be accurately calibrated. Second, the participants were not of the same height and thus had different step lengths, which probably influenced the distance measurements. Our argumentation is supported by the fact that the most accurate data were with 1.6 m tall women. Furthermore, using two units for each participant instead of one increased the interunit variability and led to the generation of more accurate data. Finally, the participants ran after each other with the same distance between them. However, small changes in the distance between the participants could cause a large difference in the CV and ICC. In future studies, each participant should wear several sensors to avoid the possible difference (1–2 m) in the distance between participants and to measure the interunit accuracy more accurately.

Conclusion

To the best of our knowledge, this is the first study to examine the reliability and validity of Polar Team Pro according to the measurements of distance and intensities during indoor conditions without the use of GPS. Based on the findings of the present study, we can conclude that Polar Team Pro underestimates the running distance and velocity when measuring a rectangular run at different speeds indoors. This underestimation takes place, especially at higher speeds, which is probably caused by the inaccuracy of the IMU algorithm that calculates the distance, as body height influences the distance and velocity measurements. The variability between the different units is, thereby, also influenced, causing variable CVs between the sensors. Test–retest variability was acceptable. Based on the findings of this study, practitioners should be cautious when measuring speed and distance using Polar Team Pro Sensors in indoor settings, as these measurements are underestimated with increasing speed.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Norwegian Center for Research Data project nr: 846452. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

Conceptualization was done by RvdT and FGP; methodology was prepared by RvdT and FGP; validation was done by RvdT; formal analysis was performed by RvdT and FGP; investigation was carried out by RvdT and FGP; writing—original draft preparation was done by RvdT and PL; writing—review and editing was done by RvdT, FGP, and PL; project administration was managed by RvdT. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Gabbett TJ, Jenkins DG. Relationship between training load and injury in professional rugby league players. J Sci Med Sport. (2011) 14:204–9. doi: 10.1016/j.jsams.2010.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Bacon CS, Mauger AR. Prediction of overuse injuries in professional U18-U21 footballers using metrics of training distance and intensity. J Strength Cond Res. (2017) 31:3067–76. doi: 10.1519/JSC.0000000000001744

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Cummins C, Welch M, Inkster B, Cupples B, Weaving D, Jones B, et al. Modelling the relationships between volume, intensity and injury-risk in professional rugby league players. J Sci Med Sport. (2019) 22:653–60. doi: 10.1016/j.jsams.2018.11.028

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Akyildiz Z, Yildiz M, Clemente FM. The reliability and accuracy of polar team pro GPS units. Proc Inst Mech Eng P J Sports Eng Technol. (2020) 236:83–9. doi: 10.1177/1754337120976660

CrossRef Full Text | Google Scholar

5. Huggins RA, Giersch GEW, Belval LN, Benjamin CL, Curtis RM, Yasuki S, et al. The validity and reliability of global positioning system units for measuring distance and velocity during linear and team sport simulated movements. J Strength Cond Res. (2020) 34:3070–7. doi: 10.1519/JSC.0000000000003787

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Fox JL, O’grady CJ, Scanlan AT, Sargent C, Stanton R. Validity of the polar team pro sensor for measuring speed and distance indoors. J Sci Med Sport. (2019) 22:1260–5. doi: 10.1016/j.jsams.2019.06.012

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Balsalobre-Fernandez C, Kuzdub M, Poveda-Ortiz P, Campo-Vecino JD. Validity and reliability of the PUSH wearable device to measure movement velocity during the back squat exercise. J Strength Cond Res. (2016) 30:1968–74. doi: 10.1519/JSC.0000000000001284

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Van Den Tillaar R, Ball N. Validity and reliability of kinematics measured with PUSH band vs. linear encoder in bench press and push-ups. Sports. (2019) 7:207. doi: 10.3390/sports7090207

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates (1988).

10. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. (2016) 15:155–63. doi: 10.1016/j.jcm.2016.02.012

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Van Den Tillaar R, Nagahara R, Gleadhill S, Jiménez-Reyes P. Step-to-step kinematic validation between an inertial measurement unit (IMU) 3D system, a combined laser + IMU system and force plates during a 50 m sprint in a cohort of sprinters. Sensors. (2021) 21:6560. doi: 10.3390/s21196560

PubMed Abstract | CrossRef Full Text | Google Scholar