This article was downloaded by: [University of Tennessee, Knoxville] On: 09 September 2014, At: 08:08 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
European Journal of Sport Science Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/tejs20
Accuracy of an UWB-based position tracking system used for time-motion analyses in game sports a
a
b
Roland Leser , Armin Schleindlhuber , Keith Lyons & Arnold Baca a
a
Centre for Sport Science and University Sports, University of Vienna, Vienna, Austria
b
National Institute of Sports Studies, University of Canberra, Bruce, ACT, Australia Published online: 10 Feb 2014.
To cite this article: Roland Leser, Armin Schleindlhuber, Keith Lyons & Arnold Baca (2014): Accuracy of an UWBbased position tracking system used for time-motion analyses in game sports, European Journal of Sport Science, DOI: 10.1080/17461391.2014.884167 To link to this article: http://dx.doi.org/10.1080/17461391.2014.884167
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
European Journal of Sport Science, 2014 http://dx.doi.org/10.1080/17461391.2014.884167
ORIGINAL ARTICLE
Accuracy of an UWB-based position tracking system used for timemotion analyses in game sports
ROLAND LESER1, ARMIN SCHLEINDLHUBER1, KEITH LYONS2, & ARNOLD BACA1 Centre for Sport Science and University Sports, University of Vienna, Vienna, Austria, 2National Institute of Sports Studies, University of Canberra, Bruce, ACT, Australia
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
1
Abstract The main aim of this study was to determine the accuracy of the ultra-wideband (UWB)-based positioning system Ubisense, which is used for time-motion analysis in sports. Furthermore, some alternatives for positioning the system’s transponders on the atheletes, as well as the accuracy depending on the location of measurement, were tested. Therefore, in a pre-study, some basic issues were examined (measurement assumptions and consistency and location of the system’s transponder used for position detection), and position measurements at the borders and in the centre of a basketball field were performed. In the main study, 13 male basketball players (15.8 years ± 0.6; 187.9 height ± 3.4; 77.5 weight ± 3.7), equipped with a Ubisense transponder mounted on top of their heads, handled a trundle wheel during simulated match play. The players with the trundle wheel participated passively in the match by following one of the ten competing players. The distance measurements of the trundle wheel were used as reference values and compared to the Ubisense distance estimations. Best results were found with the measurements of a single mounted transponder on top of the athlete’s heads. No differences were detectable in the accuracy between measurements in the centre and at the borders of the basketball field. The (Ubisense) system’s difference to the (trundle wheel) reference was 3.45 ± 1.99%, resulting in 95% limits of agreement of −0.46–7.35%. The study indicates the examined system’s sufficient accuracy for time-motion analysis in basketball. Keywords: Distance measurement, accuracy test, player tracking, sports performance, performance analysis
Introduction In contemporary game sports, position tracking systems are currently used to perform time-motion and tactical analyses (Leser, Baca, & Ogris, 2011). By means of these systems, positional information is gained automatically or semi-automatically for research purposes and for practical performance analysis in sports. In order for a proper use of these data, testing the systems’ accuracy has become a crucial issue. Accuracy studies on position tracking systems have been undertaken for many sports in the last 10 years. Barbero-Alvarez, Coutts, Granda, Barbero-Alvarez, and Castagna (2010), Coutts and Duffield (2010), and Varley, Fairweather, and Aughey (2012) performed validity and reliability studies on position tracking systems for general sports purposes. Duffield, Reid, Baker, and Spratford (2010) tested the accuracy and reliability of Global Positioning System (GPS) devices for court-based sports. Roberts,
Trewartha, and Stokes (2006) compared the position data gained by a video system with the results of notational analyses in field-based sports. Gray, Jenkins, Andrews, Taaffe, and Glover (2010) and Jennings, Cormack, Coutts, Boyd, and Aughey (2010a) examined position data for team sports. A lot of other investigations have analysed the accuracy of position tracking systems for performances in specific sports (Australian Football: Edgecomb & Norton, 2006; Jennings, Cormack, Coutts, Boyd, & Aughey, 2010b. Cricket: Petersen, Pyne, Portus, Karppinen, & Dawson, 2009. Field Hockey: MacLeod, Morris, Nevill, & Sunderland, 2009. Football: Di Salvo, Collins, McNeill, & Cardinale, 2006; Frencken, Lemmink, & Delleman, 2010. Futsal: Dogramaci, Watsford, & Murphy, 2011. Handball: Perš, Bon, Kovacic, Sibila, & Dezman, 2002). All the mentioned papers analysed the accuracy of positioning systems, referring in particular to time-motion analysis. Though the movements performed in each sport and the requirements to time-motion analysis are specific, most of
Correspondence: R. Leser, Centre for Sport Science and University Sports, University of Vienna, Auf der Schmelz 6, A-1150 Vienna, Austria. E-mail: [email protected] © 2014 European College of Sport Science
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
2
R. Leser et al.
the authors claim that their results have general validity in game sports. The above-mentioned studies applied different kinds of methods to measure the accuracy of the positioning systems. Most often the distances calculated by the position tracking systems were compared to known distances of defined runs or standard circuits (Aughey & Falloon, 2010; Dogramaci et al., 2011; Duffield et al., 2010; Edgecomb & Norton, 2006; Gray et al., 2010; Jennings et al., 2010a, 2010b; Varley et al., 2012). For straight runs and for running courses, timing gates were sometimes used for time measurement (Barbero-Alvarez et al., 2010; Coutts & Duffield, 2010; Di Salvo et al., 2006; Frencken et al., 2010; MacLeod et al., 2009; Petersen et al., 2009; Roberts et al., 2006; Waldron, Worsfold, Twist, & Lamb, 2011). Hence, speeds can also be calculated to be used for validation. In general, there are two major problems with the method using defined running paths and timing gates. First, fixed running courses cannot be considered valid for real match play. Even if a course is deduced from real game play, it lacks situational context. As opposed to running courses in real match situations, movements occur depending on the current match constellation and in a random sequence. With this, a main drawback is the defined position of the course in the observation field. Although running courses can be performed at several locations on the field (e.g. in the middle, near the borders, etc.), testing free movements allocated across the whole court is much more valid. Second, due to the position estimation at concrete instants of time, there is an inaccuracy at crossing the start and finish lines of a running course. The lower the sample rate of a system, the higher is this inaccuracy by chance. In particular for short runs, the relative error for distance estimation can be very high. In order to avoid the drawbacks with running courses, several other techniques have been applied. Siegle, Stevens, and Lames (2012) and Varley et al. (2012) used laser devices to compare the actual position data with reference values for straight runs. Since tracking systems in game sports actually measure the position of the players, the comparison of each measured position with a corresponding reference value is the most accurate validation method. However, the measurements should not only be done for straight runs but also for other movements occurring in real match play. Therefore, Randers et al. (2010) compared the position data of four different systems used concurrently in actual game play. The shortcoming of this study was the lack of accurate reference data. Ogris et al. (2012) performed a validation study by means of predefined runs, small-sided games and a motion analysis system (Vicon) as a reference system. As a result,
actual position data were analysed by means of an accurate reference system and within a valid setting. The drawbacks of this method, however, were the high degree of effort needed to gain and process the reference data, the complex process for data synchronisation and the need for a very expensive reference system. Basically, there are three types of systems available for position tracking in game sports: GPS-based, video-based and radio wave-based devices. The lastmentioned alternative is the most accurate (Ogris et al., 2012). Radio wave-based systems generally work by means of ultra-wideband (UWB) signals, which can be detected very precisely. Athletes have to wear active UWB-sending transponders, which can thus be located by base stations mounted around the playing field. The main objective of this paper was to assess the accuracy of a radio wave-based location system (Ubisense, 2013) as it is actually applied by the authors for time-motion analysis. The system is already used in the sports domain (Connaghan et al., 2009; Leser, 2012; Mucchi, Trippi, & Carpini, 2010; Tynan et al., 2009), but comprehensive validation results are still pending. The study was performed in basketball because its court uses almost the full size available with the Ubisense set-up. Additionally, basketball is a sport with very quick and variable movements and is thus also suitable as a reference for many other game sports. Due to the main use of the Ubisense system for time-motion analysis in game sports, only the parameter of covered distance – but no actual position data – was examined for the study. Leser and Schleindlhuber (2012) published some rough results in a case study for the accuracy of the Ubisense system. Therein the question of the best location/position for wearing the Ubisense transponders (measurement device) was discussed, but no answer could be provided. Furthermore, accuracy issues depending on the area (borders of a field versus the centre) of measurements were discussed. Therefore, the secondary objectives for the current study were to find a suitable mounting alternative for the Ubisense transponders and to answer the question of uniform accuracy concerning the area of measurement. Methods The investigations for this paper were conducted twofold. In a pre-study, three options for estimating the position of athletes in basketball regarding the mounting location of the measurement device were tested. Additionally, the assumptions of hardware consistency and independency of the measurements regarding their length were examined. Furthermore,
Accuracy of an UWB-based position tracking system the pre-investigations included a case study analysing the system’s accuracy concerning measurements at certain field areas (borders versus centre). Based on the results of the pre-study, the main study contained an accuracy test for the system on multiple test persons. The study was approved by the ethics committee of the University of Vienna.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
Protocol: pre-study One healthy and moderately trained former basketball player (female: 158.0 cm, 58.4 kg and 29.3 years) conducted a running protocol consisting of 20 runs in one session. All runs were performed with a trundle wheel in order to accurately measure the distances. The results gained by the trundle wheel were used as reference values for the system validation. Additionally, the subject wore four tags (transmitters for position calculation) on top of her head and two tags on each of her shoulders. The runs had a duration between 30 and 180 seconds (101.48 ± 45.83 s) and simulated typical basketball movements, including varying speeds and changing directions. Two different field areas were distinguished: 10 runs were performed in the centre of the playing field and the other 10 runs at the borders (Figure 1). The trajectory of each single run was chosen by the subject and based on her experiences as a basketball player. After a sequence of four runs, a rest of five minutes was taken for recovery each time.
3
Protocol: main study An elite male youth basketball team performed two practice matches in two different training sessions one week apart from each other. The matches lasted 40 minutes overall and were conducted as five versus five plus one player leading a trundle wheel. Based on the insights of the pre-study (see also Results and Discussion sections) the test persons only wore one tag on top of their heads. Nine additional tags were worn by other players in order to simulate the Ubisense data collection for a complete basketball match. The player with the trundle wheel followed another player as accurately as possible but did not actively participate in the match (had no ball contact). After each four-minute time interval, the trundle wheel was passed to another player for the next four minutes. This resulted in 20 data-sets containing the position data and a reference value for the covered distance for each unit. Seven players led the trundle wheel in the first match as well as in the second. Another six players led it only in one of the two matches. Thus the 20 units of data resulted from a sample of 13 male players (15.8 years ± 0.6; 187.9 height ± 3.4; 77.5 weight ± 3.7).
The measurement systems In order to gain reference values for validating the distances estimated by the location system, a trundle wheel was used (Figure 2) based on the method of
Figure 1. Typical trajectories of one centre run (left) and one border run (right) simulating a centre player respectively wing player in basketball play.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
4
R. Leser et al.
Figure 2. The measurement systems (left: outline of the Ubisense set-up; right: trundle wheel ‘Mini Rollfix, BMI’, Hersbruck, Germany).
Edgecomb and Norton (2006). A trundle wheel is a device for measuring distances by means of a wheel whose circumference is exactly known. The wheel is led on the ground surface level by a handle which is attached to the axle allowing the wheel to be held easily. According to the manufacturer the inaccuracy of the applied trundle wheel is below 0.50%. The wheel’s accuracy was also examined by the authors using a protocol of 30 distances measured by a tape and ranging from 10 to 300 metres. These distances were gauged by means of the trundle wheel which resulted in a mean difference to the reference values (tape) of 0.30 ± 0.10% (min: 0.00%; max: 0.50%). The real-time location system (RTLS) Ubisense is a position tracking system primarily used in industry for location-driven manufacturing processes (Ubisense, 2013). The study was conducted with an Ubisense set-up using six installed fixed-location base stations. The base stations were mounted in each corner of a gym (Figure 2) and in the middle of the long side walls at a height of about 5 m. One of the sensors (base stations) acted as a master sensor, sending a conventional RF signal across the observation field. If a mobile tag (transmitter) in the observation field receives a signal from the master sensor, it answers by sending an UWB pulse. For the study, the transmitter of the type Ubisense Series 7000 Compact Tag was used (dimensions: 38 × 39 × 16.5 mm; weight: 25 g). The positions of the test person were calculated in 2D via the time-differences-of-arrival of the tag-UWB-signals between the base stations and via angle-of-arrival measurements at the base stations. The mean sampling rate for the position recording was 4.17 ± 0.01 Hz per tag. The raw data was post-processed via a combination of Kalman and low-pass filters. The filter settings were based on the experiences of the authors for getting the best results with the system. For the Kalman
filter, a ratio of 1:35 was used, and the low-pass filter was set at a smoothing factor of 0.35. Due to the consecutive operation mode of the Ubisense data acquisition, the position values were interpolated in order to receive pairs of x/y-coordinates of each tag for the same instances of time.
Data analysis: pre-study The data analysis for the pre-study was done in Microsoft Excel 2010 and SPSS version 14.0 for Windows. First, the consistency of the head tags and of the shoulder tags was examined by judging the variability of the distance measurements. Therefore, the percentage differences to the reference values of the trundle wheel measurements were calculated for each run and for each tag. Then, for each of the head tags the differences between each run and the mean value of all four tags of the same run were calculated and the 95% limits of agreement (95% LoA) by means of all 80 values (4 tags times 20 runs) were computed. Analogously the same was done for the shoulder tags. The 95% LoA (representing an appropriate bandwidth for the data noise between the tag measurements) were considered as fair criterion for quantifying the consistency of the RTLS tags. The tolerance thresholds for a proper consistency of the 95% LoA difference values of the head tag measurements and of the shoulder tag measurements were set at 0 ± 5%. Second, whether the relative differences (trundle wheel distances minus tag distances – expressed as percentage) are independent from the length of the runs, which is an assumption for the system’s reliability, was examined. These two parameters (relative differences and length of the runs) were correlated.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
Accuracy of an UWB-based position tracking system Finally, the Ubisense distance estimations based on the mean x/y-coordinates of two shoulder tags were compared to the trundle wheel values in order to prove whether shoulder tag pairs are suitable measurement alternatives to the head tag measurements. Thus, four additional distances based on mean x/y-coordinates of the shoulder tags were calculated: SP1 (left shoulder tag 1 and right shoulder tag 1), SP2 (left shoulder tag 1 and right shoulder tag 2), SP3 (left shoulder tag 2 and right shoulder tag 1) and SP4 (left shoulder tag 2 and right shoulder tag 2). Based on O’Donoghue (2010) as the main accuracy parameter (or range of accuracy), the 95% limits of agreement (95% LoA) for the differences between the Ubisense distance estimations and the trundle wheel references were calculated. As tolerance thresholds for the system’s accuracy ±10% for the 95% LoA were applied (see Discussion section). An independent Student’s t-test was used to check if there are differences in the accuracy between the centre and the border runs.
5
independence between the system’s measurement error and the length of the distances was given. No differences were found in the accuracy between the centre run and border run measurements (P = 0.052). Results: main study. The mean distance measured by the trundle wheel was 418.92 ± 47.55 m, those of the Ubisense system 404.22 ± 43.95. This is a difference of 14.70 ± 9.29 m or 3.45 ± 1.99% resulting in 95% LoA of −0.46 to 7.35%, which is within the determined tolerance thresholds of ±10% for the 95% LoA. Table I gives an overview of all 20 data units. Due to the standard deviation of the measurement differences being smaller than the mean value, the mean difference can be considered as systematic bias. Thus, subtracting the mean difference (3.45%) from all measurements resulted in 95% LoA of −3.91–3.91%.
Discussion Data analysis: main study The data analysis for the main study was done in Microsoft Excel 2010. For all 20 units of data (Ubisense distance estimations plus trundle wheel measurements per four minutes time interval), the total differences (in metres) and relative differences (in percent) between the tags and the reference values of the trundle wheel were computed. In keeping with the pre-study (examination of the shoulder tag pair data), the difference values were expected to range within ±10% for the 95% LoA. Results Results: pre-study The 95% LoA for the distances of the head tag measurements ranged from −1.14% to 1.14%, and those of the shoulder tags between −8.87% and 8.87%. Thus, the head tag estimations show a very high consistency, while the shoulder tag values are beyond the tolerance thresholds (±5% difference). The distance values based on the mean coordinates of two shoulder tags had a difference to the trundle wheel measurements of 8.25 ± 4.07% (95% LoA: 0.27–16.22%) and thus were beyond the stated accuracy limits. The comparison of the relative differences of the trundle wheel and Ubisense values and the length of the runs resulted in a Pearson’s r correlation of 0.058 (P = 0.810) and hence, showed no significance. Therefore, the compliance with the requirement of
The results of the main study prove sufficient accuracy of the tested position tracking system in order to perform time-motion analysis in basketball. The pre-study indicated that the accuracy of the system is not dependent on the length of the recorded runs and that the most useful position for Table I. Results of the main study Test person
Trundle (m)
Ubisense (m)
Difference (m)
Difference (%)
Player_01 Player_02 Player_03 Player_04 Player_05 Player_06 Player_07 Player_08 Player_09 Player_10 Player_01 Player_12 Player_02 Player_14 Player_09 Player_04 Player_17 Player_08 Player_05 Player_03 Mean SD
455.30 463.80 520.20 410.00 375.00 399.90 390.10 475.10 345.60 460.00 414.50 392.60 437.30 488.30 445.60 373.80 387.10 394.80 344.50 404.9. 418.92 47.55
443.18 454.59 480.88 401.80 362.38 375.34 387.40 469.18 341.50 439.08 402.66 383.03 411.62 469.61 426.59 364.64 385.17 378.65 325.91 381.20 404.22 43.95 LoA_ − LoA_ +
12.12 9.21 39.32 8.20 12.62 24.56 2.70 5.92 4.10 20.92 11.84 9.57 25.68 18.69 19.01 9.16 1.93 16.15 18.59 23.70 14.70 9.29 −3.51 32.91
2.66 1.99 7.56 2.00 3.37 6.14 0.69 1.25 1.19 4.55 2.86 2.44 5.87 3.83 4.27 2.45 0.50 4.09 5.40 5.85 3.45 1.99 −0.46 7.35
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
6
R. Leser et al.
wearing the system’s transponders is on top of the heads of the athletes. Furthermore, no differences could be found in the accuracy between measurements performed in the centre and those performed at the borders of the playing field. The pre-study was performed in order to settle three issues. First, whether the percentage measurement error of the tested system changes with the duration/length of the measurements was examined. Due to the non-significant correlation between the relative Ubisense differences from the trundle wheel measurements and the duration of the runs, this requirement was confirmed. Second, the wearing of the Ubisense tags at the shoulders instead of the common position on top of the heads of the players was tested. Wearing the tags at the shoulders would be less intrusive and allow playing the ball with the head (e.g. in football). However, the pre-study indicated no sufficient consistency if the position data was gathered by means of one shoulder tag. If two tags are used (one tag at each of both shoulders) some additional information (e.g. line of sight and rotations) could be calculated automatically. Nevertheless, due to the fact that the system’s full sample rate is shared among all active tags, this would result in a bisection of the available sample rate per tag. Regarding the set-up of the main study, this would result in a sample rate per tag of about 2 Hz, which is considered as too low for accurate time-motion analysis. Third, differences in the accuracy between measurements performed in the centre and at the borders of the playing field were analysed. One could argue that the system’s accuracy might be lower at the borders of the playing field because the line of sight to the sensors is worse than in the centre. If this were the case, measurements for certain playing positions (e.g. wing players) would be systematically affected. However, no significant differences in the accuracy between both measurement areas were found. As a consequence of the pre-study’s results, the main study was performed with one head tag’s measurements not distinguishing between certain measurement areas. For the assessment of the system’s accuracy, in the main study accuracy thresholds had to be fixed. Referring to Edgecomb and Norton (2006), who declared their results of about 5% distance differences of the analysed GPS tracking system to a reference system as sufficient, this study also took into account a mean distance difference of 5%. But, considering that individual values can vary fairly strongly, in spite of the mean distance difference parameter, the 95% limits of agreement were used as a fair accuracy parameter (O’Donoghue, 2010). Since this parameter is a considerably stricter criterion for accuracy than the mean deviation, the
thresholds were determined with the mean difference ± 10%. This is in line with the statement of Dawson (2012) that for time-motion analysis the accuracy of a positioning system should be below 10% difference to the real value. The results of the study indicate that the accuracy of the Ubisense system is within these thresholds. Considering the systematic bias of the Ubisense measurements, the differences to the reference values are even below ±5% in the 95% LoA. However, strictly speaking the systematic bias is only suitable if the same value can be used for ongoing measurements and not only for the measurement where the bias was determined. This requirement has not been proven so far. A comparison of the current Ubisense accuracy results with other studies testing similar systems is very difficult because of the lack of standardised methods. Therefore, standards for testing positioning systems including uniform parameters for presenting the results should be developed (see below). However, Edgecomb and Norton (2006) report in a comparable study mean distance differences of 4.8% for a GPS system and 5.8% for a video-based system. This indicates that Ubisense outperforms these types of systems. The results of Ogris et al. (2012) suppose better results by radio wave-based location systems with higher sample rates than the Ubisense system. A key methodological issue of the study was the choice of a suitable reference system and proper accuracy parameters, respectively. Therefore the purpose of the tested system is a key question. If highly accurate position data (x/y- or x/y/z-coordinates) is necessary, then actual position data gained by the system has to be considered for the accuracy tests (Ogris et al., 2012). If a system is used for timemotion analysis, and measuring parameters like distances or speeds, then it is sufficient to check the accuracy of those parameters. This can be less labour-intensive than methods comparing each position measurement with an adequate reference value. Consequently, this study focused on covered distances, which also enables assessments of mean speeds in the case of time-motion analysis. For the comparison with exact distance values, a trundle wheel was used. Due to its low costs and high accuracy, this was considered as proper reference system. For the assessment of the accuracy of the position tracking system a valid, very easy-to-handle and comprehensible method was used. Considering the shortcomings of other studies (see Introduction), this method could also be used for future studies examining other location systems applied for timemotion analysis. In particular, the method accomplished the following requirements:
Accuracy of an UWB-based position tracking system 1. 2. 3. 4.
5.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
6.
7.
The method was comprehensible and repeatable. The reference values were very accurate. The data between the several measurement devices (tags) were consistent (see above). The method examined the dependency between the duration/lengths of the observed units and the size of error in order to identify a possible ‘shotgun effect’ (see above). Gaining and post-processing the test data as well as the reference data were low cost and time intensive. The movement data was gathered directly from (simulated) match play and therefore has high validity. The measurements did not focus on a specific area of the pitch but covered the whole field – including the border areas, where the highest measurement errors often occur (Ogris et al., 2012). The assumption of uniform accuracy concerning border versus centre field measurements was confirmed before.
The main limitation of the applied method, however, was that only covered distances and their derivates (covered distance per time period, average speed, etc.) could be examined. Whilst the related work (see Introduction section) shows that this is most often sufficient, specific analyses regarding the position data (x/y/z-coordinates) were not possible. Other limitations of the study were as follows: 1.
2.
3.
4.
Driving the trundle wheel during the runs interfered with the players, in particular, when performing fast path or speed changes. Due to its fixed installation, the system could only be tested indoors and with the given set-up. The study was performed for the sport of basketball. Although similar results can be expected for other sports, there is no evidence that the system performs equally for other movements. The study neither examined specific circumstances, like athlete interaction (duels) or different movements (e.g. falling down and jumping) nor did it consider different conditions like effects of temperature, pitch size, or potential radio interferences on the signals.
This paper basically addressed the problem of accuracy regarding position tracking systems in sports. Various types of these systems are currently used in the sports domain in order to perform time-motion analyses. Therefore, a key issue is the proper assessment of the accuracy of those devices. The current
7
study investigated one of the few available radio wavebased location systems for sports and confirmed sufficient accuracy for the application in basketball. Moreover, in order to make accuracy studies of different systems comparable, the authors suggested the method used for this study as a standard for accuracy tests of location systems for game sports. References Aughey, R. J., & Falloon, C. (2010). Real-time versus post-game GPS data in team sports. Journal of Science and Medicine in Sport, 13, 348–349. doi:10.1016/j.jsams.2009.01.006 Barbero-Alvarez, J. C., Coutts, A., Granda, J., Barbero-Alvarez, V., & Castagna, C. (2010). The validity and reliability of a global positioning satellite system device to assess speed and repeated sprint ability (RSA) in athletes. Journal of Science and Medicine in Sport, 13, 232–235. doi:10.1016/j.jsams.2009.02.005 Connaghan, D., Hughes, S., May, G., Kelly, P., Conaire, C., O’Connor, N., … Moyna, N. (2009). A sensing platform for physiological and contextual feedback to tennis athletes. In Proceedings of the 2009 sixth international workshop on wearable and implantable body sensor network (pp. 224–229). Washington, DC: IEEE Computer Society. Coutts, A., & Duffield, R. (2010). Validity and reliability of GPS devices for measuring movement demands of team sports. Journal of Science and Medicine in Sport, 13, 133–135. doi:10.1016/j.jsams.2008.09.015 Dawson, B. (2012, July 25–28). Movement patterns in team sports: How do they relate to performance? Presentation at the World Congress of Performance Analysis of Sport IX, Worcester, UK. Di Salvo, V., Collins, A., McNeill, B., & Cardinale, M. (2006). Validation of prozone: A new video-based performance analysis system. International Journal of Performance Analysis in Sport, 6, 108–119. Retrieved from http://www.ingentaconnect.com/con tent/uwic/ujpa/2006/00000006/00000001/art00011 Dogramaci, S. N., Watsford, M. L., & Murphy, A. J. (2011). The reliability and validity of subjective notational analysis in comparison to global positioning system tracking to assess athlete movement patterns. Journal of Strength and Conditioning Research, 25, 852–859. doi:10.1519/JSC.0b013e3181c69edd Duffield, R., Reid, M., Baker, J., & Spratford, W. (2010). Accuracy and reliability of GPS devices for measurement of movement patterns in confined spaces for court-based sports. Journal of Science and Medicine in Sport, 13, 523–525. doi:10.1016/j.jsams.2009.07.003 Edgecomb, S. J., & Norton, K. I. (2006). Comparison of global positioning and computer-based tracking systems for measuring player movement distance during Australian football. Journal of Science and Medicine Sport, 9, 25–32. Retrieved from http://www.sciencedirect.com/science/article/pii/S144024 4006000053 Frencken, W. G. P., Lemmink, K., & Delleman, N. (2010). Soccer-specific accuracy and validity of the local position measurement (LPM) system. Journal of Science Medicine and Sport, 13, 641–645. doi:10.1016/j.jsams.2010.04.003 Gray, A. J., Jenkins, D., Andrews, M. H., Taaffe, D. R., & Glover, M. L. (2010). Validity and reliability of GPS for measuring distance travelled in field-based team sports. Journal of Sports Sciences, 28, 1319–1325. doi:10.1080/02640414.2010.504783 Jennings, D., Cormack, S., Coutts, A. J., Boyd, L., & Aughey, R. J. (2010a). The validity and reliability of GPS units for measuring distance in team sport specific running patterns. International Journal of Sports Physiology and Performance, 5, 328–341. Jennings, D., Cormack, S., Coutts, A. J., Boyd, L., & Aughey, R. J. (2010b). Variability of GPS units for measuring distance in
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
8
R. Leser et al.
team sport movements. International Journal of Sports Physiology and Performance, 5, 565–569. Leser, R. (2012). A wireless position tracking system for measuring sports performance in game sports. In R. Vaeyens (Ed.), Book of abstracts of the 3rd world conference on science and soccer (p. 68). Ghent: University Press. Leser, R., Baca, A., & Ogris, G. (2011). Local positioning systems in (game) sports. Sensors, 11, 9778–9797. doi:10.3390/s111 009778 Leser, R., & Schleindlhuber, A. (2012). Accuracy test of a wireless position tracking system for measuring sports activities. In Y. Jiang & A. Baca (Eds.), Proceedings of the 2012 pre-Olympic congress on sports science and computer science in sport (pp. 73–74). Liverpool: World Academic Press. MacLeod, H., Morris, J., Nevill, A., & Sunderland, C. (2009). The validity of a nondifferential global positioning system for assessing player movement patterns in field hockey. Journal of Sports Sciences, 27, 121–128. doi:10.1080/02640410802422181 Mucchi, L., Trippi, F., & Carpini, A. (2010). Ultra wide band real-time location system for cinematic survey in sports. In Proceedings of the 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (pp. 1–6). New York: IEEE. O’Donoghue, P. (2010). Research methods for sports performance analysis. London: Routledge. Ogris, G., Leser, R., Horsak, B., Kornfeind, P., Heller, M., & Baca, A. (2012). Accuracy of the LPM tracking system considering dynamic position changes. Journal of Sports Sciences, 30, 1503–1511. doi:10.1080/02640414.2012.712712 Perš, J., Bon, M., Kovacic, S., Šibila, M., & Dežman, B. (2002). Observation and analysis of large-scale human motion. Human Movement Science, 21, 295–311. doi:10.1016/S0167-9457(02) 00096-9
Petersen, C. J., Pyne, D. B., Portus, M. R., Karppinen, S., & Dawson, B. (2009). Variability in movement patterns during one day internationals by a cricket fast bowler. International Journal of Sports Physiology and Performance, 4, 278–281. Randers, M., Mujika, I., Hewitt, A., Santisteban, J., Bischoff, R., Solano, R., …, Mohr, M. (2010). Application of four different football match analysis systems: A comparative study. Journal of Sports Sciences, 28, 171–182. doi:10.1080/02640410903428525 Roberts, S., Trewartha, G., & Stokes, K. (2006). A comparison of time-motion analysis methods for field-based sports. International Journal of Sports Physiology and Performance, 1, 388–399. Siegle, M., Stevens, T., & Lames, M. (2012). Design of an accuracy study for position detection in football. Journal of Sports Sciences, 31, 166–172. doi:10.1080/02640414.2012. 723131 Tynan, R., Schoofs, A., Muldoon, C., O’Hare, G., Ó Conaire, C., Kelly, P., & O’Connor, N. (2009). Intelligent middleware for adaptive sensing of tennis coaching sessions. In Proceedings of the 2nd international workshop on adaptation in wireless sensor networks (pp. 891–896). Vancouver, Canada. Ubisense. (2013,). Ubisense. Retrieved May 21, from http://www. ubisense.net/en Varley, M. C., Fairweather, I. H., & Aughey, R. J. (2012). Validity and reliability of GPS for measuring instantaneous velocity during acceleration, deceleration, and constant motion. Journal of Sports Sciences, 30, 121–127. doi:10.1080/ 02640414.2011.627941 Waldron, M., Worsfold, P., Twist, C., & Lamb, K. (2011). Concurrent validity and test retest reliability of a Global Positioning System (GPS) and timing gates to assess sprint performance variables. Journal of Sports Sciences, 29, 1613– 1619. doi:10.1080/02640414.2011.608703
European Journal of Sport Science Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/tejs20
Accuracy of an UWB-based position tracking system used for time-motion analyses in game sports a
a
b
Roland Leser , Armin Schleindlhuber , Keith Lyons & Arnold Baca a
a
Centre for Sport Science and University Sports, University of Vienna, Vienna, Austria
b
National Institute of Sports Studies, University of Canberra, Bruce, ACT, Australia Published online: 10 Feb 2014.
To cite this article: Roland Leser, Armin Schleindlhuber, Keith Lyons & Arnold Baca (2014): Accuracy of an UWBbased position tracking system used for time-motion analyses in game sports, European Journal of Sport Science, DOI: 10.1080/17461391.2014.884167 To link to this article: http://dx.doi.org/10.1080/17461391.2014.884167
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
European Journal of Sport Science, 2014 http://dx.doi.org/10.1080/17461391.2014.884167
ORIGINAL ARTICLE
Accuracy of an UWB-based position tracking system used for timemotion analyses in game sports
ROLAND LESER1, ARMIN SCHLEINDLHUBER1, KEITH LYONS2, & ARNOLD BACA1 Centre for Sport Science and University Sports, University of Vienna, Vienna, Austria, 2National Institute of Sports Studies, University of Canberra, Bruce, ACT, Australia
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
1
Abstract The main aim of this study was to determine the accuracy of the ultra-wideband (UWB)-based positioning system Ubisense, which is used for time-motion analysis in sports. Furthermore, some alternatives for positioning the system’s transponders on the atheletes, as well as the accuracy depending on the location of measurement, were tested. Therefore, in a pre-study, some basic issues were examined (measurement assumptions and consistency and location of the system’s transponder used for position detection), and position measurements at the borders and in the centre of a basketball field were performed. In the main study, 13 male basketball players (15.8 years ± 0.6; 187.9 height ± 3.4; 77.5 weight ± 3.7), equipped with a Ubisense transponder mounted on top of their heads, handled a trundle wheel during simulated match play. The players with the trundle wheel participated passively in the match by following one of the ten competing players. The distance measurements of the trundle wheel were used as reference values and compared to the Ubisense distance estimations. Best results were found with the measurements of a single mounted transponder on top of the athlete’s heads. No differences were detectable in the accuracy between measurements in the centre and at the borders of the basketball field. The (Ubisense) system’s difference to the (trundle wheel) reference was 3.45 ± 1.99%, resulting in 95% limits of agreement of −0.46–7.35%. The study indicates the examined system’s sufficient accuracy for time-motion analysis in basketball. Keywords: Distance measurement, accuracy test, player tracking, sports performance, performance analysis
Introduction In contemporary game sports, position tracking systems are currently used to perform time-motion and tactical analyses (Leser, Baca, & Ogris, 2011). By means of these systems, positional information is gained automatically or semi-automatically for research purposes and for practical performance analysis in sports. In order for a proper use of these data, testing the systems’ accuracy has become a crucial issue. Accuracy studies on position tracking systems have been undertaken for many sports in the last 10 years. Barbero-Alvarez, Coutts, Granda, Barbero-Alvarez, and Castagna (2010), Coutts and Duffield (2010), and Varley, Fairweather, and Aughey (2012) performed validity and reliability studies on position tracking systems for general sports purposes. Duffield, Reid, Baker, and Spratford (2010) tested the accuracy and reliability of Global Positioning System (GPS) devices for court-based sports. Roberts,
Trewartha, and Stokes (2006) compared the position data gained by a video system with the results of notational analyses in field-based sports. Gray, Jenkins, Andrews, Taaffe, and Glover (2010) and Jennings, Cormack, Coutts, Boyd, and Aughey (2010a) examined position data for team sports. A lot of other investigations have analysed the accuracy of position tracking systems for performances in specific sports (Australian Football: Edgecomb & Norton, 2006; Jennings, Cormack, Coutts, Boyd, & Aughey, 2010b. Cricket: Petersen, Pyne, Portus, Karppinen, & Dawson, 2009. Field Hockey: MacLeod, Morris, Nevill, & Sunderland, 2009. Football: Di Salvo, Collins, McNeill, & Cardinale, 2006; Frencken, Lemmink, & Delleman, 2010. Futsal: Dogramaci, Watsford, & Murphy, 2011. Handball: Perš, Bon, Kovacic, Sibila, & Dezman, 2002). All the mentioned papers analysed the accuracy of positioning systems, referring in particular to time-motion analysis. Though the movements performed in each sport and the requirements to time-motion analysis are specific, most of
Correspondence: R. Leser, Centre for Sport Science and University Sports, University of Vienna, Auf der Schmelz 6, A-1150 Vienna, Austria. E-mail: [email protected] © 2014 European College of Sport Science
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
2
R. Leser et al.
the authors claim that their results have general validity in game sports. The above-mentioned studies applied different kinds of methods to measure the accuracy of the positioning systems. Most often the distances calculated by the position tracking systems were compared to known distances of defined runs or standard circuits (Aughey & Falloon, 2010; Dogramaci et al., 2011; Duffield et al., 2010; Edgecomb & Norton, 2006; Gray et al., 2010; Jennings et al., 2010a, 2010b; Varley et al., 2012). For straight runs and for running courses, timing gates were sometimes used for time measurement (Barbero-Alvarez et al., 2010; Coutts & Duffield, 2010; Di Salvo et al., 2006; Frencken et al., 2010; MacLeod et al., 2009; Petersen et al., 2009; Roberts et al., 2006; Waldron, Worsfold, Twist, & Lamb, 2011). Hence, speeds can also be calculated to be used for validation. In general, there are two major problems with the method using defined running paths and timing gates. First, fixed running courses cannot be considered valid for real match play. Even if a course is deduced from real game play, it lacks situational context. As opposed to running courses in real match situations, movements occur depending on the current match constellation and in a random sequence. With this, a main drawback is the defined position of the course in the observation field. Although running courses can be performed at several locations on the field (e.g. in the middle, near the borders, etc.), testing free movements allocated across the whole court is much more valid. Second, due to the position estimation at concrete instants of time, there is an inaccuracy at crossing the start and finish lines of a running course. The lower the sample rate of a system, the higher is this inaccuracy by chance. In particular for short runs, the relative error for distance estimation can be very high. In order to avoid the drawbacks with running courses, several other techniques have been applied. Siegle, Stevens, and Lames (2012) and Varley et al. (2012) used laser devices to compare the actual position data with reference values for straight runs. Since tracking systems in game sports actually measure the position of the players, the comparison of each measured position with a corresponding reference value is the most accurate validation method. However, the measurements should not only be done for straight runs but also for other movements occurring in real match play. Therefore, Randers et al. (2010) compared the position data of four different systems used concurrently in actual game play. The shortcoming of this study was the lack of accurate reference data. Ogris et al. (2012) performed a validation study by means of predefined runs, small-sided games and a motion analysis system (Vicon) as a reference system. As a result,
actual position data were analysed by means of an accurate reference system and within a valid setting. The drawbacks of this method, however, were the high degree of effort needed to gain and process the reference data, the complex process for data synchronisation and the need for a very expensive reference system. Basically, there are three types of systems available for position tracking in game sports: GPS-based, video-based and radio wave-based devices. The lastmentioned alternative is the most accurate (Ogris et al., 2012). Radio wave-based systems generally work by means of ultra-wideband (UWB) signals, which can be detected very precisely. Athletes have to wear active UWB-sending transponders, which can thus be located by base stations mounted around the playing field. The main objective of this paper was to assess the accuracy of a radio wave-based location system (Ubisense, 2013) as it is actually applied by the authors for time-motion analysis. The system is already used in the sports domain (Connaghan et al., 2009; Leser, 2012; Mucchi, Trippi, & Carpini, 2010; Tynan et al., 2009), but comprehensive validation results are still pending. The study was performed in basketball because its court uses almost the full size available with the Ubisense set-up. Additionally, basketball is a sport with very quick and variable movements and is thus also suitable as a reference for many other game sports. Due to the main use of the Ubisense system for time-motion analysis in game sports, only the parameter of covered distance – but no actual position data – was examined for the study. Leser and Schleindlhuber (2012) published some rough results in a case study for the accuracy of the Ubisense system. Therein the question of the best location/position for wearing the Ubisense transponders (measurement device) was discussed, but no answer could be provided. Furthermore, accuracy issues depending on the area (borders of a field versus the centre) of measurements were discussed. Therefore, the secondary objectives for the current study were to find a suitable mounting alternative for the Ubisense transponders and to answer the question of uniform accuracy concerning the area of measurement. Methods The investigations for this paper were conducted twofold. In a pre-study, three options for estimating the position of athletes in basketball regarding the mounting location of the measurement device were tested. Additionally, the assumptions of hardware consistency and independency of the measurements regarding their length were examined. Furthermore,
Accuracy of an UWB-based position tracking system the pre-investigations included a case study analysing the system’s accuracy concerning measurements at certain field areas (borders versus centre). Based on the results of the pre-study, the main study contained an accuracy test for the system on multiple test persons. The study was approved by the ethics committee of the University of Vienna.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
Protocol: pre-study One healthy and moderately trained former basketball player (female: 158.0 cm, 58.4 kg and 29.3 years) conducted a running protocol consisting of 20 runs in one session. All runs were performed with a trundle wheel in order to accurately measure the distances. The results gained by the trundle wheel were used as reference values for the system validation. Additionally, the subject wore four tags (transmitters for position calculation) on top of her head and two tags on each of her shoulders. The runs had a duration between 30 and 180 seconds (101.48 ± 45.83 s) and simulated typical basketball movements, including varying speeds and changing directions. Two different field areas were distinguished: 10 runs were performed in the centre of the playing field and the other 10 runs at the borders (Figure 1). The trajectory of each single run was chosen by the subject and based on her experiences as a basketball player. After a sequence of four runs, a rest of five minutes was taken for recovery each time.
3
Protocol: main study An elite male youth basketball team performed two practice matches in two different training sessions one week apart from each other. The matches lasted 40 minutes overall and were conducted as five versus five plus one player leading a trundle wheel. Based on the insights of the pre-study (see also Results and Discussion sections) the test persons only wore one tag on top of their heads. Nine additional tags were worn by other players in order to simulate the Ubisense data collection for a complete basketball match. The player with the trundle wheel followed another player as accurately as possible but did not actively participate in the match (had no ball contact). After each four-minute time interval, the trundle wheel was passed to another player for the next four minutes. This resulted in 20 data-sets containing the position data and a reference value for the covered distance for each unit. Seven players led the trundle wheel in the first match as well as in the second. Another six players led it only in one of the two matches. Thus the 20 units of data resulted from a sample of 13 male players (15.8 years ± 0.6; 187.9 height ± 3.4; 77.5 weight ± 3.7).
The measurement systems In order to gain reference values for validating the distances estimated by the location system, a trundle wheel was used (Figure 2) based on the method of
Figure 1. Typical trajectories of one centre run (left) and one border run (right) simulating a centre player respectively wing player in basketball play.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
4
R. Leser et al.
Figure 2. The measurement systems (left: outline of the Ubisense set-up; right: trundle wheel ‘Mini Rollfix, BMI’, Hersbruck, Germany).
Edgecomb and Norton (2006). A trundle wheel is a device for measuring distances by means of a wheel whose circumference is exactly known. The wheel is led on the ground surface level by a handle which is attached to the axle allowing the wheel to be held easily. According to the manufacturer the inaccuracy of the applied trundle wheel is below 0.50%. The wheel’s accuracy was also examined by the authors using a protocol of 30 distances measured by a tape and ranging from 10 to 300 metres. These distances were gauged by means of the trundle wheel which resulted in a mean difference to the reference values (tape) of 0.30 ± 0.10% (min: 0.00%; max: 0.50%). The real-time location system (RTLS) Ubisense is a position tracking system primarily used in industry for location-driven manufacturing processes (Ubisense, 2013). The study was conducted with an Ubisense set-up using six installed fixed-location base stations. The base stations were mounted in each corner of a gym (Figure 2) and in the middle of the long side walls at a height of about 5 m. One of the sensors (base stations) acted as a master sensor, sending a conventional RF signal across the observation field. If a mobile tag (transmitter) in the observation field receives a signal from the master sensor, it answers by sending an UWB pulse. For the study, the transmitter of the type Ubisense Series 7000 Compact Tag was used (dimensions: 38 × 39 × 16.5 mm; weight: 25 g). The positions of the test person were calculated in 2D via the time-differences-of-arrival of the tag-UWB-signals between the base stations and via angle-of-arrival measurements at the base stations. The mean sampling rate for the position recording was 4.17 ± 0.01 Hz per tag. The raw data was post-processed via a combination of Kalman and low-pass filters. The filter settings were based on the experiences of the authors for getting the best results with the system. For the Kalman
filter, a ratio of 1:35 was used, and the low-pass filter was set at a smoothing factor of 0.35. Due to the consecutive operation mode of the Ubisense data acquisition, the position values were interpolated in order to receive pairs of x/y-coordinates of each tag for the same instances of time.
Data analysis: pre-study The data analysis for the pre-study was done in Microsoft Excel 2010 and SPSS version 14.0 for Windows. First, the consistency of the head tags and of the shoulder tags was examined by judging the variability of the distance measurements. Therefore, the percentage differences to the reference values of the trundle wheel measurements were calculated for each run and for each tag. Then, for each of the head tags the differences between each run and the mean value of all four tags of the same run were calculated and the 95% limits of agreement (95% LoA) by means of all 80 values (4 tags times 20 runs) were computed. Analogously the same was done for the shoulder tags. The 95% LoA (representing an appropriate bandwidth for the data noise between the tag measurements) were considered as fair criterion for quantifying the consistency of the RTLS tags. The tolerance thresholds for a proper consistency of the 95% LoA difference values of the head tag measurements and of the shoulder tag measurements were set at 0 ± 5%. Second, whether the relative differences (trundle wheel distances minus tag distances – expressed as percentage) are independent from the length of the runs, which is an assumption for the system’s reliability, was examined. These two parameters (relative differences and length of the runs) were correlated.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
Accuracy of an UWB-based position tracking system Finally, the Ubisense distance estimations based on the mean x/y-coordinates of two shoulder tags were compared to the trundle wheel values in order to prove whether shoulder tag pairs are suitable measurement alternatives to the head tag measurements. Thus, four additional distances based on mean x/y-coordinates of the shoulder tags were calculated: SP1 (left shoulder tag 1 and right shoulder tag 1), SP2 (left shoulder tag 1 and right shoulder tag 2), SP3 (left shoulder tag 2 and right shoulder tag 1) and SP4 (left shoulder tag 2 and right shoulder tag 2). Based on O’Donoghue (2010) as the main accuracy parameter (or range of accuracy), the 95% limits of agreement (95% LoA) for the differences between the Ubisense distance estimations and the trundle wheel references were calculated. As tolerance thresholds for the system’s accuracy ±10% for the 95% LoA were applied (see Discussion section). An independent Student’s t-test was used to check if there are differences in the accuracy between the centre and the border runs.
5
independence between the system’s measurement error and the length of the distances was given. No differences were found in the accuracy between the centre run and border run measurements (P = 0.052). Results: main study. The mean distance measured by the trundle wheel was 418.92 ± 47.55 m, those of the Ubisense system 404.22 ± 43.95. This is a difference of 14.70 ± 9.29 m or 3.45 ± 1.99% resulting in 95% LoA of −0.46 to 7.35%, which is within the determined tolerance thresholds of ±10% for the 95% LoA. Table I gives an overview of all 20 data units. Due to the standard deviation of the measurement differences being smaller than the mean value, the mean difference can be considered as systematic bias. Thus, subtracting the mean difference (3.45%) from all measurements resulted in 95% LoA of −3.91–3.91%.
Discussion Data analysis: main study The data analysis for the main study was done in Microsoft Excel 2010. For all 20 units of data (Ubisense distance estimations plus trundle wheel measurements per four minutes time interval), the total differences (in metres) and relative differences (in percent) between the tags and the reference values of the trundle wheel were computed. In keeping with the pre-study (examination of the shoulder tag pair data), the difference values were expected to range within ±10% for the 95% LoA. Results Results: pre-study The 95% LoA for the distances of the head tag measurements ranged from −1.14% to 1.14%, and those of the shoulder tags between −8.87% and 8.87%. Thus, the head tag estimations show a very high consistency, while the shoulder tag values are beyond the tolerance thresholds (±5% difference). The distance values based on the mean coordinates of two shoulder tags had a difference to the trundle wheel measurements of 8.25 ± 4.07% (95% LoA: 0.27–16.22%) and thus were beyond the stated accuracy limits. The comparison of the relative differences of the trundle wheel and Ubisense values and the length of the runs resulted in a Pearson’s r correlation of 0.058 (P = 0.810) and hence, showed no significance. Therefore, the compliance with the requirement of
The results of the main study prove sufficient accuracy of the tested position tracking system in order to perform time-motion analysis in basketball. The pre-study indicated that the accuracy of the system is not dependent on the length of the recorded runs and that the most useful position for Table I. Results of the main study Test person
Trundle (m)
Ubisense (m)
Difference (m)
Difference (%)
Player_01 Player_02 Player_03 Player_04 Player_05 Player_06 Player_07 Player_08 Player_09 Player_10 Player_01 Player_12 Player_02 Player_14 Player_09 Player_04 Player_17 Player_08 Player_05 Player_03 Mean SD
455.30 463.80 520.20 410.00 375.00 399.90 390.10 475.10 345.60 460.00 414.50 392.60 437.30 488.30 445.60 373.80 387.10 394.80 344.50 404.9. 418.92 47.55
443.18 454.59 480.88 401.80 362.38 375.34 387.40 469.18 341.50 439.08 402.66 383.03 411.62 469.61 426.59 364.64 385.17 378.65 325.91 381.20 404.22 43.95 LoA_ − LoA_ +
12.12 9.21 39.32 8.20 12.62 24.56 2.70 5.92 4.10 20.92 11.84 9.57 25.68 18.69 19.01 9.16 1.93 16.15 18.59 23.70 14.70 9.29 −3.51 32.91
2.66 1.99 7.56 2.00 3.37 6.14 0.69 1.25 1.19 4.55 2.86 2.44 5.87 3.83 4.27 2.45 0.50 4.09 5.40 5.85 3.45 1.99 −0.46 7.35
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
6
R. Leser et al.
wearing the system’s transponders is on top of the heads of the athletes. Furthermore, no differences could be found in the accuracy between measurements performed in the centre and those performed at the borders of the playing field. The pre-study was performed in order to settle three issues. First, whether the percentage measurement error of the tested system changes with the duration/length of the measurements was examined. Due to the non-significant correlation between the relative Ubisense differences from the trundle wheel measurements and the duration of the runs, this requirement was confirmed. Second, the wearing of the Ubisense tags at the shoulders instead of the common position on top of the heads of the players was tested. Wearing the tags at the shoulders would be less intrusive and allow playing the ball with the head (e.g. in football). However, the pre-study indicated no sufficient consistency if the position data was gathered by means of one shoulder tag. If two tags are used (one tag at each of both shoulders) some additional information (e.g. line of sight and rotations) could be calculated automatically. Nevertheless, due to the fact that the system’s full sample rate is shared among all active tags, this would result in a bisection of the available sample rate per tag. Regarding the set-up of the main study, this would result in a sample rate per tag of about 2 Hz, which is considered as too low for accurate time-motion analysis. Third, differences in the accuracy between measurements performed in the centre and at the borders of the playing field were analysed. One could argue that the system’s accuracy might be lower at the borders of the playing field because the line of sight to the sensors is worse than in the centre. If this were the case, measurements for certain playing positions (e.g. wing players) would be systematically affected. However, no significant differences in the accuracy between both measurement areas were found. As a consequence of the pre-study’s results, the main study was performed with one head tag’s measurements not distinguishing between certain measurement areas. For the assessment of the system’s accuracy, in the main study accuracy thresholds had to be fixed. Referring to Edgecomb and Norton (2006), who declared their results of about 5% distance differences of the analysed GPS tracking system to a reference system as sufficient, this study also took into account a mean distance difference of 5%. But, considering that individual values can vary fairly strongly, in spite of the mean distance difference parameter, the 95% limits of agreement were used as a fair accuracy parameter (O’Donoghue, 2010). Since this parameter is a considerably stricter criterion for accuracy than the mean deviation, the
thresholds were determined with the mean difference ± 10%. This is in line with the statement of Dawson (2012) that for time-motion analysis the accuracy of a positioning system should be below 10% difference to the real value. The results of the study indicate that the accuracy of the Ubisense system is within these thresholds. Considering the systematic bias of the Ubisense measurements, the differences to the reference values are even below ±5% in the 95% LoA. However, strictly speaking the systematic bias is only suitable if the same value can be used for ongoing measurements and not only for the measurement where the bias was determined. This requirement has not been proven so far. A comparison of the current Ubisense accuracy results with other studies testing similar systems is very difficult because of the lack of standardised methods. Therefore, standards for testing positioning systems including uniform parameters for presenting the results should be developed (see below). However, Edgecomb and Norton (2006) report in a comparable study mean distance differences of 4.8% for a GPS system and 5.8% for a video-based system. This indicates that Ubisense outperforms these types of systems. The results of Ogris et al. (2012) suppose better results by radio wave-based location systems with higher sample rates than the Ubisense system. A key methodological issue of the study was the choice of a suitable reference system and proper accuracy parameters, respectively. Therefore the purpose of the tested system is a key question. If highly accurate position data (x/y- or x/y/z-coordinates) is necessary, then actual position data gained by the system has to be considered for the accuracy tests (Ogris et al., 2012). If a system is used for timemotion analysis, and measuring parameters like distances or speeds, then it is sufficient to check the accuracy of those parameters. This can be less labour-intensive than methods comparing each position measurement with an adequate reference value. Consequently, this study focused on covered distances, which also enables assessments of mean speeds in the case of time-motion analysis. For the comparison with exact distance values, a trundle wheel was used. Due to its low costs and high accuracy, this was considered as proper reference system. For the assessment of the accuracy of the position tracking system a valid, very easy-to-handle and comprehensible method was used. Considering the shortcomings of other studies (see Introduction), this method could also be used for future studies examining other location systems applied for timemotion analysis. In particular, the method accomplished the following requirements:
Accuracy of an UWB-based position tracking system 1. 2. 3. 4.
5.
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
6.
7.
The method was comprehensible and repeatable. The reference values were very accurate. The data between the several measurement devices (tags) were consistent (see above). The method examined the dependency between the duration/lengths of the observed units and the size of error in order to identify a possible ‘shotgun effect’ (see above). Gaining and post-processing the test data as well as the reference data were low cost and time intensive. The movement data was gathered directly from (simulated) match play and therefore has high validity. The measurements did not focus on a specific area of the pitch but covered the whole field – including the border areas, where the highest measurement errors often occur (Ogris et al., 2012). The assumption of uniform accuracy concerning border versus centre field measurements was confirmed before.
The main limitation of the applied method, however, was that only covered distances and their derivates (covered distance per time period, average speed, etc.) could be examined. Whilst the related work (see Introduction section) shows that this is most often sufficient, specific analyses regarding the position data (x/y/z-coordinates) were not possible. Other limitations of the study were as follows: 1.
2.
3.
4.
Driving the trundle wheel during the runs interfered with the players, in particular, when performing fast path or speed changes. Due to its fixed installation, the system could only be tested indoors and with the given set-up. The study was performed for the sport of basketball. Although similar results can be expected for other sports, there is no evidence that the system performs equally for other movements. The study neither examined specific circumstances, like athlete interaction (duels) or different movements (e.g. falling down and jumping) nor did it consider different conditions like effects of temperature, pitch size, or potential radio interferences on the signals.
This paper basically addressed the problem of accuracy regarding position tracking systems in sports. Various types of these systems are currently used in the sports domain in order to perform time-motion analyses. Therefore, a key issue is the proper assessment of the accuracy of those devices. The current
7
study investigated one of the few available radio wavebased location systems for sports and confirmed sufficient accuracy for the application in basketball. Moreover, in order to make accuracy studies of different systems comparable, the authors suggested the method used for this study as a standard for accuracy tests of location systems for game sports. References Aughey, R. J., & Falloon, C. (2010). Real-time versus post-game GPS data in team sports. Journal of Science and Medicine in Sport, 13, 348–349. doi:10.1016/j.jsams.2009.01.006 Barbero-Alvarez, J. C., Coutts, A., Granda, J., Barbero-Alvarez, V., & Castagna, C. (2010). The validity and reliability of a global positioning satellite system device to assess speed and repeated sprint ability (RSA) in athletes. Journal of Science and Medicine in Sport, 13, 232–235. doi:10.1016/j.jsams.2009.02.005 Connaghan, D., Hughes, S., May, G., Kelly, P., Conaire, C., O’Connor, N., … Moyna, N. (2009). A sensing platform for physiological and contextual feedback to tennis athletes. In Proceedings of the 2009 sixth international workshop on wearable and implantable body sensor network (pp. 224–229). Washington, DC: IEEE Computer Society. Coutts, A., & Duffield, R. (2010). Validity and reliability of GPS devices for measuring movement demands of team sports. Journal of Science and Medicine in Sport, 13, 133–135. doi:10.1016/j.jsams.2008.09.015 Dawson, B. (2012, July 25–28). Movement patterns in team sports: How do they relate to performance? Presentation at the World Congress of Performance Analysis of Sport IX, Worcester, UK. Di Salvo, V., Collins, A., McNeill, B., & Cardinale, M. (2006). Validation of prozone: A new video-based performance analysis system. International Journal of Performance Analysis in Sport, 6, 108–119. Retrieved from http://www.ingentaconnect.com/con tent/uwic/ujpa/2006/00000006/00000001/art00011 Dogramaci, S. N., Watsford, M. L., & Murphy, A. J. (2011). The reliability and validity of subjective notational analysis in comparison to global positioning system tracking to assess athlete movement patterns. Journal of Strength and Conditioning Research, 25, 852–859. doi:10.1519/JSC.0b013e3181c69edd Duffield, R., Reid, M., Baker, J., & Spratford, W. (2010). Accuracy and reliability of GPS devices for measurement of movement patterns in confined spaces for court-based sports. Journal of Science and Medicine in Sport, 13, 523–525. doi:10.1016/j.jsams.2009.07.003 Edgecomb, S. J., & Norton, K. I. (2006). Comparison of global positioning and computer-based tracking systems for measuring player movement distance during Australian football. Journal of Science and Medicine Sport, 9, 25–32. Retrieved from http://www.sciencedirect.com/science/article/pii/S144024 4006000053 Frencken, W. G. P., Lemmink, K., & Delleman, N. (2010). Soccer-specific accuracy and validity of the local position measurement (LPM) system. Journal of Science Medicine and Sport, 13, 641–645. doi:10.1016/j.jsams.2010.04.003 Gray, A. J., Jenkins, D., Andrews, M. H., Taaffe, D. R., & Glover, M. L. (2010). Validity and reliability of GPS for measuring distance travelled in field-based team sports. Journal of Sports Sciences, 28, 1319–1325. doi:10.1080/02640414.2010.504783 Jennings, D., Cormack, S., Coutts, A. J., Boyd, L., & Aughey, R. J. (2010a). The validity and reliability of GPS units for measuring distance in team sport specific running patterns. International Journal of Sports Physiology and Performance, 5, 328–341. Jennings, D., Cormack, S., Coutts, A. J., Boyd, L., & Aughey, R. J. (2010b). Variability of GPS units for measuring distance in
Downloaded by [University of Tennessee, Knoxville] at 08:08 09 September 2014
8
R. Leser et al.
team sport movements. International Journal of Sports Physiology and Performance, 5, 565–569. Leser, R. (2012). A wireless position tracking system for measuring sports performance in game sports. In R. Vaeyens (Ed.), Book of abstracts of the 3rd world conference on science and soccer (p. 68). Ghent: University Press. Leser, R., Baca, A., & Ogris, G. (2011). Local positioning systems in (game) sports. Sensors, 11, 9778–9797. doi:10.3390/s111 009778 Leser, R., & Schleindlhuber, A. (2012). Accuracy test of a wireless position tracking system for measuring sports activities. In Y. Jiang & A. Baca (Eds.), Proceedings of the 2012 pre-Olympic congress on sports science and computer science in sport (pp. 73–74). Liverpool: World Academic Press. MacLeod, H., Morris, J., Nevill, A., & Sunderland, C. (2009). The validity of a nondifferential global positioning system for assessing player movement patterns in field hockey. Journal of Sports Sciences, 27, 121–128. doi:10.1080/02640410802422181 Mucchi, L., Trippi, F., & Carpini, A. (2010). Ultra wide band real-time location system for cinematic survey in sports. In Proceedings of the 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (pp. 1–6). New York: IEEE. O’Donoghue, P. (2010). Research methods for sports performance analysis. London: Routledge. Ogris, G., Leser, R., Horsak, B., Kornfeind, P., Heller, M., & Baca, A. (2012). Accuracy of the LPM tracking system considering dynamic position changes. Journal of Sports Sciences, 30, 1503–1511. doi:10.1080/02640414.2012.712712 Perš, J., Bon, M., Kovacic, S., Šibila, M., & Dežman, B. (2002). Observation and analysis of large-scale human motion. Human Movement Science, 21, 295–311. doi:10.1016/S0167-9457(02) 00096-9
Petersen, C. J., Pyne, D. B., Portus, M. R., Karppinen, S., & Dawson, B. (2009). Variability in movement patterns during one day internationals by a cricket fast bowler. International Journal of Sports Physiology and Performance, 4, 278–281. Randers, M., Mujika, I., Hewitt, A., Santisteban, J., Bischoff, R., Solano, R., …, Mohr, M. (2010). Application of four different football match analysis systems: A comparative study. Journal of Sports Sciences, 28, 171–182. doi:10.1080/02640410903428525 Roberts, S., Trewartha, G., & Stokes, K. (2006). A comparison of time-motion analysis methods for field-based sports. International Journal of Sports Physiology and Performance, 1, 388–399. Siegle, M., Stevens, T., & Lames, M. (2012). Design of an accuracy study for position detection in football. Journal of Sports Sciences, 31, 166–172. doi:10.1080/02640414.2012. 723131 Tynan, R., Schoofs, A., Muldoon, C., O’Hare, G., Ó Conaire, C., Kelly, P., & O’Connor, N. (2009). Intelligent middleware for adaptive sensing of tennis coaching sessions. In Proceedings of the 2nd international workshop on adaptation in wireless sensor networks (pp. 891–896). Vancouver, Canada. Ubisense. (2013,). Ubisense. Retrieved May 21, from http://www. ubisense.net/en Varley, M. C., Fairweather, I. H., & Aughey, R. J. (2012). Validity and reliability of GPS for measuring instantaneous velocity during acceleration, deceleration, and constant motion. Journal of Sports Sciences, 30, 121–127. doi:10.1080/ 02640414.2011.627941 Waldron, M., Worsfold, P., Twist, C., & Lamb, K. (2011). Concurrent validity and test retest reliability of a Global Positioning System (GPS) and timing gates to assess sprint performance variables. Journal of Sports Sciences, 29, 1613– 1619. doi:10.1080/02640414.2011.608703
