In court sports, the specific action that results in the highest incidence of anterior cruciate ligament (ACL) injury is landing from a jump while exhibiting dynamic knee valgus.1,2 This movement, typified by rapid excessive knee abduction, is exhibited more frequently and to higher magnitudes in females than in males.3 It is, therefore, not surprising that female adolescent athletes are 4 to 6 times more likely to suffer an ACL injury than their male counterparts.4
Because of high injury risk, it is important that female-dominated sports have efficacious, well-implemented prevention strategies. When injury prevention programs that target lower-limb neuromuscular coordination are implemented optimally, the odds of noncontact ACL injuries can be reduced by 70%.5 However, poor implementation has resulted in a lack of widespread reduction in ACL injuries.6,7 This is particularly the case for netball, a court sport played by over 70 countries, with the highest rate of female participation for multiple commonwealth countries.8,9 The risk of serious knee injury is 3.3 times higher in netball than basketball,10 with >1000 netball-related ACL reconstructions occurring each year (188/100,000 participants).11 The lack of timely and accurate feedback potentially contributes to the poor implementation of these programs. Multiple studies support the importance of providing timely feedback during neuromuscular training programs,12,13 as it may enhance motor skill development and improve joint position sense.13
Recent developments in gaming technology have facilitated the use of real-time joint kinematic tracking as a feedback tool. Such feedback has resulted in both immediate14 and short-term improvements in knee position during jump landing.15 A potential limitation is that these studies used internally focused feedback, that is, participants focused on their knee position. Benjaminse et al13 argue that this method of feedback may interfere with the natural coordination of the movement and automaticity of the skill. Virtual reality (VR) may be provide a solution to this issue by offering individuals the chance to intensively train meaningful tasks with externally focused feedback, within relatively realistic 3-dimensional (3D) environments.13,16 In the rehabilitation settings, VR results in high degrees of motivation and involvement.17,18 Although still confined to laboratory settings, innovation in portable VR headset technology is making it possible for VR to be considered as a supplement for current on-field programs. However, the ability of VR gaming technology to evaluate and provide accurate feedback on frontal plane knee position must first be established. The objectives of this study were (1) to evaluate the validity of a VR game in evaluating the projected frontal plane knee angle when landing from a jump and (2) to assess the reliability of this evaluation over time.
Methods
A cross-sectional laboratory study was used to determine the validity of a custom-built VR game in evaluating and providing feedback of the frontal plane knee angle during 3 drop vertical jumps (DVJs). Reliability was assessed by inviting a subsample (10 participants) to return 1 week later and repeat the protocol. Due to interference of the infrared cameras with the game sensor, the DVJs could not be assessed by the VR game and 3D motion analysis concurrently. To avoid potential learning effects, the order of testing was fixed such that 3 DVJs were assessed first by the 3D motion analysis and then repeated with the game. All participants were instructed to jump off a 30-cm high box and immediately perform a maximum vertical jump landing on both feet. During the jump, they were instructed to raise their arms as though they were jumping to shoot a ball. Up to 5 practice trials were granted prior to 3D motion analysis.
A convenience sample of 40 healthy individuals (61% female, age: 23.46 [2.29] y; height: 172.7 [9.9] cm; and weight: 66.87 [12.8] kg) was recruited. Participants were excluded if they had a history of knee ligament injury, patellofemoral pain, or known motion sickness when experiencing VR. Ethics approval for the study was granted through Macquarie University Human Research Ethics Committee (HREC #5201500774), and all participants provided written informed consent.
A 3D netball court was designed using VR software (Vizard Virtual Reality Engine; Worldviz, Santa Barbara, CA) and displayed on a 240° cylindrical panoramic projection system that was 8 m in diameter (Figure 1). A KINECT sensor (Microsoft, Redmond, WA), sampling at 30 Hz, was positioned 2 m in front of where participants landed from the DVJ. Embedded KINECT software algorithms calculated a 3D volume of each participant based on a fusion of the horizontal and vertical axes overlaid with a depth of field tracking of each pixel. This skeleton was transferred to the gaming software, which bound the skeleton to an avatar. A customized algorithm was written and applied to the shank and thigh vectors of participants’ avatars. The bisection of these vectors was used to calculate the projected frontal plane knee angle.19 A static calibration was utilized to determine the “0” value for each participant.
—The custom-built VR game. Participant stood on the 30-cm block and performed a maximal drop vertical jump. If the sensor (located in middle of image) detected the projected frontal plane knee angle was <9°, the ball (pictured on the left) would enter the hoop and participants would score a goal. If the knee angle detected ≥9°, then the ball would miss the hoop. VR indicates virtual reality.
Citation: Journal of Sport Rehabilitation 27, 4; 10.1123/jsr.2017-0264
Kinematic data were collected using an 8-camera motion capture system (VICON, Oxford, UK) collecting at 100 Hz. Sixteen retroreflective markers, 14 mm in diameter, were placed on both lower limbs according to the Plug-In-Gait model (Oxford Metrics, Oxford, UK). Knee angles were calculated from the XYZ cardan angles derived from the relative orientation of the femur and tibial segments.20 These were referenced to a static calibration trial. Low-frequency movement artifact was removed from marker trajectories using a generalized cross-validatory spline filter.21 Peak negative frontal plane angle was extracted for further analysis.
When playing the game, participants were instructed to shoot a “ball” toward a virtual hoop. Although instructed to ignore the feedback, if participants landed with the peak negative projected frontal plane angle of their left knee below (more positive) a predefined threshold of −9°,22,23 the game would provide positive feedback. That is, the virtual ball would enter the hoop, and the player would score a goal. Landing with a knee angle exceeding (more negative) the threshold resulted in the ball missing the hoop.
The mean peak abduction angle from the 3 DVJs from both the VR game and 3D motion analysis was used to assess the 95% limits of agreement and heteroscedasticity of the data. Consensus categories, defined as “below threshold” (average of jumps under threshold values) and “above threshold” (average of jumps exceeded threshold values), were then applied to participants. The kappa (κ) coefficient was utilized to determine agreement between systems. Coefficients were interpreted as slight (0.0–0.2), fair (0.21–0.4), moderate (0.41–0.6), substantial (0.61–0.8), and almost perfect (0.81–1.0).24 A 2 × 2 contingency table was used to quantify using specificity, sensitivity, positive likelihood ratio, and number needed to diagnose.
Reliability examined using the intraclass correlation coefficient (ICC3,1) based on the average of 3 jumps at each VR-game session.25 As the game provides pass/fail feedback, a correlation coefficient of .81 was set as the smallest threshold for acceptable retest reliability.26 Typical error and standard error of the mean were also calculated using RStudio (version 0.99.903; RStudio Inc, Boston, MA).
Results
Because of marker placement error, data from 4 participants were excluded. Raw values were inspected visually to determine if there was a systematic reduction in negative peak frontal plane angle when participants played the VR game compared with 3D motion analysis, which would indicate a learning effect (Figure 2). Based on this observation, no evidence of a systematic learning effect was present.
—Raw data of the peak negative, or lowest, knee angle evaluated from each participant’s DVJs. Each column indicates a jump. Negative angles indicate knee abduction or negative projected frontal plane knee angles. VR indicates virtual reality; 3D, 3 dimensional; DVJ, drop vertical jump.
Citation: Journal of Sport Rehabilitation 27, 4; 10.1123/jsr.2017-0264
The lower limit (−10.27°; 95% confidence interval [CI], −14.36° to −6.19°) and upper limit (17.05°; 95% CI, 12.97° to 21.14°) of the 95% limits of agreement contained 94.4% (34/36) of the difference scores. The mean difference, or bias, between the 3D motion analysis and VR game was 3.39° (95% CI, 1.03° to 5.74°; Figure 3).
—Bland–Altman plot displaying the difference in the mean of each participant’s peak knee angles as detected by the VR game and 3D analysis as function of the mean of the measures. Solid lines indicate the mean difference (95% CI), or bias, in the VR game. Broken lines and shading indicate the 95% limits of agreement and 95% CI. VR indicates virtual reality; 3D, 3 dimensional; CI, confidence interval.
Citation: Journal of Sport Rehabilitation 27, 4; 10.1123/jsr.2017-0264
There was substantial agreement (κ = .66; 95% CI, .42 to .88) between the 2 systems and high sensitivity (0.75; 95% CI, 0.43 to 0.95) and specificity (0.96; 95% CI, 0.78 to 1.0). The number needed to diagnose was 1.42 (95% CI, 1.06 to 4.79).
The VR game exhibited acceptable reliability over the 1-week retest period (ICC3,1 = .844). The typical error between participants was 2.22°, and SEM within participants was 2.36°.
Discussion
A VR game reliably measured the peak negative projected frontal plane knee angle when landing from a DVJ. There was substantial agreement between the 3D motion analysis system and VR game when a threshold of −9° was used. However, the VR game systematically reported larger knee angles than the motion analysis system, and the wide 95% limits of agreement indicate that the game had limited accuracy detecting the exact angle of peak knee abduction. Using threshold categories resulted in high sensitivity and specificity.
This limited accuracy is likely due to 2 issues. First, the motion analysis and VR game assessed consecutive and not simultaneous DVJs. Small variations between close-to-threshold jumps would influence the categorical analysis of validity, but do not explain the wide CIs of the 95% limits of agreement. This difference is likely explained by the fact that the systems are measuring different things. Although the projected frontal plane knee angle predominantly consists of knee abduction, it is also influenced by knee flexion and hip internal rotation.27 Two-dimensional motion sensors will be limited when measuring this triplanar movement. Thus, consistent with previous literature,27,28 the findings from our study indicate that the VR feedback game is sensitive to differences in angles on a continuous scale. A threshold approach may be a more feasible and valid approach for gaming technology to employ for evaluating joint position.
The important question arising from this finding is what threshold is most appropriate. The 9° threshold utilized in the present study was based on evaluations of knee abduction angles, and not a projected frontal plane angle. Although an excessive projected frontal plane knee angle is a major risk factor for ACL injuries in netball,2 handball,29 and basketball,30 the empirical values of this angle that are associated with ACL injury range from −20° to −5°.31 McLean et al32 suggests that 2-dimensional motion analysis consistently measures knee frontal plane angles as more negative than when measured using a 3D motion analysis (ie, more abduction). This positive bias, which was also present in our study, suggests that the threshold for projected frontal plane knee angles may need to be lower (ie, more negative) than peak knee abduction angles.
There is significant potential for VR games to become a tool for providing feedback on knee position during jump landing, particularly with recent innovations in portable technology. Live digital representation of participants’ body segments results in immediate improvements in knee kinematics14,33 that are maintained even once feedback is withdrawn.15 This is more promising than visual observation techniques, which have been found to have little impact on injury prevention program efficacy or rates of implementation.5 Although the current study is a preliminary investigation of the validity of the VR game in providing feedback, it suggests that further investigation is warranted into whether externally-based feedback reduces knee injury rates or improving implementation of neuromotor warm-up programs. Furthermore, this study was confined to the laboratory, and future studies exploring portable options that could provide feedback where the athlete is competing could be of greater benefit.
A limitation to this brief report is that we were unable to assess DVJs using both evaluation methods concurrently. The variability exhibited by participants when performing DVJs between the motion analysis and VR game would have influenced the accuracy of the game. However, it provided further support that a threshold cutoff, rather than a continuous scale, was less susceptible to this intraindividual variation between jumps. A further limitation was the inclusion of males in our sample. The valgus collapse mechanism associated with ACL injuries is more likely to be exhibited by females.3,22 However, the inclusion of males did not affect our ability to answer our research question and does make results more generalizable to multisex sports.
The novel biofeedback employed in the current study could identify two-thirds of individuals with peak projected frontal plane knee angle >9° during landing. Improvements in technology and wearable sensors will potentially not only make concurrent testing possible, but will also improve the accuracy of detecting frontal plane movements.
References
- 1.↑
Borowski LA, Yard EE, Fields SK, Comstock RD. The epidemiology of US high school basketball injuries, 2005–2007. Am J Sports Med. 2008;36(12):2328–2335. PubMed ID: 18765675 doi:10.1177/0363546508322893
- 2.↑
Stuelcken MC, Mellifont DB, Gorman AD, Sayers MGL. Mechanisms of anterior cruciate ligament injuries in elite women’s netball: a systematic video analysis. J Sports Sci. 2016;34(16):1516–1522. PubMed ID: 26644060 doi:10.1080/02640414.2015.1121285
- 3.↑
Malinzak RA, Colby SM, Kirkendall DT, Yu B, Garrett WE. A comparison of knee joint motion patterns between men and women in selected athletic tasks. Clin Biomech. 2001;16(5):438–445. doi:10.1016/S0268-0033(01)00019-5
- 4.↑
Holden S, Boreham C, Doherty C, Wang D, Delahunt E. Clinical assessment of countermovement jump landing kinematics in early adolescence: sex differences and normative values. Clin Biomech. 2015;30(5):469–474. doi:10.1016/j.clinbiomech.2015.03.008
- 5.↑
Taylor JB, Waxman JP, Richter SJ, Shultz SJ. Evaluation of the effectiveness of anterior cruciate ligament injury prevention programme training components: a systematic review and meta-analysis. Br J Sports Med. 2015;49(2):79–87. PubMed ID: 23922282 doi:10.1136/bjsports-2013-092358
- 6.↑
Agel J, Arendt EA, Bershadsky B. Anterior cruciate ligament injury in national collegiate athletic association basketball and soccer: a 13-year review. Am J Sports Med. 2005;33(4):524–531. PubMed ID: 15722283 doi:10.1177/0363546504269937
- 7.↑
Pfeiffer RP, Shea KG, Roberts D, Grandstrand S, Bond L. Lack of effect of a knee ligament injury prevention program on the incidence of noncontact anterior cruciate ligament injury. J Bone Joint Surg Am. 2006;88(8):1769–1774. PubMed ID: 16882900 doi:10.2106/JBJS.E.00616
- 8.↑
Australian Bureau of Statistics. Involvement in Organised Sport and Physical Activity, Australia, 2013–14. Vol 1. 1st ed. Canberra, Australia: Australian Bureau of Statistics; 2015:1–4. http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/4177.02013-14?OpenDocument. Accessed August 10, 2017.
- 9.↑
Sport New Zealand. Sport and Active Recreation in the Lives of New Zealand Adults. Wellington, New Zealand: Sport New Zealand; 2015:1–95. www.srknowledge.org.nz/researchseries/active-new-zealand-20132014/. Accessed August 10, 2017.
- 10.↑
McKay GD, Payne WR, Goldie PA, Oakes BW, Stanley JJ. A comparison of the injuries sustained by female basketball and netball players. Aust J Sci Med Sport. 1996;28(1):12–17. PubMed ID: 8742861
- 11.↑
Janssen KW, Orchard JW, Driscoll TR, van Mechelen W. High incidence and costs for anterior cruciate ligament reconstructions performed in Australia from 2003–2004 to 2007–2008: time for an anterior cruciate ligament register by Scandinavian model? Scand J Med Sci Sports. 2012;22(4):495–501. PubMed ID: 21210852 doi:10.1111/j.1600-0838.2010.01253.x
- 12.↑
Myklebust G, Engebretsen L, Braekken IH, Skjølberg A, Olsen O-E, Bahr R. Prevention of anterior cruciate ligament injuries in female team handball players: a prospective intervention study over three seasons. Clin J Sport Med. 2003;13(2):71–78. PubMed ID: 12629423 doi:10.1097/00042752-200303000-00002
- 13.↑
Benjaminse A, Gokeler A, Dowling AV, et al. Optimization of the anterior cruciate ligament injury prevention paradigm: novel feedback techniques to enhance motor learning and reduce injury risk. J Orthop Sports Phys Ther. 2015;45(3):170–182. PubMed ID: 25627151 doi:10.2519/jospt.2015.4986
- 14.↑
Ford KR, DiCesare CA, Myer GD, Hewett TE. Real-time biofeedback to target risk of anterior cruciate ligament injury: a technical report for injury prevention and rehabilitation. J Sport Rehabil. 2015;Technical Notes 13. doi:10.1123/jsr.2013-0138
- 15.↑
Nyman E, Armstrong CW. Real-time feedback during drop landing training improves subsequent frontal and sagittal plane knee kinematics. Clin Biomech. 2015;30(9):988–994. doi:10.1016/j.clinbiomech.2015.06.018
- 16.↑
Crosbie JH, Lennon S, Basford JR, McDonough SM. Virtual reality in stroke rehabilitation: still more virtual than real. Disabil Rehabil. 2007;29(14):1139–1146. PubMed ID: 17613000 doi:10.1080/09638280600960909
- 17.↑
Olivieri I, Chiappedi M, Meriggi P, Mazzola M, Grandi A, Angelini L. Rehabilitation of children with hemiparesis: a pilot study on the use of virtual reality. Biomed Res Int. 2013;2013:1–5. PubMed ID: 24199197 doi:10.1155/2013/695935
- 18.↑
Riva G. Virtual reality in rehabilitation of spinal cord injuries: a case report. Rehabil Psychol. 2000;45(1):81–88. doi:10.1037/0090-5550.45.1.81
- 19.↑
Bonnechère B, Jansen B, Salvia P, et al. Validity and reliability of the kinect within functional assessment activities: comparison with standard stereophotogrammetry. Gait Posture. 2014;39(1):593–598. doi:10.1016/j.gaitpost.2013.09.018
- 20.↑
Davis RB III, Ounpuu S, Tyburski D. A gait analysis data collection and reduction technique. Hum Mov Sci. 1991;10:575–587. doi:10.1016/0167-9457(91)90046-Z
- 21.↑
Woltring HJ. A Fortran package for generalized, cross-validatory spline smoothing and differentiation. Adv Eng Softw. 1986;8(2):104–113. doi:10.1016/0141-1195(86)90098-7
- 22.↑
Hewett TE, Myer GD, Ford KR, et al. Biomechanical measures of neuromuscular control and valgus loading of the knee predict anterior cruciate ligament injury risk in female athletes: a prospective study. Am J Sports Med. 2005;33(4):492–501. PubMed ID: 15722287 doi:10.1177/0363546504269591
- 23.↑
Myer GD, Ford KR, Di Stasi SL, Foss KDB, Micheli LJ, Hewett TE. High knee abduction moments are common risk factors for patellofemoral pain (PFP) and anterior cruciate ligament (ACL) injury in girls: is PFP itself a predictor for subsequent ACL injury? Br J Sports Med. 2015;49(2):118–122. PubMed ID: 24687011 doi:10.1136/bjsports-2013-092536
- 24.↑
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. PubMed ID: 843571 doi:10.2307/2529310
- 25.↑
Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005;19(1):231–240. PubMed ID: 15705040 doi:10.1519/15184.1
- 26.↑
Hopkins WG. Measures of reliability in sports medicine and science. Sports Med. 2000;30(1):1–15. PubMed ID: 10907753 doi:10.2165/00007256-200030010-00001
- 27.↑
Sorenson B, Kernozek TW, Willson JD, Ragan R, Hove J. Two- and three-dimensional relationships between knee and hip kinematic motion analysis: single-leg drop-jump landings. J Sport Rehabil. 2015;24(4):363–372. PubMed ID: 25658442 doi:10.1123/jsr.2014-0206
- 28.↑
Huber ME, Seitz AL, Leeser M, Sternad D. Validity and reliability of Kinect skeleton for measuring shoulder joint angles: a feasibility study. Physiotherapy. 2015;101(4):389–393. PubMed ID: 26050135 doi:10.1016/j.physio.2015.02.002
- 29.↑
Olsen OE, Myklebust G, Engebretsen L, Bahr R. Injury pattern in youth team handball: a comparison of two prospective registration methods. Scand J Med Sci Sports. 2006;16(6):426–432. PubMed ID: 17121645 doi:10.1111/j.1600-0838.2005.00484.x
- 30.↑
Krosshaug T, Nakamae A, Boden BP, et al. Mechanisms of anterior cruciate ligament injury in basketball: video analysis of 39 cases. Am J Sports Med. 2007;35(3):359–367. PubMed ID: 17092928 doi:10.1177/0363546506293899
- 31.↑
Olsen O-E, Myklebust G, Engebretsen L, Bahr R. Injury mechanisms for anterior cruciate ligament injuries in team handball: a systematic video analysis. Am J Sports Med. 2004;32(4):1002–1012. PubMed ID: 15150050 doi:10.1177/0363546503261724
- 32.↑
McLean SG, Walker K, Ford KR, Myer GD, Hewett TE, van den Bogert AJ. Evaluation of a two dimensional analysis method as a screening and evaluation tool for anterior cruciate ligament injury. Br J Sports Med. 2005;39(6):355–362. PubMed ID: 15911607 doi:10.1136/bjsm.2005.018598
- 33.↑
Ericksen HM, Thomas AC, Gribble PA, Doebel SC, Pietrosimone BG. Immediate effects of real-time feedback on jump-landing kinematics. J Orthop Sports Phys Ther. 2015;45(2):112–118. PubMed ID: 25552287 doi:10.2519/jospt.2015.4997
