Sensitivity and Specificity of a Multimodal Approach for Concussion Assessment in Youth Athletes

in Journal of Sport Rehabilitation
Restricted access

Purchase article

USD  $24.95

Student 1 year online subscription

USD  $76.00

1 year online subscription

USD  $101.00

Student 2 year online subscription

USD  $144.00

2 year online subscription

USD  $192.00

Context: Current international consensus endorses a multimodal approach to concussion assessment. However, the psychometric evaluation of clinical measures used to identify postconcussion performance deficits once an athlete is asymptomatic remains limited, particularly in the pediatric population. Objective: To describe and compare the sensitivity and specificity of a multimodal assessment battery (balance, cognition, and upper and lower body strength) versus individual clinical measures at discriminating between concussed youth athletes and noninjured controls when asymptomatic. Design: Prospective cohort study. Setting: Hospital laboratory setting. Participants: A total of 32 youth athletes with a concussion and 32 matched (age and sex) noninjured control participants aged 10–18 years. Intervention(s): Participants were administered preinjury (baseline) assessments of cognition (Immediate Post-Concussion Assessment and Cognitive Testing [ImPACT]), balance (BioSway), and upper and lower body strength (grip strength and standing long jump). Assessments were readministered when concussed participants reported symptom resolution (asymptomatic time point). Noninjured control participants were reassessed using the same time interval as their concussion matched pair. Sensitivity and specificity were calculated using standardized regression-based methods and receiver operating characteristic curves. Main Outcome Measures: Outcome measures included baseline and postinjury ImPACT, BioSway, grip strength, and standing long jump scores. Results: When asymptomatic, declines in performance on each individual clinical measure were seen in 3% to 22% of the concussion group (sensitivity = 3%–22%) compared with 3% to 13% of the noninjured control group (specificity = 87%–97%) (90% confidence interval). The multimodal battery of all combined clinical measures yielded a sensitivity of 41% and a specificity of 77% (90% confidence interval). Based on discriminative analyses, the multimodal approach was statistically superior compared with an individual measures approach for balance and upper and lower body strength, but not for cognition. Conclusions: Results provide a foundation for understanding which domains of assessment (cognition, balance, and strength) may be sensitive and specific to deficits once symptoms resolve in youth athletes. More work is needed prior to clinical implementation of a preinjury (baseline) to postinjury multimodal approach to assessment following concussion in youth athletes.

Toong, Hunt, Scratch, and Reed are with the Rehabilitation Sciences Institute, Faculty of Medicine, University of Toronto, Toronto, ON, Canada. Toong, Wilson, Hunt, Scratch, and Reed are with the Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada. Hunt and Reed are also with the Department of Occupational Science and Occupational Therapy, Faculty of Medicine, University of Toronto, Toronto, ON, Canada. Scratch is also with the Department of Pediatrics, University of Toronto, Toronto, ON, Canada. DeMatteo is with the School of Rehabilitation Sciences, McMaster University, Hamilton, ON, Canada.

Reed (nick.reed@utoronto.ca) is corresponding author.
  • 1.

    Zemek R, Grool A, Duque D, DeMatteo C. Annual and seasonal trends in ambulatory visits for pediatric concussion in Ontario between 2003 and 2013. J Pediatr. 2016;181:222228.e2.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2.

    McCrory P, Meeuwisse W, Dvorak J, et al. Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017:51(11):838847.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    Bressan S, Babl FE. Diagnosis and management of paediatric concussion. J Paediatr Child Health. 2016;52(2):151157. PubMed ID: 26174579 doi:

  • 4.

    Cusimano MD, Topolovec-Vranic J, Zhang S, Mullen SJ, Wong M, Ilie G. Factors influencing the underreporting of concussion in sports: a qualitative study of minor hockey participants. Clin J Sport Med. 2017;27(4):375380. PubMed ID: 28653965 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Register-Mihalik JK, Guskiewicz KM, McLeod TCV, Linnan LA, Mueller FO, Marshall SW. Knowledge, attitude, and concussion-reporting behaviors among high school athletes: a preliminary study. J Athl Train. 2013;48(5):645653. PubMed ID: 23848520 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    Chrisman SP, Quitiquit C, Rivara FP. Qualitative study of barriers to concussive symptom reporting in high school athletics. J Adolesc Heal. 2013;52(3):330335.e3. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    Feddermann-Demont N, Echemendia RJ, Schneider KJ, et al. What domains of clinical function should be assessed after sport-related concussion? A systematic review. Br J Sports Med. 2017;51(11):903918. PubMed ID: 29098983 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Reed N, Taha T, Monette G, Keightley M. A preliminary exploration of concussion and strength performance in youth Ice Hockey players. Int J Sports Med. 2016;37(9):708713. PubMed ID: 27191209 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Haaland KY, Temkin N, Randahl G, Dikmen S. Recovery of simple motor skills after head injury. J Clin Exp Neuropsychol. 1994;16(3):448456. PubMed ID: 7929712 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Drijkoningen D, Caeyenberghs K, Vander Linden C, Van Herpe K, Duysens J, Swinnen SP. Associations between muscle strength asymmetry and impairments in gait and posture in young brain-injured patients. J Neurotrauma. 2015;32(17):13241332. PubMed ID: 25738975 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Katz-Leurer M, Rotem H, Keren O, Meyer S. The relationship between step variability, muscle strength and functional walking performance in children with post-traumatic brain injury. Gait Posture. 2009;29(1):154157. PubMed ID: 18701292 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12.

    Tremblay S, Vincent B, Sebastien P, et al. Multimodal assessment of primary motor cortex integrity following sport concussion in asymptomatic athletes. Clin Neurophysiol. 2014;125(7):13711379. PubMed ID: 24462505 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13.

    Evans TM, Van Remmen H, Purkar A, et al. Microwave & Magnetic (M2) proteomics of a mouse model of mild traumatic brain injury. Transl proteomics. 2014;3:1021. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 14.

    Powers KC, Cinelli ME, Kalmar JM. Cortical hypoexcitability persists beyond the symptomatic phase of a concussion. Brain Inj. 2014;28(4):465471. PubMed ID: 24702432 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15.

    Kamins J, Bigler E, Covassin T, et al. What is the physiological time to recovery after concussion? A systematic review. Br J Sports Med. 2017;51(12):935940. PubMed ID: 28455363 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16.

    Haider MN, Leddy JL, Pavlesen S, et al. A systematic review of criteria used to define recovery from sport-related concussion in youth athletes. Br J Sport Med. 2017;0(July):114.

    • Search Google Scholar
    • Export Citation
  • 17.

    Bianco A, Jemni M, Thomas E, et al. A systematic review to determine reliability and usefulness of the field-based test batteries for the assessment of physical fitness in adolescents—the ASSO project. 2015;28:445478. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18.

    Fernandez-Santos JR, Ruiz JR, Cohen DD, Gonzalez-Montesinos JL, Castro-Piñero J. Reliability and validity of tests to assess lower-body muscular power in children. J Strength Cond Res. 2015;29(8):22772285. PubMed ID: 25647647 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 19.

    Giza CC. Pediatric issues in sports concussions. Contin Lifelong Learn Neurol. 2014;20(December):15701587. doi:

  • 20.

    Valovich McLeod TC, Barr WB, McCrea M, Guskiewicz KM. Psychometric and measurement properties of concussion assessment tools in youth sports. J Athl Train. 2006;41(4):399408. http://www.ncbi.nlm.nih.gov/pubmed/17273465. Accessed June 18, 2018. PubMed ID: 17273465

    • Search Google Scholar
    • Export Citation
  • 21.

    Kirkwood MW, Randolph C, Yeates KO. Returning pediatric athletes to play after concussion: the evidence (or lack thereof) behind baseline neuropsychological testing. Acta Paediatr Int J Paediatr. 2009;98(9):14091411. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22.

    Kirkwood M, Yeates K, Wilson P. Pediatric sport-related concussion: a review of the clinical management of an oft-neglected population. Pediatrics. 2006;117:13591371. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23.

    Davis GA, Anderson V, Babl FE, et al. What is the difference in concussion management in children as compared with adults? A systematic review. Br J Sports Med. 2017;51:949957. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24.

    Lalkhen AG, McCluskey A. Clinical tests: sensitivity and specificity. Contin Educ Anaesth Crit Care Pain. 2008;8(6):221223. doi:

  • 25.

    Schatz P, Pardini JE, Lovell MR, Collins MW, Podell K. Sensitivity and specificity of the ImPACT Test Battery for concussion in athletes. Arch Clin Neuropsychol. 2006;21(1):9199. PubMed ID: 16143492 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26.

    Van Kampen DA, Lovell MR, Pardini JE, Collins MW, Fu FH. The “Value Added” of neurocognitive testing after sports-related concussion. Am J Sports Med. 2006;34(10):16301635. PubMed ID: 16816151 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 27.

    King LA, Horak FB, Mancini M, et al. Instrumenting the balance error scoring system for use with patients reporting persistent balance problems after mild traumatic brain injury. Arch Phys Med Rehabil. 2014;95(2):353359. PubMed ID: 24200875 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 28.

    Furman GR, Lin CC, Bellanca JL, Marchetti GF, Collins MW, Whitney SL. Comparison of the balance accelerometer measure and balance error scoring system in adolescent concussions in sports. Am J Sports Med. 2013;41(6):14041410. PubMed ID: 23585486 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 29.

    Sady M, Vaughan C. Psychometric characteristics of the postconcussion symptom inventory in children and adolescents. Arch Clin. 2014. http://acn.oxfordjournals.org/content/early/2014/04/15/arclin.acu014.short. Accessed November 26, 2016.

    • Search Google Scholar
    • Export Citation
  • 30.

    Resch JE, Brown CN, Schmidt J, et al. The sensitivity and specificity of clinical measures of sport concussion: three tests are better than one. BMJ Open Sport Exerc Med. 2016;2(1):e000012. PubMed ID: 27900145 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 31.

    Register-Mihalik JK, Guskiewicz KM, Mihalik JP, Schmidt JD, Kerr ZY, McCrea MA. Reliable change, sensitivity, and specificity of a multidimensional concussion assessment battery. J Head Trauma Rehabil. 2013;28(4):274283. PubMed ID: 22691965 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32.

    Broglio SP, Harezlak J, Katz B, et al. Acute sport concussion assessment optimization: a prospective assessment from the CARE consortium. Sport Med. 2019;49(12):19771987. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 33.

    Eckner JT, Kutcher JS, Broglio SP, Richardson JK. Effect of sport-related concussion on clinically measured simple reaction time. Br J Sports Med. 2014;48(2):112118. PubMed ID: 23314889 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34.

    Randolph C, McCrea M, Barr WB. Is neuropsychological testing useful in the management of sport-related concussion? J Athl Train. 2005;40(3):139152. PubMed ID: 16284633

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 35.

    Kriz PK, Mannix R, Taylor AM, Ruggieri D, Meehan WP. Neurocognitive deficits of concussed adolescent athletes at self-reported symptom resolution in the Zurich guidelines era. Orthop J Sport Med. 2017;5(11):232596711773730. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 36.

    Broglio SP, Macciocchi SN, Ferrara MS. Neurocognitive performance of concussed athletes when symptom free. J Athl Train. 2007;42(4):504508. http://www.ncbi.nlm.nih.gov/pubmed/18174938. Accessed June 19, 2018. PubMed ID: 18174938

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 37.

    Rochefort C, Walters-Stewart C, Aglipay M, Barrowman N, Zemek R, Sveistrup H. Self-reported balance status is not a reliable indicator of balance performance in adolescents at one-month post-concussion. J Sci Med Sport. 2017;20(11):970975. PubMed ID: 28495201 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 38.

    Servatius RJ, Spiegler KM, Handy JD, Pang KCH, Tsao JW, Mazzola CA. Neurocognitive and fine motor deficits in asymptomatic adolescents during the subacute period after concussion. J Neurotrauma. 2018;35(8):10081014. PubMed ID: 29278998 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 39.

    ImPACT Applications Inc. ImPACT Version 2.1, Clinical User’s Manual. Pittsburgh, PA; 2002.

  • 40.

    Biodex Medical Systems Inc. BioSway portable balance system operation manual. Vol 6339. http://www.biodex.com/sites/default/files/950460man_10202revd.pdf.

    • Search Google Scholar
    • Export Citation
  • 41.

    Reed N, Murphy J, Dick T, et al. A multi-modal approach to assessing recovery in youth athletes following concussion. J Vis Exp. 2014;(91):19.

    • Search Google Scholar
    • Export Citation
  • 42.

    McCrory P, Meeuwisse W, Aubry M. Consensus statement on concussion in sport: the 4th International Conference on Concussion in Sport held in Zurich, November 2012. Br J Sport. 2013. http://bjsm.bmj.com/content/47/5/250.short. Accessed October 24, 2016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 43.

    McCrea M, Barr WB, Guskiewicz K, et al. Standard regression-based methods for measuring recovery after sport-related concussion. J Int Neuropsychol Soc. 2005;11(1):5869. PubMed ID: 15686609 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 44.

    Barr WB, McCrea M. Sensitivity and specificity of standardized neurocognitive testing immediately following sports concussion. J Int Neuropsychol Soc. 2001;7(6):693702. PubMed ID: 11575591 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 45.

    Nelson LD, Laroche AA, Pfaller AY, et al. Prospective, head-to-head study of three computerized neurocognitive assessment tools (CNTs): Reliability and validity for the assessment of sport-related concussion. J Int Neuropsychol Soc. 2015;22(1):2437. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 46.

    Nelson LD. False-positive rates of reliable change indices for concussion test batteries: a Monte Carlo simulation. J Athl Train. 2015;50(12):13191322. PubMed ID: 26678291 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 47.

    Hosmer D, Lemeshow S. Applied Logistic Regression. 2nd ed. (Wiley, ed.). New York: Wiley; 2000.

  • 48.

    DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845. PubMed ID: 3203132 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 49.

    R Foundation for statistical computing. R: A language and environment for statistical computing. 2013. http://www.r-project.org/.

  • 50.

    Makdissi M, Davis G, McCrory P. Clinical challenges in the diagnosis and assessment of sports-related concussion. Neurol Clin Pract. 2015;5(1):25. PubMed ID: 29443157 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 51.

    Downey RI, Hutchison MG, Comper P. Determining sensitivity and specificity of the Sport Concussion Assessment Tool 3 (SCAT3) components in university athletes. Brain Inj. 2018;32(11):13451352. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 52.

    Eisenberg MA, Meehan WP, Mannix R. Duration and course of post-concussive symptoms. Pediatrics. 2014;133(6):9991006. PubMed ID: 24819569 doi:

  • 53.

    Henry LC, Elbin RJ, Collins MW, Marchetti G, Kontos AP. Examining recovery trajectories after sport-related concussion with a multimodal clinical assessment approach. Neurosurgery. 2016;78(2):232241. PubMed ID: 26445375 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 54.

    McDevitt J, Appiah-Kubi K, Tierney R, Wright W. Vestibular and oculomotor assessments may increase accuracy of subacute concussion assessment. Int J Sports Med. 2016;37(9):738747. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 55.

    Howell DR, Myer GD, Brilliant A, Barber Foss K, Meehan WP. Quantitative multimodal assessment of concussion recovery in youth athletes. Clin J Sport Med. February 2019:16.

    • Search Google Scholar
    • Export Citation
  • 56.

    Slobounov S, Cao C, Sebastianelli W, Slobounov E, Newell K. Residual deficits from concussion as revealed by virtual time-to-contact measures of postural stability. Clin Neurophysiol. 2008;119(2):281289. PubMed ID: 18068428 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 57.

    Lovell MR, Solomon GS. Neurocognitive test performance and symptom reporting in cheerleaders with concussions. J Pediatr. 2013;163(4):1192-5.e1. doi:

  • 58.

    Lynall RC, Mauntel TC, Pohlig RT, et al. Lower extremity musculoskeletal injury risk after concussion recovery in high school athletes. J Athl Train. 2017;52(11):10281034. PubMed ID: 29140128 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 59.

    McPherson AL, Nagai T, Webster KE, Hewett TE. Musculoskeletal injury risk after sport-related concussion: a systematic review and meta-analysis. Am J Sports Med. 2018:036354651878590. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 60.

    Howell DR, Buckley TA, Lynall RC, Meehan WP. Worsening dual-task gait costs after concussion and their association with subsequent sport-related injury. J Neurotrauma. 2018;35(14):16301636. PubMed ID: 29490564 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 61.

    Oldham JR, Howell DR, Knight CA, Crenshaw JR, Buckley TA. Gait performance is associated with subsequent lower extremity injury following concussionMed Sci Sports Exerc. 2020;52(11):22792285. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 62.

    Littleton AC, Register-Mihalik JK, Guskiewicz KM. Test-retest reliability of a computerized concussion test: CNS vital signs. Sports Health. 2015;7(5):443447. PubMed ID: 26502421 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 63.

    Makdissi M, Schneider KJ, Feddermann-Demont N, et al. Approach to investigation and treatment of persistent symptoms following sport-related concussion: A systematic review. Br J Sports Med. 2017;51(12):958968. PubMed ID: 28483928 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 546 546 35
Full Text Views 21 21 5
PDF Downloads 22 22 4