Test–Retest Reliability of a Functional Reaction Time Assessment Battery

in Journal of Sport Rehabilitation

Context: Reaction time is commonly assessed postconcussion through a computerized neurocognitive battery. Although this measure is sensitive to postconcussion deficits, it is not clear if computerized reaction time reflects the dynamic reaction time necessary to compete effectively and safely during sporting activities. Functional reaction time assessments may be useful postconcussion, but reliability must be determined before clinical implementation. Objective: To determine the test–retest reliability of a functional reaction time assessment battery and to determine if reaction time improved between sessions. Design: Cohort. Setting: Laboratory. Participants: Forty-one participants (21 men and 20 women) completed 2 time points. Participants, on average, were 22.5 (2.1) years old, 72.5 (11.9) cm tall, had a mass of 71.0 (13.7) kg, and were mostly right leg and hand dominant (92.7%). Interventions: Participants completed 2 clinical reaction time tests (computerized Stroop and drop stick) and 5 functional reaction time tests (gait, jump landing, single-leg hop, anticipated cut, and unanticipated cut) across 2 sessions. Drop stick and functional reaction time assessments were performed in single (motor task only) and dual task (motor task with cognitive task). Main Outcome Measures: Reaction time (in seconds) was calculated during all assessments. Test–retest reliability was determined using 2-way mixed-effects intraclass correlation coefficients (3, k). Paired samples t tests compared mean reaction time between sessions. Results: Test–retest reliability was moderate to excellent for all reaction time outcomes (intraclass correlation coefficients [3, k] range = .766–.925). Several statistically significant between-session mean differences were observed, but effect sizes were negligible to small (d range = 0.05–0.44). Conclusions: The functional reaction time assessment battery displayed similar reliability to the standard computerized reaction time assessment battery and may provide important postinjury information, but more research is needed to determine clinical utility.

Reaction time (RT) is commonly assessed after injury and as a sport performance indicator. RT is negatively affected after concussion1 and orthopedic injury.2 Quicker RT is associated with better professional baseball batting performance3 and improved sport performance in elite youth soccer players.4

RT is often assessed clinically in a stationary manner, but during sport, athletes react in a highly dynamic environment with ever-changing physical and mental stimuli. Functional RT is not correlated with clinically assessed RT,5 but the reliability of this functional RT battery is unclear, impacting its clinical utility. The purpose of this study was to determine the test–retest reliability of a functional RT test battery and to determine if RT improved between the 2 sessions.

Methods

Study Design

We utilized an observational research design with participants tested twice, leaving at least 7 days between testing sessions. Dependent variables included RT measures from each task as described below. Detailed study methodology has been described elsewhere.5

Participants

Forty-one participants signed informed consent documentation approved by the University of Georgia Institutional Review Board. Participants must have been healthy young adults between the ages of 18 and 30 years who participated in ≥30 minutes of moderate or vigorous physical activity ≥3 times or more per week. Participants were excluded if they self-reported lower-extremity injury resulting in physical activity time loss of at least 3 days in the last 6 months, any history of lower-extremity or low-back surgery, a concussion within the past year, ≥3 previous concussions, or any other physical or mental health condition that could impact RT.5 Participants received $50 total for completing both sessions.

Procedures

All participants reported to the laboratory for 2 identical testing sessions separated by at least 7 days (8.1 [1.9] d [range = 7–14 d]). Participants completed 2 clinical RT assessment batteries (computerized Stroop task and drop stick test) and 5 functional RT assessments (gait, jump landing, single-leg hop, anticipated cut, and unanticipated cut); battery order was block randomized.

The computerized Stroop task was completed per standard protocol to mimic clinical testing. Participants were seated in front of a computer and struck the keyboard immediately following the on-screen stimulus. For computerized simple RT, participants struck the spacebar following presentation of any word. For computerized complex RT, participants struck the spacebar when a color word was written in the corresponding color (eg, “red” written in the color red). For computerized Stroop RT, participants struck the spacebar when a color word was not written in the corresponding color (eg, “red” written in the color green). The computerized composite RT score was a combination of the computerized complex and Stroop RT.5

During the drop stick test,5 participants were seated with their dominant forearm and hypothenar eminence resting on a table. A research team member held the instrument (wooden rod embedded in a hockey puck) vertically between the participant’s open hand. The researcher released the stick randomly between 2 and 5 seconds after the participant was told to “get set.” Participants only used a single hand to catch the falling stick. Dropped puck distance was recorded and converted to RT with the following formula: distance = ½ (gravity) × (time).2

All functional RT assessments are described in Table 1. The drop stick test and the functional RT test battery, except for unanticipated cut, were conducted in single and dual task (randomized within each task).5 Single task was the movement portion alone, while dual task was the movement portion with a concurrent cognitive challenge. The cognitive challenge was subtraction from a random starting number between 90 and 150 by either 6s or 7s. Subtraction was initiated prior to trial initiation (ie, when instructed to “get set”) and stopped after successful completion (ie, movement task completely finished). Unanticipated cutting was only performed in single task as pilot testing revealed it was too difficult to complete in dual task.

Table 1

Description of the Functional Reaction Time Assessments

AssessmentStarting positionCueDescriptionReaction time calculation
GaitStandardized mat location while standing still with feet approximately shoulder-width apart.An audible cue was triggered randomly by the research team between 2 and 5 s after the participant was told to “get set.” Participants initiated walking immediately after hearing the audible stimulus.The participant was told to begin walking as quickly as possible after the audible cue but to walk at a comfortable pace. Trials: 5 per session.Time between the audible stimulus and the participant’s center of pressure moving >3 cm in either the frontal or sagittal plane, whichever occurred first.5
Jump landingStanding on a 30-cm box at 50% of their body height behind the target. Participants took an athletic stance after being told to “get set.”A visual cue was triggered randomly by the research team between 2 and 5 s after the participant got into the starting position and was told to “get set.” Participants moved as quickly as possible after the visual stimulus.Participants jumped forward off of the box landing on both legs before immediately completing a maximum vertical jump. Trials: 4 per session.The time between the visual stimulus and sacral marker movement ≥3 cm in either the sagittal or frontal plane, whichever occurred first, from its mean position 0.5 s prior to the visual stimulus.5,6
Single-leg hopStanding on a 30-cm box at 50% of their body height behind the target. Participants took an athletic stance after being told to “get set.”A visual cue was triggered randomly by the research team between 2 and 5 s after the participant got into the starting position and was told to “get set.” Participants moved as quickly as possible after the visual stimulus.Participants jumped off of the box landing on a single leg. They maintained a hands-on hips and single-leg stance immediately upon foot contact for at least 10 s. Trials: 8 total per session, 4 each leg.The time between the visual stimulus and sacral marker movement ≥3 cm in either the sagittal or frontal plane, whichever occurred first, from its mean position 0.5 s prior to the visual stimulus.5,6
Anticipated cutStanding on a 30-cm box at 50% of their body height behind the target. Participants took an athletic stance after being told to “get set.”A visual cue was triggered randomly by the research team between 2 and 5 s after the participant got into the starting position and was told to “get set.” Participants moved as quickly as possible after the visual stimulus.Participants jumped forward off of the box, performing a 45° cutting motion in the direction provided prior to the trial after landing on a single leg. Trials: 8 total per session, 4 each direction.The time between the visual stimulus and sacral marker movement ≥3 cm in either the sagittal or frontal plane, whichever occurred first, from its mean position 0.5 s prior to the visual stimulus.5,6
Unanticipated cutStanding on a 30-cm box at 50% of their body height behind the target. Participants took an athletic stance after being told to “get set.”A visual cue was triggered randomly by the research team between 2 and 5 s after the participant got into the starting position and was told to “get set.” Participants moved as quickly as possible after the visual stimulus.Same movement as above, but the cutting direction was unknown until the participant left the box and triggered a starting gate (0.76 m behind the target), which triggered a visual stimulus indicating the direction the participant should cut. Trials: 10 total per session, 5 each direction.The time between the visual stimulus and sacral marker movement ≥3 cm in either the sagittal or frontal plane, whichever occurred first, from its mean position 0.5 s prior to the visual stimulus.5,6

High-speed motion capture (150 Hz, Qualisys Miqus; Qualisys Motion Capture Systems, Goteborg, Sweden) was used during the functional RT assessments, except gait (Table 1). Each participant had a retroreflective marker placed over their sacral body. Marker data were processed and filtered with a fourth order, low-pass Butterworth filter with a 10 Hz cutoff frequency.6 Gait was assessed using a reliable and valid instrumented mat (4.9-m long by 0.6-m wide, Protokinetics Zeno Walkway; Protokinetics LLC, Havertown, PA) that measured center of pressure at 120 Hz.7 Custom MATLAB code was used to determine functional RT (detailed in Table 1).5,6

Statistical Analyses

Test–retest reliability was determined for each RT assessment and cognitive condition (single or dual task) separately, using 2-way mixed effects intraclass correlation coefficients (3, k) and corresponding 95% confidence intervals.8 We defined poor reliability as intraclass correlation coefficient values <.5, moderate reliability as 0.5 to 0.75, good reliability as 0.75 to 0.89, and excellent reliability as ≥0.9.8 Standard error of the measurement and 95% confidence interval minimal detectable change9 were calculated for each RT assessment and cognitive condition. Paired samples t tests (a priori α = .05) compared mean RT between sessions within a given RT assessment and cognitive condition, and Cohen d effect sizes were calculated (0–0.49, small effect; 0.5–0.79, medium effect; and ≥0.8, large effect).

Results

Twenty-one male and 20 female participants completed the study protocol. Participants were 22.5 (2.1) years old, 72.5 (11.9) cm tall, had a mass of 71.0 (13.7) kg, and were mostly right leg and hand dominant (92.7%). Five participants reported a concussion history with 4 reporting 1, and 1 reporting 2 previous concussions.5

Overall, test–retest reliability was moderate to excellent for all RT outcomes with most assessments falling in the good range (Table 2). Several statistically significant between-session mean differences were observed, but effect sizes overall were small with a single medium effect for dual-task single-leg hop (Table 2).

Table 2

Test–Retest Reliability and Between-Session Comparisons for All Reaction Time Assessments

Reaction time taskMean (SD)aICC3,k (95% CI)SEMMDCP valuebCohen d
Computerized simple
 Session 10.306 (0.042).841 (.701–.915)0.0170.047.6160.05
 Session 20.308 (0.045)
Computerized complex
 Session 10.581 (0.066).833 (.687–.911)0.0270.075<.0010.44
 Session 20.552 (0.066)
Computerized Stroop
 Session 10.655 (0.080).901 (.815–.947)0.0260.072.0380.20
 Session 20.639 (0.084)
Computerized composite
 Session 10.619 (0.066).925 (.859–.960)0.0190.053<.0010.34
 Session 20.596 (0.071)
Drop stick
 Session 1—single task0.202 (0.031).776 (.580–.881)0.0150.042.0470.29
 Session 2—single task0.193 (0.031)
 Session 1—dual task0.216 (0.034).875 (.765–.933)0.0120.033.0040.31
 Session 2—dual task0.205 (0.036)
Gait
 Session 1—single task0.243 (0.042).766 (.560–.875)0.0280.078.2340.18
 Session 2—single task0.253 (0.072)
 Session 1—dual task0.341 (0.103).782 (.592–.884)0.0430.119.0050.40
 Session 2—dual task0.305 (0.079)
Jump landing
 Session 1—single task0.479 (0.090).775 (.578–.880)0.0430.119.0820.24
 Session 2—single task0.457 (0.090)
 Session 1—dual task0.576 (0.103).808 (.639–.897)0.0490.136.0410.27
 Session 2—dual task0.546 (0.119)
Single-leg hop
 Session 1—single task0.522 (0.095).883 (.780–.937)0.0330.091.0190.25
 Session 2—single task0.498 (0.099)
 Session 1—dual task0.636 (0.120).866 (.749–.929)0.0410.114<.0010.53
 Session 2—dual task0.577 (0.104)
Anticipated cut
 Session 1—single task0.445 (0.087).745 (.522–.864)0.0410.114.5280.09
 Session 2—single task0.452 (0.075)
 Session 1—dual task0.540 (0.101).909 (.830–.952)0.0300.083.0660.17
 Session 2—dual task0.523 (0.100)
Unanticipated cutc
 Session 1—single task0.523 (0.103).879 (.772–.935)0.0330.091.0930.17
 Session 2—single task0.507 (0.086)

Abbreviations: CI, confidence interval; ICC, intraclass correlation coefficient; MDC, 95% confidence interval minimal detectable change.

aAll reaction time outcomes are reported in seconds. bP values are from paired samples t tests between sessions for each outcome. cNo dual task unanticipated cuts were performed due to high task difficulty.

Discussion

Functional RT demonstrated good test–retest reliability with some small improvements observed between sessions. Test–retest reliability for single- and dual-task functional RT was similar to a standard computerized RT assessment. Though more research is needed, employing functional RT assessments after orthopedic injuries and concussions may better reflect the on-field scenarios athletes will experience upon return to play.

Except for single-leg hop, test–retest reliability was slightly better during dual task compared with single task during functional RT assessments. The mean differences between cognitive conditions were small, and the clinical implications are not immediately clear. With no cognitive component, it is possible participants put too much focus on a motor task that is not overly complicated for a healthy individual. Functional RT was significantly faster during single task compared with dual task.5 The slightly better reliability during dual task compared with single task may be driven by the reduced mental focus on the motor task itself, allowing for better task repeatability.

We did observe some small but statistically significant between-session differences for several tasks, including computerized complex RT, computerized Stroop RT, computerized composite RT, single- and dual-task drop stick, single-task single-leg hop, and all dual-task functional RT assessments except anticipated cut. However, the largest effect size noted for these statistical differences was 0.53 (dual-task single-leg hop). The minimal detectable change for the dual-task single-leg hop was 0.114 seconds, while the mean difference observed between sessions was 0.017 seconds. Given the medium effect (all other effects were small) and the fact that the mean difference was well less than the minimal detectable change, we believe these between-session differences are negligible, and the statistically significant findings were likely due to a fairly robust sample and outcome measures with minimal variability. Even if these are spurious findings, researchers and clinicians should be careful to take steps to minimize potential learning effects, especially in dual-task situations, which are much more novel to most patients, by fully explaining all tasks and including multiple practice trials.

There are many potential future applications for a functional RT assessment battery. Those with a recent concussion history may demonstrate altered RT compared with healthy controls during functional assessment.6 Slower RT on preinjury neurocognitive assessments may be predictive of musculoskeletal injury risk,10 and athletes who tore their anterior cruciate ligament had slower preinjury RT than those who did not.11 These results suggest RT may provide important information about musculoskeletal injury risk, and it is possible a more functional, sport-like RT assessment may be more sensitive to injury risk than stationary clinical RT. Given the associations between RT and sport performance,3,4 incorporating functional RT assessments may better identify performance deficits that could be improved through targeted rehabilitation.

Our study findings were limited by examining only healthy, physically active individuals. Injury status and athlete competitive level may impact the functional RT battery. Other RT assessments exist but were not investigated here. The clinical utility of this functional RT battery is not fully understood yet and must be further studied before these RT assessments are implemented.

Conclusions

The functional RT tasks utilized here demonstrated moderate to excellent test–retest reliability. In general, dual-task functional assessments were slightly more reliable than their corresponding single-task functional assessment, although the reasons for this are not clear. Functional RT assessments may better reflect sport activities than traditional computerized RT assessments. Future research should focus on functional RT outcomes postinjury to determine how such assessments may contribute to understanding full recovery and successful return to play.

Acknowledgments

This study was supported by a grant from the Mary Frances Early College of Education at the University of Georgia. The authors have no conflict of interest to disclose.

References

  • 1.

    Lempke LB, Howell DR, Eckner JT, Lynall RC. Examination of reaction time deficits following concussion: a systematic review and meta-analysis. Sports Med. 2020;50(7):13411359. PubMed ID: 32162242 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2.

    Armitano-Lago CN, Morrison S, Hoch JM, Bennett HJ, Russell DM. Anterior cruciate ligament reconstructed individuals demonstrate slower reactions during a dynamic postural task. Scand J Med Sci Sports. 2020; 30(8):15181528. PubMed ID: 32343021 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    Laby DM, Kirschen DG, Govindarajulu U, DeLand P. The hand-eye coordination of professional baseball players: the relationship to batting. Optom Vis Sci. 2018;95(7):557567. PubMed ID: 29985271 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4.

    Song YH, Ha SM, Yook JS, Ha MS. Interactive improvements of visual and auditory function for enhancing performance in youth soccer players. Int J Environ Res Public Health. 2019;16(24). doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Lempke LB, Johnson RS, Schmidt JD, Lynall RC. Clinical versus functional reaction time: implications for postconcussion management. Med Sci Sports Exerc. 2020;52(8):16501657. PubMed ID: 32053547 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    Lynall RC, Blackburn JT, Guskiewicz KM, Marshall SW, Plummer P, Mihalik JP. Reaction time and joint kinematics during functional movement in recently concussed individuals. Arch Phys Med Rehabil. 2018;99(5):880886. PubMed ID: 29337022 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 7.

    Lynall RC, Zukowski LA, Plummer P, Mihalik JP. Reliability and validity of the protokinetics movement analysis software in measuring center of pressure during walking. Gait Posture. 2017;52:308311. PubMed ID: 28033577 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155163. PubMed ID: 27330520 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Riemann BL, Lininger MR. Statistical primer for athletic trainers: the essentials of understanding measures of reliability and minimal important change. J Athl Train. 2018;53(1):98103. PubMed ID: 29332472 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    McDonald AA, Wilkerson GB, McDermott BP, Bonacci JA. Risk factors for initial and subsequent core or lower extremity sprain or strain among collegiate football players. J Athl Train. 2019;54(5):489496. PubMed ID: 31084505 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Swanik CB, Covassin T, Stearne DJ, Schatz P. The relationship between neurocognitive function and noncontact anterior cruciate ligament injuries. Am J Sports Med. 2007;35(6):943948. PubMed ID: 17369562 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

The authors are with the UGA Biomechanics Laboratory, Department of Kinesiology, Mary Frances Early College of Education, University of Georgia, Athens, GA, USA.

Lynall (rlynall@uga.edu) is a corresponding author.
  • 1.

    Lempke LB, Howell DR, Eckner JT, Lynall RC. Examination of reaction time deficits following concussion: a systematic review and meta-analysis. Sports Med. 2020;50(7):13411359. PubMed ID: 32162242 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2.

    Armitano-Lago CN, Morrison S, Hoch JM, Bennett HJ, Russell DM. Anterior cruciate ligament reconstructed individuals demonstrate slower reactions during a dynamic postural task. Scand J Med Sci Sports. 2020; 30(8):15181528. PubMed ID: 32343021 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    Laby DM, Kirschen DG, Govindarajulu U, DeLand P. The hand-eye coordination of professional baseball players: the relationship to batting. Optom Vis Sci. 2018;95(7):557567. PubMed ID: 29985271 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4.

    Song YH, Ha SM, Yook JS, Ha MS. Interactive improvements of visual and auditory function for enhancing performance in youth soccer players. Int J Environ Res Public Health. 2019;16(24). doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Lempke LB, Johnson RS, Schmidt JD, Lynall RC. Clinical versus functional reaction time: implications for postconcussion management. Med Sci Sports Exerc. 2020;52(8):16501657. PubMed ID: 32053547 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    Lynall RC, Blackburn JT, Guskiewicz KM, Marshall SW, Plummer P, Mihalik JP. Reaction time and joint kinematics during functional movement in recently concussed individuals. Arch Phys Med Rehabil. 2018;99(5):880886. PubMed ID: 29337022 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 7.

    Lynall RC, Zukowski LA, Plummer P, Mihalik JP. Reliability and validity of the protokinetics movement analysis software in measuring center of pressure during walking. Gait Posture. 2017;52:308311. PubMed ID: 28033577 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155163. PubMed ID: 27330520 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Riemann BL, Lininger MR. Statistical primer for athletic trainers: the essentials of understanding measures of reliability and minimal important change. J Athl Train. 2018;53(1):98103. PubMed ID: 29332472 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    McDonald AA, Wilkerson GB, McDermott BP, Bonacci JA. Risk factors for initial and subsequent core or lower extremity sprain or strain among collegiate football players. J Athl Train. 2019;54(5):489496. PubMed ID: 31084505 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Swanik CB, Covassin T, Stearne DJ, Schatz P. The relationship between neurocognitive function and noncontact anterior cruciate ligament injuries. Am J Sports Med. 2007;35(6):943948. PubMed ID: 17369562 doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 293 293 58
PDF Downloads 114 114 35