In a recent editorial,1 we presented the many lives that a Critically Appraised Topic (CAT) can take upon publication. Through the lens of education, research, and clinical practice, CATs can provide a framework for advancing the external evidence by steering educational content, future research endeavors, and clinical decisions. Within this issue of IJATT, we present an additional consideration for CATs.
As readers are well aware, CATs typically follow the Patient, Intervention, Control/Comparison, Outcome (PICO) format for a focused clinical question and then set out to answer that question. These CATs are a snapshot of the most current research relevant to that question, which is appraised for validity and then synthesized in a clinical bottom line (the answer). By definition, CATs represent smaller scale systematic reviews and include a strong clinical focus.
In fall 2018, I was tasked with reviewing research abstracts for the Eastern Athletic Trainers’ Association (EATA) annual meeting. I also had several students who submitted abstracts that fall. I opened my packet of abstracts to review and was reading the first one . . . to my surprise, it appeared that my student had changed the title of his abstract. Weird, we had been happy with it. I read a little further, and saw a few changes to the original and was about to call my student to see why he revised his abstract after I approved it. I also noted that there was a conflict of interest here and was about to contact the chair of research abstracts in order to recuse myself from scoring this abstract. It wasn’t until the analysis that I realized that this actually wasn’t my student’s abstract, but a very similarly performed CAT on the same topic. Even better, both authors presented their posters at EATA that year at the same time . . . (JMM)
Within this issue, two CATs (McGinnis et al.3 and Booth et al.4) are featured in which the authors sought to answer the same clinical question: “Does vestibular rehabilitation improve dizziness and balance outcomes for athletes who suffered a concussion?” Based on this question, both groups separately developed similar selection criteria and search strategies, and obtained much of the same external evidence. However, it was the analysis of outcomes that resulted in two different CATs (see Figure 1).
In the McGinnis et al.3 CAT, the authors assessed the evidence for the potential benefits of vestibular rehabilitation for improving self-reported measures of dizziness and balance, and also for objective measures of balance. Based on the group means and standard deviations, each of the outcomes demonstrated a substantial change from pre- to posttest in favor of vestibular rehabilitation therapy. However, based on (1) the quality of the included studies and (2) the width of the confidence intervals around the unstandardized changes in the outcomes (95% confidence around the mean difference), there was uncertainty as to whether vestibular rehabilitation would result, on average, in beneficial changes for every athlete who suffered a concussion. In other words, there appeared to be a trend in improvement at the group level, but this was relatively unpredictable at the level of the individual patient.5 Therefore, based on the uncertainty of the results, vestibular rehabilitation should be determined based on an individual patient basis as opposed to being part of the standard of care for everyone.
In the Booth et al.4 CAT, the authors also assessed the evidence for the potential benefits of vestibular rehabilitation for improving self-reported measures of dizziness and balance, and included return to play as an additional outcome of interest. However, these authors chose a different approach in their analysis, examining the (1) magnitude of change, based on the standardized mean difference; (2) the relative chance of improvement due to vestibular rehabilitation; and (3) the numbers needed to treat to benefit (NNTB) for those who underwent vestibular rehabilitation. All these effect sizes were interpreted based on their respective point measures, as well as the credible lower limit (CLL95%). The CLL95% is the lower boundary of the 95% confidence interval. It represents the smallest probable effect for a patient undergoing a treatment6 and allows for a prediction of possible “worst results” or nonresults for a treatment.
The insights gained from the Booth et al.4 CAT help to contextualize the findings from the McGinnis et al.3 CAT. While there is uncertainty in the average improvement with vestibular rehabilitation therapy (McGinnis et al.3), the chance of improvement for an individual patient can be substantial (Booth et al.4). There is a chance for little-to-no improvement with vestibular rehabilitation therapy, but there is also little chance of poor or bad outcomes. In reading these two CATs, be sure to compare and contrast the strategies used for extracting the evidence, and for generating the outcomes to answer very similar clinical questions. Also, we encourage an examination of how these outcomes influenced the separate clinical bottom lines when taken in context of the external evidence quality.5,7
CATs play a critical role in athletic training education, research, and practice.1 When reading a CAT, consider how the authors chose to select and analyze the outcomes for answering the posed question. There may be different ways of extracting study results to construct outcomes, which then may influence the clinical bottom line that answers that clinical question. These two studies (McGinnis et al.3 and Booth et al.4) are examples of CATs that were developed independently in the same time frame, presented at the same conference, and published in the same issue of IJATT. Though complementary, they had differing analyses, results, and interpretations. Be mindful of this anecdote moving forward and be creative in developing new CATs or implementing a different analysis on the outcomes used from the same external evidence. It just goes to show you, there really is more than one way to skin a CAT!
Although Medina McKeon and McKeon were co-authors on the Booth et al.4 article, Johanna Hoch, IJATT associate editor, oversaw the processing of that manuscript in order to maintain the blinded peer-review process.
McKeon PO, Medina McKeon JM. How many lives does a cat really have? Int J Athl Ther Train. 2018;23(5):185–186. doi:10.1123/ijatt.2018-0093
Medina McKeon JM, McKeon PO. PICO: a hot topic in evidence-based practice. Int J Athl Ther Train. 2015;20(1):1–3. doi:10.1123/ijatt.2014-0141
McGinnis I, Cobb J, Tierney R, Russ A. Vestibular rehabilitation for dizziness and imbalance following concussion: a critically appraised topic. Int J Athl Ther Train. 2019;24(3). doi:10.1123/ijatt.2018-0007
Booth M, Powell J, McKeon PO, Medina McKeon JM. Vestibular rehabilitation therapy for management of concussion: a critically appraised topic. Int J Athl Ther Train. 2019;24(3). doi:10.1123/ijatt.2018-0057
Medina McKeon JM, McKeon PO. From strength of recommendation to clinical implementation: weighing and considering the evidence. Int J Athl Ther Train. 2019;24(1):1–4. doi:10.1123/ijatt.2018-0130
Wilkerson GB, Denegar CR. A growing consensus for change in interpretation of clinical research evidence. J Athl Train. 2018;53(3):320–326. PubMed ID: 29624454 doi:10.4085/1062-6050-8-17