Exertional heat stroke (EHS) is the leading cause of preventable death in athletics.1,2 Athletic trainers (ATs) are often the first responders on the scene of an environmental emergency. Given their vital position for early recognition and management of EHS, athletic trainers must be confident in recognizing, diagnosing, and treating EHS.
A total of three studies have explored athletic training students’ self-confidence with emergency cardiovascular care skills and assessment, but minimal research exists on self-confidence recognizing exertional heat stroke.3–5 In total, one pilot study found that four simulation labs improved athletic training students’ self-confidence in recognizing and managing exertional heat stroke.6 Another study identified entry-level ATs’ confidence levels for diagnosing nonorthopedic conditions were lower than those of orthopedic conditions.7
The Athletic Trainers’ Self-Confidence Scale (ATSCS; Appendix) was developed to measure certified ATs’ self-confidence in skills related to recognizing and managing exertional heat illnesses.8 In their study, Griffes et al.8 found that certified ATs had low self-confidence in their abilities to insert a rectal thermistor and implement cold water immersion. This finding agrees with recent literature identifying a lack of self-confidence for not utilizing best practices for recognizing and managing EHS.9 Athletic training programs should prepare students with the skills to respond to EHS emergencies. However, we can see from previous studies that low self-confidence may impact ATs’ performance. To better understand the impact of EHS education methods, athletic training educators need a reliable and consistent tool to assess athletic training students’ self-confidence. While Griffes et al.8 identified that the ATSCS was internally consistent (Cronbach’s alpha coefficient = .82) and had convergent validity for each item of the nine-item scale; the scale was not evaluated for its test–retest reliability. Therefore, the purpose of this study was to assess the reliability and consistency of the ATSCS when completed by professional master of science in athletic training (MSAT) students. This was an exploratory test–retest reliability research design evaluating the internal consistency, test–retest reliability, and minimal detectable change (MDC) of the ATSCS.
Methods
Procedures
The Weber State University institutional review board approved this study (No. AY21-22-60). Each participant received an email from Qualtrics XM (Qualtrics) with the ATSCS survey on the day they were to complete it. The survey’s first question included the informed consent document for participants to read and decide whether or not to consent to participate in the study. All testing sessions were conducted remotely with the participants’ personal computers. Once the participants completed the survey, they would wait 48 hr, and then a new ATSCS survey link was sent to them. The 48-hr interval was chosen to establish the reliability of the survey as a measure of self-confidence following an educational intervention.
Participants
Professional MSAT students in their first or second year of a Commission on Accreditation of Athletic Training Education accredited program were recruited for participation in this study. All students had previously completed a course where they learned recognition and management of exertional heat illnesses at least four months before participating in this study. To avoid selection bias, we invited all MSAT students who met the inclusion criteria to participate. Inclusion criteria were MSAT students who completed a course with learning outcomes related to recognizing and managing exertional heat illnesses. Exclusion criteria were MSAT students who had not yet taken a class on exertional heat illnesses.
Instrumentation
The ATSCS was delivered via a Qualtrics XM (Qualtrics) survey and distributed to all eligible students. The ATSCS contains nine items asking participants to indicate their level of agreement with each statement of self-confidence in recognizing and managing exertional heat illnesses. This questionnaire utilizes a 7-point Likert scale with 1 (strongly agree) and 7 (strongly disagree).8 The lower the composite score, the higher the respondent’s self-confidence level. All participants completed three testing sessions of the ATSCS with 48 hr between sessions.
Statistical Analysis
The data were analyzed using Statistical Package for the Social Sciences (SPSS, version 25.0) for Macintosh. Collected data included the average item scores and the composite scores from each of the 18 participants over three testing sessions of the ATSCS. Descriptive analyses were conducted for the nine individual items and composite scores for all participants who completed the study (N = 18). A Cronbach’s alpha coefficient was conducted to establish the internal consistency between the nine items of the ATSCS. The following scale was used to establish internal consistency in accordance with George and Mallery.10 Scores of >.9 = excellent, >.8 = good, >.7 = acceptable, >.6 = questionable, >.5 = poor, and ≤.5 = unacceptable reliability. An intraclass correlation coefficient (ICC) was calculated on the 18 participants’ composite scores to determine the reliability of the ATSCS between testing sessions for all participants. As defined by Portney and Watkins,11 we considered test–retest reliability as poor (ICC < .50), moderate (ICC .50–.75), or good (ICC > .75). The MDC was calculated with the following formulas: standard error of the measurement
Results
In total, eighteen professional MSAT students (nine first-year and nine second-year students) met the inclusion criteria and completed all three testing sessions of the ATSCS. Descriptives for the individual items and composite scores can be found in Tables 1 and 2. In our sample, Cronbach’s alpha determined the nine items of the ATSCS had good internal consistency (α = .86; 95% CI [ .78, .94]). The ICC indicated moderate test–retest reliability from the first session to the third session, ICC = .75 (95% CI [.497, .893]). The calculated MDC for the composite change score is 6.19.
Individual Item Descriptives
| ATSCS questions | ATSCS test | Mean | SD | n | SEm | Min | Max |
|---|---|---|---|---|---|---|---|
| 1 | Test 1 | 2.33 | 0.59 | 18 | 0.14 | 1.00 | 3.00 |
| Test 2 | 2.22 | 0.73 | 18 | 0.17 | 1.00 | 4.00 | |
| Test 3 | 2.00 | .49 | 18 | 0.11 | 1.00 | 3.00 | |
| 2 | Test 1 | 2.44 | 0.70 | 18 | 0.17 | 1.00 | 4.00 |
| Test 2 | 2.06 | 0.54 | 18 | 0.13 | 1.00 | 3.00 | |
| Test 3 | 1.94 | 0.42 | 18 | 0.10 | 1.00 | 3.00 | |
| 3 | Test 1 | 2.50 | 0.71 | 18 | 0.17 | 1.00 | 4.00 |
| Test 2 | 2.11 | 0.47 | 18 | 0.11 | 1.00 | 3.00 | |
| Test 3 | 2.11 | 0.47 | 18 | 0.11 | 1.00 | 3.00 | |
| 4 | Test 1 | 2.22 | 0.65 | 18 | 0.15 | 1.00 | 3.00 |
| Test 2 | 2.06 | 0.64 | 18 | 0.15 | 1.00 | 3.00 | |
| Test 3 | 1.89 | 0.58 | 18 | 0.14 | 1.00 | 3.00 | |
| 5 | Test 1 | 1.39 | 0.70 | 18 | 0.16 | 1.00 | 3.00 |
| Test 2 | 1.39 | 0.61 | 18 | 0.14 | 1.00 | 3.00 | |
| Test 3 | 1.33 | 0.49 | 18 | 0.11 | 1.00 | 2.00 | |
| 6 | Test 1 | 2.17 | 0.71 | 18 | 0.17 | 1.00 | 3.00 |
| Test 2 | 2.06 | 0.87 | 18 | 0.21 | 1.00 | 4.00 | |
| Test 3 | 1.89 | 0.68 | 18 | 0.16 | 1.00 | 3.00 | |
| 7 | Test 1 | 2.17 | 0.71 | 18 | 0.17 | 1.00 | 3.00 |
| Test 2 | 2.11 | 0.83 | 18 | 0.20 | 1.00 | 3.00 | |
| Test 3 | 1.78 | 0.65 | 18 | 0.15 | 1.00 | 3.00 | |
| 8 | Test 1 | 1.78 | 0.81 | 18 | 0.19 | 1.00 | 3.00 |
| Test 2 | 1.67 | 0.91 | 18 | 0.21 | 1.00 | 4.00 | |
| Test 3 | 1.61 | 0.70 | 18 | 0.16 | 1.00 | 3.00 | |
| 9 | Test 1 | 2.00 | 0.59 | 18 | 0.14 | 1.00 | 3.00 |
| Test 2 | 2.00 | 0.69 | 18 | 0.16 | 1.00 | 3.00 | |
| Test 3 | 1.83 | 0.71 | 18 | 0.17 | 1.00 | 3.00 |
Note. SEm = standard error of measurement; ATSCS = Athletic Trainers’ Self-Confidence Scale; Min = minimum; Max = maximum.
Composite Score Descriptives
| ATSCS test | Mean | SD | n | SEm | Min | Max |
|---|---|---|---|---|---|---|
| 1 | 19.00 | 4.75 | 18 | 1.12 | 10.00 | 28.00 |
| 2 | 17.67 | 5.08 | 18 | 1.20 | 9.00 | 28.00 |
| 3 | 16.39 | 3.62 | 18 | 0.85 | 9.00 | 22.00 |
Note. SEm = standard error of measurement; ATSCS = Athletic Trainers’ Self-Confidence Scale; Min = minimum; Max = maximum.
Discussion
Athletic training educators must prepare athletic training students to perform lifesaving, critical skills confidently. Using assessment tools to establish the current self-confidence of athletic training students can inform educators on effective pedagogical strategies. This study’s descriptive statistics indicate our participants range from somewhat agree to strongly agree with each of the statements on their confidence to recognize and manage exertional heat illnesses. It is important to note that all participants had completed extensive didactic and hands-on experience on exertional heat illnesses between 4 and 16 months before beginning this study. This education included the recognition of exertional heat stroke through practicing rectal thermometry on low-fidelity task trainers, high-fidelity simulation manikins, and standardized patients followed by the management of exertional heat illnesses. Future use of the ATSCS may involve issuing a preintervention survey to establish a baseline for students’ self-confidence and then a postintervention survey to identify changes the intervention had on self-confidence. Based on the results of this test–retest study, the pre- and postintervention surveys may be issued within 48 hr of each other and still identify reliable results.
Our analysis found the ATSCS to be a consistent and reliable scale to measure students’ self-confidence over time. The calculated Cronbach’s alpha for the ATSCS indicates that each of the nine items measures a single construct, self-confidence in recognizing and managing exertional heat illnesses. These results are consistent with the original internal consistency report by Griffes et al.8 on the assessment of certified ATs’ self-confidence. Moderate test–retest reliability from the first test session to the third test session informs researchers that the ATSCS should identify actual changes in self-confidence that are not due to chance alone.
While the ICC indicates the ATSCS is moderately reliable, it does not offer interpretation for a meaningful change when performing repeated testing sessions. The MDC represents clinical significance, and it refers to the smallest amount of difference that would indicate an actual change and not a random variation.12 The MDC should be utilized in future studies when interpreting results and making a practical significance decision on if an educational intervention truly causes a change in athletic training students’ self-confidence. The difference between the means of the initial and the final tests of the ATSCS in this test–retest reliability study was less than 3 points which did not meet the threshold for clinical significance. The MDC indicates that for clinically significant differences in athletic training students’ self-confidence, there must be a change of at least 6.19 points in the ATSCS score when conducting repeated testing.
Clinical Implications
Athletic trainers are required to recognize and manage life-threatening conditions such as EHS and cardiac emergencies. Educators must prepare their students to confidently perform critical, lifesaving tasks such as rectal thermometry and cold water immersion. Athletic training educators can use the ATSCS to establish the level of an athletic training student’s self-confidence in recognizing and managing EHS. While the results of this study show promise, we cannot assume that athletic training students’ self-confidence will directly correlate to clinical competence. Athletic training educators need to ensure athletic training students also have the competence in performing skills related to recognizing and managing EHS.
Limitations
Limitations of this study include threats to external validity. Since the participants in this study were selected from one MSAT program, they may not be representative of the entire athletic training student population. Given this study sampled only one MSAT program, we do not yet know if these findings and MDC will remain consistent across more programs. In addition, the sample size for this study was only 18 participants. While the sample size may look small, a review conducted in 2017 indicated for three observations, our sample size was sufficient to reach an ICC of .70 or higher.13 Last, we chose a narrow window of 48 hr between survey administrations for this study. This is not a long enough window for participants to forget their answers to each question. Therefore, we do not yet know if the 48-hr interval is the most appropriate interval to assess self-confidence changes due to the possibility of participant recall. However, given the purpose of the ATSCS as an educational intervention outcome, we felt the 48-hr intervals were justified. Future research is recommended to explore educational interventions’ immediate and prolonged effect on athletic training student self-confidence in recognizing and managing exertional heat illnesses.
Conclusion
Athletic training educators are required to prepare athletic training students with the competence and confidence to recognize and treat exertional heat illnesses. This study indicates the ATSCS is a reliable and consistent tool for determining the level of athletic training students’ self-confidence. Educators and researchers should use the calculated MDC to assess the effect of educational interventions on athletic training students’ self-confidence.
References
- 1.↑
Casa DJ, DeMartini JK, Bergeron MF, et al. National athletic trainers’ association position statement: exertional heat illnesses. J Athl Train. 2015;50(9):986–1000. doi:10.4085/1062-6050-50.9.07
- 2.↑
Belval LN, Casa DJ, Adams WM, et al. Consensus statement-prehospital care of exertional heat stroke. Prehospital Emerg Care. 2018;22(3):392–397. doi:10.1080/10903127.2017.1392666
- 3.↑
Paloncy KA, Georges L, Liggett AJ. A high-fidelity simulation is effective in improving athletic training students’ self-efficacy with emergency cardiovascular care skills. Athl Train Educ J. 2019;14(2):108–116. doi:10.4085/1402108
- 4.
Tivener KA, Gloe DS. The effect of high-fidelity cardiopulmonary resuscitation (CPR) simulation on athletic training student knowledge, confidence, emotions, and experiences. Athl Train Educ J. 2015;10(2):103–112. doi:10.4085/1002103
- 5.↑
Doherty-Restrepo JL, Harrelson KE, Swinnie T, Montalvo AM. Does simulation-based training increase athletic training students’ clinical confidence and competence in performing a cardiovascular screening? J Allied Health. 2017;46(3):171–177. PubMed ID: 28889167
- 6.↑
Stedge HL, Herzog V. Self-confidence and perceptions of athletic training students following simulated experiences: a mixed-methods pilot study. Internet J Allied Health Sci Pract. 2021;19(3):1–10. doi:10.46743/1540-580X/2021.1973
- 7.↑
Morin GE, Misasi S, Davis C, Hannah C, Rothbard M. Entry-level athletic trainers’ self-confidence in clinical skill preparedness for treating athletic and emergent settings populations. Athl Train Educ J. 2014;9(4):166–173. doi:10.4085/0904166
- 8.↑
Griffes LS, Kutz M, Paule-Koba A. Athletic trainer’s self-confidence and experience level in managing exertional heat related illnesses. JSMAHS. 2016;2(1):2. doi:10.25035/jsmahs.02.01.02
- 9.↑
Schellhase KC, Plant J, Mazerolle SM. Athletic trainers’ attitudes and perceptions regarding exertional heat stroke before and after an educational intervention. Athl Train Educ J. 2017;12(3):179–187. doi:10.4085/1203179
- 10.↑
George D, Mallery P. IBM SPSS Statistics 25 Step by Step: A Simple Guide and Reference. 15th ed. Routledge; 2019.
- 11.↑
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 3rd ed. Pearson/Prentice Hall; 2009.
- 12.↑
Haley SM, Fragala-Pinkham MA. Interpreting change scores of tests and measures used in physical therapy. Phys Ther. 2006;86(5):735–743. PubMed ID: 16649896 doi:10.1093/ptj/86.5.735
- 13.↑
Bujang MA, Baharum N. A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: a review. Arch Orofac Sci. 2017;12(1):1–11.
Appendix: The Athletic Trainer’s Self-Confidence Scale
Please answer the questions based on your capabilities right now as a student athletic trainer and how confident you are that you can do the following tasks at the present time.
- 1.I can always manage to solve difficult problems when working with an exertional heat illness patient.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 2.If a heat illness emergency arises, I can identify the problem in a timely manner so no one is harmed.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 3.I can establish return to activity goals for individuals who have recently encountered a heat illness.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 4.I am confident that I can efficiently respond to unexpected exertional heat illness events and be able to recognize, treat, and prevent them in a proper amount of time.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 5.I know the proper way to administer a rectal thermometer.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 6.I can create a heat illness action plan for patients’ practices and games.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 7.I remain calm if a patient were facing a life-threatening heat illness condition because I can rely on my coping skills
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 8.When confronted with a patient exhibiting signs/symptoms of heat illness, I know how to correctly use the rectal thermometer to determine if the patient is suffering from exertional heat stroke and when to remove them from an ice bath based on their core temperature.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
- 9.If a patient is in trouble, I can think of an emergency plan.
- •Strongly agree
- •Agree
- •Somewhat agree
- •Undecided
- •Somewhat disagree
- •Disagree
- •Strongly disagree
