Collaborate With a Statistician or Related Methodologist

Statistical errors are pervasive in the biomedical literature, including sport science. Including statisticians or related methodologists as part of the research team can help reduce potential errors in the design, analysis, and reporting of studies. Unfortunately, statistical collaboration remains uncommon in sport science.¹ We recognize that involving methodologists requires effort and resources, as well as shifts in incentives and culture.² While this wish list is “aspirational,” we assert that sport-science journals can help. For example, journal editors and editorial boards can build bridges with statisticians and implement statistical review of selected manuscripts in the first instance, and hopefully to most of them at a later stage. We also encourage editors to explore a peer-review process involving statistical reviewers who would help authors improve the statistics in their papers through a collaborative effort.³

Share Data and Code

Lack of statistical transparency also contributes to the preponderance of statistical errors in the biomedical literature, including sport science. Without access to the underlying data and computer code, it is difficult for peer reviewers to properly vet analyses or detect errors in the data. Unfortunately, data and code sharing remain rare in sport science.⁴ Sport scientists may have legitimate privacy concerns when it comes to data sharing. Indeed, data sets of small samples of elite athletes may be difficult to fully deidentify. However, there are ways to remove the threat of identifiability, such as releasing only a subset of the data or randomly perturbing values by a small amount.⁴ We also recognize that code-based statistical analysis programs (such as SAS or R) may represent a barrier for some sport scientists. However, we believe that the code is critical as it serves as a precise record of the statistical approach. Deidentifying data and writing code are 2 areas where statisticians can help.

Calculate the Needed Sample Size During Study Planning

Small studies predominate in sport science and a priori sample-size calculations are rarely reported.^5,6 Small studies are likely to be underpowered for drawing conclusions beyond the sample. Underpowered studies are wasteful—they may miss effects or estimate effect sizes so imprecisely that the estimates are effectively useless. Furthermore, when an underpowered study finds a significant effect, this finding is less reliable than a significant effect from a well-powered study—the effect is more likely to be spurious and may substantially overestimate the true effect size.⁵ We encourage sport scientists to perform and report formal a priori sample-size calculations, particularly for experimental studies. These calculations aim to ensure sufficient statistical power for a specific hypothesis test or sufficient precision for estimating effect sizes.⁵ Statisticians can help with such calculations. We recognize that achieving adequate sample sizes may not be easy, especially when it comes to elite athlete cohorts, but increasing sample sizes would go a long way toward improving the reliability of sport-science studies.

Provide Data Visualization for Every Statistical Model

Many papers in sport science report correlation or regression analyses without an accompanying plot. A plot is essential to ensure that the model is a good representation of the data. For example, consider a hypothetical data set with 2 continuous variables, x and y. The Pearson’ correlation coefficient between x and y is .64, P = .002. In the absence of a plot, readers may believe that this result indicates a robust relationship. However, the visualization of the plot (Figure 1) reveals that this result is predominantly driven by a single outlier. Therefore, we recommend that every model be accompanied by a figure showing individual-level data points so that reviewers and readers can judge for themselves whether an apparent relationship is robust. Sharing the underlying data also gives reviewers and readers access to this key information.

View Full Size

—Example data set that generates a significant linear relationship between x and y (Pearson correlation coefficient r = .64, P = .002) but is predominantly driven by a single outlier (top right corner of the plot).

Citation: International Journal of Sports Physiology and Performance 17, 5; 10.1123/ijspp.2022-0023

• Download Figure
• Download figure as PowerPoint slide

Account for Multiple Testing

Multiple testing increases the chance of making a type I error. Researchers engage in multiple testing when they test multiple outcome variables, time points, subgroups, or definitions of exposures or outcomes. Multiple testing is common in sport science: One review reported that 232 sport-science studies had run a median of 30 statistical tests related to outcomes; and only 14% had specified a primary outcome.⁶ When conducting experimental studies, we encourage sport scientists to either identify a primary outcome and time point a priori or use formal statistical methods that properly account for multiple testing. While most sport scientists have likely heard of Bonferroni corrections, they may be unaware of more efficient alternatives to Bonferroni. This is another area where a statistician can help. When conducting observational or exploratory studies, accounting for multiple testing may be more informal. For example, researchers may simply put the results in context, as in “We ran over 100 tests comparing the groups and thus would expect to find 5 significant differences just by chance if there were actually no differences between the groups and the tests were independent.”

We recognize that getting more researchers to follow these recommendations will require changes in incentives and culture.² Present times are all about publishing fast.⁷ But we believe that journals such as IJSPP can play a key role in driving change.⁸ Journal policies, practices, and recommendations could go a long way toward realizing this wish list:

1. •Collaborate with a statistician or related methodologist.
2. •Share data and code.
3. •Calculate the needed sample size during study planning.
4. •Provide data visualization for every statistical model.
5. •Account for multiple testing.