Strengthening the Practice of Exercise and Sport-Science Research

in International Journal of Sports Physiology and Performance
Restricted access

Purchase article

USD  $24.95

Student 1 year subscription

USD  $107.00

1 year subscription

USD  $142.00

Student 2 year subscription

USD  $203.00

2 year subscription

USD  $265.00

Exercise and sport sciences continue to grow as a collective set of disciplines investigating a broad array of basic and applied research questions. Despite the progress, there is room for improvement. A number of problems pertaining to reliability and validity of research practices hinder advancement and the potential impact of the field. These problems include inadequate validation of surrogate outcomes, too few longitudinal and replication studies, limited reporting of null or trivial results, and insufficient scientific transparency. The purpose of this review is to discuss these problems as they pertain to exercise and sport sciences based on their treatment in other disciplines, namely psychology and medicine, and to propose a number of solutions and recommendations.

Halperin and Pyne are with the Physiology Discipline, Australian Inst of Sport, Canberra, ACT, Australia. Halperin is with the Centre for Exercise and Sport Science Research, Edith Cowan University, Joondalup, WA, Australia. Vigotsky is with the Dept of Biomedical Engineering, Northwestern University, Evanston, IL. Foster is with the Dept of Exercise and Sport Science, University of Wisconsin–La Crosse, La Crosse, WI.

Halperin (Israel.Halperin@ausport.gov.au) is corresponding author.
  • 1.

    Hopkins W, Marshall S, Batterham A, Hanin J. Progressive statistics for studies in sports medicine and exercise science. Med Sci Sports Exerc. 2009;41:3–13. PubMed doi:10.1249/MSS.0b013e31818cb278

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2.

    Halperin I, Pyne DB, Martin DT. Threats to internal validity in exercise science: a review of overlooked confounding variables. Int J Sports Physiol Perform. 2015;10(7):823–829. PubMed doi:10.1123/ijspp.2014-0566

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 3.

    Andersen MB, McCullagh P, Wilson GJ. But what do the numbers really tell us?: arbitrary metrics and effect size reporting in sport psychology research. J Sport Exerc Psychol. 2007;29(5):664–672. PubMed doi:10.1123/jsep.29.5.664

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4.

    Ivarsson A, Andersen MB, Stenling A, Johnson U, Lindwall M. Things we still haven’t learned (so far). J Sport Exerc Psychol. 2015;37(4):449–461. PubMed doi:10.1123/jsep.2015-0015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Schweizer G, Furley P. Reproducible research in sport and exercise psychology: the role of sample sizes. Psychol Sport Exerc. 2016;23:114–122. doi:10.1016/j.psychsport.2015.11.005

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 6.

    Ivarsson A, Andersen MB. What counts as “evidence” in evidence-based practice?: searching for some fire behind all the smoke. J Sport Psychol Action. 2016;7(1):11–22. doi:10.1080/21520704.2015.1123206

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):124. doi:10.1371/journal.pmed.0020124

  • 8.

    Earp BD, Trafimow D. Replication, falsification, and the crisis of confidence in social psychology. Front Psychol. 2015;6(621):1–11. PubMed doi:10.3389/fpsyg.2015.00621

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 9.

    Koole SL, Lakens D. Rewarding replications: a sure and simple way to improve psychological science. Perspect Psychol Sci. 2012;7(6):608–614. PubMed doi:10.1177/1745691612462586

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Kidwell MC, Lazarević LB, Baranski E, et al. Badges to acknowledge open practices: a simple, low-cost, effective method for increasing transparency. PLoS Biol. 2016;14(5):e1002456. PubMed. doi:10.1371/journal.pbio.1002456

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Collaboration OS. An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspect Psychol Sci. 2012;7(6):657–660. PubMed doi:10.1177/1745691612462588

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12.

    Nimmo WS, Tucker GT. Clinical Measurement in Drug Evaluation. New York, NY: Wiley; 1995.

  • 13.

    Fleming TR, Powers JH. Biomarkers and surrogate endpoints in clinical trials. Stat Med. 2012;31(25):2973–2984. PubMed doi:10.1002/sim.5403

  • 14.

    Fleming TR. Surrogate endpoints and FDA’s accelerated approval process. Health Aff. 2005;24(1):67–78. PubMed doi:10.1377/hlthaff.24.1.67

  • 15.

    Aronson J. Biomarkers and surrogate endpoints. Br J Clin Pharmacol. 2005;59(5):491–494. PubMed doi:10.1111/j.1365-2125.2005.02435.x

  • 16.

    Stastny P, Gołas´ A, Blazek D, et al. A systematic review of surface electromyography analyses of the bench press movement task. PLoS ONE. 2017;12(2):e0171632. PubMed doi:10.1371/journal.pone.0171632

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    Clark DR, Lambert MI, Hunter AM. Muscle activation in the loaded free barbell squat: a brief review. J Strength Cond Res. 2012;26(4):1169–1178. PubMed doi:10.1519/JSC.0b013e31822d533d

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18.

    Journal of Orthopaedic & Sports Physical Therapy. Strengthening your hip muscles: some exercises may be better than others. J Orthop Sports Phys Ther. 2013;43(2):65. doi:10.2519/jospt.2013.0501

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 19.

    Halperin I, Vigotsky AD. The mind–muscle connection in resistance training: friend or foe? Eur J Appl Physiol. 2016;116(4):863–864. PubMed doi:10.1007/s00421-016-3341-y

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20.

    Echt DS, Liebson PR, Mitchell LB, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. N Engl J Med. 1991;324(12):781–788. PubMed doi:10.1056/NEJM199103213241201

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21.

    Phillips SM. Strength and hypertrophy with resistance training: chasing a hormonal ghost. Eur J Appl Physiol. 2012;112(5):1981–1983. PubMed doi:10.1007/s00421-011-2148-0

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22.

    Schoenfeld BJ. Postexercise hypertrophic adaptations: a reexamination of the hormone hypothesis and its applicability to resistance training program design. J Strength Cond Res. 2013;27(6):1720–1730. PubMed doi:10.1519/JSC.0b013e31828ddd53

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23.

    Damas F, Phillips S, Vechin FC, Ugrinowitsch C. A review of resistance training-induced changes in skeletal muscle protein synthesis and their contribution to hypertrophy. Sports Med. 2015;45(6):801–807. PubMed doi:10.1007/s40279-015-0320-0

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24.

    Atherton PJ, Miller BF, Burd NA, et al. Commentaries on viewpoint: what is the relationship between acute measure of muscle protein synthesis and changes in muscle mass? J Appl Physiol. 2015;118(4):498–503. doi:10.1152/japplphysiol.01069.2014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25.

    Rothwell PM. Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials. 2006;1(1):e9. doi:10.1371/journal.pctr.0010009

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26.

    Pincus T, Stein C. Why randomized controlled clinical trials do not depict accurately long-term outcomes in rheumatoid arthritis: some explanations and suggestions for future studies. Clin Exp Rheumatol. 1996;15:S27–28. doi:10.1007/BF03342657

    • Search Google Scholar
    • Export Citation
  • 27.

    Kiely J. Periodization paradigms in the 21st century: evidence-led or tradition-driven? Int J Sports Physiol Perform. 2012;7(3):242–250. PubMed doi:10.1123/ijspp.7.3.242

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 28.

    Harries SK, Lubans DR, Callister R. Systematic review and meta-analysis of linear and undulating periodized resistance training programs on muscular strength. J Strength Cond Res. 2015;29(4):1113–1125. PubMed doi:10.1519/JSC.0000000000000712

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 29.

    Rhea MR, Ball SD, Phillips WT, Burkett LN. A comparison of linear and daily undulating periodized programs with equated volume and intensity for strength. J Strength Cond Res. 2002;16(2):250–255. PubMed

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Wulf G. Attentional focus and motor learning: a review of 15 years. Int J Sports Psychol. 2013;6(1):77–104. doi:10.1080/1750984x.2012.723728

    • Search Google Scholar
    • Export Citation
  • 31.

    Porter J, Wu W, Partridge J. Focus of attention and verbal instructions: strategies of elite track and field coaches and athletes. Sport Sci Rev. 2010;19(4):77–89.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 32.

    Halperin I, Chapman DW, Martin DT, Abbiss C, Wulf G. Coaching cues in amateur boxing: an analysis of ringside feedback provided between rounds of competition. Psychol Sport Exerc. 2016;25:44–50. doi:10.1016/j.psychsport.2016.04.003

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 33.

    Simmons JP, Nelson LD, Simonsohn U. False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22(11):1359–1366. PubMed doi:10.1177/0956797611417632

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34.

    Forstmeier W, Wagenmakers EJ, Parker TH. Detecting and avoiding likely false-positive findings—a practical guide. Biol Rev Camb Philos Soc. 2016;92(4):1941–1968. PubMed doi:10.1111/brv.12315

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 35.

    Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2011;90(3):891–904. doi:10.1007/s11192-011-0494-7

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 36.

    Nosek BA, Spies JR, Motyl M. Scientific utopia II: restructuring incentives and practices to promote truth over publishability. Perspect Psychol Sci. 2012;7(6):615–631. doi:10.1177/1745691612459058

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 37.

    Fanelli D. “Positive” results increase down the hierarchy of the sciences. PLoS ONE. 2010;5(4):e10068. PubMed doi:10.1371/journal.pone.0010068

  • 38.

    Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):1502–1505. PubMed doi:10.1126/science.1255484

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 39.

    Fanelli D. How many scientists fabricate and falsify research?: a systematic review and meta-analysis of survey data. PLoS ONE. 2009;4(5):e5738. PubMed doi:10.1371/journal.pone.0005738

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 40.

    Kerr NL. HARKing: hypothesizing after the results are known. Pers Soc Psychol Rev. 1998;2(3):196–217. PubMed doi:10.1207/s15327957pspr0203_4

  • 41.

    Chambers CD, Feredoes E, Muthukumaraswamy SD, Etchells P. Instead of “playing the game” it is time to change the rules: registered reports at AIMS Neuroscience and beyond. AIMS Neurosci. 2014;1(1):4–17. doi:10.3934/Neuroscience.2014.1.4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 42.

    Matosin N, Frank E, Engel M, Lum JS, Newell KA. Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture. Dis Model Mech. 2014;7(2):171–173. PubMed doi:10.1242/dmm.015123

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 43.

    Chambers C. Registered reports: a step change in scientific publishing. Elsevier website. http://www.elsevier.com/reviewers-update/story/innovation-in-publishing/registered-reports-a-step-change-in-scientific-publishing; 2014.

    • Search Google Scholar
    • Export Citation
  • 44.

    Nosek BA, Lakens D. Registered reports. Soc Psychol. 2014;45(3):137–141. doi:10.1027/1864-9335/a000192

  • 45.

    Association WM. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191. doi:10.1001/jama.2013.281053

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 46.

    Dessing JC, Beek PJ. Human Movement Science adopts registered reports for hypothesis-driven research. Hum Mov Sci. 2015;44:A1–A2. doi:10.1016/j.humov.2015.09.011

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 47.

    Kaplan RM, Irvin VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS ONE. 2015;10(8):e0132382. PubMed doi:10.1371/journal.pone.0132382

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 48.

    Button KS, Bal L, Clark A, Shipley T. Preventing the ends from justifying the means: withholding results to address publication bias in peer-review. BMC Psychol. 2016;4(1):59. PubMed doi:10.1186/s40359-016-0167-7

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 49.

    Simons DJ. The value of direct replication. Perspect Psychol Sci. 2014;9(1):76–80. PubMed doi:10.1177/1745691613514755

  • 50.

    Schmidt S. Shall we really do it again?: the powerful concept of replication is neglected in the social sciences. Rev Gen Psychol. 2009;13(2):90–100. doi:10.1037/a0015108

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 51.

    Pashler H, Harris CR. Is the replicability crisis overblown?: three arguments examined. Perspect Psychol Sci. 2012;7(6):531–536. PubMed doi:10.1177/1745691612463401

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 52.

    Stroebe W, Strack F. The alleged crisis and the illusion of exact replication. Perspect Psychol Sci. 2014;9(1):59–71. PubMed doi:10.1177/1745691613514450

  • 53.

    Nosek BA, Errington TM. Making sense of replications. Elife. 2017;6:e23383. PubMed doi:10.7554/eLife.23383

  • 54.

    Collaboration OS. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716

  • 55.

    Editorial. Receptive to replication. Nat Biotechnol. 2013;31(11):943. PubMed doi:10.1038/nbt.2748

  • 56.

    Pashler H, Wagenmakers EJ. Editors’ introduction to the special section on replicability in psychological science: a crisis of confidence? Perspect Psychol Sci. 2012;7(6):528–530. PubMed doi:10.1177/1745691612465253

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 57.

    Nosek BA, Bar-Anan Y. Scientific utopia: I: opening scientific communication. Psychol Inq. 2012;23(3):217–243. doi:10.1080/1047840X.2012.692215

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 58.

    Morey RD, Chambers CD, Etchells PJ, et al. The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review. R Soc Open Sci. 2016;3(1):150547. PubMed doi:10.1098/rsos.150547

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 59.

    McKiernan EC, Bourne PE, Brown CT, et al. How open science helps researchers succeed. Elife. 2016;5:e16800. PubMed doi:10.7554/eLife.16800

  • 60.

    Vanpaemel W, Vermorgen M, Deriemaecker L, Storms G. Are we wasting a good crisis?: the availability of psychological research data after the storm. Collabra Psychol. 2015;1(1):1–5. doi:10.1525/collabra.13

    • Search Google Scholar
    • Export Citation
  • 61.

    Bouchard C, Rankinen T. Individual differences in response to regular physical activity. Med Sci Sports Exerc. 2001;33(6):S446–S451. doi:10.1097/00005768-200106001-00013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 62.

    Weissgerber TL, Milic NM, Winham SJ, Garovic VD. Beyond bar and line graphs: time for a new data presentation paradigm. PLoS Biol. 2015;13(4):e1002128. doi:10.1371/journal.pbio.1002128

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 63.

    Saxon E. Beyond bar charts. BMC Biol. 2015;13(1):60. doi:10.1186/s12915-015-0169-6

  • 64.

    Schoenfeld BJ, Aragon A, Wilborn C, Urbina SL, Hayward SE, Krieger J. Pre- versus post-exercise protein intake has similar effects on muscular adaptations. Peer J. 2017;5:e2825. doi:10.7717/peerj.2825

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 576 576 70
Full Text Views 38 38 11
PDF Downloads 21 21 8