Strengthening the Practice of Exercise and Sport-Science Research

in International Journal of Sports Physiology and Performance
Restricted access

Purchase article

USD $24.95

Student 1 year subscription

USD $107.00

1 year subscription

USD $142.00

Student 2 year subscription

USD $203.00

2 year subscription

USD $265.00

Exercise and sport sciences continue to grow as a collective set of disciplines investigating a broad array of basic and applied research questions. Despite the progress, there is room for improvement. A number of problems pertaining to reliability and validity of research practices hinder advancement and the potential impact of the field. These problems include inadequate validation of surrogate outcomes, too few longitudinal and replication studies, limited reporting of null or trivial results, and insufficient scientific transparency. The purpose of this review is to discuss these problems as they pertain to exercise and sport sciences based on their treatment in other disciplines, namely psychology and medicine, and to propose a number of solutions and recommendations.

Halperin and Pyne are with the Physiology Discipline, Australian Inst of Sport, Canberra, ACT, Australia. Halperin is with the Centre for Exercise and Sport Science Research, Edith Cowan University, Joondalup, WA, Australia. Vigotsky is with the Dept of Biomedical Engineering, Northwestern University, Evanston, IL. Foster is with the Dept of Exercise and Sport Science, University of Wisconsin–La Crosse, La Crosse, WI.

Halperin (Israel.Halperin@ausport.gov.au) is corresponding author.
International Journal of Sports Physiology and Performance

Article Sections

References

  • 1.

    Hopkins WMarshall SBatterham AHanin J. Progressive statistics for studies in sports medicine and exercise science. Med Sci Sports Exerc. 2009;41:313. PubMed doi:10.1249/MSS.0b013e31818cb278

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2.

    Halperin IPyne DBMartin DT. Threats to internal validity in exercise science: a review of overlooked confounding variables. Int J Sports Physiol Perform. 2015;10(7):823829. PubMed doi:10.1123/ijspp.2014-0566

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 3.

    Andersen MBMcCullagh PWilson GJ. But what do the numbers really tell us?: arbitrary metrics and effect size reporting in sport psychology research. J Sport Exerc Psychol. 2007;29(5):664672. PubMed doi:10.1123/jsep.29.5.664

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4.

    Ivarsson AAndersen MBStenling AJohnson ULindwall M. Things we still haven’t learned (so far). J Sport Exerc Psychol. 2015;37(4):449461. PubMed doi:10.1123/jsep.2015-0015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Schweizer GFurley P. Reproducible research in sport and exercise psychology: the role of sample sizes. Psychol Sport Exerc. 2016;23:114122. doi:10.1016/j.psychsport.2015.11.005

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 6.

    Ivarsson AAndersen MB. What counts as “evidence” in evidence-based practice?: searching for some fire behind all the smoke. J Sport Psychol Action. 2016;7(1):1122. doi:10.1080/21520704.2015.1123206

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):124. doi:10.1371/journal.pmed.0020124

  • 8.

    Earp BDTrafimow D. Replication, falsification, and the crisis of confidence in social psychology. Front Psychol. 2015;6(621):111. PubMed doi:10.3389/fpsyg.2015.00621

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 9.

    Koole SLLakens D. Rewarding replications: a sure and simple way to improve psychological science. Perspect Psychol Sci. 2012;7(6):608614. PubMed doi:10.1177/1745691612462586

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Kidwell MCLazarević LBBaranski Eet al. Badges to acknowledge open practices: a simple, low-cost, effective method for increasing transparency. PLoS Biol. 2016;14(5):e1002456. PubMed. doi:10.1371/journal.pbio.1002456

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Collaboration OS. An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspect Psychol Sci. 2012;7(6):657660. PubMed doi:10.1177/1745691612462588

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12.

    Nimmo WSTucker GT. Clinical Measurement in Drug Evaluation. New York, NY: Wiley; 1995.

  • 13.

    Fleming TRPowers JH. Biomarkers and surrogate endpoints in clinical trials. Stat Med. 2012;31(25):29732984. PubMed doi:10.1002/sim.5403

  • 14.

    Fleming TR. Surrogate endpoints and FDA’s accelerated approval process. Health Aff. 2005;24(1):6778. PubMed doi:10.1377/hlthaff.24.1.67

  • 15.

    Aronson J. Biomarkers and surrogate endpoints. Br J Clin Pharmacol. 2005;59(5):491494. PubMed doi:10.1111/j.1365-2125.2005.02435.x

  • 16.

    Stastny PGołas´ ABlazek Det al. A systematic review of surface electromyography analyses of the bench press movement task. PLoS ONE. 2017;12(2):e0171632. PubMed doi:10.1371/journal.pone.0171632

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    Clark DRLambert MIHunter AM. Muscle activation in the loaded free barbell squat: a brief review. J Strength Cond Res. 2012;26(4):11691178. PubMed doi:10.1519/JSC.0b013e31822d533d

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18.

    Journal of Orthopaedic & Sports Physical Therapy. Strengthening your hip muscles: some exercises may be better than others. J Orthop Sports Phys Ther. 2013;43(2):65. doi:10.2519/jospt.2013.0501

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 19.

    Halperin IVigotsky AD. The mind–muscle connection in resistance training: friend or foe? Eur J Appl Physiol. 2016;116(4):863864. PubMed doi:10.1007/s00421-016-3341-y

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20.

    Echt DSLiebson PRMitchell LBet al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. N Engl J Med. 1991;324(12):781788. PubMed doi:10.1056/NEJM199103213241201

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21.

    Phillips SM. Strength and hypertrophy with resistance training: chasing a hormonal ghost. Eur J Appl Physiol. 2012;112(5):19811983. PubMed doi:10.1007/s00421-011-2148-0

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22.

    Schoenfeld BJ. Postexercise hypertrophic adaptations: a reexamination of the hormone hypothesis and its applicability to resistance training program design. J Strength Cond Res. 2013;27(6):17201730. PubMed doi:10.1519/JSC.0b013e31828ddd53

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23.

    Damas FPhillips SVechin FCUgrinowitsch C. A review of resistance training-induced changes in skeletal muscle protein synthesis and their contribution to hypertrophy. Sports Med. 2015;45(6):801807. PubMed doi:10.1007/s40279-015-0320-0

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24.

    Atherton PJMiller BFBurd NAet al. Commentaries on viewpoint: what is the relationship between acute measure of muscle protein synthesis and changes in muscle mass? J Appl Physiol. 2015;118(4):498503. doi:10.1152/japplphysiol.01069.2014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25.

    Rothwell PM. Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials. 2006;1(1):e9. doi:10.1371/journal.pctr.0010009

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26.

    Pincus TStein C. Why randomized controlled clinical trials do not depict accurately long-term outcomes in rheumatoid arthritis: some explanations and suggestions for future studies. Clin Exp Rheumatol. 1996;15:S2728. doi:10.1007/BF03342657

    • Search Google Scholar
    • Export Citation
  • 27.

    Kiely J. Periodization paradigms in the 21st century: evidence-led or tradition-driven? Int J Sports Physiol Perform. 2012;7(3):242250. PubMed doi:10.1123/ijspp.7.3.242

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 28.

    Harries SKLubans DRCallister R. Systematic review and meta-analysis of linear and undulating periodized resistance training programs on muscular strength. J Strength Cond Res. 2015;29(4):11131125. PubMed doi:10.1519/JSC.0000000000000712

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 29.

    Rhea MRBall SDPhillips WTBurkett LN. A comparison of linear and daily undulating periodized programs with equated volume and intensity for strength. J Strength Cond Res. 2002;16(2):250255. PubMed

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Wulf G. Attentional focus and motor learning: a review of 15 years. Int J Sports Psychol. 2013;6(1):77104. doi:10.1080/1750984x.2012.723728

    • Search Google Scholar
    • Export Citation
  • 31.

    Porter JWu WPartridge J. Focus of attention and verbal instructions: strategies of elite track and field coaches and athletes. Sport Sci Rev. 2010;19(4):7789.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 32.

    Halperin IChapman DWMartin DTAbbiss CWulf G. Coaching cues in amateur boxing: an analysis of ringside feedback provided between rounds of competition. Psychol Sport Exerc. 2016;25:4450. doi:10.1016/j.psychsport.2016.04.003

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 33.

    Simmons JPNelson LDSimonsohn U. False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22(11):13591366. PubMed doi:10.1177/0956797611417632

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34.

    Forstmeier WWagenmakers EJParker TH. Detecting and avoiding likely false-positive findings—a practical guide. Biol Rev Camb Philos Soc. 2016;92(4):19411968. PubMed doi:10.1111/brv.12315

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 35.

    Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2011;90(3):891904. doi:10.1007/s11192-011-0494-7

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 36.

    Nosek BASpies JRMotyl M. Scientific utopia II: restructuring incentives and practices to promote truth over publishability. Perspect Psychol Sci. 2012;7(6):615631. doi:10.1177/1745691612459058

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 37.

    Fanelli D. “Positive” results increase down the hierarchy of the sciences. PLoS ONE. 2010;5(4):e10068. PubMed doi:10.1371/journal.pone.0010068

  • 38.

    Franco AMalhotra NSimonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):15021505. PubMed doi:10.1126/science.1255484

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 39.

    Fanelli D. How many scientists fabricate and falsify research?: a systematic review and meta-analysis of survey data. PLoS ONE. 2009;4(5):e5738. PubMed doi:10.1371/journal.pone.0005738

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 40.

    Kerr NL. HARKing: hypothesizing after the results are known. Pers Soc Psychol Rev. 1998;2(3):196217. PubMed doi:10.1207/s15327957pspr0203_4

  • 41.

    Chambers CDFeredoes EMuthukumaraswamy SDEtchells P. Instead of “playing the game” it is time to change the rules: registered reports at AIMS Neuroscience and beyond. AIMS Neurosci. 2014;1(1):417. doi:10.3934/Neuroscience.2014.1.4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 42.

    Matosin NFrank EEngel MLum JSNewell KA. Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture. Dis Model Mech. 2014;7(2):171173. PubMed doi:10.1242/dmm.015123

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 43.

    Chambers C. Registered reports: a step change in scientific publishing. Elsevier website. http://www.elsevier.com/reviewers-update/story/innovation-in-publishing/registered-reports-a-step-change-in-scientific-publishing; 2014.

    • Search Google Scholar
    • Export Citation
  • 44.

    Nosek BALakens D. Registered reports. Soc Psychol. 2014;45(3):137141. doi:10.1027/1864-9335/a000192

  • 45.

    Association WM. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191. doi:10.1001/jama.2013.281053

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 46.

    Dessing JCBeek PJ. Human Movement Science adopts registered reports for hypothesis-driven research. Hum Mov Sci. 2015;44:A1A2. doi:10.1016/j.humov.2015.09.011

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 47.

    Kaplan RMIrvin VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS ONE. 2015;10(8):e0132382. PubMed doi:10.1371/journal.pone.0132382

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 48.

    Button KSBal LClark AShipley T. Preventing the ends from justifying the means: withholding results to address publication bias in peer-review. BMC Psychol. 2016;4(1):59. PubMed doi:10.1186/s40359-016-0167-7

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 49.

    Simons DJ. The value of direct replication. Perspect Psychol Sci. 2014;9(1):7680. PubMed doi:10.1177/1745691613514755

  • 50.

    Schmidt S. Shall we really do it again?: the powerful concept of replication is neglected in the social sciences. Rev Gen Psychol. 2009;13(2):90100. doi:10.1037/a0015108

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 51.

    Pashler HHarris CR. Is the replicability crisis overblown?: three arguments examined. Perspect Psychol Sci. 2012;7(6):531536. PubMed doi:10.1177/1745691612463401

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 52.

    Stroebe WStrack F. The alleged crisis and the illusion of exact replication. Perspect Psychol Sci. 2014;9(1):5971. PubMed doi:10.1177/1745691613514450

  • 53.

    Nosek BAErrington TM. Making sense of replications. Elife. 2017;6:e23383. PubMed doi:10.7554/eLife.23383

  • 54.

    Collaboration OS. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716

  • 55.

    Editorial. Receptive to replication. Nat Biotechnol. 2013;31(11):943. PubMed doi:10.1038/nbt.2748

  • 56.

    Pashler HWagenmakers EJ. Editors’ introduction to the special section on replicability in psychological science: a crisis of confidence? Perspect Psychol Sci. 2012;7(6):528530. PubMed doi:10.1177/1745691612465253

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 57.

    Nosek BABar-Anan Y. Scientific utopia: I: opening scientific communication. Psychol Inq. 2012;23(3):217243. doi:10.1080/1047840X.2012.692215

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 58.

    Morey RDChambers CDEtchells PJet al. The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review. R Soc Open Sci. 2016;3(1):150547. PubMed doi:10.1098/rsos.150547

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 59.

    McKiernan ECBourne PEBrown CTet al. How open science helps researchers succeed. Elife. 2016;5:e16800. PubMed doi:10.7554/eLife.16800

  • 60.

    Vanpaemel WVermorgen MDeriemaecker LStorms G. Are we wasting a good crisis?: the availability of psychological research data after the storm. Collabra Psychol. 2015;1(1):15. doi:10.1525/collabra.13

    • Search Google Scholar
    • Export Citation
  • 61.

    Bouchard CRankinen T. Individual differences in response to regular physical activity. Med Sci Sports Exerc. 2001;33(6):S446S451. doi:10.1097/00005768-200106001-00013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 62.

    Weissgerber TLMilic NMWinham SJGarovic VD. Beyond bar and line graphs: time for a new data presentation paradigm. PLoS Biol. 2015;13(4):e1002128. doi:10.1371/journal.pbio.1002128

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 63.

    Saxon E. Beyond bar charts. BMC Biol. 2015;13(1):60. doi:10.1186/s12915-015-0169-6

  • 64.

    Schoenfeld BJAragon AWilborn CUrbina SLHayward SEKrieger J. Pre- versus post-exercise protein intake has similar effects on muscular adaptations. Peer J. 2017;5:e2825. doi:10.7717/peerj.2825

    • Crossref
    • Search Google Scholar
    • Export Citation

Article Metrics

All Time Past Year Past 30 Days
Abstract Views 69 69 51
Full Text Views 4 4 1
PDF Downloads 4 4 2

Altmetric Badge

PubMed

Google Scholar

Cited By