Identifying and Quantifying the Unintended Variability in Common Systematic Observation Instruments to Measure Youth Physical Activity

in Journal of Physical Activity and Health
Restricted access

Purchase article

USD $24.95

Student 1 year subscription

USD $115.00

1 year subscription

USD $153.00

Student 2 year subscription

USD $218.00

2 year subscription

USD $285.00

Background: Direct observation protocols may introduce variability in physical activity estimates. Methods: Thirty-five physical education lessons were video recorded and coded using the System for Observing Fitness Instruction Time (SOFIT). A multistep process examined variability in moderate to vigorous physical activity (MVPA%; walking + vigorous/total scans). Initially, per-SOFIT protocol MVPA% (MVPA%SOFIT) estimates were produced for each lesson. Second, true MVPA% (mean MVPA% of all students using all observations, MVPA%true) estimates were calculated. Third, MVPA% (MVPA%perm) was calculated based on all permutations of students and observation order. Fourth, physical education lessons were divided into 2 groups with 5 lessons from each group randomly selected 10,000 times. Group MVPA%perm differences between the 10 selected lessons were compared with the MVPA%true difference between group 1 and group 2. Results: Across all lessons, 10,212,600 permutations were possible (average 291,789 combinations per lesson; range = 73,440–570,024). Across lessons, the average absolute difference between MVPA%true and MVPA%SOFIT estimates was ±4.8% (range = 0.1%–17.5%). Permutations, based on students selected and observation order, indicated that the mean range of MVPA%perm estimates was 41.6% within a lesson (range = 29.8%–55.9%). Differences in MVPA% estimates between the randomly selected groups of lessons varied by 32.0%. Conclusion: MVPA% estimates from focal child observation should be interpreted with caution.

Weaver, Whitfield, and Beets are with the Dept of Exercise Science, Arnold School of Public Health, University of South Carolina, Columbia, SC. Beighle and Erwin are with the Dept of Kinesiology and Health Promotion, University of Kentucky, Lexington, KY. Hardin is with the Dept of Epidemiology & Biostatistics, Arnold School of Public Health, University of South Carolina, Columbia, SC.

Weaver (weaverrg@mailbox.sc.edu) is corresponding author.
Journal of Physical Activity and Health
Article Sections
References
  • 1.

    Sirard JRPate RR. Physical activity assessment in children and adolescents. Sports Med. 2001;31(6):439454. PubMed doi:10.2165/00007256-200131060-00004

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2.

    McKenzie Tvan der Mars H. Top 10 research questions related to assessing physical activity and its contexts using systematic observation. Res Q Exerc Sport. 2015;86(1):1329. PubMed doi:10.1080/02701367.2015.991264

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    McKenzie TSallis JFNader PR. SOFIT—system for observing fitness instruction time. J Teach Phys Educ. 1991;11(2):195205. doi:10.1123/jtpe.11.2.195

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 4.

    Ridgers NDStratton GMcKenzie T. Reliability and validity of the system for observing children’s activity and relationships during play (SOCARP). J Phys Act Health. 2010;7:1725. PubMed doi:10.1123/jpah.7.1.17

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Brown WHPfeiffer KAMcIver KLDowda MAlmeida MJCAPate RR. Assessing preschool children’s physical activity: the observational system for recording physical activity in children-preschool version. Res Q Exerc Sport. 2006;77(2):167176. PubMed

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    McNamee JVan Der Mars H. Accuracy of momentary time sampling: a comparison of varying interval lengths using SOFIT. J Teach Phys Educ. 2005;24(3):282292. doi:10.1123/jtpe.24.3.282

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    McKenzie TSallis JFNader PRet al. BEACHES: an observational system for assessing children’s eating and physical activity behaviors and associated events. J Appl Behav Anal. 1991;24(1):141151. PubMed doi:10.1901/jaba.1991.24-141

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Mcclain JAbraham TBrusseau TTudor-Locke C. Epoch length and accelerometer outputs in children: comparison to direct observation. Med Sci Sports Exerc. 2008;40(12):20802087. PubMed doi:10.1249/MSS.0b013e3181824d98

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Saint-Maurice PWelk GIhmels MKrapfl J. Validation of the SOPLAY direct observation tool with an accelerometry-based physical activity monitor. J Phys Act Health. 2011;8(8):11081116. PubMed doi:10.1123/jpah.8.8.1108

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Heath EMColeman KJLensegrav TLFallon JA. Using momentary time sampling to estimate minutes of physical activity in physical education: validation of scores for the system for observing fitness instruction time. Res Q Exerc Sport. 2006;77(1):142146. PubMed

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Pope RPColeman KJGonzalez ECBarron FHeath EM. Validity of a revised system for observing fitness instruction time (SOFIT). Pediatr Exerc Sci. 2002;14(2):135146. doi:10.1123/pes.14.2.135

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12.

    Rowe Pvan der Mars HSchuldheisz JFox S. Measuring students’ physical activity levels: validating SOFIT for use with high-school students. J Teach Phys Educ. 2004;23(3):235251. doi:10.1123/jtpe.23.3.235

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13.

    Scruggs PW. A comparative analysis of pedometry in measuring physical activity of children. Med Sci Sports Exerc. 2007;39(10):18371846. PubMed doi:10.1249/mss.0b013e318126c1aa

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14.

    McIver KLBrown WHPfeiffer KADowda MPate RR. Assessing children’s physical activity in their homes: the observational system for recording physical activity in children-home. J Appl Behav Anal. 2009;42(1):116. PubMed doi:10.1901/jaba.2009.42-1

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15.

    McKenzie T. SOFIT—system for observing fitness instruction time: SOFIT protocol. San Diego, CA: San Diego State University; 2015.

  • 16.

    Donnelly JEGreene JLGibson CAet al. Physical activity across the curriculum (PAAC): a randomized controlled trial to promote physical activity and diminish overweight and obesity in elementary school children. Prev Med. 2009;49(4):336341. PubMed doi:10.1016/j.ypmed.2009.07.022

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    Sallis JFMcKenzie TAlcaraz JBohdan KFaucette NHovell M. The effects of a 2-year physical education program (SPARK) on physical activity and fitness in elementary school students. Am J Public Health. 1997;87(8):13281334. doi:10.2105/AJPH.87.8.1328

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18.

    Luepker RVPerry CLMcKinlay SMet al. Outcomes of a field trial to improve children’s dietary patterns and physical activity. JAMA. 1996;275(10):768776. PubMed doi:10.1001/jama.1996.03530340032026

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 19.

    McKenzie TSallis JFProchaska JJConway TLMarshall SJRosengard P. Evaluation of a two-year middle-school physical education intervention: M-SPAN. Med Sci Sports Exerc. 2004;36(8):13821388. PubMed doi:10.1249/01.MSS.0000135792.20358.4D

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20.

    Lonsdale CRosenkranz RRPeralta LRBennie AFahey PLubans DR. A systematic review and meta-analysis of interventions designed to increase moderate-to-vigorous physical activity in school physical education lessons. Prev Med. 2013;56(2):152161. PubMed doi:10.1016/j.ypmed.2012.12.004

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21.

    Chuang R-JSharma SVPerry CDiamond P. Does the CATCH early childhood program increase physical activity among low-income preschoolers?—results from a pilot study. Am J Health Promot. 2018;32(2):344348. doi:10.1177/0890117117700952

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22.

    Cardon GVerstraete SDe Clercq DDe Bourdeaudhuij I. Research note: physical activity levels in elementary-school physical education: a comparison of swimming and nonswimming classes. J Teach Phys Educ. 2004;23(3):252263. doi:10.1123/jtpe.23.3.252

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23.

    Chow BCMcKenzie TLLouie L. Children’s physical activity and environmental influences during elementary school physical education. J Teach Phys Educ. 2008;27(1):3850. doi:10.1123/jtpe.27.1.38

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24.

    Chow BCMcKenzie TLLouie L. Physical activity and environmental influences during secondary school physical education. J Teach Phys Educ. 2009;28(1):2137. doi:10.1123/jtpe.28.1.21

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25.

    Coleman KJGeller KSRosenkranz RRDzewaltowski DA. Physical activity and healthy eating in the after-school environment. J School Health. 2008;78(12):633640. PubMed doi:10.1111/j.1746-1561.2008.00359.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 26.

    Hollis JLWilliams AJSutherland Ret al. A systematic review and meta-analysis of moderate-to-vigorous physical activity levels in elementary school physical education lessons. Prev Med. 2016;86:3454. doi:10.1016/j.ypmed.2015.11.018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27.

    Keating XDKulinna PHSilverman S. Measuring teaching behaviors, lesson context, and physical activity in school physical education programs: comparing the SOFIT and the C-SOFIT instruments. Meas Phys Educ Exerc Sci. 1999;3(4):207220. doi:10.1207/s15327841mpee0304_2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 28.

    Roberts SFairclough S. A five-stage process for the development and validation of a systematic observation instrument the system for observing the teaching of games in physical education (SOTG-PE). Eur Phys Educ Rev. 2012;18(1):97113. doi:10.1177/1356336X11430653

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 29.

    Capio CMSit CHAbernethy B. Physical activity measurement using MTI (actigraph) among children with cerebral palsy. Arch Phys Med Rehabil. 2010;91(8):12831290. PubMed doi:10.1016/j.apmr.2010.04.026

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Rowe PJSchuldheisz JMVan Der Mars H. Validation of SOFIT for measuring physical activity of first-to eighth-grade students. Pediatr Exerc Sci. 1997;9(2):136149. doi:10.1123/pes.9.2.136

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 31.

    Troiano RPBerrigan DDodd KWMâsse LCTilert TMcDowell M. Physical activity in the United States measured by accelerometer. Med Sci Sports Exerc. 2008;40(1):181188. PubMed doi:10.1249/mss.0b013e31815a51b3

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32.

    Sallis JFProchaska JJTaylor WC. A review of correlates of physical activity of children and adolescents. Med Sci Sports Exerc. 2000;32(5):963975. PubMed doi:10.1097/00005768-200005000-00014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33.

    Sterdt ELiersch SWalter U. Correlates of physical activity of children and adolescents: a systematic review of reviews. Health Educ J. 2014;73(1):7289. doi:10.1177/0017896912469578

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 34.

    Nettlefold LMcKay HWarburton DMcGuire KBredin SNaylor P. The challenge of low physical activity during the school day: at recess, lunch and in physical education. Br J Sports Med. 2011;45(10):813819. PubMed doi:10.1136/bjsm.2009.068072

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 35.

    Tudor-Locke CLee SMMorgan CFBeighle APangrazi RP. Children’s pedometer-determined physical activity during the segmented school day. Med Sci Sports Exerc. 2006;38(10):17321738. PubMed doi:10.1249/01.mss.0000230212.55119.98

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 36.

    McKenzie TNader PRStrikmiller PKet al. School physical education: effect of the Child and Adolescent Trial for Cardiovascular Health. Prev Med. 1996;25(4):423431. PubMed doi:10.1006/pmed.1996.0074

    • Crossref
    • Search Google Scholar
    • Export Citation
Article Metrics
All Time Past Year Past 30 Days
Abstract Views 50 50 16
Full Text Views 4 4 0
PDF Downloads 5 5 0
Altmetric Badge
PubMed
Google Scholar