Identifying and Quantifying the Unintended Variability in Common Systematic Observation Instruments to Measure Youth Physical Activity

in Journal of Physical Activity and Health
Restricted access

Purchase article

USD  $24.95

Student 1 year subscription

USD  $115.00

1 year subscription

USD  $153.00

Student 2 year subscription

USD  $218.00

2 year subscription

USD  $285.00

Background: Direct observation protocols may introduce variability in physical activity estimates. Methods: Thirty-five physical education lessons were video recorded and coded using the System for Observing Fitness Instruction Time (SOFIT). A multistep process examined variability in moderate to vigorous physical activity (MVPA%; walking + vigorous/total scans). Initially, per-SOFIT protocol MVPA% (MVPA%SOFIT) estimates were produced for each lesson. Second, true MVPA% (mean MVPA% of all students using all observations, MVPA%true) estimates were calculated. Third, MVPA% (MVPA%perm) was calculated based on all permutations of students and observation order. Fourth, physical education lessons were divided into 2 groups with 5 lessons from each group randomly selected 10,000 times. Group MVPA%perm differences between the 10 selected lessons were compared with the MVPA%true difference between group 1 and group 2. Results: Across all lessons, 10,212,600 permutations were possible (average 291,789 combinations per lesson; range = 73,440–570,024). Across lessons, the average absolute difference between MVPA%true and MVPA%SOFIT estimates was ±4.8% (range = 0.1%–17.5%). Permutations, based on students selected and observation order, indicated that the mean range of MVPA%perm estimates was 41.6% within a lesson (range = 29.8%–55.9%). Differences in MVPA% estimates between the randomly selected groups of lessons varied by 32.0%. Conclusion: MVPA% estimates from focal child observation should be interpreted with caution.

Weaver, Whitfield, and Beets are with the Dept of Exercise Science, Arnold School of Public Health, University of South Carolina, Columbia, SC. Beighle and Erwin are with the Dept of Kinesiology and Health Promotion, University of Kentucky, Lexington, KY. Hardin is with the Dept of Epidemiology & Biostatistics, Arnold School of Public Health, University of South Carolina, Columbia, SC.

Weaver (weaverrg@mailbox.sc.edu) is corresponding author.
  • 1.

    Sirard JR, Pate RR. Physical activity assessment in children and adolescents. Sports Med. 2001;31(6):439–454. PubMed doi:10.2165/00007256-200131060-00004

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2.

    McKenzie T, van der Mars H. Top 10 research questions related to assessing physical activity and its contexts using systematic observation. Res Q Exerc Sport. 2015;86(1):13–29. PubMed doi:10.1080/02701367.2015.991264

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    McKenzie T, Sallis JF, Nader PR. SOFIT—system for observing fitness instruction time. J Teach Phys Educ. 1991;11(2):195–205. doi:10.1123/jtpe.11.2.195

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 4.

    Ridgers ND, Stratton G, McKenzie T. Reliability and validity of the system for observing children’s activity and relationships during play (SOCARP). J Phys Act Health. 2010;7:17–25. PubMed doi:10.1123/jpah.7.1.17

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Brown WH, Pfeiffer KA, McIver KL, Dowda M, Almeida MJCA, Pate RR. Assessing preschool children’s physical activity: the observational system for recording physical activity in children-preschool version. Res Q Exerc Sport. 2006;77(2):167–176. PubMed

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    McNamee J, Van Der Mars H. Accuracy of momentary time sampling: a comparison of varying interval lengths using SOFIT. J Teach Phys Educ. 2005;24(3):282–292. doi:10.1123/jtpe.24.3.282

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    McKenzie T, Sallis JF, Nader PR, et al. BEACHES: an observational system for assessing children’s eating and physical activity behaviors and associated events. J Appl Behav Anal. 1991;24(1):141–151. PubMed doi:10.1901/jaba.1991.24-141

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Mcclain J, Abraham T, Brusseau T, Tudor-Locke C. Epoch length and accelerometer outputs in children: comparison to direct observation. Med Sci Sports Exerc. 2008;40(12):2080–2087. PubMed doi:10.1249/MSS.0b013e3181824d98

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Saint-Maurice P, Welk G, Ihmels M, Krapfl J. Validation of the SOPLAY direct observation tool with an accelerometry-based physical activity monitor. J Phys Act Health. 2011;8(8):1108–1116. PubMed doi:10.1123/jpah.8.8.1108

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Heath EM, Coleman KJ, Lensegrav TL, Fallon JA. Using momentary time sampling to estimate minutes of physical activity in physical education: validation of scores for the system for observing fitness instruction time. Res Q Exerc Sport. 2006;77(1):142–146. PubMed

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Pope RP, Coleman KJ, Gonzalez EC, Barron F, Heath EM. Validity of a revised system for observing fitness instruction time (SOFIT). Pediatr Exerc Sci. 2002;14(2):135–146. doi:10.1123/pes.14.2.135

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12.

    Rowe P, van der Mars H, Schuldheisz J, Fox S. Measuring students’ physical activity levels: validating SOFIT for use with high-school students. J Teach Phys Educ. 2004;23(3):235–251. doi:10.1123/jtpe.23.3.235

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13.

    Scruggs PW. A comparative analysis of pedometry in measuring physical activity of children. Med Sci Sports Exerc. 2007;39(10):1837–1846. PubMed doi:10.1249/mss.0b013e318126c1aa

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14.

    McIver KL, Brown WH, Pfeiffer KA, Dowda M, Pate RR. Assessing children’s physical activity in their homes: the observational system for recording physical activity in children-home. J Appl Behav Anal. 2009;42(1):1–16. PubMed doi:10.1901/jaba.2009.42-1

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15.

    McKenzie T. SOFIT—system for observing fitness instruction time: SOFIT protocol. San Diego, CA: San Diego State University; 2015.

  • 16.

    Donnelly JE, Greene JL, Gibson CA, et al. Physical activity across the curriculum (PAAC): a randomized controlled trial to promote physical activity and diminish overweight and obesity in elementary school children. Prev Med. 2009;49(4):336–341. PubMed doi:10.1016/j.ypmed.2009.07.022

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    Sallis JF, McKenzie T, Alcaraz J, Bohdan K, Faucette N, Hovell M. The effects of a 2-year physical education program (SPARK) on physical activity and fitness in elementary school students. Am J Public Health. 1997;87(8):1328–1334. doi:10.2105/AJPH.87.8.1328

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18.

    Luepker RV, Perry CL, McKinlay SM, et al. Outcomes of a field trial to improve children’s dietary patterns and physical activity. JAMA. 1996;275(10):768–776. PubMed doi:10.1001/jama.1996.03530340032026

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 19.

    McKenzie T, Sallis JF, Prochaska JJ, Conway TL, Marshall SJ, Rosengard P. Evaluation of a two-year middle-school physical education intervention: M-SPAN. Med Sci Sports Exerc. 2004;36(8):1382–1388. PubMed doi:10.1249/01.MSS.0000135792.20358.4D

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20.

    Lonsdale C, Rosenkranz RR, Peralta LR, Bennie A, Fahey P, Lubans DR. A systematic review and meta-analysis of interventions designed to increase moderate-to-vigorous physical activity in school physical education lessons. Prev Med. 2013;56(2):152–161. PubMed doi:10.1016/j.ypmed.2012.12.004

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21.

    Chuang R-J, Sharma SV, Perry C, Diamond P. Does the CATCH early childhood program increase physical activity among low-income preschoolers?—results from a pilot study. Am J Health Promot. 2018;32(2):344–348. doi:10.1177/0890117117700952

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22.

    Cardon G, Verstraete S, De Clercq D, De Bourdeaudhuij I. Research note: physical activity levels in elementary-school physical education: a comparison of swimming and nonswimming classes. J Teach Phys Educ. 2004;23(3):252–263. doi:10.1123/jtpe.23.3.252

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23.

    Chow BC, McKenzie TL, Louie L. Children’s physical activity and environmental influences during elementary school physical education. J Teach Phys Educ. 2008;27(1):38–50. doi:10.1123/jtpe.27.1.38

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24.

    Chow BC, McKenzie TL, Louie L. Physical activity and environmental influences during secondary school physical education. J Teach Phys Educ. 2009;28(1):21–37. doi:10.1123/jtpe.28.1.21

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25.

    Coleman KJ, Geller KS, Rosenkranz RR, Dzewaltowski DA. Physical activity and healthy eating in the after-school environment. J School Health. 2008;78(12):633–640. PubMed doi:10.1111/j.1746-1561.2008.00359.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 26.

    Hollis JL, Williams AJ, Sutherland R, et al. A systematic review and meta-analysis of moderate-to-vigorous physical activity levels in elementary school physical education lessons. Prev Med. 2016;86:34–54. doi:10.1016/j.ypmed.2015.11.018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27.

    Keating XD, Kulinna PH, Silverman S. Measuring teaching behaviors, lesson context, and physical activity in school physical education programs: comparing the SOFIT and the C-SOFIT instruments. Meas Phys Educ Exerc Sci. 1999;3(4):207–220. doi:10.1207/s15327841mpee0304_2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 28.

    Roberts S, Fairclough S. A five-stage process for the development and validation of a systematic observation instrument the system for observing the teaching of games in physical education (SOTG-PE). Eur Phys Educ Rev. 2012;18(1):97–113. doi:10.1177/1356336X11430653

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 29.

    Capio CM, Sit CH, Abernethy B. Physical activity measurement using MTI (actigraph) among children with cerebral palsy. Arch Phys Med Rehabil. 2010;91(8):1283–1290. PubMed doi:10.1016/j.apmr.2010.04.026

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Rowe PJ, Schuldheisz JM, Van Der Mars H. Validation of SOFIT for measuring physical activity of first-to eighth-grade students. Pediatr Exerc Sci. 1997;9(2):136–149. doi:10.1123/pes.9.2.136

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 31.

    Troiano RP, Berrigan D, Dodd KW, Mâsse LC, Tilert T, McDowell M. Physical activity in the United States measured by accelerometer. Med Sci Sports Exerc. 2008;40(1):181–188. PubMed doi:10.1249/mss.0b013e31815a51b3

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32.

    Sallis JF, Prochaska JJ, Taylor WC. A review of correlates of physical activity of children and adolescents. Med Sci Sports Exerc. 2000;32(5):963–975. PubMed doi:10.1097/00005768-200005000-00014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33.

    Sterdt E, Liersch S, Walter U. Correlates of physical activity of children and adolescents: a systematic review of reviews. Health Educ J. 2014;73(1):72–89. doi:10.1177/0017896912469578

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 34.

    Nettlefold L, McKay H, Warburton D, McGuire K, Bredin S, Naylor P. The challenge of low physical activity during the school day: at recess, lunch and in physical education. Br J Sports Med. 2011;45(10):813–819. PubMed doi:10.1136/bjsm.2009.068072

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 35.

    Tudor-Locke C, Lee SM, Morgan CF, Beighle A, Pangrazi RP. Children’s pedometer-determined physical activity during the segmented school day. Med Sci Sports Exerc. 2006;38(10):1732–1738. PubMed doi:10.1249/01.mss.0000230212.55119.98

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 36.

    McKenzie T, Nader PR, Strikmiller PK, et al. School physical education: effect of the Child and Adolescent Trial for Cardiovascular Health. Prev Med. 1996;25(4):423–431. PubMed doi:10.1006/pmed.1996.0074

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 55 55 3
Full Text Views 4 4 0
PDF Downloads 5 5 0