Identifying and Quantifying the Unintended Variability in Common Systematic Observation Instruments to Measure Youth Physical Activity
in Journal of Physical Activity and HealthClick name to view affiliation
Background: Direct observation protocols may introduce variability in physical activity estimates. Methods: Thirty-five physical education lessons were video recorded and coded using the System for Observing Fitness Instruction Time (SOFIT). A multistep process examined variability in moderate to vigorous physical activity (MVPA%; walking + vigorous/total scans). Initially, per-SOFIT protocol MVPA% (MVPA%SOFIT) estimates were produced for each lesson. Second, true MVPA% (mean MVPA% of all students using all observations, MVPA%true) estimates were calculated. Third, MVPA% (MVPA%perm) was calculated based on all permutations of students and observation order. Fourth, physical education lessons were divided into 2 groups with 5 lessons from each group randomly selected 10,000 times. Group MVPA%perm differences between the 10 selected lessons were compared with the MVPA%true difference between group 1 and group 2. Results: Across all lessons, 10,212,600 permutations were possible (average 291,789 combinations per lesson; range = 73,440–570,024). Across lessons, the average absolute difference between MVPA%true and MVPA%SOFIT estimates was ±4.8% (range = 0.1%–17.5%). Permutations, based on students selected and observation order, indicated that the mean range of MVPA%perm estimates was 41.6% within a lesson (range = 29.8%–55.9%). Differences in MVPA% estimates between the randomly selected groups of lessons varied by 32.0%. Conclusion: MVPA% estimates from focal child observation should be interpreted with caution.
Weaver, Whitfield, and Beets are with the Dept of Exercise Science, Arnold School of Public Health, University of South Carolina, Columbia, SC. Beighle and Erwin are with the Dept of Kinesiology and Health Promotion, University of Kentucky, Lexington, KY. Hardin is with the Dept of Epidemiology & Biostatistics, Arnold School of Public Health, University of South Carolina, Columbia, SC.
© 2023 Human Kinetics