Cross-Validation and Comparison of Energy Expenditure Prediction Models Using Count-Based and Raw Accelerometer Data in Youth

in Journal for the Measurement of Physical Behaviour
Restricted access

Purchase article

USD  $24.95

Student 1 year subscription

USD  $37.00

1 year subscription

USD  $50.00

Student 2 year subscription

USD  $71.00

2 year subscription

USD  $93.00

Background: Machine learning may improve energy expenditure (EE) prediction from body-worn accelerometers. However, machine learning models are rarely cross-validated in an independent sample, and the use of machine learning raises additional questions including the effect of accelerometer placement and data type (count vs. raw) for optimal EE prediction. Purpose: To assess the accuracy of artificial neural network (ANN) models for EE prediction in youth using count-based or raw data from accelerometers worn on the hip, wrist, or in combination, and compare these to count-based, EE regression equations. Methods: Data were collected in two settings; one (n = 27) to calibrate the EE prediction models, and the other (n = 34) for model cross-validation. Participants wore a portable metabolic analyzer (EE criterion) and accelerometers on the left wrist and right hip while completing 30 minutes of exergames (calibration, cross-validation) and a maximal exercise test (calibration only). Six ANNs were created from the calibration data, separately by accelerometer placement (hip, wrist, combination) and data format (count-based, raw) to predict EE (15-second epochs). Three count-based linear regression equations were also developed for comparison to the ANNs. Results: The count-based, hip ANN demonstrated lower error (RMSE: 1.2 METs) than all other ANNs (RMSE: 1.7–3.6 METs) and EE regression equations (RMSE: 1.5–3.2 METs). However, all models showed bias toward the mean. Conclusion: An ANN developed for hip-worn accelerometers had higher accuracy for EE prediction during an exergame session than wrist or combination ANNs, and ANNs developed using count-based data had higher accuracy than ANNs developed using raw data.

Montoye is with Integrative Physiology and Health Science, Alma College, Alma, MI. Clevenger and Pfeiffer are with the Department of Kinesiology, Michigan State University, East Lansing, MI. Mackintosh and McNarry are with The Applied Sports Technology Exercise and Medicine Research Centre, Swansea University, Swansea, Wales, United Kingdom.

Montoye (montoyeah@alma.edu) is corresponding author.
  • ActiGraph. (2016). What are counts? Retrieved from https://actigraphcorp.force.com/support/s/article/What-are-counts

  • Bai, J., Di, C., Xiao, L., Evenson, K.R., LaCroix, A.Z., Crainiceanu, C.M., & Buchner, D.M. (2016). An activity index for raw accelerometry data and its comparison with other activity metrics. PLoS One, 11, e0160644. PubMed ID: 27513333. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bailey, R.C., Olson, J., Pepper, S.L., Porszasz, J., Barstow, T.J., & Cooper, D.M. (1995). The level and tempo of children’s physical activities: An observational study. Medicine & Science in Sports & Exercise, 27(7), 1033–1041. PubMed ID: 7564970. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bakrania, K., Yates, T., Rowlands, A.V., Esliger, D.W., Bunnewell, S., Sanders, J., . . . Edwardson, C.L. (2016). Intensity thresholds on raw acceleration data: Euclidean norm minus one (ENMO) and mean amplitude deviation (MAD) approaches. PLoS One, 11, e0164045. PubMed ID: 27706241. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barkman, J., Pfeiffer, K.A., Diltz, A., & Peng, W. (2016). Examining energy expenditure in youth Using Xbox Kinect: Differences by player mode. Journal of Physical Activity and Health, 13, S41–S43. PubMed ID: 27392377. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bland, J.M., & Altman, D.G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307–310. PubMed ID: 2868172

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bouten, C.V., Sauren, A.A., Verduin, M., & Janssen, J.D. (1997). Effects of placement and orientation of body-fixed accelerometers on the assessment of energy expenditure during walking. Medical & Biological Engineering & Computing, 35(1), 50–56. PubMed ID: 9136191. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brønd, J.C., Andersen, L.B., & Arvidsson, D. (2017). Generating ActiGraph counts from raw acceleration recorded by an alternative monitor. Medicine & Science in Sports & Exercise, 49(11), 2351–2360. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brønd, J.C., & Arvidsson, D. (2016). Sampling frequency affects the processing of ActiGraph raw acceleration data to activity counts. Journal of Applied Physiology, 120(3), 362–369. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clevenger, K.A., & Howe, C.A. (2015). Energy cost and enjoyment of active videogames in children and teens: Xbox 360 Kinect. Games for Health Journal, 4, 318–324. PubMed ID: 26182220. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crouter, S.E., Flynn, J.I., & Bassett, D.R., Jr. (2015). Estimating physical activity in youth using a wrist accelerometer. Medicine & Science in Sports & Exercise, 47(5), 944–951. PubMed ID: 25207928. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crouter, S.E., Horton, M., & Bassett, D.R., Jr. (2012). Use of a two-regression model for estimating energy expenditure in children. Medicine & Science in Sports & Exercise, 44(6), 1177–1185. PubMed ID: 22143114. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dannecker, K.L., Sazonova, N.A., Melanson, E.L., Sazonov, E.S., & Browning, R.C. (2013). A comparison of energy expenditure estimation of several physical activity monitors. Medicine & Science in Sports & Exercise, 45, 2105–2112. PubMed ID: 23669877. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, B., Montoye, A., Moore, R., Pfeiffer, K., & Biswas, S. (2013). Energy-aware activity classification using wearable sensor networks. Proceedings of the SPIE International Society for Optical Engineering, 2013, 87230Y.

    • Search Google Scholar
    • Export Citation
  • Dong, B., Biswas, S., Montoye, A., & Pfeiffer, K. (2013). Comparing metabolic energy expenditure estimation using wearable multi-sensor network and single accelerometer. Conference Proceedings of the IEEE Engineering in Medicine and Biology Society, 2013, 2866–2869.

    • Search Google Scholar
    • Export Citation
  • Ellis, K., Kerr, J., Godbole, S., Lanckriet, G., Wing, D., & Marshall, S. (2014). A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers. Physiological Measurement, 35(11), 2191–2203. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Esteban-Cornejo, I., Tejero-Gonzalez, C.M., Sallis, J.F., & Veiga, O.L. (2015). Physical activity and cognition in adolescents: A systematic review. Journal of Science and Medicine in Sport, 18, 534–539. PubMed ID: 25108657. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • FAO/WHO/UNU Expert Consultation. (2001). Energy requirements of adults. Retrieved from http://www.fao.org/docrep/007/y5686e/y5686e07.htm

    • Search Google Scholar
    • Export Citation
  • Freedson, P., Pober, D., & Janz, K.F. (2005). Calibration of accelerometer output for children. Medicine & Science in Sports & Exercise, 37, S523–S530. PubMed ID: 16294115. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Glickman, M.E., Rao, S.R., & Schultz, M.R. (2014). False discovery rate control is a recommended alternative to Bonferroni-type adjustments in health studies. Journal of Clinical Epidemiology, 67(8), 850–857. PubMed ID: 24831050. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Graves, L.E., Ridgers, N.D., & Stratton, G. (2008). The contribution of upper limb and total body movement to adolescents’ energy expenditure whilst playing Nintendo Wii. European Journal of Applied Physiology, 104, 617–623. PubMed ID: 18607619. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gyllensten, I.C., & Bonomi, A.G. (2011). Identifying types of physical activity with a single accelerometer: Evaluating laboratory-trained algorithms in daily life. IEEE Transactions on Biomedical Engineering, 58, 2656–2663. PubMed ID: 21712150. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hibbing, P.R., Ellingson, L.D., Dixon, P.M., & Welk, G.J. (2018). Adapted sojourn models to estimate activity intensity in youth: A suite of tools. Medicine & Science in Sports & Exercise, 50(4), 846–854. PubMed ID: 29135657. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hibbing, P.R., LaMunion, S.R., Kaplan, A.S., & Crouter, S.E. (2018). Estimating energy expenditure with ActiGraph GT9X inertial measurement unit. Medicine & Science in Sports & Exercise, 50(5), 1093–1102. PubMed ID: 29271847. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hwang, J., Fernandez, A.M., & Lu, A.S. (2018). Application and validation of activity monitors’ epoch lengths and placement sites for physical activity assessment in exergaming. Journal of Clinical Medicine, 7(9), E268. PubMed ID: 30208567. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • John, D., & Freedson, P. (2012). ActiGraph and Actical physical activity monitors: A peek under the hood. Medicine & Science in Sports & Exercise, 44, S86–S89. PubMed ID: 22157779. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kerr, J., Patterson, R.E., Ellis, K., Godbole, S., Johnson, E., Lanckriet, G., & Staudenmayer, J. (2016). Objective assessment of physical activity: Classifiers for public health. Medicine & Science in Sports & Exercise, 48(5), 951–957. PubMed ID: 27089222. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lof, M., Henriksson, H., & Forsum, E. (2013). Evaluations of Actiheart, IDEEA and RT3 monitors for estimating activity energy expenditure in free-living women. Journal of Nutritional Sciences, 2, e31. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lyden, K., Keadle, S.K., Staudenmayer, J., & Freedson, P.S. (2014). A method to estimate free-living active and sedentary behavior from an accelerometer. Medicine & Science in Sports & Exercise, 46, 386–397. PubMed ID: 23860415. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lyons, R. (2013). Understanding digital signal processing (3rd ed.). Upper Saddle River, NJ: Prentice Hall.

  • Mackintosh, K.A., Montoye, A.H.K., Pfeiffer, K.A., & McNarry, M. (2016). Investigating optimal accelerometer placement for energy expenditure prediction in children using a machine learning approach. Physiological Measurement, 37, 1728–1740. PubMed ID: 27653339. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McClain, J.J., Abraham, T.L., Brusseau, T.A., Jr, & Tudor-Locke, C. (2008). Epoch length and accelerometer outputs in children: comparison to direct observation. Medicine & Science in Sports & Exercise, 40(12), 2080–2087. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McManus, C. (1991). The inheritance of left-handedness. In: G.R. Bock & J. Marsh (Eds.), Ciba Foundation Symposium: Biological Asymmetry and Handedness. West Sussex, England: John Wiley & Sons Ltd.

    • Search Google Scholar
    • Export Citation
  • McMurray, R.G., Butte, N.F., Crouter, S.E., Trost, S.G., Pfeiffer, K.A., Bassett, D.R., . . . Fulton, J.E. (2015). Exploring metrics to express energy expenditure of physical activity in youth. PLoS One, 10, e0130869. PubMed ID: 26102204. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Montoye, A.H.K., Begum, M., Henning, Z., & Pfeiffer, K.A. (2017). Comparison of linear and non-linear models for predicting energy expenditure from raw accelerometer data. Physiological Measurement, 38, 343–357. PubMed ID: 28107205. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Montoye, A.H.K., Moore, R.W., Bowles, H.R., Korycinski, R., & Pfeiffer, K.A. (2018). Reporting accelerometer methods in physical activity intervention studies: a systematic review and recommendations for authors. British Journal of Sports Medicine, 52(23), 1507–1516. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Montoye, A.H.K., Mudd, L.M., Biswas, S., & Pfeiffer, K.A. (2015). Energy expenditure prediction using raw accelerometer data in simulated free living. Medicine & Science in Sports & Exercise, 47, 1735–1746. PubMed ID: 25494392. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Montoye, A.H.K., Pivarnik, J.M., Mudd, L.M., Biswas, S., & Pfeiffer, K.A. (2016). Wrist-independent energy expenditure prediction models from raw accelerometer data. Physiological Measurement, 37, 1770–1784. PubMed ID: 27653642. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • O’Driscoll, R., Turicchi, J., Beaulieu, K., Scott, S., Matu, J., Deighton, K., . . . Stubbs, J. (2018). How well do activity monitors estimate energy expenditure? A systematic review and meta-analysis of the validity of current technologies. British Journal of Sports Medicine. Advance online publication. doi:

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • Preece, S.J., Goulermas, J.Y., Kenney, L.P., Howard, D., Meijer, K., & Crompton, R. (2009). Activity identification using body-mounted sensors: A review of classification techniques. Physiological Measurement, 30(4), R1–R33. PubMed ID: 19342767 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ripley, B., & Venables, W. (2016). Package ‘nnet’: Feed-forward neural networks and multinomial log-linear models. Retrieved from https://cran.r-project.org/web/packages/nnet/nnet.pdf

    • Search Google Scholar
    • Export Citation
  • Rosdahl, H., Gullstrand, L., Salier-Eriksson, J., Johansson, P., & Schantz, P. (2010). Evaluation of the Oxycon Mobile metabolic system against the Douglas bag method. European Journal of Applied Physiology, 109(2), 159–171. PubMed ID: 20043228. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rosenberg, M., Lay, B., Lee, M., Derbyshire, A., Kur, J., Ferguson, R., . . . Braham, R. (2013). New-generation active videogaming maintains energy expenditure in children across repeated bouts. Games for Health Journal, 2, 274–279. PubMed ID: 26196928. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ryan, J., & Gormley, J. (2013). An evaluation of energy expenditure estimation by three activity monitors. European Journal of Sports Science, 13, 681–688. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sasaki, J.E., John, D., & Freedson, P.S. (2011). Validation and comparison of ActiGraph activity monitors. Journal of Science and Medicine in Sport, 14(5), 411–416. PubMed ID: 21616714. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sasaki, J.E., Hickey, A.M., Staudenmayer, J.W., John, D., Kent, J.A., & Freedson, P.S. (2016). Performance of activity classification algorithms in free-living older adults. Medicine & Science in Sports & Exercise, 48, 941–950. PubMed ID: 26673129. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schofield, W.N. (1985). Predicting basal metabolic rate, new standards and review of previous work. Human Nutrition and Clinical Nutrition, 39(Suppl. 1), 5–41.

    • Search Google Scholar
    • Export Citation
  • Shao, J. (1993). Linear model selection by cross-validation. Journal of the American Statistical Association, 88, 486–494. doi:

  • Staudenmayer, J., He, S., Hickey, A., Sasaki, J., & Freedson, P. (2015). Methods to estimate aspects of physical activity and sedentary behavior from high-frequency wrist accelerometer measurements. Journal of Applied Physiology, 119, 396–403. PubMed ID: 26112238. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Staudenmayer, J., Pober, D., Crouter, S., Bassett, D., & Freedson, P. (2009). An artificial neural network to estimate physical activity energy expenditure and identify physical activity type from an accelerometer. Journal of Applied Physiology, 107, 1300–1307. PubMed ID: 19644028. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Swartz, A.M., Strath, S.J., Bassett, D.R., Jr., O’Brien, W.L., King, G.A., & Ainsworth, B.E. (2000). Estimation of energy expenditure using CSA accelerometers at hip and wrist sites. Medicine & Science in Sports & Exercise, 32, S450–S456. PubMed ID: 10993414. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Troiano, R.P., Berrigan, D., Dodd, K.W., Masse, L.C., Tilert, T., & McDowell, M. (2008). Physical activity in the United States measured by accelerometer. Medicine & Science in Sports & Exercise, 40, 181–188. PubMed ID: 18091006. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Troiano, R.P., McClain, J.J., Brychta, R.J., & Chen, K.Y. (2014). Evolution of accelerometer methods for physical activity research. British Journal of Sports Medicine, 48, 1019–1023. PubMed ID: 24782483. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Trost, S.G., Wong, W.K., Pfeiffer, K.A., & Zheng, Y. (2012). Artificial neural networks to predict activity type and energy expenditure in youth. Medicine & Science in Sports & Exercise, 44, 1801–1809. PubMed ID: 22525766. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • US Department of Health and Human Services. (2018). Physical Activity Guidelines Advisory Committee: 2018 Physical Activity Guidelines for Americans (2nd ed.). Rockville, MD: Office of Disease Prevention and Health Promotion. Retrieved from www.health.gov/paguidelines

    • Search Google Scholar
    • Export Citation
  • van Hees, V.T., Fang, Z., Langford, J., Assah, F., Mohammad, A., da Silva, I.C., . . . Brage, S. (2014). Autocalibration of accelerometer data for free-living physical activity assessment using local gravity and temperature: an evaluation on four continents. Journal of Applied Physiology, 117(7), 738–744. PubMed ID: 25103964. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T., Gorzelniak, L., Dean Leon, E.C., Eder, M., Pias, M., Taherian, S., . . . Brage, S. (2013). Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity. PLoS One, 8(4), e61691. PubMed ID: 23626718. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T., Sabia, S., Anderson, K.N., Denton, S.J., Oliver, J., Catt, M., . . . Singh-Manoux, A. (2015). A novel, open access method to assess sleep duration using a wrist-worn accelerometer. PLoS One, 10(11), e0142533. PubMed ID: 26569414. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, J., Redmond, S.J., Voleno, M., Narayanan, M.R., Wang, N., Cerutti, S., & Lovell, N.H. (2012). Energy expenditure estimation during normal ambulation using triaxial accelerometry and barometric pressure. Physiological Measurement, 33(11), 1811–1830. PubMed ID: 23110944. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 0 0 0
PDF Downloads 0 0 0