Application of Convolutional Neural Network Algorithms for Advancing Sedentary and Activity Bout Classification

in Journal for the Measurement of Physical Behaviour
View More View Less
  • 1 University of California San Diego
  • 2 Children’s Mercy Hospital
  • 3 Kaiser Permanente Washington Health Research Institute
Restricted access

Purchase article

USD  $24.95

Student 1 year online subscription

USD  $38.00

1 year online subscription

USD  $51.00

Student 2 year online subscription

USD  $73.00

2 year online subscription

USD  $97.00

Background: Machine learning has been used for classification of physical behavior bouts from hip-worn accelerometers; however, this research has been limited due to the challenges of directly observing and coding human behavior “in the wild.” Deep learning algorithms, such as convolutional neural networks (CNNs), may offer better representation of data than other machine learning algorithms without the need for engineered features and may be better suited to dealing with free-living data. The purpose of this study was to develop a modeling pipeline for evaluation of a CNN model on a free-living data set and compare CNN inputs and results with the commonly used machine learning random forest and logistic regression algorithms. Method: Twenty-eight free-living women wore an ActiGraph GT3X+ accelerometer on their right hip for 7 days. A concurrently worn thigh-mounted activPAL device captured ground truth activity labels. The authors evaluated logistic regression, random forest, and CNN models for classifying sitting, standing, and stepping bouts. The authors also assessed the benefit of performing feature engineering for this task. Results: The CNN classifier performed best (average balanced accuracy for bout classification of sitting, standing, and stepping was 84%) compared with the other methods (56% for logistic regression and 76% for random forest), even without performing any feature engineering. Conclusion: Using the recent advancements in deep neural networks, the authors showed that a CNN model can outperform other methods even without feature engineering. This has important implications for both the model’s ability to deal with the complexity of free-living data and its potential transferability to new populations.

Nakandala and Kumar are with the Department of Computer Science and Engineering, University of California San Diego, La Jolla, CA, USA. Jankowska is with the Qualcomm Institute/Calit2, University of California San Diego, La Jolla, CA, USA. Tuz-Zahra, Bellettiere, LaCroix, Hartman, Zou, and Natarajan are with The Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla, CA, USA. Carlson is with the Center for Children’s Healthy Lifestyles and Nutrition, Children’s Mercy Hospital, Kansas City, MO, USA, and the Department of Pediatrics, Children’s Mercy Hospital and University of Missouri Kansas City, Kansas City, MO, USA. Rosenberg is with the Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA.

Jankowska (majankowska@ucsd.edu) is corresponding author.

Supplementary Materials

    • Supplementary Figure 1 (pdf 49 KB)
    • Supplementary Table 1 (pdf 127 KB)
    • Supplementary Table 2 (pdf 185 KB)
  • Atkin, A.J., Gorely, T., Clemes, S.A., Yates, T., Edwardson, C., Brage, S., . . . Biddle, S.J.H. (2012). Methods of measurement in epidemiology: Sedentary behaviour. International Journal of Epidemiology, 41(5), 14601471. PubMed ID: 23045206 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barreira, T.V., Zderic, T.W., Schuna, J.M., Hamilton, M.T., & Tudor-Locke, C. (2015). Free-living activity counts-derived breaks in sedentary time: Are they real transitions from sitting to standing? Gait and Posture, 42(1), 7072. PubMed ID: 25953504 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bastian, T., Maire, A., Dugas, J., Ataya, A., Villars, C., Gris, F., . . . Simon, C. (2015). Automatic identification of physical activity types and sedentary behaviors from triaxial accelerometer: Laboratory-based calibrations are not enough. Journal of Applied Physiology, 118(6), 716722. PubMed ID: 25593289 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bellettiere, J., Healy, G.N., LaMonte, M.J., Kerr, J., Evenson, K.R., Rillamas-Sun, E., . . . LaCroix, A.Z. (2019). Sedentary behavior and prevalent diabetes in 6,166 older women: The objective physical activity and cardiovascular health study. Journals of Gerontology—Series A Biological Sciences and Medical Sciences, 74(3), 387395. PubMed ID: 29726906 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carlson, J.A., Bellettiere, J., Kerr, J., Salmon, J., Timperio, A., Verswijveren, S.J.J.M., & Ridgers, N.D. (2019). Day-level sedentary pattern estimates derived from hip-worn accelerometer cut-points in 8–12-year-olds: Do they reflect postural transitions? Journal of Sports Sciences, 37(16), 18991909. PubMed ID: 31002287 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chang, Y., Bellettiere, J., Godbole, S., Keshavarz, S., Maestas, J.P., Unkart, J.T., . . . Sears, D.D. (2020). Total sitting time and sitting pattern in postmenopausal women differ by Hispanic ethnicity and are associated with cardiometabolic risk biomarkers. Journal of the American Heart Association, 9(4), e013403. PubMed ID: 32063113 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cheng, H.T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., . . . Anil, R. (2016). Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, ACM. New York, NY :Association for Computing Machinery (pp. 710).

    • Search Google Scholar
    • Export Citation
  • Choi, L., Ward, S.C., Schnelle, J.F., & Buchowski, M.S. (2012). Assessment of wear/nonwear time classification algorithms for triaxial accelerometer. Medicine & Science in Sports & Exercise, 44(10), 20092016. PubMed ID: 22525772 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ellis, K., Godbole, S., Marshall, S., Lanckriet, G., Staudenmayer, J., & Kerr, J. (2014). Identifying active travel behaviors in challenging environments using GPS, accelerometers, and machine learning algorithms. Frontiers in Public Health, 2, 36. PubMed ID: 24795875 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ellis, K., Kerr, J., Godbole, S., Lanckriet, G., Wing, D., & Marshall, S. (2014). A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers. Physiological Measurement, 35(11), 21912203. PubMed ID: 25340969 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Farrahi, V., Niemelä, M., Kangas, M., Korpelainen, R., & Jämsä, T. (2019). Calibration and validation of accelerometer-based activity monitors: A systematic review of machine-learning approaches. Gait and Posture, 68, 285299. PubMed ID: 30579037 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guan, Y., & Plötz, T. (2017). Ensembles of deep LSTM Learners for activity recognition using wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2), 128. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hartman, S.J., Marinac, C.R., Cadmus-Bertram, L., Kerr, J., Natarajan, L., Godbole, S., . . . Sears, D.D. (2018). Sedentary behaviors and biomarkers among breast cancer survivors. Journal of Physical Activity and Health, 15(1), 16. PubMed ID: 28682735 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Keadle, S.K., Lyden, K.A., Strath, S.J., Staudenmayer, J.W., & Freedson, P.S. (2019). A framework to evaluate devices that assess physical behavior. Exercise and Sport Sciences Reviews, 47(4), 206214. PubMed ID: 31524786 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kerr, J., Carlson, J., Godbole, S., Cadmus-Bertram, L., Bellettiere, J., & Hartman, S. (2018). Improving hip-worn accelerometer estimates of sitting using machine learning methods. Medicine & Science in Sports & Exercise, 50(7), 15181524. PubMed ID: 29443824 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kerr, J., Patterson, R.E., Ellis, K., Godbole, S., Johnson, E., Lanckriet, G., & Staudenmayer, J. (2016). Objective assessment of physical activity: Classifiers for public health. Medicine & Science in Sports & Exercise, 48(5), 951957. PubMed ID: 27089222 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kingma, D.P., & Ba, J.L. (2015, May 7–9). Adam: A method for stochastic gradient descent. ICLR: International Conference on Learning Representations, San Diego, CA.

    • Search Google Scholar
    • Export Citation
  • Knaeps, S., Bourgois, J.G., Charlier, R., Mertens, E., Lefevre, J., & Wijndaele, K. (2018). Ten-year change in sedentary behaviour, moderate-to-vigorous physical activity, cardiorespiratory fitness and cardiometabolic risk: Independent associations and mediation analysis. British Journal of Sports Medicine, 52(16), 10631068. PubMed ID: 27491779 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krizhevsky, A., Sutskever, I., & Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25(2012), 1097–1105.

    • Search Google Scholar
    • Export Citation
  • LaMonte, M.J., Lee, I.-M., Rillamas-Sun, E., Bellettiere, J., Evenson, K.R., Buchner, D.M., . . . LaCroix, A.Z. (2019). Comparison of questionnaire and device measures of physical activity and sedentary behavior in a multi-ethnic cohort of older women. Journal for the Measurement of Physical Behaviour, 2(2), 8293. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Migueles, J.H., Cadenas-Sanchez, C., Ekelund, U., Delisle Nyström, C., Mora-Gonzalez, J., Löf, M., . . . Ortega, F.B. (2017). Accelerometer data collection and processing criteria to assess physical activity and other outcomes: A systematic review and practical considerations. Sports Medicine, 47(9), 18211845. PubMed ID: 28303543 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nweke, H.F., Teh, Y.W., Al-Garadi, M.A., & Alo, U.R. (2018). Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Systems with Applications, 105, 233261. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Patterson, R., McNamara, E., Tainio, M., de Sá, T.H., Smith, A.D., Sharp, S.J., . . . Wijndaele, K. (2018). Sedentary behaviour and risk of all-cause, cardiovascular and cancer mortality, and incident type 2 diabetes: A systematic review and dose response meta-analysis. European Journal of Epidemiology, 33(9), 811829. PubMed ID: 29589226 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Powell, C., Herring, M.P., Dowd, K.P., Donnelly, A.E., & Carson, B.P. (2018). The cross-sectional associations between objectively measured sedentary time and cardiometabolic health markers in adults—A systematic review with meta-analysis component. Obesity Reviews, 19(3), 381395. PubMed ID: 29178252 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ramasamy Ramamurthy, S., & Roy, N. (2018). Recent trends in machine learning for human activity recognition—A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1254. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rokni, S.A., Nourollahi, M., & Ghasemzadeh, H. (2018, February 2–7). Personalized human activity recognition using convolutional neural networks. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, New Orleans, LA.

    • Search Google Scholar
    • Export Citation
  • Saeedi, R., Norgaard, S., & Gebremedhin, A.H. (2018). A closed-loop deep learning architecture for robust activity recognition using wearable sensors. Proceedings—2017 IEEE International Conference on Big Data, Big Data 2017, Boston, MA, December 11–14, 2017. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sasaki, J.E., Hickey, A.M., Staudenmayer, J.W., John, D., Kent, J.A., & Freedson, P.S. (2016). Performance of activity classification algorithms in free-living older adults. Medicine & Science in Sports & Exercise, 48(5), 941950. PubMed ID: 26673129 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1), 19291958.

    • Search Google Scholar
    • Export Citation
  • Steeves, J.A., Bowles, H.R., McClain, J.J., Dodd, K.W., Brychta, R.J., Wang, J., & Chen, K.Y. (2015). Ability of thigh-worn ActiGraph and activPAL monitors to classify posture and motion. Medicine & Science in Sports & Exercise, 47(5), 952959. PubMed ID: 25202847 doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019). Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters, 119, 311. doi:

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 286 286 125
Full Text Views 7 7 3
PDF Downloads 10 10 7