The Assessment of 24-Hr Physical Behavior in Children and Adolescents via Wearables: A Systematic Review of Laboratory Validation Studies

in Journal for the Measurement of Physical Behaviour

Click name to view affiliation

Marco GiurgiuDepartment of Sports and Sports Science, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Department of Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany

Search for other papers by Marco Giurgiu in
Current site
Google Scholar
PubMedClose
*
,
Carina NiggDepartment of Sports and Sports Science, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Sport Pedagogy Department, Institute of Sport Science, University of Bern, Bern, Switzerland

Search for other papers by Carina Nigg in
Current site
Google Scholar
PubMedClose
,
Janis FiedlerDepartment of Sports and Sports Science, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Search for other papers by Janis Fiedler in
Current site
Google Scholar
PubMedClose
,
Irina TimmDepartment of Sports and Sports Science, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Search for other papers by Irina Timm in
Current site
Google Scholar
PubMedClose
,
Ellen RulfDepartment of Sports and Sports Science, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Search for other papers by Ellen Rulf in
Current site
Google Scholar
PubMedClose
,
Johannes B.J. BussmannDepartment of Rehabilitation Medicine, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands

Search for other papers by Johannes B.J. Bussmann in
Current site
Google Scholar
PubMedClose
,
Claudio R. NiggHealth Science Department, Institute of Sport Science, University of Bern, Bern, Switzerland

Search for other papers by Claudio R. Nigg in
Current site
Google Scholar
PubMedClose
,
Alexander WollDepartment of Sports and Sports Science, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Search for other papers by Alexander Woll in
Current site
Google Scholar
PubMedClose
, and
Ulrich W. Ebner-PriemerDepartment of Sports and Sports Science, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Department of Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany

Search for other papers by Ulrich W. Ebner-Priemer in
Current site
Google Scholar
PubMedClose
DOI:
https://doi.org/10.1123/jmpb.2022-0014
Keywords:
accelerometry; physical activity; sleep; sedentary behavior
First Published Online:
30 Sep 2022
In Print:
Volume 5: Issue 4
Page Range:
263–276
Restricted access

Purpose: To raise attention to the quality of published validation protocols while comparing (in)consistencies and providing an overview on wearables, and whether they show promise or not. Methods: Searches from five electronic databases were included concerning the following eligibility criteria: (a) laboratory conditions with humans (<18 years), (b) device outcome must belong to one dimension of the 24-hr physical behavior construct (i.e., intensity, posture/activity type outcomes, biological state), (c) must include a criterion measure, and (d) published in a peer-reviewed English language journal between 1980 and 2021. Results: Out of 13,285 unique search results, 123 articles were included. In 86 studies, children <13 years were recruited, whereas in 26 studies adolescents (13–18 years) were recruited. Most studies (73.2%) validated an intensity outcome such as energy expenditure; only 20.3% and 13.8% of studies validated biological state or posture/activity type outcomes, respectively. We identified 14 wearables that had been used to validate outcomes from two or three different dimensions. Most (n = 72) of the identified 88 wearables were only validated once. Risk of bias assessment resulted in 7.3% of studies being classified as “low risk,” 28.5% as “some concerns,” and 71.5% as “high risk.” Conclusion: Overall, laboratory validation studies of wearables are characterized by low methodological quality, large variability in design, and a focus on intensity. No identified wearable provides valid results across all three dimensions of the 24-hr physical behavior construct. Future research should more strongly aim at biological state and posture/activity type outcomes, and strive for standardized protocols embedded in a validation framework.

Supplementary Materials

    • Supplementary Material S1 (PDF 277 KB)
    • Supplementary Material S2 (PDF 336 KB)
    • Supplementary Material S3 (PDF 336 KB)
    • Supplementary Material S4 (PDF 650 KB)
    • Supplementary Material S5 (PDF 368 KB)
    • Supplementary Material S6 (PDF 314 KB)
    • Supplementary Material S7 (PDF 346 KB)
  • Collapse
  • Expand

Journal for the Measurement of Physical Behaviour

Cover Journal for the Measurement of Physical Behaviour

  • Ahn, S., & Fedewa, A.L. (2011). A meta-analysis of the relationship between children’s physical activity and mental health. Journal of Pediatric Psychology, 36(4), 385–397. https://doi.org/10.1093/jpepsy/jsq107

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ancoli-Israel, S., Martin, J.L., Blackswell, T., Buenaver, L., Liu, L., Meltzer, L.J., Sadeh, A., Spira, A.P., & Taylor, D.J. (2015). The SBSM guide to actigraphy monitoring: Clinical and research applications. Behavioral Sleep Medicine, 13(Suppl. 1), S4–S38. https://doi.org/10.1080/15402002.2015.1046356

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Babic, M.J., Morgan, P.J., Plotnikoff, R.C., Lonsdale, C., White, R.L., & Lubans, D.R. (2014). Physical activity and physical self-concept in youth: Systematic review and meta-analysis. Sports Medicine, 44(11), 1589–1601. https://doi.org/10.1007/s40279-014-0229-z

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bassett, D.R., Rowlands, A., & Trost, S.G. (2012). Calibration and validation of wearable monitors. Medicine & Science in Sports & Exercise, 44(1 Suppl. 1), S32–S38. https://doi.org/10.1249/MSS.0b013e3182399cf7

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bull, F.C., Al-Ansari, S.S., Biddle, S., Borodulin, K., Buman, M.P., Cardon, G., Carty, C., Chaput, J.-P., Chastin, S., Chou, R., Dempsey, P.C., DiPietro, L., Ekelund, U., Firth, J., Friedenreich, C.M., Garcia, L., Gichu, M., Jago, R., Katzmarzyk, P.T., . . . Willumsen, J.F. (2020). World Health Organization 2020 guidelines on physical activity and sedentary behaviour. British Journal of Sports Medicine, 54(24), 1451–1462. https://doi.org/10.1136/bjsports-2020-102955

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Burchartz, A., Anedda, B., Auerswald, T., Giurgiu, M., Hill, H., Ketelhut, S., Kolb, S., Mall, C., Manz, K., Nigg, C.R., Reichert, M., Sprengeler, O., Wunsch, K., & Matthews, C.E. (2020). Assessing physical behavior through accelerometry—State of the science, best practices and future directions. Psychology of Sport and Exercise, 49, Article 101703. https://doi.org/10.1016/j.psychsport.2020.101703

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Choi, L., Liu, Z., Matthews, C.E., & Buchowski, M.S. (2011). Validation of accelerometer wear and nonwear time classification algorithm. Medicine & Science in Sports & Exercise, 43(2), 357–364. https://doi.org/10.1249/MSS.0b013e3181ed61a3

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Dunn, J., Kidzinski, L., Runge, R., Witt, D., Hicks, J.L., Schüssler-Fiorenza Rose, S.M., Li, X., Bahmani, A., Delp, S.L., Hastie, T., & Snyder, M.P. (2021). Wearable sensors enable personalized predictions of clinical laboratory measurements. Nature Medicine, 27(6), 1105–1112. https://doi.org/10.1038/s41591-021-01339-0

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ellis, K., Kerr, J., Godbole, S., Lanckriet, G., Wing, D., & Marshall, S. (2014). A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers. Physiological Measurement, 35(11), 2191–2203. https://doi.org/10.1088/0967-3334/35/11/2191

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Evenson, K.R., Goto, M.M., & Furberg, R.D. (2015). Systematic review of the validity and reliability of consumer-wearable activity trackers. The International Journal of Behavioral Nutrition and Physical Activity, 12, Article 159. https://doi.org/10.1186/s12966-015-0314-1

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Fairclough, S., Noonan, R., Rowlands, A., van Hees, V., Knowles, Z., & Boddy, L. (2016). Wear compliance and activity in children wearing wrist and hip mounted accelerometers. Medicine & Science in Sports & Exercise, 48(2), 245–253. https://doi.org/10.1249/MSS.0000000000000771

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Farrahi, V., Niemelä, M., Kangas, M., Korpelainen, R., & Jämsä, T. (2019). Calibration and validation of accelerometer-based activity monitors: A systematic review of machine-learning approaches. Gait & Posture, 68, 285–299. https://doi.org/10.1016/j.gaitpost.2018.12.003

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Fatima, Y., Doi, S.A.R., & Mamun, A.A. (2015). Longitudinal impact of sleep on overweight and obesity in children and adolescents: A systematic review and bias‐adjusted meta‐analysis. Obesity Reviews, 16(2), 137–149. https://doi.org/10.1111/obr.12245

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Feehan, L.M., Geldman, J., Sayre, E.C., Park, C., Ezzat, A.M., Yoo, J.Y., Hamilton, C.B., & Li, L.C. (2018). Accuracy of fitbit devices: Systematic review and narrative syntheses of quantitative data. JMIR MHealth and UHealth, 6(8), Article e10527. https://doi.org/10.2196/10527

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Freedson, P., Bowles, H.R., Troiano, R., & Haskell, W. (2012). Assessment of physical activity using wearable monitors: Recommendations for monitor calibration and use in the field. Medicine & Science in Sports & Exercise, 44(1 Suppl. 1), S1–S4. https://doi.org/10.1249/MSS.0b013e3182399b7e

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Fuller, D., Colwell, E., Low, J., Orychock, K., Tobin, M.A., Simango, B., Buote, R., van Heerden, D., Luan, H., Cullen, K., Slade, L., & Taylor, N.G.A. (2020). Reliability and validity of commercially available wearable devices for measuring steps, energy expenditure, and heart rate: Systematic review. JMIR MHealth and UHealth, 8(9), Article e18694. https://doi.org/10.2196/18694

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hewitt, L., Stanley, R.M., Cliff, D., & Okely, A.D. (2019). Objective measurement of tummy time in infants (0–6 months): A validation study. PLoS One, 14(2), Article e0210977. https://doi.org/10.1371/journal.pone.0210977

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Jeffreys, R.M., Inge, T.H., Jenkins, T.M., King, W.C., Oruc, V., Douglas, A.D., & Bray, M.S. (2015). Physical activity monitoring in extremely obese adolescents from the teen-laboratories study. Journal of Physical Activity and Health, 12(1), 132–138. https://doi.org/10.1123/jpah.2013-0006

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Johnston, W., Judice, P.B., Molina García, P., Mühlen, J.M., Lykke Skovgaard, E., Stang, J., Schumann, M., Cheng, S., Bloch, W., Brønd, J.C., Ekelund, U., Grøntved, A., Caulfield, B., Ortega, F.B., & Sardinha, L.B. (2021). Recommendations for determining the validity of consumer wearable and smartphone step count: Expert statement and checklist of the INTERLIVE network. British Journal of Sports Medicine, 55(14), 780–793. https://doi.org/10.1136/bjsports-2020-103147

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Kang, M., & Rowe, D.A. (2015). Issues and challenges in sedentary behavior measurement. Measurement in Physical Education and Exercise Science, 19(3), 105–115. https://doi.org/10.1080/1091367X.2015.1055566

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Keadle, S.K., Bustamante, E.E., & Buman, M.P. (2021). Physical activity and public health: Four decades of progress. Kinesiology Review, 10(3), 319–330. https://doi.org/10.1123/kr.2021-0028

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Keadle, S.K., Lyden, K.A., Strath, S.J., Staudenmayer, J.W., & Freedson, P.S. (2019). A framework to evaluate devices that assess physical behavior. Exercise and Sport Sciences Reviews, 47(4), 206–214. https://doi.org/10.1249/JES.0000000000000206

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mokkink, L.B., Terwee, C.B., Patrick, D.L., Alonso, J., Stratford, P.W., Knol, D.L., Bouter, L.M., & Vet, H.C.W. (2010). The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study. Quality of Life Research, 19(4), 539–549. https://doi.org/10.1007/s11136-010-9606-8

    • Crossref
    • Search Google Scholar
    • Export Citation

  • O’Brien, M.W. (2021). Implications and recommendations for equivalence testing in measures of movement behaviors: A scoping review. Journal for the Measurement of Physical Behaviour, 4(4), 353–362. https://doi.org/10.1123/jmpb.2021-0021

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J., Grimshaw, J.M., Hróbjartsson, A., Lalu, M.M., Li, T., Loder, E.W., Mayo-Wilson, E., McDonald, S., . . . Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, Article n71. https://doi.org/10.1136/bmj.n71

    • Search Google Scholar
    • Export Citation

  • Phillips, S.M., Summerbell, C., Hobbs, M., Hesketh, K.R., Saxena, S., Muir, C., & Hillier-Brown, F.C. (2021). A systematic review of the validity, reliability, and feasibility of measurement tools used to assess the physical activity and sedentary behaviour of pre-school aged children. The International Journal of Behavioral Nutrition and Physical Activity, 18(1), Article 141. https://doi.org/10.1186/s12966-021-01132-9

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Quante, M., Kaplan, E.R., Rueschman, M., Cailler, M., Buxton, O.M., & Redline, S. (2015). Practical considerations in using accelerometers to assess physical activity, sedentary behavior, and sleep. Sleep Health, 1(4), 275–284. https://doi.org/10.1016/j.sleh.2015.09.002

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Rodriguez-Ayllon, M., Cadenas-Sánchez, C., Estévez-López, F., Muñoz, N.E., Mora-Gonzalez, J., Migueles, J.H., . . . Esteban-Cornejo, I. (2019). Role of physical activity and sedentary behavior in the mental health of preschoolers, children and adolescents: A systematic review and meta-analysis. Sports Medicine, 49(9), 1383–1410. https://doi.org/10.1007/s40279-019-01099-5

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Rosenberger, M.E., Fulton, J.E., Buman, M.P., Troiano, R.P., Grandner, M.A., Buchner, D.M., & Haskell, W.L. (2019). The 24-hour activity cycle: A new paradigm for physical activity. Medicine & Science in Sports & Exercise, 51(3), 454–464. https://doi.org/10.1249/MSS.0000000000001811

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Schardt, C., Adams, M.B., Owens, T., Keitz, S., & Fontelo, P. (2007). Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Medical Informatics and Decision Making, 7, Article 16. https://doi.org/10.1186/1472-6947-7-16

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sperlich, B., & Holmberg, H.-C. (2017). Wearable, yes, but able. . .? It is time for evidence-based marketing claims! British Journal of Sports Medicine, 51(16), Article 1240. https://doi.org/10.1136/bjsports-2016-097295

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Stamatakis, E., Koster, A., Hamer, M., Rangul, V., Lee, I.-M., Bauman, A.E., Atkin, A.J., Aadahl, M., Matthews, C.E., Mork, P.J., Askie, L., Cistulli, P., Granat, M., Palm, P., Crowley, P.J., Stevens, M., Gupta, N., Pulakka, A., Stenholm, S., . . . Holtermann, A. (2019). Emerging collaborative research platforms for the next generation of physical activity, sleep and exercise medicine guidelines: The Prospective Physical Activity, Sitting, and Sleep consortium (ProPASS). British Journal of Sports Medicine, 54(8), 435–437. https://doi.org/10.1136/bjsports-2019-100786

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Staudenmayer, J., Zhu, W., & Catellier, D.J. (2012). Statistical considerations in the analysis of accelerometry-based activity monitor data. Medicine & Science in Sports & Exercise, 44(1 Suppl. 1), S61–S67. https://doi.org/10.1249/mss.0b013e3182399e0f

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Stevens, M.L., Gupta, N., Inan Eroglu, E., Crowley, P.J., Eroglu, B., Bauman, A., Granat, M., Straker, L., Palm, P., Stenholm, S., Aadahl, M., Mork, P., Chastin, S., Rangul, V., Hamer, M., Koster, A., Holtermann, A., & Stamatakis, E. (2020). Thigh-worn accelerometry for measuring movement and posture across the 24-hour cycle: A scoping review and expert statement. BMJ Open Sport & Exercise Medicine, 6(1), Article e000874. https://doi.org/10.1136/bmjsem-2020-000874

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Tremblay, M.S., Aubert, S., Barnes, J.D., Saunders, T.J., Carson, V., Latimer-Cheung, A.E., Chastin, S.F.M., Altenburg, T.M., & Chinapaw, M.J.M. (2017). Sedentary Behavior Research Network (SBRN)—Terminology Consensus Project process and outcome. The International Journal of Behavioral Nutrition and Physical Activity, 14(1), Article 75. https://doi.org/10.1186/s12966-017-0525-8

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Tremblay, M.S., Carson, V., Chaput, J.-P., Connor Gorber, S., Dinh, T., Duggan, M., Faulkner, G., Gray, C.E., Gruber, R., Janson, K., Janssen, I., Katzmarzyk, P.T., Kho, M.E., Latimer-Cheung, A.E., LeBlanc, C., Okely, A.D., Olds, T., Pate, R.R., Phillips, A., . . . Zehr, L. (2016). Canadian 24-Hour movement guidelines for children and youth: An integration of physical activity. Sedentary Behaviour, and Sleep. Applied Physiology, Nutrition, and Metabolism, 41(6 Suppl. 3), S311–S27. https://doi.org/10.1139/apnm-2016-0151

    • Search Google Scholar
    • Export Citation

  • Troiano, R.P., McClain, J.J., Brychta, R.J., & Chen, K.Y. (2014). Evolution of accelerometer methods for physical activity research. British Journal of Sports Medicine, 48(13), 1019–1023. https://doi.org/10.1136/bjsports-2014-093546

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Troiano, R.P., Stamatakis, E., & Bull, F.C. (2020). How can global physical activity surveillance adapt to evolving physical activity guidelines? Needs, challenges and future directions. British Journal of Sports Medicine, 54(24), 1468–1473. https://doi.org/10.1136/bjsports-2020-102621

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Trost, S.G. (2020). Population-level physical activity surveillance in young people: Are accelerometer-based measures ready for prime time? The International Journal of Behavioral Nutrition and Physical Activity, 17(1), Article 28. https://doi.org/10.1186/s12966-020-00929-4

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Umemneku Chikere, C.M., Wilson, K., Graziadio, S., Vale, L., & Allen, A.J. (2019). Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard—An update. PLoS One, 14(10), Article e0223832. https://doi.org/10.1371/journal.pone.0223832

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Warburton, D.E., Katzmarzyk, P.T., Rhodes, R.E., & Shephard, R.J. (2007). Evidence-informed physical activity guidelines for Canadian adults. Applied Physiology, Nutrition, and Metabolism, 32(S2E), S16–S68. https://doi.org/10.1139/H07-123

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Welk, G.J., Bai, Y., Lee, J.-M., Godino, J., Saint-Maurice, P.F., & Carr, L. (2019). Standardizing analytic methods and reporting in activity monitor validation studies. Medicine & Science in Sports & Exercise, 51(8), 1767–1780. https://doi.org/10.1249/MSS.0000000000001966

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Whiting, P.F., Rutjes, A.W., Westwood, M.E., Mallett, S., Deeks, J.J., Reitsma, J.B., . . . QUADAS-2 Group. (2011). QUADAS-2: A revised tool for the Quality Assessment of Diagnostic Accuracy Studies. Annals of Internal Medicine, 155(8), 529–536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009

    • Crossref
    • Search Google Scholar
    • Export Citation

  • World Health Organization. (2019). WHO guidelines on physical activity, sedentary behaviour and sleep for children under 5 years of age.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 979 979 67
Full Text Views 28 28 2
PDF Downloads 34 34 2