Wearable Device Validity in Measuring Steps, Energy Expenditure, and Heart Rate Across Age, Gender, and Body Mass Index: Data Analysis From a Systematic Review

in Journal of Physical Activity and Health

Click name to view affiliation

Sumayyah B. MusaSchool of Human Kinetics and Recreation, Memorial University of Newfoundland, St. John’s, NL, Canada

Search for other papers by Sumayyah B. Musa in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0003-3155-0825
,
Ryan EllisSchool of Human Kinetics and Recreation, Memorial University of Newfoundland, St. John’s, NL, Canada

Search for other papers by Ryan Ellis in
Current site
Google Scholar
PubMed
Close
,
Brianne ChafeSchool of Human Kinetics and Recreation, Memorial University of Newfoundland, St. John’s, NL, Canada

Search for other papers by Brianne Chafe in
Current site
Google Scholar
PubMed
Close
,
Shelby L. SturrockDalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

Search for other papers by Shelby L. Sturrock in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0003-4795-3258
,
Rebecca Ann MaherSchool of Human Kinetics and Recreation, Memorial University of Newfoundland, St. John’s, NL, Canada

Search for other papers by Rebecca Ann Maher in
Current site
Google Scholar
PubMed
Close
,
Kim CullenSchool of Human Kinetics and Recreation, Memorial University of Newfoundland, St. John’s, NL, Canada

Search for other papers by Kim Cullen in
Current site
Google Scholar
PubMed
Close
, and
Daniel FullerDepartment of Community Health and Epidemiology, College of Medicine, University of Saskatchewan, Saskatoon, SK, Canada

Search for other papers by Daniel Fuller in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-2015-2955*
Restricted access

Background: This paper examined whether the criterion validity of step count (SC), energy expenditure (EE), and heart rate (HR) varied across studies depending on the average age, body mass index (BMI), and predominant gender of participants. Methods: Data from 1536 studies examining the validity of various wearable devices were used. Separate multilevel regression models examined the associations among age, gender, and BMI with device criterion validity assessed using mean absolute percent error (MAPE) at the study level. Results: MAPE values were reported in 970 studies for SC, 328 for EE, and 238 for HR, respectively. There were several significant differences in MAPE between age, gender, and BMI categories for SC, EE, and HR. SC MAPE was significantly different for older adults compared with adults. Compared with studies among normal-weight populations, MAPE was greater among studies with overweight samples for SC, HR, and EE. Comparing studies with more women than men, MAPE was significantly greater for EE and HR. Conclusions: There are important differences in the criterion validity of commercial wearable devices across studies of varying ages, BMIs, and genders. Few studies have examined differences in error between different age groups, particularly for EE and HR.

  • Collapse
  • Expand
  • 1.

    World Health Organization. Global Recommendations on Physical Activity for Health.

  • 2.

    Kyu HH, Bachman VF, Alexander LT, et al. Physical activity and risk of breast cancer, colon cancer, diabetes, ischemic heart disease, and ischemic stroke events: systematic review and dose-response meta-analysis for the global burden of disease study 2013. BMJ. 2016;354:i3857. doi:10.1136/bmj.i3857

    • Search Google Scholar
    • Export Citation
  • 3.

    Melaku YA, Wassie MM, Gill TK, et al. Burden of disease attributable to suboptimal diet, metabolic risks and low physical activity in Ethiopia and comparison with Eastern Sub-Saharan African countries, 1990–2015: findings from the global burden of disease study 2015. BMC Public Health. 2018;18(1):552. doi:10.1186/s12889-018-5438-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 4.

    Lee J, Finkelstein J. Activity trackers: a critical review. In: Studies in Health Technology and Informatics. Vol 205. IOS Press; 2014:558562. doi:10.3233/978-1-61499-432-9-558

    • Search Google Scholar
    • Export Citation
  • 5.

    Feehan LM, Geldman J, Sayre EC, et al. Accuracy of fitbit devices: systematic review and narrative syntheses of quantitative data. J Med Internet Res. 2018;6(8):e10527. doi:10.2196/10527

    • Search Google Scholar
    • Export Citation
  • 6.

    Tudor-Locke C, Han H, Aguiar EJ, et al. How fast is fast enough? Walking cadence (steps/min) as a practical estimate of intensity in adults: a narrative review. Br J Sports Med. 2018;52(12):776788. PubMed ID: 29858465 doi:10.1136/bjsports-2017-097628

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    Tudor-Locke C, Craig CL, Brown WJ, et al. How many steps/day are enough? For adults. Int J Behav Nutr Phys Act. 2011;8:79. doi:10.1186/1479-5868-8-79

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8.

    Hansen BH, Dalene KE, Ekelund U, et al. Step by step: association of device-measured daily steps with all-cause mortality—A prospective cohort Study. Scand J Med Sci Sport. 2020;30(9):17051711. doi:10.1111/sms.13726

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 9.

    Nelson N, Asplund CA. Exercise testing: who, when, and why? Adv Sport Med Concepts Controv. 2016;8(3 suppl 1):S16S23. doi:10.1016/j.pmrj.2015.10.019

    • Search Google Scholar
    • Export Citation
  • 10.

    Maynard T. Exercise: part II. Translating the exercise prescription. Diabetes Educ. 1991;17(5):384393. PubMed ID: 1879281 doi:10.1177/014572179101700511

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11.

    Fuller D, Colwell E, Low J, et al. Systematic review of the reliability and validity of commercially available wearable devices for measuring steps, energy expenditure, and heart rate. JMIR mHealth uHealth. 2020;8(9):e18694. PubMed ID: 32897239 doi:10.2196/18694

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12.

    Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutr Phys Act. 2015;12:159. doi:10.1186/s12966-015-0314-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13.

    Van Remoortel H, Giavedoni S, Raste Y, et al. Validity of activity monitors in health and chronic disease: a systematic review. Int J Behav Nutr Phys Act. 2012;9(1):84. doi:10.1186/1479-5868-9-84

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 14.

    Boyer KA, Johnson RT, Banks JJ, Jewell C, Hafer JF. Systematic review and meta-analysis of gait mechanics in young and older adults. Exp Gerontol. 2017;95:6370. PubMed ID: 28499954 doi:10.1016/j.exger.2017.05.005

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15.

    Molina‐Garcia P, Migueles JH, Cadenas‐Sanchez C, et al. A systematic review on biomechanical characteristics of walking in children and adolescents with overweight/obesity: possible implications for the development of musculoskeletal disorders. Obes Rev. 2019;20(7):10331044. PubMed ID: 30942558 doi:10.1111/obr.12848

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16.

    Telfer S, Bigham JJ. The influence of population characteristics and measurement system on barefoot plantar pressures: a systematic review and meta-regression analysis. Gait Posture. 2019;67:269276. PubMed ID: 30391749 doi:10.1016/j.gaitpost.2018.10.030

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17.

    O’Connell S, Ólaighin G, Quinlan LR. When a step is not a step! Specificity analysis of five physical activity monitors. PLoS One. 2017;12(1):114. doi:10.1371/journal.pone.0169616

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18.

    Zhang Y, Weaver RG, Armstrong B, Burkart S, Zhang S, Beets MW. Validity of wrist-worn photoplethysmography devices to measure heart rate: a systematic review and meta-analysis. J Sports Sci. 2020;38(17):20212034. PubMed ID: 32552580 doi:10.1080/02640414.2020.1767348

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19.

    Rughwani N. Normal anatomic and physiologic changes with aging and related disease outcomes: a refresher. Mt Sinai J Med. 2011;78(4):509514. PubMed ID: 21748740 doi:10.1002/msj.20271

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 20.

    Graf A, Judge JO, Õunpuu S, Thelen DG. The effect of walking speed on lower-extremity joint powers among elderly adults who exhibit low physical performance. Arch Phys Med Rehabil. 2005;86(11):21772183. PubMed ID: 16271567 doi:10.1016/j.apmr.2005.06.007

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 21.

    Kang HG, Dingwell JB. Dynamics and stability of muscle activations during walking in healthy young and older adults. J Biomech. 2009;42(14):22312237. PubMed ID: 19664776 doi:10.1016/j.jbiomech.2009.06.038

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22.

    Storti KL, Pettee KK, Brach JS, Talkowski JB, Richardson CR, Kriska AM. Gait speed and step-count monitor accuracy in community-dwelling older adults. Med Sci Sports Exerc. 2008;40(1):5964. PubMed ID: 18091020 doi:10.1249/mss.0b013e318158b504

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23.

    Cyarto EV, Myers AM, Tudor-Locke C. Pedometer accuracy in nursing home and community-dwelling older adults. Med Sci Sports Exerc. 2004;36(2):205209. PubMed ID: 14767241 doi:10.1249/01.MSS.0000113476.62469.98

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24.

    Maganja SA, Clarke DC, Lear SA, Mackey DC. Formative evaluation of consumer-grade activity monitors worn by older adults: test-retest reliability and criterion validity of step counts. JMIR Form Res. 2020;4(8):e16537. PubMed ID: 32651956 doi:10.2196/16537

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25.

    Fokkema T, Kooiman TJM, Krijnen WP, Van Der Schans CP, De Groot M. Reliability and validity of ten consumer activity trackers depend on walking speed. Med Sci Sports Exerc. 2017;49(4):793800. PubMed ID: 28319983 doi:10.1249/MSS.0000000000001146

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 26.

    Renner K, Queen R. Detection of age and gender differences in walking using mobile wearable sensors. Gait Posture. 2021;87(August 2020):5964. doi:10.1016/j.gaitpost.2021.04.017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 27.

    Cleland I, Nugent CD, Finlay DD, et al. Effects of BMI and abdominal volume on the accuracy of step count obtained from a tri-axial accelerometer. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS. 2011;2011:36563659. doi:10.1109/IEMBS.2011.6090616

    • Search Google Scholar
    • Export Citation
  • 28.

    Feito Y, Bassett DR, Thompson DL, Tyo BM. Effects of body mass index on step count accuracy of physical activity monitors. J Phys Act Heal. 2012;9(4):594600. doi:10.1123/jpah.9.4.594

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 29.

    Tyo BM, Fitzhugh EC, Bassett DR, John D, Feito Y, Thompson DL. Effects of body mass index and step rate on pedometer error in a free-living environment. Med Sci Sports Exerc. 2011;43(2):350356. PubMed ID: 20543755 doi:10.1249/MSS.0b013e3181e9b133

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 30.

    Fuller D.Replication Data for: Systematic Review of the Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate. Harvard Dataverse; 2019. https://doi.org/10.7910/DVN/O7GQIM. Accessed March 2, 2022.

    • Search Google Scholar
    • Export Citation
  • 31.

    Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. doi:10.1186/2046-4053-4-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 32.

    Centers for Disease Control and Prevention. Defining Adult Overweight & Obesity. 2021. https://www.cdc.gov/obesity/adult/defining.html. Accessed March 17, 2022.

    • Search Google Scholar
    • Export Citation
  • 33.

    Musa S, Fuller D. Wearable Device Validity Analysis. Published 2021. https://github.com/walkabillylab/wearable-validity-by-age-sex-bmi/blob/main/wearable-validity.md. Accessed March 2, 2022.

    • Search Google Scholar
    • Export Citation
  • 34.

    Patterson JG, Jabson JM, Bowen DJ. Measuring sexual and gender minority populations in health surveillance. LGBT Heal. 2017;4(2):82105. doi:10.1089/lgbt.2016.0026

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 35.

    Springer KW, Mager Stellman J, Jordan-Young RM. Beyond a catalogue of differences: a theoretical frame and good practice guidelines for researching sex/gender in human health. Soc Sci Med. 2012;74(11):18171824. PubMed ID: 21724313 doi:10.1016/j.socscimed.2011.05.033

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 36.

    Liwander A, Pederson A, Boyle E. Why the Canadian sedentary behaviour guidelines should reflect sex and gender. Can J Public Heal. 2013;104(7):479481. doi:10.17269/cjph.104.4108

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 1067 1066 96
Full Text Views 52 52 5
PDF Downloads 82 82 6