Methods for Activity Monitor Validation Studies: An Example With the Fitbit Charge

in Journal for the Measurement of Physical Behaviour
Restricted access

Purchase article

USD  $24.95

Student 1 year subscription

USD  $37.00

1 year subscription

USD  $50.00

Student 2 year subscription

USD  $71.00

2 year subscription

USD  $93.00

Purpose: To advance research practices with consumer monitors, standard validation methods are needed. This study provides an example of best practices through systematically evaluating the validity of the Fitbit Charge (FBC) under free-living conditions using a strong reference measure and robust measurement agreement methods. Methods: 94 healthy participants (Mage 41.8 ±9.3 yrs) wore a FBC and two research grade accelerometers (Actigraph GT3X and activPAL) as they went about normal activities for a week. Estimated daily minutes of moderate to vigorous physical activity (MVPA) from the FBC were compared against reference estimates obtained from the Sojourns Including Posture (SIP) methodology, while daily step counts were compared against the activPAL. Results: Correlations with reference indicators were high for average daily MVPA (r = 0.8; p < .0001) and steps (r = 0.76; p < .0001), but the FBC overestimated time spent in MVPA by 56% and steps by 15%. The mean absolute percent errors of MVPA and steps estimated by FBC were 71.5% and 30.0%, respectively. Neither of the MVPA and step estimates from the FBC fell into the ±10% equivalence zone set by the criterion. The Kappa statistics of the classification agreement between the two MVPA assessment methods was 0.32 with a low sensitivity of 30.1% but a high specificity of 96.7%. Conclusion: The FBC overestimated minutes of MVPA and steps when compared to both reference assessments in free-living conditions. Standardized reporting in future studies will facilitate comparisons with other monitors and with future versions of the FBC.

DeShaw, Ellingson, Lansing, Perez, and Welk are with the Iowa State University, Ames, IA. Bai is with the University of Vermont, Burlington, VT.

DeShaw (kjsouth@iastate.edu) is corresponding author.
  • An, H.S., Jones, G.C., Kang, S.K., Welk, G.J., & Lee, J.M. (2017). How valid are wearable physical activity trackers for measuring steps? European Journal of Sport Science, 17(3), 360–368. PubMed ID: 27912681 doi:10.1080/17461391.2016.1255261

    • Crossref
    • Search Google Scholar
    • Export Citation
  • An, H.S., Kim, Y., & Lee, J.M. (2017). Accuracy of inclinometer functions of the activPAL and ActiGraph GT3X+: A focus on physical activity. Gait & Posture, 51, 174–180. PubMed ID: 27780084 doi:10.1016/j.gaitpost.2016.10.014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bai, Y., Hibbing, P., Mantis, C., & Welk, G.J. (2017). Comparative evaluation of heart rate-based monitors: Apple watch vs. Fitbit Charge HR. Journal of Sports Sciences, 36(15), 1734–1741. PubMed ID: 29210326 doi:10.1080/02640414.2017.1412235

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bai, Y., Welk, G.J., Nam, Y.H., Lee, J.A., Lee, J.M., Kim, Y., … Dixon, P.M. (2016). Comparison of consumer and research monitors under semistructured settings. Medicine & Science in Sports & Exercise, 48(1), 151–158. PubMed ID: 26154336 doi:10.1249/MSS.0000000000000727

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Boudreaux, B.D., Hebert, E.P., Hollander, D.B., Williams, B.M., Cormier, C.L., Naquin, M.R., … Kraemer, R.R. (2018). Validity of wearable activity monitors during cycling and resistance exercise. Medicine & Science in Sports & Exercise, 50(3), 624–633. PubMed ID: 29189666 doi:10.1249/MSS.0000000000001471

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brooke, S.M., An, H.S., Kang, S.K., Noble, J.M., Berg, K.E., & Lee, J.M. (2017). Concurrent validity of wearable activity trackers under free-living conditions. Journal of Strength & Conditioning Research, 31(4), 1097–1106. PubMed ID: 27465631 doi:10.1519/JSC.0000000000001571

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Calabró, M.A., Stewart, J.M., & Welk, G.J. (2013). Validation of pattern-recognition monitors in children using doubly labeled water. Medicine & Science in Sports & Exercise, 45(7), 1313–1322. doi:10.1249/MSS.0b013e31828579c3

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chu, A.H., Ng, S.H., Paknezhad, M., Gauterin, A., Koh, D., Brown, M.S., & Müller-Riemenschneider, F. (2017). Comparison of wrist-worn Fitbit Flex and waist-worn ActiGraph for measuring steps in free-living adults. PLoS ONE, 12(2), 0172535. PubMed ID: 28234953 doi:10.1371/journal.pone.0172535

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dixon, P.M., Saint-Maurice, P.F., Kim, Y., Hibbing, P., Bai, Y., & Welk, G.J. (2018). A primer on the use of equivalence testing for evaluating measurement agreement. Medicine & Science in Sports & Exercise, 50(4), 837–845. doi:10.1249/MSS.0000000000001481

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ellingson, L.D., Schwabacher, I.J., Kim, Y., Welk, G.J., & Cook, D.B. (2016). Validity of an integrative method for processing physical activity data. Medicine & Science in Sports & Exercise, 48(8), 1629–1638. PubMed ID: 27015380 doi:10.1249/MSS.0000000000000915

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Evenson, K.R., Goto, M.M., & Furberg, R.D. (2015). Systematic review of the validity and reliability of consumer-wearable activity trackers. International Journal of Behavioral Nutrition and Physical Activity, 12, 159. PubMed ID: 26684758 doi:10.1186/s12966-015-0314-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johannsen, D.L., Calabro, M.A., Stewart, J., Franke, W., Rood, J.C., & Welk, G.J. (2010). Accuracy of armband monitors for measuring daily energy expenditure in healthy adults. Medicine & Science in Sports & Exercise, 42(11), 2134–2140. PubMed ID: 20386334 doi:10.1249/MSS.0b013e3181e0b3ff

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kooiman, T.J., Dontje, M.L., Sprenger, S.R., Krijnen, W.P., van der Schans, C.P., & de Groot, M. (2015). Reliability and validity of ten consumer activity trackers. BMC Sports Science, Medicine and Rehabilitation, 7, 24. PubMed ID: 26464801 doi:10.1186/s13102-015-0018-5

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lakens, D. (2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355–362. PubMed ID: 28736600 doi:10.1177/1948550617697177

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lee, J.A., & Laurson, K.R. (2015). Validity of the SenseWear armband step count measure during controlled and free-living conditions. Journal of Exercise Science & Fitness, 13(1), 16–23. PubMed ID: 29541094 doi:10.1016/j.jesf.2014.11.002

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lee, J.M., Kim, Y., & Welk, G.J. (2014). Validity of consumer-based physical activity monitors, 1840–1848. Medicine & Science in Sports & Exercise, 46(9), 1840–1848. PubMed ID: 24777201 doi:10.1249/MSS.0000000000000287

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lyden, K., Keadle, S.K., Staudenmayer, J., & Freedson, P.S. (2014). A method to estimate free-living active and sedentary behavior from an accelerometer. Medicine & Science in Sports & Exercise, 46(2), 386–397. doi:10.1249/MSS.0b013e3182a42a2d

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Plasqui, G. (2017). Smart approaches for assessing free-living energy expenditure following identification of types of physical activity. Obesity Reviews, 18(Suppl. 1), 50–55. doi:10.1111/obr.12506

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sushames, A., Edwards, A., Thompson, F., McDermott, R., & Gebel, K. (2016). Validity and reliability of Fitbit flex for step count, moderate to vigorous physical activity and activity energy expenditure. PLoS ONE, 11(9), e0161224. PubMed ID: 27589592 doi:10.1371/journal.pone.0161224

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 162 162 38
Full Text Views 9 9 3
PDF Downloads 6 6 0