Application of an Ecological Momentary Assessment Protocol in a Workplace Intervention: Assessing Compliance, Criterion Validity, and Reactivity

in Journal of Physical Activity and Health
Restricted access

Purchase article

USD  $24.95

Student 1 year subscription

USD  $115.00

1 year subscription

USD  $153.00

Student 2 year subscription

USD  $218.00

2 year subscription

USD  $285.00

Background: Ecological momentary assessment (EMA) is a method of collecting behavioral data in real time. The purpose of this study was to examine EMA compliance, identify factors predicting compliance, assess criterion validity of, and reactivity to, using EMA in a workplace intervention study. Methods: Forty-five adults (91.1% female, 39.7 [9.6] y) were recruited for a workplace standing desk intervention. Participants received 5 surveys each day for 5 workdays via smartphone application. EMA items assessed current position (sitting/standing/stepping). EMA responses were time matched to objectively measured time in each position before and after each prompt. Multilevel logistic regression models estimated factors influencing EMA response. Cohen kappa measured interrater agreement between EMA-reported and device-measured position. Reactivity was assessed by comparing objectively measured sitting/standing/stepping in the 15 minutes before and after each EMA prompt using multilevel repeated-measures models. Results: Participants answered 81.4% of EMA prompts. Differences in compliance differed by position. There was substantial agreement between EMA-reported and device-measured position (κ = .713; P < .001). Following the EMA prompt, participants sat 0.87 minutes more than before the prompt (P < .01). Conclusion: The use of EMA is a valid assessment of position when used in an intervention to reduce occupational sitting and did not appear to disrupt sitting in favor of the targeted outcome.

Weatherson, Yun, Wunderlich, and Puterman are with the University of British Columbia, Vancouver, BC, Canada. Faulkner is with the School of Kinesiology, Faculty of Education, University of British Columbia, Vancouver, BC, Canada.

Faulkner (guy.faulkner@ubc.ca) is corresponding author.
  • 1.

    Tremblay M, Aubert S, Barnes J, et al. Sedentary behavior research network (SBRN)—Terminology consensus project process and outcome. Int J Behav Nutr Phys Act. 2017;14:1–17. doi:10.1186/s12966-016-0456-9

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2.

    Biswas A, Oh P, Faulkner G, Bajaj R, Silver M, Mitchell M, Alter D. Sedentary time and its association with risk for disease incidence, mortality, and hospitalization in adults A systematic review and meta-analysis. Ann Intern Med. 2015;162:123–132. PubMed ID: 25599350 doi:10.7326/M14-1651

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    Roberts K, Butler G, Branchard B, et al. The physical activity, sedentary behavior and sleep (PASS) indicator framework. Health Promot Chron. 2017;37:252–256. doi:10.24095/hpcdp.37.8.04

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 4.

    Victorian Health Promotion Foundation. Reducing prolonged sitting in the workplace. An evidence review: summary report. 2012. http://www.vichealth.vic.gov.au.ezproxy.library.ubc.ca/~/media/ResourceCentre/PublicationsandResources/Economic%20participation/2012%20workplace/VH_Reducing_prolonged_sitting_07.ashx. Accessed February 04, 2019.

    • Export Citation
  • 5.

    Shrestha N, Kukkonen-Harjula K, Verbeek J, Ijaz S, Hermans V, Bhaumik S. Workplace interventions for reducing sitting at work. Cochrane Database Syst Rev. 2018;6:CD010912. PubMed ID: 29926475 doi:10.1002/14651858.CD010912.pub3

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    Troiano R, Gabriel K, Welk G, Owen N, Sternfeld B. Reported physical activity and sedentary behavior: why do you ask? J Phys Act Health. 2012;9:S68–S75. PubMed ID: 22287450 doi:10.1123/jpah.9.s1.s68

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    Loney T, Standage M, Thompson D, Sebire S, Cumming S. Self-report vs objectively assessed physical activity: which is right for public health? J Phys Act Health. 2011;8:62–70. PubMed ID: 21297186 doi:10.1123/jpah.8.1.62

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Troiano R, Berrigan D, Dodd K, Masse L, Tilert T, Mcdowell M. Physical activity in the United States measured by accelerometer. Med Sci Sports Exer. 2008;40:181–188. doi:10.1249/mss.0b013e31815a51b3

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 9.

    Colley R, Gorber S, Tremblay M. Quality control and data reduction procedures for accelerometry-derived measures of physical activity. Health Rep. 2010;21:63. PubMed ID: 20426228

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Stone AA, Shiffman S. Ecological momentary assessment in behavioral medicine. Ann Behav Med. 1994;16:199–202. doi:10.1093/abm/16.3.199

  • 11.

    Dunton G. Ecological momentary assessment in physical activity research. Exerc Sport Sci Rev. 2017;45:48–54. PubMed ID: 27741022 doi:10.1249/JES.0000000000000092

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12.

    Shiffman S, Stone AA, Hufford M. Ecological momentary assessment. Annu Rev Clin Psycho. 2008;4:1–32. doi:10.1146/annurev.clinpsy.3.022806.091415

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13.

    Ram N, Brinberg M, Pincus AL, Conroy DE. The questionable ecological validity of ecological momentary assessment: considerations for design and analysis. Res Hum Dev. 2017;14(3):253–270. PubMed ID: 30613195 doi:10.1080/15427609.2017.1340052

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14.

    Liao Y, Skelton K, Dunton G, Bruening M. A systematic review of methods and procedures used in ecological momentary assessments of diet and physical activity research in youth: an adapted STROBE checklist for reporting EMA studies (CREMAS). J Med Internet Res. 2016;18(6):e151. PubMed ID: 27328833 doi:10.2196/jmir.4954

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15.

    Romanzini CLP, Romanzini M, Batista MB, et al. Methodology used in ecological momentary assessment studies about sedentary behavior in children, adolescents, and adults: systematic review using the checklist for reporting ecological momentary assessment studies. J Med Internet Res. 2019;21(5):e11967. PubMed ID: 31094349 doi:10.2196/11967

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16.

    Engelen L, Chau JY, Burks-Young S, Bauman A. Application of ecological momentary assessment in workplace health evaluation. Health Promot J Austr. 2016;27:259–263. PubMed ID: 27596817 doi:10.1071/HE16043

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    Pronk NP, Katz AS, Lowry M, Payfer JR. Reducing occupational sitting time and improving worker health: the Take-a-Stand Project, 2011. Prev Chronic Dis. 2012;9:110323. doi:10.5888/pcd9.110323

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18.

    Graves LEF, Murphy RC, Shepherd SO, Cabot J, Hopkins ND. Evaluation of sit-stand workstations in an office setting: a randomised controlled trial. BMC Public Health. 2015;15(1):1145. doi:10.1186/s12889-015-2469-8

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19.

    Wen CKF, Schneider S, Stone AA, Spruijt-Metz D. Compliance with mobile ecological momentary assessment protocols in children and adolescents: a systematic review and meta-analysis. J Med Internet Res. 2017;19(4):e132. PubMed ID: 28446418 doi:10.2196/jmir.6641

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20.

    McCarthy DE, Minami H, Yeh VM, Bold KW. An experimental investigation of reactivity to ecological momentary assessment frequency among adults trying to quit smoking. Addiction. 2015;110:1549–1560. PubMed ID: 26011583 doi:10.1111/add.12996

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21.

    Maher J, Rebar A, Dunton G. Ecological momentary assessment is a feasible and valid methodological tool to measure older adults’ physical activity and sedentary behavior. Front Psychol. 2018;9:1485. PubMed ID: 30158891 doi:10.3389/fpsyg.2018.01485

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22.

    Kozey-Keadle S, Libertine A, Lyden K, Staudenmayer J, Freedson P. Validation of wearable monitors for assessing sedentary behavior. Med Sci Sports Exer. 2011;43:1561–1567. doi:10.1249/MSS.0b013e31820ce174

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23.

    Grant P, Ryan C, Tigbe W, Granat M. The validation of a novel activity monitor in the measurement of posture and motion during everyday activities. Br J Sports Med. 2006;40:992–997. doi:10.1136/bjsm.2006.030262

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24.

    Dunton GF, Liao Y, Kawabata K, Intille S. Momentary assessment of adults’ physical activity and sedentary behavior: feasibility and validity. Front Psychol. 2012;3:260. PubMed ID: 22866046 doi:10.3389/fpsyg.2012.00260

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25.

    Hox JJ. Multilevel Analysis: Techniques and Applications. Mahwah, NJ: Erlbaum; 2002.

  • 26.

    McCoach DB. Hierarchical linear modeling. In: Hancock GR, Mueller RO, eds. The Reviewer’s Guide to Quantitative Methods in the Social Sciences. New York, NY: Routledge; 2010:123–140.

    • Search Google Scholar
    • Export Citation
  • 27.

    Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd ed. Thousand Oaks, CA: Sage Publications; 2002.

    • Search Google Scholar
    • Export Citation
  • 28.

    Smith L, McCourt O, Sawyer A, et al. A review of occupational physical activity and sedentary behavior correlates. Occup Med. 2016;66(3):185–192. doi:10.1093/occmed/kqv164

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 29.

    Knell G, Gabriel K, Businelle M, Shuval K, Wetter D, Kendzor D. Ecological momentary assessment of physical activity: validation study. J Med Internet Res. 2017;19:e253. PubMed ID: 28720556 doi:10.2196/jmir.7602

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Zink J, Belcher B, Dzubur E, et al. Association between self-reported and objective activity levels by demographic factors: ecological momentary assessment study in children. JMIR Mhealth Uhealth. 2018;6:e150. PubMed ID: 29954723 doi:10.2196/mhealth.9592

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 31.

    Stone AA, Shiffman S. Capturing momentary, self-report data: a proposal for reporting guidelines. Ann Behav Med. 2002;24(3):236–243. PubMed ID: 12173681 doi:10.1207/S15324796ABM2403_09

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 32.

    Morren M, van Dulmen S, Ouwerkerk J, Bensing J. Compliance with momentary pain measurement using electronic diaries: a systematic review. Eur J Pain. 2009;13(4):354–365. PubMed ID: 18603458 doi:10.1016/j.ejpain.2008.05.010

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33.

    Hufford M. Special methodological challenges and opportunities in ecological momentary assessment. In: Stone AA, ed. The Science of Real-Time Data Capture: Self-Reports in Health Research. Oxford: Oxford University Press; 2007:54–75.

    • Search Google Scholar
    • Export Citation
  • 34.

    Liao Y, Intille SS, Dunton GF. Using ecological momentary assessment to understand where and with whom adults’ physical and sedentary activity occur. Int J Behav Med. 2015;22(1):51–61. PubMed ID: 24639067 doi:10.1007/s12529-014-9400-z

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 35.

    Neuhaus M, Eakin EG, Straker L, et al. Reducing occupational sedentary time: a systematic review and meta‐analysis of evidence on activity‐permissive workstations. Obes Rev. 2014;15:822–838. PubMed ID: 25040784 doi:10.1111/obr.12201

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 241 240 135
Full Text Views 19 19 7
PDF Downloads 15 15 8