Unique Views on Obesity-Related Behaviors and Environments: Research Using Still and Video Images

in Journal for the Measurement of Physical Behaviour
Restricted access

Purchase article

USD  $24.95

Student 1 year subscription

USD  $37.00

1 year subscription

USD  $50.00

Student 2 year subscription

USD  $71.00

2 year subscription

USD  $93.00

Objectives: To document challenges to and benefits from research involving the use of images by capturing examples of such research to assess physical activity– or nutrition-related behaviors and/or environments. Methods: Researchers (i.e., key informants) using image capture in their research were identified through knowledge and networks of the authors of this paper and through literature search. Twenty-nine key informants completed a survey covering the type of research, source of images, and challenges and benefits experienced, developed specifically for this study. Results: Most respondents used still images in their research, with only 26.7% using video. Image sources were categorized as participant generated (n = 13; e.g., participants using smartphones for dietary assessment), researcher generated (n = 10; e.g., wearable cameras with automatic image capture), or curated from third parties (n = 7; e.g., Google Street View). Two of the major challenges that emerged included the need for automated processing of large datasets (58.8%) and participant recruitment/compliance (41.2%). Benefit-related themes included greater perspectives on obesity with increased data coverage (34.6%) and improved accuracy of behavior and environment assessment (34.6%). Conclusions: Technological advances will support the increased use of images in the assessment of physical activity, nutrition behaviors, and environments. To advance this area of research, more effective collaborations are needed between health and computer scientists. In particular development of automated data extraction methods for diverse aspects of behavior, environment, and food characteristics are needed. Additionally, progress in standards for addressing ethical issues related to image capture for research purposes is critical.

Carlson is with the Center for Children’s Healthy Lifestyles and Nutrition, Children’s Mercy Kansas City, Kansas City, MO. Hipp is with the Dept. of Parks, Recreation, and Tourism Management, College of Natural Resources, and a Fellow at the Center for Geospatial Analytics, North Carolina State University, Raleigh, NC. Kerr is with the Dept. of Family Medicine and Public Health, Moores Cancer Center, University of California San Diego, La Jolla, CA. Horowitz and Berrigan are with the Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD. Carlson and Hipp are co-first authors.

Carlson (jacarlson@cmh.edu) is corresponding author.
  • Bader, M.D.M., Mooney, S.J., Bennett, B., & Rundle, A.G. (2017). The promise, practicalities, and perils of virtually auditing neighborhoods using Google street view. The ANNALS of the American Academy of Political and Social Science, 669(1), 18–40. doi:10.1177/0002716216681488

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bader, M.D.M., Mooney, S.J., Lee, Y.J., Sheehan, D., Neckerman, K.M., Rundle, A.G., & Teitler, J.O. (2015). Development and deployment of the Computer Assisted Neighborhood Visual Assessment System (CANVAS) to measure health-related neighborhood conditions. Health & Place, 31, 163–172. PubMed ID: 25545769 doi:10.1016/j.healthplace.2014.10.012

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barrett, D.P., Xu, R., Yu, H., & Siskind, J.M. (2016). Collecting and annotating the large continuous action dataset. Machine Vision and Applications, 27(7), 983–995. doi:10.1007/s00138-016-0768-4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bossard, L., Guillaumin, M., & Van Gool, L. (2014). Food-101 – Mining discriminative components with random forests. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), Computer vision – ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6–12, 2014, proceedings, part VI (pp. 446–461). Cham, Switzerland: Springer International Publishing.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Boushey, C.J., Spoden, M., Delp, E.J., Zhu, F., Bosch, M., Ahmad, Z., … Kerr, D. (2017). Reported energy intake accuracy compared to doubly labeled water and usability of the mobile food record among community dwelling adults. Nutrients, 9(3), 312. doi:10.3390/nu9030312

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Boushey, C.J., Spoden, M., Zhu, F.M., Delp, E.J., & Kerr, D.A. (2016). New mobile methods for dietary assessment: Review of image-assisted and image-based dietary assessment methods. Proceedings of the Nutrition Society, 76(3), 283–294. PubMed ID: 27938425 doi:10.1017/S0029665116002913

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carlson, J.A., Liu, B., Sallis, J.F., Kerr, J., Hipp, J.A., Staggs, V.S., … Vasconcelos, N.M. (2017). Automated ecological assessment of physical activity: Advancing direct observation. International Journal of Environmental Research and Public Health, 14(12), 1487. PubMed ID: 29194358 doi:10.3390/ijerph14121487

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Colabianchi, N. (2015). Improving environmental measures in obesity research using innovative technology. (1R21CA188481-01-A1). Washington, DC: NIH, Research Portfolio Online Reporting Tools (RePORT); Retrieved from https://projectreporter.nih.gov/.

    • Search Google Scholar
    • Export Citation
  • Cowburn, G., Matthews, A., Doherty, A., Hamilton, A., Kelly, P., Williams, J., … Nelson, M. (2015). Exploring the opportunities for food and drink purchasing and consumption by teenagers during their journeys between home and school: A feasibility study using a novel method. Public Health Nutrition, 19(1), 93–103. PubMed ID: 25874731 doi:10.1017/S1368980015000889

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davis, D.S., Goldmon, M.V., & Coker-Appiah, D.S. (2011). Using a community-based participatory research approach to develop a faith-based obesity intervention for african american children. Health Promotion Practice, 12(6), 811–822. PubMed ID: 21540194 doi:10.1177/1524839910376162

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deng, J., Dong, W., Socher, R., Li, L.J., Kai, L., & Li, F.F. (2009, June 20–25). ImageNet: A large-scale hierarchical image database. Paper presented at the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL.

    • Search Google Scholar
    • Export Citation
  • Evenson, K.R., Jones, S.A., Holliday, K.M., Cohen, D.A., & McKenzie, T.L. (2016). Park characteristics, use, and physical activity: A review of studies using SOPARC (System for Observing Play and Recreation in Communities). Preventive Medicine, 86, 153–166. PubMed ID: 26946365 doi:10.1016/j.ypmed.2016.02.029

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eyler, A.A., Blanck, H.M., Gittelsohn, J., Karpyn, A., McKenzie, T.L., Partington, S., … Winters, M. (2015). Physical activity and food environment assessments: Implications for practice. American Journal of Preventive Medicine, 48(5), 639–645. PubMed ID: 25891064 doi:10.1016/j.amepre.2014.10.008

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fulton, J.E., Carlson, S.A., Ainsworth, B.E., Berrigan, D., Carlson, C., Dorn, J.M., … Wendel, A. (2016). Strategic priorities for physical activity surveillance in the United States. Translational Journal of the American College of Sports Medicine, 1(13), 111–123. doi:10.1249/tjx.0000000000000020

    • Search Google Scholar
    • Export Citation
  • Glanz, K., Sallis, J.F., Saelens, B.E., & Frank, L.D. (2007). Nutrition Environment Measures Survey in Stores (NEMS-S): Development and evaluation. American Journal of Preventive Medicine, 32(4), 282–289. PubMed ID: 17383559 doi:10.1016/j.amepre.2006.12.019

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Graham, D.J., & Hipp, J.A. (2014). Emerging technologies to promote and evaluate physical activity: Cutting-edge research and future directions. Front Public Health, 2, 66. PubMed ID: 25019066 doi:10.3389/fpubh.2014.00066

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Greene, M.R. (2016). Estimations of object frequency are frequently overestimated. Cognition, 149, 6–10. PubMed ID: 26774103 doi:10.1016/j.cognition.2015.12.011

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hall, K.L., Vogel, A.L., Stipelman, B., Stokols, D., Morgan, G., & Gehlert, S. (2012). A four-phase model of transdisciplinary team-based research: Goals, team processes, and strategies. Translational Behavioral Medicine, 2(4), 415–430. PubMed ID: 23483588 doi:10.1007/s13142-012-0167-y

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hipp, J.A., Adlakha, D., Eyler, A.A., Chang, B., & Pless, R. (2013). Emerging technologies: Webcams and crowd-sourcing to identify active transportation. American Journal of Preventive Medicine, 44(1), 96–97. PubMed ID: 23253658 doi:10.1016/j.amepre.2012.09.051

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hipp, J.A., Adlakha, D., Eyler, A.A., Gernes, R., Kargol, A., Stylianou, A.H., & Pless, R. (2017). Learning from outdoor webcams: Surveillance of physical activity across environments. In P. Thakuriah, N. Tilahun, & M. Zellner (Eds.), Seeing cities through big data: Research, methods and applications in urban informatics (pp. 471–490). Cham, Switzerland: Springer International Publishing.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • James, P., Jankowska, M., Marx, C., Hart, J.E., Berrigan, D., Kerr, J., … Laden, F. (2016). “Spatial Energetics”: Integrating data from GPS, accelerometry, and GIS to address obesity and inactivity. American Journal of Preventive Medicine, 51(5), 792–800. PubMed ID: 27528538 doi:10.1016/j.amepre.2016.06.006

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jenny, Y., Russell, B., Ce, L., & Torralba, A. (2009, September 29–October 2). LabelMe video: Building a video database with human annotations. Paper presented at the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.

    • Search Google Scholar
    • Export Citation
  • Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception and Psychophysics, 14(2), 201–211. doi:10.3758/bf03212378

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Joseph, R.P., & Maddock, J.E. (2016). Observational park-based physical activity studies: A systematic review of the literature. Preventive Medicine, 89, 257–277. PubMed ID: 27311337 doi:10.1016/j.ypmed.2016.06.016

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kelly, C.M., Wilson, J.S., Baker, E.A., Miller, D.K., & Schootman, M. (2013). Using Google street view to audit the built environment: Inter-rater reliability results. Annals of Behavioral Medicine, 45(Suppl. 1), S108–112. doi:10.1007/s12160-012-9419-9

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kelly, P., Marshall, S.J., Badland, H., Kerr, J., Oliver, M., Doherty, A.R., & Foster, C. (2013). An ethical framework for automated, wearable cameras in health behavior research. American Journal of Preventive Medicine, 44(3), 314–319. PubMed ID: 23415131 doi:10.1016/j.amepre.2012.11.006

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kerr, J., Duncan, S., & Schipperjin, J. (2011). Using global positioning systems in health research: A practical approach to data collection and processing. American Journal of Preventive Medicine, 41(5), 532–540. PubMed ID: 22011426 doi:10.1016/j.amepre.2011.07.017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kerr, J., Marshall, S.J., Godbole, S., Chen, J., Legge, A., Doherty, A.R., … Foster, C. (2013). Using the SenseCam to improve classifications of sedentary behavior in free-living settings. American Journal of Preventive Medicine, 44(3), 290–296. PubMed ID: 23415127 doi:10.1016/j.amepre.2012.11.004

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, A.C., Winter, S.J., Sheats, J.L., Rosas, L.G., Buman, M.P., Salvo, D., … Dommarco, J.R. (2016). Leveraging citizen science and information technology for population physical activity promotion. Translational Journal of the American College of Sports Medicine, 1(4), 30–44. PubMed ID: 27525309 doi:10.1249/tjx.0000000000000003

    • Search Google Scholar
    • Export Citation
  • Krizhevsky, A., Sutskever, I., & Hinton, G.E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. doi:10.1145/3065386

    • Crossref
    • Search Google Scholar
    • Export Citation
  • LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436. PubMed ID: 26017442 doi:10.1038/nature14539

  • Loveday, A., Sherar, B.L., Sanders, P.J., Sanderson, W.P., & Esliger, W.D. (2015). Technologies that assess the location of physical activity and sedentary behavior: A systematic review. Journal of Medical Internet Research, 17(8), e192. PubMed ID: 26245157 doi:10.2196/jmir.4761

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Macer, P.J., Thomas, P.J., Chalabi, N., & Meech, J.F. (1996). Finding the cut of the wrong trousers: Fast video search using automatic storyboard generation. Paper presented at the Conference Companion on Human Factors in Computing Systems, Vancouver, Canada.

    • Search Google Scholar
    • Export Citation
  • Martin, C.K., Han, H., Coulon, S.M., Allen, H.R., Champagne, C.M., & Anton, S.D. (2009). A novel method to remotely measure food intake of free-living individuals in real time: The remote food photography method. British Journal of Nutrition, 101(3), 446–456. PubMed ID: 18616837 doi:10.1017/s0007114508027438

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Paper presented at the Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, Vancouver, BC.

    • Search Google Scholar
    • Export Citation
  • McKenzie, T.L., Cohen, D.A., Sehgal, A., Williamson, S., & Golinelli, D. (2006). System for Observing Play and Recreation in Communities (SOPARC): Reliability and feasibility measures. Journal of Physical Activity and Health, 3(s1), S208–S222. PubMed ID: 28834508 doi:10.1123/jpah.3.s1.s208

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Medler, D.A. (1998). A brief history of connectionism. Neural Computing Surveys, 1, 61–101.

  • Moghimi, M., Kerr, J., Johnson, E., Godbole, S., & Belongie, S. (2015). Discriminative regions: A substrate for analyzing life-logging image sequences. In X. He, S. Luo, D. Tao, C. Xu, J. Yang, & M.A. Hasan (Eds.), MultiMedia Modeling: 21st International Conference, MMM 2015, Sydney, NSW, Australia, January 5-7, 2015, Proceedings, Part II (pp. 357–368). Cham, Switzerland: Springer International Publishing.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mooney, S.J., Bader, M.D.M., Lovasi, G.S., Teitler, J.O., Koenen, K.C., Aiello, A.E., … Rundle, A.G. (2017). Street audits to measure neighborhood disorder: Virtual or in-person? American Journal of Epidemiology, 186(3), 265–273. PubMed ID: 28899028 doi:10.1093/aje/kwx004

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nahum-Shani, I., Hekler, E.B., & Spruijt-Metz, D. (2015). Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association, 34(0), 1209–1219. doi:10.1037/hea0000306

    • Crossref
    • Search Google Scholar
    • Export Citation
  • National Institutes of Health. (2017). Certificates of Confidentiality (CoC). Retrieved from https://humansubjects.nih.gov/coc/index

  • Nebeker, C., Lagare, T., Takemoto, M., Lewars, B., Crist, K., Bloss, C.S., & Kerr, J. (2016). Engaging research participants to inform the ethical conduct of mobile imaging, pervasive sensing, and location tracking research. Translational Behavioral Medicine, 6(4), 577–586. PubMed ID: 27688250 doi:10.1007/s13142-016-0426-4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Oliva, A., & Torralba, A. (2006). Building the gist of a scene: The role of global image features in recognition. Progress in Brain Research, 155, 23–36. PubMed ID: 17027377 doi:10.1016/S0079-6123(06)55002-2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Park, K., & Ewing, R. (2017). The usability of Unmanned Aerial Vehicles (UAVs) for measuring park-based physical activity. Landscape and Urban Planning, 167, 157–164. doi:10.1016/j.landurbplan.2017.06.010

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pless, R., & Souvenir, R. (2009). A survey of manifold learning for images. IPSJ Transactions on Computer Vision and Applications, 1, 83–94. doi:10.2197/ipsjtcva.1.83

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ramanan, D. (2012). Detecting activities of daily living in first-person camera views. Paper presented at the Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI.

    • Search Google Scholar
    • Export Citation
  • Riley, W.T., Rivera, D.E., Atienza, A.A., Nilsen, W., Allison, S.M., & Mermelstein, R. (2011). Health behavior models in the age of mobile interventions: are our theories up to the task? Translational Behavioral Medicine, 1(1), 53–71. PubMed ID: 21796270 doi:10.1007/s13142-011-0021-7

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rui, Y., & Anandan, P. (2000). Segmenting visual actions based on spatio-temporal motion patterns. Paper presented at the Proceedings IEEE Conference on Computer Vision and Pattern Recognition. (Cat. No. PR00662), Hilton Head Island, SC.

    • Search Google Scholar
    • Export Citation
  • Spiegel, P.K. (1995). The first clinical X-ray made in America--100 years. American Journal of Roentgenology, 164(1), 241–243. PubMed ID: 7998549 doi:10.2214/ajr.164.1.7998549

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Streuber, S., Quiros-Ramirez, M.A., Hill, M.Q., Hahn, C.A., Zuffi, S., O’Toole, A., & Black, M.J. (2016). Body talk: Crowdshaping realistic 3D avatars with words. ACM Translation Graph, 35(4), 1–14. doi:10.1145/2897824.2925981

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thornton, I.M., Rensink, R.A., & Shiffrar, M. (2002). Active versus passive processing of biological motion. Perception, 31(7), 837–853. PubMed ID: 12206531 doi:10.1068/p3072

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Torralba, A., Russell, B.C., & Yuen, J. (2010). LabelMe: Online image annotation and applications. Proceedings of the IEEE, 98(8), 1467–1484. doi:10.1109/JPROC.2010.2050290

    • Crossref
    • Search Google Scholar
    • Export Citation
  • U.S. Department of Health and Human Services. (2016). Child Welfare Information Gateway: Mandatory reporters of child abuse and neglect. Retrieved from https://www.childwelfare.gov/

    • Search Google Scholar
    • Export Citation
  • Vaca-Castano, G., Das, S., Sousa, J.P., Lobo, N.D., & Shah, M. (2017). Improved scene identification and object detection on egocentric vision of daily activities. Computer Vision and Image Understanding, 156, 92–103. doi:10.1016/j.cviu.2016.10.016

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C., & Burris, M.A. (1997). Photovoice: Concept, methodology, and use for participatory needs assessment. Health Education and Behavior, 24(3), 369–387. PubMed ID: 9158980 doi:10.1177/109019819702400309

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wang, C.C., & Pies, C.A. (2004). Family, maternal, and child health through photovoice. Maternal and Child Health Journal, 8(2), 95–102. PubMed ID: 15198177 doi:10.1023/B:MACI.0000025732.32293.4f

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Whyte, W.H. (1980). The social life of small urban spaces (2nd ed.). New York, NY: Project for Public Spaces.

  • Wood, G., Lynch, T.P., Devine, C., Keller, K., & Figueira, W. (2016). High-resolution photo-mosaic time-series imagery for monitoring human use of an artificial reef. Ecology and Evolution, 6(19), 6963–6968. PubMed ID: 28725373 doi:10.1002/ece3.2342

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wood, S.A., Guerry, A.D., Silver, J.M., & Lacayo, M. (2013). Using social media to quantify nature-based tourism and recreation. Scientific Reports, 3, 2976. PubMed ID: 24131963 doi:10.1038/srep02976

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zacks, J.M., & Swallow, K.M. (2007). Event segmentation. Current Directions in Psychological Science, 16(2), 80–84. PubMed ID: 22468032 doi:10.1111/j.1467-8721.2007.00480.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, Z., Hu, Y., Chan, S., & Chia, L.T. (2008). Motion context: A new representation for human action recognition. In D. Forsyth, P. Torr, & A. Zisserman (Eds.), Computer Vision– ECCV 2008: 10th European Conference on Computer Vision, Marseille, France, October 12–18, 2008, Proceedings, Part IV (pp. 817–829). Berlin, Heidelberg: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhuang, Y., Belkin, M., & Dennis, S. (2013). Metric based automatic event segmentation. In D. Uhler, K. Mehta, & J.L. Wong (Eds.), Mobile Computing, Applications, and Services: 4th International Conference, MobiCASE 2012, Seattle, WA, USA, October 11–12, 2012. Revised Selected Papers (pp. 129–148). Berlin, Heidelberg: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 82 82 6
Full Text Views 1 1 0
PDF Downloads 1 1 0