Comparison of Sedentary Time Between Thigh-Worn and Wrist-Worn Accelerometers

Click name to view affiliation

Kristin Suorsa University of Turku
Turku University Hospital

Search for other papers by Kristin Suorsa in
Current site
Google Scholar
PubMed
Close
*
,
Anna Pulakka University of Turku
Turku University Hospital
Finnish Institute for Health and Welfare

Search for other papers by Anna Pulakka in
Current site
Google Scholar
PubMed
Close
*
,
Tuija Leskinen University of Turku
Turku University Hospital

Search for other papers by Tuija Leskinen in
Current site
Google Scholar
PubMed
Close
*
,
Jaana Pentti University of Turku
Turku University Hospital
University of Helsinki

Search for other papers by Jaana Pentti in
Current site
Google Scholar
PubMed
Close
*
,
Andreas Holtermann National Research Centre for the Working Environment

Search for other papers by Andreas Holtermann in
Current site
Google Scholar
PubMed
Close
*
,
Olli J. Heinonen University of Turku

Search for other papers by Olli J. Heinonen in
Current site
Google Scholar
PubMed
Close
*
,
Juha Sunikka University of Turku
Turku University Hospital

Search for other papers by Juha Sunikka in
Current site
Google Scholar
PubMed
Close
*
,
Jussi Vahtera University of Turku
Turku University Hospital

Search for other papers by Jussi Vahtera in
Current site
Google Scholar
PubMed
Close
*
, and
Sari Stenholm University of Turku
Turku University Hospital

Search for other papers by Sari Stenholm in
Current site
Google Scholar
PubMed
Close
*
Open access

Background: The accuracy of wrist-worn accelerometers in identifying sedentary time has been scarcely studied in free-living conditions. The aim of this study was to compare daily sedentary time estimates between a thigh-worn accelerometer, which measured sitting and lying postures, and a wrist-worn accelerometer, which measured low levels of movement. Methods: The study population consisted of 259 participants (Mage = 62.8 years, SD = 0.9) from the Finnish Retirement and Aging Study (FIREA). Participants wore an Axivity AX3 accelerometer on their mid-thigh and an Actigraph wActiSleep-BT accelerometer on their non-dominant wrist simultaneously for a minimum of 4 days in free-living conditions. Two definitions to estimate daily sedentary time were used for data from the wrist-worn accelerometer: 1) the count cutpoint, ≤1853 counts per minute; and 2) the Euclidean Norm Minus One (ENMO) cutpoint, <30 mg. Results: Compared to the thigh-worn accelerometer, daily sedentary time estimate was 63 min (95% confidence interval [CI] = −53 to −73) lower by the count cutpoint and 50 min (95% CI = 34 to 67) lower by the ENMO cutpoint. The limits of agreement in daily sedentary time estimates between the thigh- and cutpoint methods for wrist-worn accelerometers were wide (the count cutpoint: −117 to 243, the ENMO cutpoint: −212 to 313 min). Conclusions: Currently established cutpoint-based methods to estimate sedentary time from wrist-worn accelerometers result in underestimation of daily sedentary time compared to posture-based estimates of thigh-worn accelerometers. Thus, sedentary time estimates obtained from wrist-worn accelerometers using currently available cutpoint-based methods should be interpreted with caution and future work is needed to improve their accuracy.

Sedentary behavior refers to waking behavior while sitting, reclining, or lying down with energy expenditure no more than 1.5 metabolic units (METs) (Tremblay et al., 2017). Daily sedentary time during waking hours has been estimated to be considerable, 7.7 to 9.4 hr among adults in Western countries (Harvey, Chastin, & Skelton, 2015; Matthews et al., 2008). A high amount of daily sedentary time has been shown to be associated with cardiometabolic risk factors (Healy, Matthews, Dunstan, Winkler, & Owen, 2011; Powell, Herring, Dowd, Donnelly, & Carson, 2018), risk of cardiovascular diseases (Vasankari et al., 2017), and mortality (Biswas et al., 2015; Ekelund et al., 2016).

Recently, accelerometers have provided a feasible way to estimate sedentary time either by measuring low levels of or absence of movement or specific postures. Although these methods are not perfectly in line with the official definition of sedentary behavior, which is based on the energy expenditure and posture, they have been used to estimate sedentary behavior nonetheless. Accelerometers mounted on the hip have been used extensively to provide movement-based estimates of sedentary time, but wrist-worn accelerometers have recently also gained popularity in large epidemiological studies such as the National Health and Nutrition Examination Survey (NHANES) (Troiano, McClain, Brychta, & Chen, 2014) and the UK Biobank study (Plotz et al., 2017). In addition to movement-based hip- and wrist-worn accelerometers, sedentary time has been estimated by posture-based thigh-worn accelerometers (Edwardson et al., 2017). Thigh-worn accelerometers provide information of the inclination of the thigh, which is suitable for distinguishing between postures, and have shown high accuracy for capturing time spent sitting and lying down and therefore for estimating sedentary time (Kozey Keadle, Libertine, Lyden, Staudenmayer, & Freedson, 2011; Lyden, Kozey Keadle, Staudenmayer, & Freedson, 2012; Skotte, Korshøj, Kristiansen, Hanisch, & Holtermann, 2014; Stemland et al., 2015). High accuracy for posture detection has been reported especially for data processing algorithms in the PAL Technologies software designed for thigh-worn activPAL accelerometers (Kozey Keadle et al., 2011; Lyden et al., 2012), and in the Acti4 software (Kongsvold, 2016; Skotte et al., 2014; Stemland et al., 2015), which can be used to process raw data from several thigh-worn accelerometer brands such as activPAL, ActiGraph, and Axivity with good equivalence between accelerometer brands in estimates of time spent sitting and lying (Crowley et al., 2019).

Growing interest in wrist placement stems from increased participant compliance, ability to detect reliably light physical activity related to daily tasks, and the possibility to measure sleep with the same device, creating opportunities for 24-hr physical behavior measurements (Freedson & John, 2013; Quante et al., 2015; Schrack et al., 2016). However, the weakness of wrist-worn accelerometers is that they cannot detect lower body posture, which is an important part of the sedentary behavior definition. Moreover, accelerometer-measured sedentary time is most commonly estimated with a cutpoint approach (i.e., determination of cutpoint values that reflect activities with an energy expenditure no more than 1.5 METs; Watson, Carlson, Carroll, & Fulton, 2014). The cutpoint approach has been used traditionally with hip-worn accelerometers due to close location of the center of mass of the body and, thus, relatively high correspondence to whole body movements (Shiroma et al., 2016). However, with wrist-worn accelerometers there are inherent limitations due to low correspondence between wrist movements and whole body movements. Despite these weaknesses, sedentary cutpoint values for wrist-worn accelerometers have been developed (Flórez-Pregonero, Buman, & Ainsworth, 2018; Koster et al., 2016). Koster et al. (2016) compared an ActiGraph accelerometer worn on the non-dominant wrist to a thigh-worn activPAL and found the most optimal vector magnitude (VM) cutoff point to differentiate between sedentary behavior and light activity to be <1853 counts per minute (CPM). However, since studies comparing sedentary estimates from thigh- and wrist-worn accelerometers in free-living conditions are few (Koster et al., 2016; Pavey, Gomersall, Clark, & Brown, 2016), and the study population of Koster et al. (2016) consisted of a small sample of older adults (Mage 78 years), the accuracy of this cutoff point needs to be evaluated in a larger population of adults free of aging-related mobility limitations.

In addition to relying on device-specific activity counts, several research groups have recently developed methods to analyze raw triaxial acceleration data from wrist-worn accelerometers to enable data harmonization between studies using different accelerometer devices. However, only a few studies have analyzed raw acceleration data, such as Euclidean Norm Minus One (ENMO), and examined wrist accelerometer threshold values for estimating sedentary time in adults (Bakrania et al., 2016; Hildebrand, Hansen, van Hees, & Ekelund, 2017; Rowlands et al., 2018; Sanders et al., 2019), and these studies have suggested various threshold values for adults in free-living environments (Hildebrand et al., 2017; Rowlands et al., 2018). As the use of raw acceleration data and wrist-worn accelerometers is increasing, evaluation of the suggested threshold values in free-living conditions is needed. Therefore, it is essential to compare movement-based sedentary estimates from wrist-worn accelerometers to identify more reliable methods of estimating sedentary time, such as postural-based thigh-worn accelerometers. The aims of the current study were to compare daily sedentary time estimates from a wrist-worn ActiGraph wActiSleep-BT accelerometer using 1) a cutoff point value of 1853 CPM by Koster et al. (2016) and 2) a sedentary threshold of 30 mg for raw ENMO acceleration values based on the data processing algorithm in the GGIR-package (Rowlands et al., 2018), to daily sedentary estimates from a thigh-worn Axivity AX3 accelerometer processed by the validated data processing algorithm in Acti4 software (Skotte et al., 2014; Stemland et al., 2015) as the reference method in free-living conditions.

Methods

Participants

The study population consisted of participants from the Finnish Retirement and Aging Study (FIREA), an ongoing longitudinal cohort study of older adults in Finland established in 2013. Details of the design and implementation of the FIREA study have been reported elsewhere (Leskinen et al., 2018; Pulakka et al., 2019). Briefly, participants were first contacted 18 months prior to their estimated retirement date by sending a questionnaire. After responding to the questionnaire, Finnish-speaking participants with an estimated retirement date between 2017 and 2019 who lived in Southwest Finland and were still working were invited to participate in the clinical sub-study (n = 773). Of them, 290 participated in the study between September 2015 and May 2018. Informed consent was obtained from all participants. The FIREA was conducted in line with the Declaration of Helsinki and was approved by the ethics committee of Hospital District of Southwest Finland.

All 290 participants were provided with two accelerometer devices (a thigh-worn Axivity and a wrist-worn ActiGraph). We excluded 18 participants due to device malfunction or missing accelerometer or diary data. In addition, to minimize the effects to sedentary estimates caused by differences in wear and sleep time between methods, we excluded participants who had large differences (>10 min during all the days) in daily wear time between the methods (the count cutpoint vs. thigh comparison: 12 persons; the ENMO cutpoint vs. thigh comparison: 71 persons). After that, we excluded participants who had less than 1 valid day with ≥10 hr of waking wear time from both thigh- and wrist-worn accelerometers (the count cutpoint comparison: 1 person; the ENMO cutpoint comparison: 2 persons), resulting in an analytic sample of 259 persons for the comparison between the thigh-worn accelerometer and the count cutpoint method and an analytic sample of 199 persons for the comparison between the thigh-worn accelerometer and the ENMO cutpoint method.

Participants Characteristics

Sex, date of birth, and occupational status were obtained from the register of Keva, Finland’s largest pension provider which administers the pensions of local and state government. Occupational status was categorized based on the International Standard Classification of Occupations (ISCO; Statistics Finland, 2010) into three groups according to the occupational titles by the last known occupation: managers and professionals (ISCO classes 1–2), associate professionals and office workers (ISCO classes 3–4), and service and manual workers (ISCO classes 5–9).

Accelerometer Measurements

A study nurse fastened the triaxial accelerometer and inclinometer Axivity AX3 (Axivity Ltd., Newcastle, UK), initialized to record at 100 Hz, to the participant’s right thigh during the clinical examination visit. The Axivity accelerometer was fastened with adhesive waterproof film dressing (Opsite, Smith & Nephew, London, UK) directly to the skin on the medial front of the right thigh, midway between the hip and knee joints, which is a standardized position (Skotte et al., 2014). Participants were also instructed to wear the triaxial Actigraph wActiSleep-BT accelerometer (ActiGraph, Pensacola, FL, US) on the non-dominant wrist using a wrist band provided with the accelerometer. The ActiGraph was initialized to record at 80 Hz.

A study nurse gave the participants both oral and written information on how to use the accelerometers. Participants were asked to wear both accelerometers at least four days and nights including at least two working days and two days off. Participants were instructed to wear the devices at all times, including during water-based activities such as swimming, but to remove them for sauna bathing. In addition, participants were instructed to perform a reference measurement in a standing upright position for 15 sec every day. Participants were also asked to record date, waking time, bedtime, reference measurement times, and information about working day (working day or day off) for each day that they wore the devices on a daily log. After the measurement period, the participants returned devices and daily logs by mail.

Accelerometer Data Processing

Thigh-worn Accelerometers

To derive the posture-based estimates of sedentary time, data from the thigh-worn Axivity accelerometers were downloaded through Open Movement software (version 1.0.0.37; Open Movement, Newcastle University, UK). The raw data were further processed and analyzed using a customized MATLAB program, Acti4, which determines the type and duration of different activities and body postures with a high sensitivity and specificity (Skotte et al., 2014; Stemland et al., 2015). The Acti4 software is freely available upon request from the National Research Centre for the Working Environment in Copenhagen, Denmark. The epoch length was set to 5 sec for sitting and lying in the Acti4 software. The detailed data analysis procedures in the Acti4 software are described elsewhere (Gupta et al., 2015; Skotte et al., 2014). We included wear time between the first and last date and time recorded in the daily log after excluding sleep time, based on bedtime recorded in the daily log. Non-wear time was excluded based on the definition of the Acti4 software: periods longer than 60 min without movement and also periods between 10 and 60 min if the standard deviation (SD) in x, y, and z axes were higher than 0.5 g for any second during a 5-s interval immediately before the period without movement (raw and unfiltered data were used) (Skotte et al., 2014). In the Acti4 software, the daily sedentary time estimate was defined based on postures, as daily time spent sitting and lying. Sitting postures or lying were detected when the thigh inclination was above 45°.

Wrist-worn Accelerometers: The Count Cutpoint Method

For the movement-based count cutpoint method, data from the wrist-worn Actigraph accelerometers were downloaded and converted into 60-s epochs in ActiLife software, version 6.13 (ActiGraph, Pensacola, FL, US). We calculated the VM CPM as the square root of the sum of the squares of the activity counts of the three axes. Similarly, as with data from the thigh-worn accelerometers, we included wear time between the first and last date and time recorded in the daily log after excluding sleep time, based on bedtime recorded in the daily log. We further excluded non-wear time using the algorithm developed by Choi et al. (2011, 2012) that has been validated for wrist-worn triaxial accelerometers. (Choi et al., 2012) Daily sedentary time estimate was defined as VM CPM <1853 (Koster et al., 2016).

Wrist-worn Accelerometers: The ENMO Cutpoint Method

For the movement-based ENMO cutpoint method, we used open source R-package GGIR (version 1.7-1) to analyze raw acceleration data from the wrist-worn accelerometers in R statistical software (version 3.5.1; R Foundation for Statistical Computing, Vienna, Austria, https://cran.r-project.org/). The R-package GGIR script that we used is shown in Supplementary Material 1 (available online). The data processing in the GGIR package included 1) autocalibration according to the local gravity, 2) detection of sustained abnormally high values and non-wear time, and 3) calculation of the average magnitude of dynamic acceleration expressed as ENMO (as mg [milligravitational units] where g = 9.81 m/s2) over 5-s epochs with negative values rounded to zero (Migueles, Rowlands, Huber, Sabia, & van Hees, 2019; Sabia et al., 2014; van Hees, 2014; van Hees et al., 2013; van Hees et al., 2015). The ENMO cutpoint method is described in detail elsewhere (van Hees, 2014; van Hees et al., 2013). Sleep time was detected based on the combination of the daily logs and algorithm of the GGIR package (van Hees et al., 2015) so that sleep was defined as periods of time within the in-bed and out-bed times reported in the daily logs during which there was no change larger than 5° in the arm angle over at least 5 min. Non-wear time was classified using 15-min time blocks based on the characteristics of the 60-min time window centered at these 15 min. A block was classified as non-wear time if the standard deviation of the 60-min window was less than 13.0 mg for at least two out of the three axes, or if the value range for at least two out of three axes was less than 50 mg (Sabia et al., 2014; van Hees et al., 2013). We defined sedentary time using a previously proposed threshold of 30 mg (Rowlands et al., 2018). To increase the correspondence to the previously suggested and commonly used 60-s epoch length for wrist-worn accelerometers (Heesch, Hill, Aguilar-Farias, van Uffelen, & Pavey, 2018), we included in our sedentary time estimates only bouts of a minimum of 60 s in which a minimum of 90% of the time met the threshold criteria (<30 mg). In the supplemental analyses, we also derived sedentary time estimates without bout restriction (i.e., directly from 5-s epoch values).

Harmonization of the Sleep and Non-wear Time Detection Methods

We used developed and commonly used data processing methods because we aimed to describe differences that could be expected between these methods. Because non-wear time was identified by different algorithms in the thigh- and wrist-worn accelerometers, we excluded days with a more than 10-min difference in waking wear time between the methods (the count cutpoint vs. thigh comparison: 214 days for the ENMO cutpoint vs. 646 days for the thigh comparison). For sleep detection, there are no established practices or commonly used algorithms; therefore, we harmonized sleep detection by using self-reported in-bed times from the daily logs as basis for defining sleep time from the thigh- and the wrist-worn accelerometers. In the final analytical sample of the comparison between the count cutpoint method and the thigh-worn accelerometer there were no differences in sleep time but minor differences in non-wear time (<10 min) because we used the exact same sleep detection method (self-reported in-bed times) and accepted less than 10-min difference in waking wear time. In the final analytical sample of the comparison between the ENMO cutpoint method and the thigh-worn accelerometer, the differences in sleep and non-wear time were only minor, as the ENMO cutpoint method gave slightly less sleep time (M difference = 3 min, SD = 61 min) and non-wear time (M difference = 4 min, SD = 60 min) than the thigh-worn accelerometer.

Statistical Analyses

Characteristics of the study population are shown as percentages for categorical variables and M (SD) for continuous variables. Pearson correlations were used to examine the relative agreement between the posture-based thigh-worn accelerometer (reference method) and both movement-based methods for the wrist-worn accelerometer (count cutpoint and ENMO cutpoint method) for mean daily sedentary time. A Pearson correlation of <0.5 was determined to be low, 0.5–0.75 moderate, and >0.75 high (Chastin, Culhane, & Dall, 2014). In supplemental analyses, we also examined the relative agreements between the thigh- and the wrist-worn accelerometers using Pearson correlations separately for working days and days off.

We used linear mixed models to compare mean estimates and 95% confidence intervals (95% CI) for daily sedentary time by the movement-based count cutpoint and by the movement-based ENMO cutpoint methods to the daily sedentary time estimates from the posture-based thigh-worn accelerometer. Further, to visualize the magnitude of the pairwise differences, we used Bland-Altman analysis for paired measurements of a varying true value (Bland & Altman, 2007) to compare daily sedentary time estimates by the movement-based count cutpoint and the movement-based ENMO cutpoint method to the daily sedentary time estimates from the posture-based thigh-worn accelerometer. In supplemental analyses, we conducted the Bland-Altman analysis to compare daily sedentary time estimates by the ENMO cutpoint method without bout restriction to the daily sedentary time estimates from the thigh-worn accelerometer. Furthermore, we examined accuracy of the wrist-worn accelerometer in estimating within-individual differences in sedentary time. This was done by comparing sedentary time estimates between working days and days off for each method by using linear mixed models because daily sedentary time has been observed to differ between working days and days off (Pulakka, Stenholm, et al., 2018). The results are shown as mean differences and 95% limits of agreement.

All statistical analyses were performed using SAS statistical software (version 9.4; SAS Institute, Inc., Cary, NC, US).

Results

Mean age of the participants was 62.8 years (SD = 0.9) and 82% of participants were women (Table 1). Details related to duration of the measurement are shown in Table 1.

Table 1

Characteristics of the Participants in Comparisons Between Wrist-Worn and Thigh-Worn Accelerometer

CharacteristicsComparison Between the Count Cutpoint (Wrist) and ThighComparison Between the ENMO Cutpoint (Wrist) and Thigh
N259199
Age, M (SD)62.8 (0.9)62.8 (1.0)
Gender, n (%)
 Women213 (82)162 (81)
 Men46 (18)37 (19)
Occupational status, n (%)
 Managers, professionals93 (36)72 (36)
 Associate professionals, office workers89 (34)70 (35)
 Service and manual workers77 (30)57 (29)
Number of measurement days, M (SD)3.2 (1.3)2.0 (1.0)
 Working days2.0 (0.9)1.5 (0.7)
 Days off1.7 (0.7)1.2 (0.5)
Mean daily wake wear time, min (95% CI)
 Thigh944 (936 to 952)941 (928 to 954)
 Wrist944 (936 to 952)942 (929 to 955)

Note. SD = standard deviation, CI = confidence interval, ENMO = Euclidean Norm Minus One.

Correlations between daily sedentary time estimates obtained from the thigh- and wrist-worn accelerometers are illustrated in Figure 1. Pearson’s correlation was high for the movement-based count cutpoint (0.78, 95% CI = 0.75 to 0.80) and moderate for the movement-based ENMO cutpoint method (0.62, 95% CI = 0.56 to 0.68) when compared against the posture-based thigh-worn accelerometer.

Figure 1
Figure 1

—Correlation between daily sedentary time estimates from the thigh-worn and wrist-worn accelerometers using (A) the count cutpoint and (B) the Euclidean Norm Minus One cutpoint.

Citation: Journal for the Measurement of Physical Behaviour 3, 3; 10.1123/jmpb.2019-0052

Mean daily sedentary time estimates obtained from the thigh- and wrist-worn accelerometers and the mean difference between the methods are shown in Table 2. The mean daily sedentary time estimate was 556 min (95% CI = 543 to 569) from the posture-based thigh-worn accelerometer. Compared to the thigh-worn accelerometer, the mean daily sedentary time estimate was 63 min lower (95% CI =  −73 to −53) by the movement-based count cutpoint and 50 min lower (95% CI = −67 to −34) by the movement-based ENMO cutpoint method.

Table 2

Comparison of Daily Sedentary Minutes Between Wrist-Worn and Thigh-Worn Accelerometer on All Days, Working Days, and Days Off

All Days (min/day)Working Days (min/day)Days Off (min/day)Difference, Working Days vs. Days Off (min/day)
M95% CIM95% CIM95% CIM95% CI
Count cutpoint (wrist) vs. thigh
 Thigh (ref)556543 to 569578561 to 595532517 to 5464830 to 67
 Wrist493480 to 506516499 to 533468453 to 4824933 to 64
 Difference−63−73 to −53−62−73 to −50−64−76 to −510−12 to 14
ENMO cutpoint (wrist) vs. thigh
 Thigh (ref)562544 to 581579556 to 603534510 to 5574921 to 76
 Wrist512493 to 530523499 to 547493569 to 517343 to 65
 Difference−50−67 to −34−56−78 to −35−41−60 to −22−11−40 to 17

Note. ENMO = Euclidean Norm Minus One, min = minutes, 95% CI = 95% confidence interval.

Figure 2 shows agreement between the thigh- and wrist-worn accelerometers based on Bland-Altman analysis in which differences between the measurement methods are plotted against their average. For the movement-based count cutpoint, the 95% limits of agreement were from −117 min to 243 min and the mean difference between the methods did not depend on the level of the estimated mean daily sedentary time. The 95% limits of agreement were larger for the movement-based ENMO cutpoint method (−212 min to 313 min).

Figure 2
Figure 2

—The Bland-Altman plots describing the level of agreement in daily sedentary time estimates by different methods. The difference in daily sedentary time estimates between the thigh-worn accelerometers and (A) the count cutpoint and (B) the Euclidean Norm Minus One cutpoint is plotted against the mean of daily sedentary time obtained by these two methods. Solid gray line is the zero bias line representing the mean difference and dashed lines present the 95% limits of agreement. The histogram describes the number of observations (i.e., number of days for each value of difference between daily sedentary time estimates obtained from the thigh- and wrist-worn accelerometers).

Citation: Journal for the Measurement of Physical Behaviour 3, 3; 10.1123/jmpb.2019-0052

Correlations and mean differences between the methods in estimating sedentary time were also examined separately on working days and days off. Similarly with the main analyses, correlations with the thigh-worn accelerometer were higher for the movement-based count cutpoint (working day r = 0.75, 95% CI = 0.71 to 0.79; days off r = 0.79, 95% CI = 0.75 to 0.83) than for the movement-based ENMO cutpoint method (working day r = 0.59, 95% CI = 0.49 to 0.67; days off r = 0.68, 95% CI = 0.59 to 0.75) (Supplementary Material 2 [available online]). Although the absolute level of daily sedentary time estimate differed between the posture-based and movement-based methods on working days and days off, almost the same difference in daily sedentary time estimates between working days and days off, approximately 50 min, was observed by the movement-based count cutpoint method and the posture-based thigh-worn accelerometer (Table 2). By the movement-based ENMO cutpoint method, the difference in the sedentary time estimate between working days and days off was observed to be slightly less, 34 min, when compared to the posture-based thigh-worn accelerometer (Table 2).

In the supplemental analyses, calculation of the sedentary time estimate directly from 5-s epoch values resulted in overestimation of daily sedentary time by 59 min by the movement-based ENMO method (Supplementary Material 3 [available online]).

Discussion

In this study we compared daily sedentary time estimates from two movement-based cutpoint methods applied to a wrist-worn accelerometer to estimates from a posture-based thigh-worn accelerometer. The thigh-worn accelerometer has previously shown high accuracy to detect time spent sitting and lying and therefore to estimate sedentary time. So far the accuracy of wrist-worn accelerometers using cutpoint-based methods has been scarcely studied in free-living conditions. Our results show that using currently established methods, a wrist-worn accelerometer underestimates daily sedentary time by approximately 1 hr when compared to the posture-based thigh-worn accelerometer in free-living conditions. This observed discrepancy in sedentary time estimates is clinically meaningful and therefore the results from wrist-worn accelerometers using currently available cutpoint-based methods should be interpreted with caution.

Movement-based wrist-worn accelerometers and posture-based thigh-worn accelerometers have been used in epidemiological studies to estimate sedentary time despite the fact that none of these methods are suitable for capturing both metabolic and postural components of the official definition of sedentary behavior. Differences between the thigh- and wrist-worn accelerometers were expected, because of the different estimation (movement vs. posture) and because wrist movements can be independent of body posture. Therefore, when using a wrist-worn accelerometer, the false negative classifications of sedentary time increase when a person is sitting and moving arms, for instance while doing handcraft. On the other hand, the false positive classifications of sedentary time increase when a person is standing arms still. In addition to the location of the device, different data processing methods result in different estimates of daily sedentary time. Despite the inherent limitations of wrist-worn accelerometers to detect full body movements, wrist-worn accelerometers are used in large epidemiological studies (Plotz et al., 2017; Troiano et al., 2014) due to the convenience to capture 24-h physical behavior and increased participant compliance. Thus, methods for wrist-worn accelerometers to estimate sedentary time, which is one important part of the 24-h physical behavior composition, have to be as accurate as possible.

To the best of our knowledge, the validation study by Koster et al. (2016), in which the count cutpoint method was developed, is so far the only study in which sedentary estimates between posture-based thigh- and movement-based wrist-worn accelerometers were compared in free-living conditions. As in that study, we also observed an underestimation of daily sedentary time by the wrist-worn vs. thigh-worn accelerometer when using the movement-based count cutpoint method, but to a larger extent (63 min vs. 23 min). This may be due to the differences of the study populations: our participants were younger (Mage 63 years) and still working, whereas the study participants of the validation study were older (78 years) and retired (Koster et al., 2016). Therefore, our study participants might have spent more time in sedentary activities which include arm movements (e.g., working with computer), and these sitting activities might not be detected as sedentary time by the movement-based wrist-worn accelerometer. Moreover, differences in the data processing (Acti4 vs. PAL) and sleep detection methods can partly explain the observed differences. Another explanation might be the chosen epoch lengths which are reported to affect sedentary estimates (Heesch et al., 2018). Koster et al. (2016) tested the epoch lengths of 15 and 60 s and observed that 60 s was more accurate in assessing sedentary time. In our study, we used previously established methods in which the epoch length was 5 s for the thigh and 60 s for the wrist-worn accelerometer. The optimal epoch lengths for estimating sedentary time by accelerometers worn in different locations are still under debate, but an epoch length of 60 s for wrist-worn accelerometers may be more accurate compared to shorter epoch lengths when evaluated against a thigh-worn accelerometer (Heesch et al., 2018; Koster et al., 2016) and direct observation (Flórez-Pregonero et al., 2018).

Use of wrist-worn accelerometers and especially raw acceleration data is becoming more common and therefore knowledge of the accuracy of the movement-based ENMO cutpoint methods for the wrist-worn accelerometers is essential. The ENMO cutpoint method resulted also in an underestimation of daily sedentary time by 1 hr compared to the thigh-worn accelerometer. We decided to estimate daily sedentary time from 60-s bouts in which 90% of the minimum required time met the sedentary threshold criteria, because 60-s epoch length for wrist accelerometer has been reported to be more accurate compared to shorter epoch lengths (Heesch et al., 2018; Koster et al., 2016). In addition, a 5-s epoch length or shorter for wrist-worn accelerometers resulted in large overestimation of daily sedentary time, from 60 to 140 min, when compared to thigh-worn accelerometers in our supplemental analyses (Supplementary Material 3 [available online]) and also in other adult study populations (Hildebrand et al., 2017). It should be noted that the threshold value of 30 mg that we used was developed among a small group of young adults (N = 20) during 2 hr of activities in supervised free-living conditions (Rowlands et al., 2018). Movement-based sedentary thresholds may not be easily transferred from young adults to older adults because of different movement patterns and gait speed (Heesch et al., 2018). Therefore, additional research to determine a more optimal movement-based threshold value for sedentary behavior when using the ENMO cutpoint method for wrist-worn accelerometers might be needed. A possible future direction for research may focus on threshold values for “stationary behaviors” including sitting, lying, and standing still, because movement-based methods for wrist-worn accelerometers may be more suitable for estimating these stationary behaviors than for estimating sedentary behaviors (Flo´rez-Pregonero et al., 2018; Tremblay et al., 2017). Although, as pointed out, there are challenges associated with threshold-based methods, and machine learning methods may offer a more accurate approach for processing acceleration data from wrist-worn accelerometers (Heesch et al., 2018). However, machine learning methods still need considerable field-based validation for each target population (Heesch et al., 2018; Sasaki et al., 2016).

In addition to examining differences in daily sedentary time estimates obtained from the wrist- and thigh-worn accelerometers, we also investigated within-individual changes in sedentary time between work days and days off when assessed by the different methods. We found that estimates of within-individual changes in daily sedentary time were very similar, independent of the device location or data processing method. This suggests that although the movement-based wrist-worn accelerometer underestimates absolute daily sedentary time compared to a posture-based method, it can reliably estimate individual changes in daily sedentary time.

Our study has several important strengths. First, we included a large number of participants who wore a thigh- and wrist-worn accelerometer simultaneously for up to 7 24-h measurement days in free-living conditions with high compliance. Therefore, our study has an advantage over laboratory studies in which activities are usually simple and of short duration and may not represent complex activities of daily living. Furthermore, our reference method was a thigh-worn accelerometer, a device that has shown high accuracy in detecting time spent sitting and lying down and therefore to produce estimates of sedentary time due to thigh inclination information (Kozey Keadle et al., 2011; Lyden et al., 2012; Skotte et al., 2014; Stemland et al., 2015). The decision tree applied in the Acti4 software, the raw data processing method for thigh-worn accelerometer data used in this study, has a high sensitivity and specificity to detect sitting and lying down (i.e., postural component of sedentary behavior) both in semi-standardized protocol in comparison to video observations (Crowley et al., 2019) and in free-living conditions (Skotte et al., 2014; Stemland et al., 2015); although the distinction between sitting and lying postures is not possible when using only the thigh-worn accelerometer. Nevertheless, our aim was to compare a movement-based wrist-worn accelerometer to a more accurate posture-based thigh-worn accelerometer, as there are no field-based tools to measure directly or accurately the metabolic component of sedentary behavior. Furthermore, Acti4 software has usually been used to process data from thigh-worn ActiGraph GT3X+ accelerometers (Skotte et al., 2014; Stemland et al., 2015), but it has good equivalence when processing data from ActiGraph GT3X+ and Axivity AX3 accelerometers (Crowley et al., 2019; Rowlands et al., 2018).

Our study also has some limitations that should be addressed. Due to different accelerometer wear locations, accelerometer brands, epoch lengths, and algorithms to detect non-wear and sleep time used in our study, there are several possible factors contributing to the observed differences in daily sedentary time. Generally, our aim was to examine differences in daily sedentary time estimates between the previously established methods in their standard form; nonetheless, we harmonized the methods as much as we could within the limits of the algorithms. Epoch length (i.e., sampling frequency) has been observed to affect generation of counts from raw acceleration data (Brønd & Arvidsson, 2016), and, thus, it may also affect the daily sedentary time estimates (Heesch et al., 2018). Epoch length is probably more critical for wrist-worn accelerometers than for thigh-worn accelerometers, because wrist movements can be expected to be more frequent compared to thigh movements during sedentary activities. We used a 60-s epoch length (count cutpoint method) as well as a 60-s bout length (ENMO cutpoint method) for the wrist, which has been reported to be more accurate for wrist-worn accelerometers compared to shorter epoch lengths (Heesch et al., 2018; Koster et al., 2016). Sleep detection methods have also been reported to affect daily sedentary time estimates (Meredith Jones, Williams, Galland, Kennedy, & Taylor, 2016; Pulakka, Shiroma, et al., 2018), whereas methods to detect non-wear time also have an effect, but to a lesser extent (Pulakka, Shiroma, et al., 2018). The thigh- and wrist-worn accelerometers may have had different wear times because a wrist-worn accelerometer is easily removed and therefore it is possible that the wrist-worn accelerometer was not worn while the thigh-worn accelerometer was in use and attached to the thigh. To minimize the effects of differences in wear or sleep time in our study, we included only days with a maximum of 10 min of difference in daily waking wear time between thigh- and wrist-worn accelerometers. In addition, we used exactly same sleep detection method in the comparison between the count cutpoint method and the thigh-worn accelerometer; thus, there were no differences in sleep time and only minor differences in non-wear time in this comparison. In the comparison between the ENMO cutpoint method and the thigh-worn accelerometer there were only minor differences both in sleep time and non-wear time.

Generalizability of our results may be limited, because our study population was homogenous regarding age and ethnicity. Therefore, future research should determine accuracy of wrist-worn accelerometers in estimating sedentary time in more diverse populations.

Conclusions

The results of this study of working-age adults in free-living conditions show that compared to a position-sensitive thigh-worn accelerometer, a wrist-worn accelerometer, by which sedentary time is estimated based on movement, underestimates daily sedentary time by 1 hr. This should be taken into consideration when comparing results of sedentary time between studies that have used different estimation methods (posture- or movement-based) and data processing methods to estimate sedentary time. As wrist-worn accelerometers are increasingly used in the field of physical activity and epidemiological research, sedentary time estimates obtained from wrist-worn accelerometers using currently available movement-based cutpoint methods should be interpreted with caution. Machine learning methods offer a promising tool to improve accuracy of wrist-worn accelerometers. Future research is needed to improve accuracy of wrist-worn accelerometers in estimating sedentary time in free-living conditions.

Acknowledgments

This work was supported by the Academy of Finland (Grants 286294, 294154 and 319246 to SS; 309526 to TL); Finnish Ministry of Education and Culture (to SS), Nordforsk (to JV), Hospital District of South-West Finland (SS, TL and AP), Juho Vainio Foundation (KS), the Department of Clinical Medicine, University of Turku (KS), and Päivikki and Sakari Sohlberg Foundation (AP). The authors want to thank the FIREA participants for their willingness to participate in the study and the FIREA study staff members for their contribution in the data collection. In addition, the authors wish to acknowledge CSC – IT Center for Science, Finland, for computational resources and support. The authors declare no conflict of interest.

References

  • Bakrania, K., Yates, T., Rowlands, A., Esliger, D., Bunnewell, S., Sanders, J., . . . Edwardson, C. (2016). Intensity thresholds on raw acceleration data: Euclidean norm minus one (ENMO) and mean amplitude deviation (MAD) approaches. PLoS ONE, 11(10), e0164045. PubMed ID: 27706241 doi:10.1371/journal.pone.0164045

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Biswas, A., Oh, P.I., Faulkner, G.E., Bajaj, R.R., Silver, M.A., Mitchell, M.S., & Alter, D.A. (2015). Sedentary time and its association with risk for disease incidence, mortality, and hospitalization in adults: A systematic review and meta-analysis. Annals of Internal Medicine, 162(2), 123132. PubMed ID: 25599350 doi:10.7326/M14-1651

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bland, J.M., & Altman, D. (2007). Agreement between methods of measurement with multiple observations per individual. Journal of Biopharmaceutical Statistics, 17(4), 571582. PubMed ID: 17613642 doi:10.1080/10543400701329422

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brønd, J., & Arvidsson, D. (2016). Sampling frequency affects the processing of actigraph raw acceleration data to activity counts. Journal of Applied Physiology, 120(3), 362369. doi:10.1152/japplphysiol.00628.2015

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chastin, S., Culhane, B., & Dall, P.M. (2014). Comparison of self-reported measure of sitting time (IPAQ) with objective measurement (activPAL). Physiological Measurement, 35(11), 23192328. PubMed ID: 25341050 doi:10.1088/0967-3334/35/11/2319

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Choi, L., Liu, Z., Matthews, C., & Buchowski, M. (2011). Validation of accelerometer wear and nonwear time classification algorithm. Medicine & Science in Sports & Exercise, 43(2), 357364. PubMed ID: 20581716 doi:10.1249/MSS.0b013e3181ed61a3

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Choi, L., Ward, S., Schnelle, J., & Buchowski, M. (2012). Assessment of wear/nonwear time classification algorithms for triaxial accelerometer. Medicine & Science in Sports & Exercise, 44(10), 20092016. PubMed ID: 22525772 doi:10.1249/MSS.0b013e318258cb36

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crowley, P., Skotte, J., Stamatakis, E., Hamer, M., Aadahl, M., Stevens, M., . . . Holtermann, A. (2019). Comparison of physical behavior estimates from three different thigh-worn accelerometers brands: A proof-of-concept for the prospective physical activity, sitting, and sleep consortium (ProPASS). International Journal of Behavioral Nutrition and Physical Activity, 16(1), 65. PubMed ID: 31419998 doi:10.1186/s12966-019-0835-0

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Edwardson, C., Winkler, E.A.H., Bodicoat, D., Yates, T., Davies, M., Dunstan, D., & Healy, G. (2017). Considerations when using the activPAL monitor in field-based research with adult populations. Journal of Sport and Health Science(Jshs), 6(2), 162178. doi:10.1016/j.jshs.2016.02.002

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ekelund, U., Steene Johannessen, J., Brown, W.J., Fagerland, M., Owen, N., Powell, K., . . . Lee, I. (2016). Does physical activity attenuate, or even eliminate, the detrimental association of sitting time with mortality? A harmonised meta-analysis of data from more than 1 million men and women. The Lancet, 388(10051), 13021310. doi:10.1016/S0140-6736(16)30370-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Flórez-Pregonero, A., Buman, M.S., & Ainsworth, B.E. (2018). The accuracy of the placement of wearable monitors to classify sedentary and stationary time under free-living conditions. Journal for the Measurement of Physical Behaviour, 1, 165173. doi:10.1123/jmpb.2018-0022

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Freedson, P., & John, D. (2013). Comment on “estimating activity and sedentary behavior from an accelerometer on the hip and wrist”. Medicine & Science in Sports & Exercise, 45(5), 962963. PubMed ID: 23594509 doi:10.1249/MSS.0b013e31827f024d

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gupta, N., Stordal, C.C., Hallman, D., Korshøj, M., Gomes, C.I., & Holtermann, A. (2015). Is objectively measured sitting time associated with low back pain? A cross-sectional investigation in the NOMAD study. PLoS ONE, 10(3), e0121159. PubMed ID: 25806808 doi:10.1371/journal.pone.0121159

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harvey, J.A., Chastin, S.F., & Skelton, D.A. (2015). How sedentary are older people? A systematic review of the amount of sedentary behavior. Journal of Aging and Physical Activity, 23(3), 471487. PubMed ID: 25387160 doi:10.1123/japa.2014-0164

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Healy, G.N., Matthews, C.E., Dunstan, D.W., Winkler, E.A., & Owen, N. (2011). Sedentary time and cardio-metabolic biomarkers in US adults: NHANES 2003-06. European Heart Journal, 32(5), 590597. PubMed ID: 21224291 doi:10.1093/eurheartj/ehq451

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Heesch, K.C., Hill, R.L., Aguilar-Farias, N., van Uffelen, J., & Pavey, T. (2018). Validity of objective methods for measuring sedentary behaviour in older adults: A systematic review. International Journal of Behavioral Nutrition and Physical Activity, 15, 119. PubMed ID: 30477509 doi:10.1186/s12966-018-0749-2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hildebrand, M., Hansen, B.H., van Hees, V.T., & Ekelund, U. (2017). Evaluation of raw acceleration sedentary thresholds in children and adults. Scandinavian Journal of Medicine & Science in Sports, 27(12), 18141823. PubMed ID: 27878845 doi:10.1111/sms.12795

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kongsvold, A.M. (2016). Validation of the AX3 accelerometer for detection of common daily activties and postures. Norges teknisk-naturvitenskapelige universitet Open archive. Retrieved from http://hdl.handle.net/11250/2400440

    • Search Google Scholar
    • Export Citation
  • Koster, A., Shiroma, E.J., Caserotti, P., Matthews, C.E., Chen, K.Y., Glynn, N.W., & Harris, T.B. (2016). Comparison of sedentary estimates between activPAL and hip- and wrist-worn ActiGraph. Medicine & Science in Sports & Exercise, 48(8), 15141522. PubMed ID: 27031744 doi:10.1249/MSS.0000000000000924

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kozey Keadle, S., Libertine, A., Lyden, K., Staudenmayer, J., & Freedson, P. (2011). Validation of wearable monitors for assessing sedentary behavior. Medicine & Science in Sports & Exercise, 43(8), 15611567. PubMed ID: 21233777 doi:10.1249/MSS.0b013e31820ce174

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leskinen, T., Pulakka, A., Heinonen, O.J., Pentti, J., Kivimäki, M., Vahtera, J., & Stenholm, S. (2018). Changes in non-occupational sedentary behaviours across the retirement transition: The finnish retirement and aging (FIREA) study. Journal of Epidemiology and Community Health, 72, 695701. PubMed ID: 29636399 doi:10.1136/jech-2017-209958

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lyden, K., Kozey Keadle, S., Staudenmayer, J., & Freedson, P. (2012). Validity of two wearable monitors to estimate breaks from sedentary time. Medicine & Science in Sports & Exercise, 44(11), 22432252. PubMed ID: 22648343 doi:10.1249/MSS.0b013e318260c477

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Matthews, C., Chen, K., Freedson, P., Buchowski, M., Beech, B., Pate, R., & Troiano, R. (2008). Amount of time spent in sedentary behaviors in the united states, 2003-2004. American Journal of Epidemiology, 167(7), 875881. PubMed ID: 18303006 doi:10.1093/aje/kwm390

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meredith Jones, K., Williams, S., Galland, B., Kennedy, G., & Taylor, R. (2016). 24 h accelerometry: Impact of sleep-screening methods on estimates of sedentary behaviour and physical activity while awake. Journal of Sports Sciences, 34(7), 679685. PubMed ID: 26194337 doi:10.1080/02640414.2015.1068438

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Migueles, J.H., Rowlands, A.V., Huber, F., Sabia, S., & van Hees, V.T. (2019). GGIR: A research Community-Driven open source R package for generating physical activity and sleep outcomes from multi-day raw accelerometer data. Journal for the Measurement of Physical Behaviour, 2(3), 188196. Retrieved from https://journals.humankinetics.com/view/journals/jmpb/2/3/article-p188.xml

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pavey, T., Gomersall, S., Clark, B., & Brown, W. (2016). The validity of the GENEActiv wrist-worn accelerometer for measuring adult sedentary time in free living. Journal of Science and Medicine in Sport, 19(5), 395399. PubMed ID: 25956687 doi:10.1016/j.jsams.2015.04.007

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Plotz, T., Doherty, A., Jackson, D., Hammerla, N., Plötz, T., Olivier, P., . . . Wareham, N. (2017). Large scale population assessment of physical activity using wrist worn accelerometers: The UK biobank study. PLoS ONE, 12(2), e0169649. doi:10.1371/journal.pone.0169649

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Powell, C., Herring, M.P., Dowd, K.P., Donnelly, A.E., & Carson, B.P. (2018). The cross-sectional associations between objectively measured sedentary time and cardiometabolic health markers in adults - a systematic review with meta-analysis component. Obesity Reviews, 19(3), 381395. PubMed ID: 29178252 doi:10.1111/obr.12642

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pulakka, A., Leskinen, T., Koster, A., Pentti, J., Vahtera, J., & Stenholm, S. (2019). Daily physical activity patterns among aging workers: The finnish retirement and aging study (FIREA). Occupational and Environmental Medicine, 76(1), 3339. PubMed ID: 30352811 doi:10.1136/oemed-2018-105266

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pulakka, A., Shiroma, E., Harris, T., Pentti, J., Vahtera, J., & Stenholm, S. (2018). Classification and processing of 24-hour wrist accelerometer data. Journal for the Measurement of Physical Behaviour, 1(2), 51. doi:10.1123/jmpb.2017-0008

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pulakka, A., Stenholm, S., Bosma, H., Schaper, N., Savelberg, H., Stehouwer, C.D.A., . . . Koster, A. (2018). Association between employment status and objectively measured physical activity and sedentary behavior: The maastricht study. Occupational and Environmental Medicine, 60(4), 309315. doi:10.1097/JOM.0000000000001254

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Quante, M., Kaplan, E., Rueschman, M., Cailler, M., Buxton, O., & Redline, S. (2015). Practical considerations in using accelerometers to assess physical activity, sedentary behavior, and sleep. Sleep Health, 1(4), 275284. PubMed ID: 29073403 doi:10.1016/j.sleh.2015.09.002

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rowlands, A., Mirkes, E., Yates, T., Clemes, S., Davies, M., Khunti, K., & Edwardson, C. (2018). Accelerometer-assessed physical activity in epidemiology: Are monitors equivalent? Medicine & Science in Sports & Exercise, 50(2), 257265. PubMed ID: 28976493 doi:10.1249/MSS.0000000000001435

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sabia, S., van Hees, V.T., Shipley, M., Trenell, M., Hagger, J.G., Elbaz, A., . . . Singh Manoux, A. (2014). Association between questionnaire- and accelerometer-assessed physical activity: The role of sociodemographic factors. American Journal of Epidemiology, 179(6), 781790. PubMed ID: 24500862 doi:10.1093/aje/kwt330

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanders, G., Boddy, L., Sparks, S.A., Curry, W., Roe, B., Kaehne, A., & Fairclough, S. (2019). Evaluation of wrist and hip sedentary behaviour and moderate-to-vigorous physical activity raw acceleration cutpoints in older adults. Journal of Sports Sciences, 37(11), 12701279. PubMed ID: 30558487 doi:10.1080/02640414.2018.1555904

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sasaki, J., Hickey, A., Staudenmayer, J., John, D., Kent, J., & Freedson, P. (2016). Performance of activity classification algorithms in free-living older adults. Medicine & Science in Sports & Exercise, 48(5), 941950. PubMed ID: 26673129 doi:10.1249/MSS.0000000000000844

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schrack, J., Cooper, R., Koster, A., Shiroma, E., Murabito, J., Rejeski, W.J., . . . Harris, T. (2016). Assessing daily physical activity in older adults: Unraveling the complexity of monitors, measures, and methods. The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences, 71(8), 10391048. PubMed ID: 26957472 doi:10.1093/gerona/glw026

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shiroma, E.J., Schepps, M.A., Harezlak, J., Chen, K.Y., Matthews, C.E., Koster, A., . . . Harris, T.B. (2016). Daily physical activity patterns from hip- and wrist-worn accelerometers. Physiological Measurement, 37(10), 18521861. PubMed ID: 27654140 doi:10.1088/0967-3334/37/10/1852

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skotte, J., Korshøj, M., Kristiansen, J., Hanisch, C., & Holtermann, A. (2014). Detection of physical activity types using triaxial accelerometers. Journal of Physical Activity and Health, 11(1), 7684. PubMed ID: 23249722 doi:10.1123/jpah.2011-0347

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Statistics Finland. (2010). Classification of occupations. Retrieved from http://www.stat.fi/meta/luokitukset/ammatti/001-2010/index_en.html

    • Search Google Scholar
    • Export Citation
  • Stemland, I., Ingebrigtsen, J., Christiansen, C., Jensen, B., Hanisch, C., Skotte, J., & Holtermann, A. (2015). Validity of the Acti4 method for detection of physical activity types in free-living settings: Comparison with video analysis. Ergonomics, 58(6), 953965. PubMed ID: 25588819 doi:10.1080/00140139.2014.998724

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, M., Aubert, S., Barnes, J., Saunders, T., Carson, V., Latimer Cheung, A., . . . Chinapaw, M.J.M. (2017). Sedentary behavior research network (SBRN) - terminology consensus project process and outcome. International Journal of Behavioral Nutrition and Physical Activity, 14(1), 75. PubMed ID: 28599680 doi:10.1186/s12966-017-0525-8

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Troiano, R., McClain, J., Brychta, R., & Chen, K. (2014). Evolution of accelerometer methods for physical activity research. British Journal of Sports Medicine, 48(13), 10191023. PubMed ID: 24782483 doi:10.1136/bjsports-2014-093546

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T. (2014). Autocalibration of accelerometer data for free-living physical activity assessment using local gravity and temperature: An evaluation on four continents. Journal of Applied Physiology, 117(7), 738. PubMed ID: 25103964 doi:10.1152/japplphysiol.00421.2014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T., Gorzelniak, L., Dean León, E., Eder, M., Pias, M., Taherian, S., . . . Brage, S. (2013). Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity. PLoS ONE, 8(4), e61691. PubMed ID: 23626718 doi:10.1371/journal.pone.0061691

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T., Sabia, S., Anderson, K., Denton, S., Oliver, J., Catt, M., . . . Singh Manoux, A. (2015). A novel, open access method to assess sleep duration using a wrist-worn accelerometer. PLoS ONE, 10(11), e0142533. PubMed ID: 26569414 doi:10.1371/journal.pone.0142533

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vasankari, V., Husu, P., Vaha-Ypya, H., Suni, J., Tokola, K., Halonen, J., . . . Vasankari, T. (2017). Association of objectively measured sedentary behaviour and physical activity with cardiovascular disease risk. European Journal of Preventive Cardiology, 24(12), 13111318. PubMed ID: 28530126 doi:10.1177/2047487317711048

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Watson, K., Carlson, S., Carroll, D., & Fulton, J. (2014). Comparison of accelerometer cut points to estimate physical activity in US adults. Journal of Sports Sciences, 32(7), 660669. PubMed ID: 24188163 doi:10.1080/02640414.2013.847278

    • Crossref
    • Search Google Scholar
    • Export Citation

Suorsa and Pulakka share first authorship. Suorsa, Pulakka, Leskinen, Pentti, Sunikka, Vahtera, and Stenholm are with the Department of Public Health and the Centre for Population Health Research, University of Turku, Turku, Finland; and Turku University Hospital, Turku, Finland. Pulakka is also with the Department of Public Health Solutions, Public Health Promotion Unit, Finnish Institute for Health and Welfare, Helsinki, Finland. Pentti is also with the Department of Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland. Holtermann is with the National Research Centre for the Working Environment, Copenhagen, Denmark. Heinonen is with the Department of Health and Physical Activity, Paavo Nurmi Centre, University of Turku, Turku, Finland.

Suorsa (kristin.suorsa@utu.fi) is corresponding author.
  • Collapse
  • Expand
  • Figure 1

    —Correlation between daily sedentary time estimates from the thigh-worn and wrist-worn accelerometers using (A) the count cutpoint and (B) the Euclidean Norm Minus One cutpoint.

  • Figure 2

    —The Bland-Altman plots describing the level of agreement in daily sedentary time estimates by different methods. The difference in daily sedentary time estimates between the thigh-worn accelerometers and (A) the count cutpoint and (B) the Euclidean Norm Minus One cutpoint is plotted against the mean of daily sedentary time obtained by these two methods. Solid gray line is the zero bias line representing the mean difference and dashed lines present the 95% limits of agreement. The histogram describes the number of observations (i.e., number of days for each value of difference between daily sedentary time estimates obtained from the thigh- and wrist-worn accelerometers).

  • Bakrania, K., Yates, T., Rowlands, A., Esliger, D., Bunnewell, S., Sanders, J., . . . Edwardson, C. (2016). Intensity thresholds on raw acceleration data: Euclidean norm minus one (ENMO) and mean amplitude deviation (MAD) approaches. PLoS ONE, 11(10), e0164045. PubMed ID: 27706241 doi:10.1371/journal.pone.0164045

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Biswas, A., Oh, P.I., Faulkner, G.E., Bajaj, R.R., Silver, M.A., Mitchell, M.S., & Alter, D.A. (2015). Sedentary time and its association with risk for disease incidence, mortality, and hospitalization in adults: A systematic review and meta-analysis. Annals of Internal Medicine, 162(2), 123132. PubMed ID: 25599350 doi:10.7326/M14-1651

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bland, J.M., & Altman, D. (2007). Agreement between methods of measurement with multiple observations per individual. Journal of Biopharmaceutical Statistics, 17(4), 571582. PubMed ID: 17613642 doi:10.1080/10543400701329422

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brønd, J., & Arvidsson, D. (2016). Sampling frequency affects the processing of actigraph raw acceleration data to activity counts. Journal of Applied Physiology, 120(3), 362369. doi:10.1152/japplphysiol.00628.2015

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chastin, S., Culhane, B., & Dall, P.M. (2014). Comparison of self-reported measure of sitting time (IPAQ) with objective measurement (activPAL). Physiological Measurement, 35(11), 23192328. PubMed ID: 25341050 doi:10.1088/0967-3334/35/11/2319

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Choi, L., Liu, Z., Matthews, C., & Buchowski, M. (2011). Validation of accelerometer wear and nonwear time classification algorithm. Medicine & Science in Sports & Exercise, 43(2), 357364. PubMed ID: 20581716 doi:10.1249/MSS.0b013e3181ed61a3

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Choi, L., Ward, S., Schnelle, J., & Buchowski, M. (2012). Assessment of wear/nonwear time classification algorithms for triaxial accelerometer. Medicine & Science in Sports & Exercise, 44(10), 20092016. PubMed ID: 22525772 doi:10.1249/MSS.0b013e318258cb36

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Crowley, P., Skotte, J., Stamatakis, E., Hamer, M., Aadahl, M., Stevens, M., . . . Holtermann, A. (2019). Comparison of physical behavior estimates from three different thigh-worn accelerometers brands: A proof-of-concept for the prospective physical activity, sitting, and sleep consortium (ProPASS). International Journal of Behavioral Nutrition and Physical Activity, 16(1), 65. PubMed ID: 31419998 doi:10.1186/s12966-019-0835-0

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Edwardson, C., Winkler, E.A.H., Bodicoat, D., Yates, T., Davies, M., Dunstan, D., & Healy, G. (2017). Considerations when using the activPAL monitor in field-based research with adult populations. Journal of Sport and Health Science(Jshs), 6(2), 162178. doi:10.1016/j.jshs.2016.02.002

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ekelund, U., Steene Johannessen, J., Brown, W.J., Fagerland, M., Owen, N., Powell, K., . . . Lee, I. (2016). Does physical activity attenuate, or even eliminate, the detrimental association of sitting time with mortality? A harmonised meta-analysis of data from more than 1 million men and women. The Lancet, 388(10051), 13021310. doi:10.1016/S0140-6736(16)30370-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Flórez-Pregonero, A., Buman, M.S., & Ainsworth, B.E. (2018). The accuracy of the placement of wearable monitors to classify sedentary and stationary time under free-living conditions. Journal for the Measurement of Physical Behaviour, 1, 165173. doi:10.1123/jmpb.2018-0022

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Freedson, P., & John, D. (2013). Comment on “estimating activity and sedentary behavior from an accelerometer on the hip and wrist”. Medicine & Science in Sports & Exercise, 45(5), 962963. PubMed ID: 23594509 doi:10.1249/MSS.0b013e31827f024d

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gupta, N., Stordal, C.C., Hallman, D., Korshøj, M., Gomes, C.I., & Holtermann, A. (2015). Is objectively measured sitting time associated with low back pain? A cross-sectional investigation in the NOMAD study. PLoS ONE, 10(3), e0121159. PubMed ID: 25806808 doi:10.1371/journal.pone.0121159

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harvey, J.A., Chastin, S.F., & Skelton, D.A. (2015). How sedentary are older people? A systematic review of the amount of sedentary behavior. Journal of Aging and Physical Activity, 23(3), 471487. PubMed ID: 25387160 doi:10.1123/japa.2014-0164

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Healy, G.N., Matthews, C.E., Dunstan, D.W., Winkler, E.A., & Owen, N. (2011). Sedentary time and cardio-metabolic biomarkers in US adults: NHANES 2003-06. European Heart Journal, 32(5), 590597. PubMed ID: 21224291 doi:10.1093/eurheartj/ehq451

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Heesch, K.C., Hill, R.L., Aguilar-Farias, N., van Uffelen, J., & Pavey, T. (2018). Validity of objective methods for measuring sedentary behaviour in older adults: A systematic review. International Journal of Behavioral Nutrition and Physical Activity, 15, 119. PubMed ID: 30477509 doi:10.1186/s12966-018-0749-2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hildebrand, M., Hansen, B.H., van Hees, V.T., & Ekelund, U. (2017). Evaluation of raw acceleration sedentary thresholds in children and adults. Scandinavian Journal of Medicine & Science in Sports, 27(12), 18141823. PubMed ID: 27878845 doi:10.1111/sms.12795

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kongsvold, A.M. (2016). Validation of the AX3 accelerometer for detection of common daily activties and postures. Norges teknisk-naturvitenskapelige universitet Open archive. Retrieved from http://hdl.handle.net/11250/2400440

    • Search Google Scholar
    • Export Citation
  • Koster, A., Shiroma, E.J., Caserotti, P., Matthews, C.E., Chen, K.Y., Glynn, N.W., & Harris, T.B. (2016). Comparison of sedentary estimates between activPAL and hip- and wrist-worn ActiGraph. Medicine & Science in Sports & Exercise, 48(8), 15141522. PubMed ID: 27031744 doi:10.1249/MSS.0000000000000924

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kozey Keadle, S., Libertine, A., Lyden, K., Staudenmayer, J., & Freedson, P. (2011). Validation of wearable monitors for assessing sedentary behavior. Medicine & Science in Sports & Exercise, 43(8), 15611567. PubMed ID: 21233777 doi:10.1249/MSS.0b013e31820ce174

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leskinen, T., Pulakka, A., Heinonen, O.J., Pentti, J., Kivimäki, M., Vahtera, J., & Stenholm, S. (2018). Changes in non-occupational sedentary behaviours across the retirement transition: The finnish retirement and aging (FIREA) study. Journal of Epidemiology and Community Health, 72, 695701. PubMed ID: 29636399 doi:10.1136/jech-2017-209958

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lyden, K., Kozey Keadle, S., Staudenmayer, J., & Freedson, P. (2012). Validity of two wearable monitors to estimate breaks from sedentary time. Medicine & Science in Sports & Exercise, 44(11), 22432252. PubMed ID: 22648343 doi:10.1249/MSS.0b013e318260c477

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Matthews, C., Chen, K., Freedson, P., Buchowski, M., Beech, B., Pate, R., & Troiano, R. (2008). Amount of time spent in sedentary behaviors in the united states, 2003-2004. American Journal of Epidemiology, 167(7), 875881. PubMed ID: 18303006 doi:10.1093/aje/kwm390

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meredith Jones, K., Williams, S., Galland, B., Kennedy, G., & Taylor, R. (2016). 24 h accelerometry: Impact of sleep-screening methods on estimates of sedentary behaviour and physical activity while awake. Journal of Sports Sciences, 34(7), 679685. PubMed ID: 26194337 doi:10.1080/02640414.2015.1068438

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Migueles, J.H., Rowlands, A.V., Huber, F., Sabia, S., & van Hees, V.T. (2019). GGIR: A research Community-Driven open source R package for generating physical activity and sleep outcomes from multi-day raw accelerometer data. Journal for the Measurement of Physical Behaviour, 2(3), 188196. Retrieved from https://journals.humankinetics.com/view/journals/jmpb/2/3/article-p188.xml

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pavey, T., Gomersall, S., Clark, B., & Brown, W. (2016). The validity of the GENEActiv wrist-worn accelerometer for measuring adult sedentary time in free living. Journal of Science and Medicine in Sport, 19(5), 395399. PubMed ID: 25956687 doi:10.1016/j.jsams.2015.04.007

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Plotz, T., Doherty, A., Jackson, D., Hammerla, N., Plötz, T., Olivier, P., . . . Wareham, N. (2017). Large scale population assessment of physical activity using wrist worn accelerometers: The UK biobank study. PLoS ONE, 12(2), e0169649. doi:10.1371/journal.pone.0169649

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Powell, C., Herring, M.P., Dowd, K.P., Donnelly, A.E., & Carson, B.P. (2018). The cross-sectional associations between objectively measured sedentary time and cardiometabolic health markers in adults - a systematic review with meta-analysis component. Obesity Reviews, 19(3), 381395. PubMed ID: 29178252 doi:10.1111/obr.12642

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pulakka, A., Leskinen, T., Koster, A., Pentti, J., Vahtera, J., & Stenholm, S. (2019). Daily physical activity patterns among aging workers: The finnish retirement and aging study (FIREA). Occupational and Environmental Medicine, 76(1), 3339. PubMed ID: 30352811 doi:10.1136/oemed-2018-105266

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pulakka, A., Shiroma, E., Harris, T., Pentti, J., Vahtera, J., & Stenholm, S. (2018). Classification and processing of 24-hour wrist accelerometer data. Journal for the Measurement of Physical Behaviour, 1(2), 51. doi:10.1123/jmpb.2017-0008

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pulakka, A., Stenholm, S., Bosma, H., Schaper, N., Savelberg, H., Stehouwer, C.D.A., . . . Koster, A. (2018). Association between employment status and objectively measured physical activity and sedentary behavior: The maastricht study. Occupational and Environmental Medicine, 60(4), 309315. doi:10.1097/JOM.0000000000001254

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Quante, M., Kaplan, E., Rueschman, M., Cailler, M., Buxton, O., & Redline, S. (2015). Practical considerations in using accelerometers to assess physical activity, sedentary behavior, and sleep. Sleep Health, 1(4), 275284. PubMed ID: 29073403 doi:10.1016/j.sleh.2015.09.002

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rowlands, A., Mirkes, E., Yates, T., Clemes, S., Davies, M., Khunti, K., & Edwardson, C. (2018). Accelerometer-assessed physical activity in epidemiology: Are monitors equivalent? Medicine & Science in Sports & Exercise, 50(2), 257265. PubMed ID: 28976493 doi:10.1249/MSS.0000000000001435

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sabia, S., van Hees, V.T., Shipley, M., Trenell, M., Hagger, J.G., Elbaz, A., . . . Singh Manoux, A. (2014). Association between questionnaire- and accelerometer-assessed physical activity: The role of sociodemographic factors. American Journal of Epidemiology, 179(6), 781790. PubMed ID: 24500862 doi:10.1093/aje/kwt330

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sanders, G., Boddy, L., Sparks, S.A., Curry, W., Roe, B., Kaehne, A., & Fairclough, S. (2019). Evaluation of wrist and hip sedentary behaviour and moderate-to-vigorous physical activity raw acceleration cutpoints in older adults. Journal of Sports Sciences, 37(11), 12701279. PubMed ID: 30558487 doi:10.1080/02640414.2018.1555904

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sasaki, J., Hickey, A., Staudenmayer, J., John, D., Kent, J., & Freedson, P. (2016). Performance of activity classification algorithms in free-living older adults. Medicine & Science in Sports & Exercise, 48(5), 941950. PubMed ID: 26673129 doi:10.1249/MSS.0000000000000844

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schrack, J., Cooper, R., Koster, A., Shiroma, E., Murabito, J., Rejeski, W.J., . . . Harris, T. (2016). Assessing daily physical activity in older adults: Unraveling the complexity of monitors, measures, and methods. The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences, 71(8), 10391048. PubMed ID: 26957472 doi:10.1093/gerona/glw026

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shiroma, E.J., Schepps, M.A., Harezlak, J., Chen, K.Y., Matthews, C.E., Koster, A., . . . Harris, T.B. (2016). Daily physical activity patterns from hip- and wrist-worn accelerometers. Physiological Measurement, 37(10), 18521861. PubMed ID: 27654140 doi:10.1088/0967-3334/37/10/1852

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Skotte, J., Korshøj, M., Kristiansen, J., Hanisch, C., & Holtermann, A. (2014). Detection of physical activity types using triaxial accelerometers. Journal of Physical Activity and Health, 11(1), 7684. PubMed ID: 23249722 doi:10.1123/jpah.2011-0347

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Statistics Finland. (2010). Classification of occupations. Retrieved from http://www.stat.fi/meta/luokitukset/ammatti/001-2010/index_en.html

    • Search Google Scholar
    • Export Citation
  • Stemland, I., Ingebrigtsen, J., Christiansen, C., Jensen, B., Hanisch, C., Skotte, J., & Holtermann, A. (2015). Validity of the Acti4 method for detection of physical activity types in free-living settings: Comparison with video analysis. Ergonomics, 58(6), 953965. PubMed ID: 25588819 doi:10.1080/00140139.2014.998724

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tremblay, M., Aubert, S., Barnes, J., Saunders, T., Carson, V., Latimer Cheung, A., . . . Chinapaw, M.J.M. (2017). Sedentary behavior research network (SBRN) - terminology consensus project process and outcome. International Journal of Behavioral Nutrition and Physical Activity, 14(1), 75. PubMed ID: 28599680 doi:10.1186/s12966-017-0525-8

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Troiano, R., McClain, J., Brychta, R., & Chen, K. (2014). Evolution of accelerometer methods for physical activity research. British Journal of Sports Medicine, 48(13), 10191023. PubMed ID: 24782483 doi:10.1136/bjsports-2014-093546

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T. (2014). Autocalibration of accelerometer data for free-living physical activity assessment using local gravity and temperature: An evaluation on four continents. Journal of Applied Physiology, 117(7), 738. PubMed ID: 25103964 doi:10.1152/japplphysiol.00421.2014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T., Gorzelniak, L., Dean León, E., Eder, M., Pias, M., Taherian, S., . . . Brage, S. (2013). Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity. PLoS ONE, 8(4), e61691. PubMed ID: 23626718 doi:10.1371/journal.pone.0061691

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Hees, V.T., Sabia, S., Anderson, K., Denton, S., Oliver, J., Catt, M., . . . Singh Manoux, A. (2015). A novel, open access method to assess sleep duration using a wrist-worn accelerometer. PLoS ONE, 10(11), e0142533. PubMed ID: 26569414 doi:10.1371/journal.pone.0142533

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vasankari, V., Husu, P., Vaha-Ypya, H., Suni, J., Tokola, K., Halonen, J., . . . Vasankari, T. (2017). Association of objectively measured sedentary behaviour and physical activity with cardiovascular disease risk. European Journal of Preventive Cardiology, 24(12), 13111318. PubMed ID: 28530126 doi:10.1177/2047487317711048

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Watson, K., Carlson, S., Carroll, D., & Fulton, J. (2014). Comparison of accelerometer cut points to estimate physical activity in US adults. Journal of Sports Sciences, 32(7), 660669. PubMed ID: 24188163 doi:10.1080/02640414.2013.847278

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 5055 1652 209
PDF Downloads 1956 490 24