In February 2022, ActiGraph made their long-proprietary algorithm for generating activity counts publicly available, with Python code for generating counts provided by the company (Neishabouri et al., 2022). Later, R versions were also created, such as by Jairo Migueles (https://github.com/jhmigueles/actilifecounts), which was integrated into the often-used GGIR package and by Brian Helsel (https://github.com/bhelsel/agcounts) which, in a paper within this special issue, was confirmed to produce comparable counts to the ActiLife software (Montoye et al., 2024). The 2024 special issue of the Journal for the Measurement of Physical Behavior (JMPB) explores various aspects of this open-source method of generating ActiGraph activity counts. As the research field strives to understand the implications of counts being made open-source, we wanted to provide a forum where invited experts who have long utilized ActiGraph counts and other methods of analyzing accelerometer data in their research could provide feedback and initial reactions to this development. The purpose of this Expert Question and Answer, therefore, was to solicit and summarize the responses of several noted experts in physical activity (PA) measurement and interpretation.
Invitation for Feedback
In the summer and fall of 2023, the authors compiled lists of frequently cited authors in papers accepted in the JMPB special issue as well as noted experts in physical behavior measurement. From these lists, roughly 10 individuals were contacted to provide responses to a prepared set of questions. It was asked that these experts answer at least two of the five questions posed, and they were given approximately 3 weeks to provide responses. Most accepted the invitation to contribute comments; we have avoided providing exact numbers as some experts requested anonymity for their involvement in providing responses. Respondents answered anywhere from two to five of the provided questions. To the extent possible, we left responses unedited, other than to correct grammatical errors. If editing was needed, for example, to provide clarification or link to a response from a previous question, we sent the edited responses back to the expert for approval. Following all expert responses, we offer brief comments to provide clarification or to point out patterns or similarities/differences in expert responses and how they may inform future directions for the field.
Expert Responses
Expert responses to our questions are provided in this section, question by question. Responses are anonymous and ordered to provide the best possible flow and clarity with as little editorialization as possible.
Question 1: What Was Your Initial Reaction When You Found Out That ActiGraph Released Code to Produce Open-Source ActiGraph Counts Without Needing ActiLife Software or ActiGraph Accelerometers?
- •Finally! It is consistent with the field progressing toward open-source code.
- •A major problem in PA research using wearable devices is the plethora of different devices produced by manufacturers around the world. In addition to that, researchers often take the raw signals from these accelerometer-based PA measurement devices and process them using different methods. Furthermore, many of the devices can be affixed to different sites on the body (e.g., wrist, waist, ankle, and thigh). Due to all of these factors, it has been extremely difficult to compare and/or compile the results of different studies, because of the heterogeneity in how data are being collected, processed, and analyzed. Thus, it was a significant advancement when in 2022, ActiGraph made their counts open-source so other companies could generate ActiGraph counts with raw data from other accelerometer-based devices using the same method as ActiGraph. In a paper in the journal Scientific Reports, Neishabouri et al. (2022) describe precisely how raw acceleration signals are digitized, filtered, full-wave rectified, and integrated to arrive at the nonproprietary counts. This has potential benefits for data harmonization and comparability across studies. However, it does not solve the problems related to different “wear locations.”
- •ActiGraph has been responsive to needs and requests of the research community. For example, they and other research device manufacturers provided unprocessed acceleration data soon after the 2009 workshop, “Objective Measurement of Physical Activity: Best Practices and Future Directions,” called for making these data available. The ActiLife software was updated to allow users to create metrics from the unprocessed data and apply various algorithms to the resulting data to generate measures of physical behavior and sleep. A further concern of the research community was the proprietary nature of ActiGraph counts and other manufacturer-specific metrics. Provision of the code to produce counts is another welcome example of the responsiveness of ActiGraph.
- •I was very pleased to hear this since it is helpful for researchers to have this transparency. Also, it helps remove the barrier of cost for purchasing ActiLife software. In my experience, ActiGraph has been a very good research partner, and this open-source decision attests further to that.
- •I am a bit skeptical as to the potential value of generating counts from raw acceleration data. When raw data became available, one of the advantages highlighted was removing the proprietary nature of counts. A key recommendation from the proceedings from the workshop “Objective Measurement of Physical Activity: Best Practices and Future Direction,” held in 2009 stated “Monitor data should be collected and saved as raw signals with postprocessing used for data transformation” and that “this approach will allow data from different monitors to be directly compared and avoids the uncertainty of the meaning of preprocessed data (e.g., counts)” (Freedson et al., 2012). While releasing the code to generate open-source ActiGraph counts from different monitors could address the lack of comparability between devices afforded by proprietary counts, there were other limitations. For example, the frequency-dependent filtering applied in the generation of ActiGraph counts leads to underestimation of higher intensity activity, such as running, as shown by several groups (Brage, Wedderkopp, Andersen, & Froberg, 2003; Brage, Wedderkopp, Franks, et al., 2003; Rowlands et al., 2007). There are other open-source metrics that can be generated from raw acceleration data collected using different monitors (e.g., Euclidean minus one [ENMO], mean amplitude deviation, and monitor-independent movement summary [MIMS]), some of which have already been used extensively. Therefore, I question what is gained by converting raw acceleration to a metric we already know underestimates higher-intensity PA.
- •The release of the counts algorithm felt incomplete. For example, it did not acknowledge that several researchers have already tried to reverse engineer ActiGraph counts, or that counts from the ActiGraph are not fully compatible with older accelerometers such as the 7,164. Only one paper has attempted to reverse engineer the original device-based counts and by that attempt to preserve backward compatibility with ActiGraph devices from before ∼2014 (van Hees et al., 2010); later work, including the recent publication by Neishabouri (Neishabouri et al., 2022), gave up on backward compatibility with the older ActiGraph generations. As a result, the current algorithm is incomplete as it provides no guarantee that Freedson (Freedson et al., 1998) and Troiano (Troiano et al., 2008) cut-points can safely be applied since those cut-points were developed with early models of ActiGraph accelerometer. Note that this is an issue with generating counts both with ActiLife and with open-source methods.
Question 2: Can You Comment on the Importance of Transparency in Accelerometer Data Processing/Interpretation and in Comparability of Data Across Accelerometer Brands?
- •As many in the field of physical behavior assessment have noted, transparency (e.g., in providing an open-source method to generate ActiGraph counts) is critical to be able to interpret how the digital data represent human behavior and to support comparisons of data across studies and over time. The inability to compare outcome metrics from the various research devices in use prior to the availability of unprocessed acceleration data was a clear impediment to research and understanding of PA and its effects.
- •It is consistent with new National Institutes of Health data sharing policies. Additionally, such transparency prevents mistakes from individual investigators working independently.
- •It is critically important to have transparency in accelerometer data processing/interpretation because the scientific research community is skeptical of wearable devices that use proprietary accelerometer counts and proprietary algorithms to process the data. Researchers often say that a “black box” approach is disadvantageous because it does not provide a complete understanding of how the raw signals are being analyzed, and companies could change their proprietary methods at any time, without warning and without informing end users. This issue is particularly problematic when a study uses a longitudinal study design to track participants over many years. It is also problematic for meta-analytic reviews that attempt to combine cross-sectional data across multiple years. Careful studies are needed to demonstrate that, when worn simultaneously on individuals performing PA, these devices produced by different manufacturers yield open-source counts that are equivalent to each other. In addition, shaker-table studies are needed to demonstrate that devices produced by different manufacturers yield the same open-source counts.
- •Transparency—in the sense of knowing exactly what has been done, rather than any brand is trying to “hide” data—is essential for research so that investigators can make comparisons across studies and across time. This allows combining data—apples to apples—to provide a totality of evidence.
Question 3: What Do You See as Some Advantages of Generating Open-Source ActiGraph Counts From Accelerometer Data When Trying to Understand Physical Behaviors of Those Wearing Accelerometers? Conversely, What Do You See As Some Disadvantages of Generating Open-Source ActiGraph Counts From Accelerometer Data?
- •A potential advantage could be comparability with historical ActiGraph count data; Neishabouri et al. (2022) report that at the end of 2021, there had been >20,000 papers published using ActiGraph devices. However, I am unsure how this could work in practice as accelerometer data collection protocols have changed. For example, ActiGraph count data were predominantly collected using waist-worn devices using a waking wear protocol, for example, in the National Health and Nutrition Examination Survey (NHANES; Troiano et al., 2008). Yet, since 2010, accelerometers have been increasingly worn continuously on the wrist (Belcher et al., 2021; Doherty et al., 2017) or thigh (Stevens et al., 2020), enabling measurement of the full 24-hr period for seven days a week or longer. These data, even if converted to ActiGraph counts, would not be comparable with historical count data collected from the waist site during waking only protocols. So, while being firmly in favor of open-source methods, I am unsure of the value of generating open-source ActiGraph counts moving forward.
- •A major advantage of generating open-source ActiGraph counts is the ability to compare data collected with different device models, even if from different manufacturers, if they provide unprocessed acceleration data. Researchers still need to understand the potential effects of variations in data collection protocols, whether they are at the device level (e.g., sampling frequency) or user protocols (e.g., wear location). Such effects may be trivial or extensive, but they need to be considered. In my opinion, the disadvantage is not particular to open-source ActiGraph counts. My concern is that we are approaching a repeat of the “cut-point conundrum,” where the plethora of available intensity cut-points led to confusion in research interpretation and application. Multiple metrics are available from unprocessed acceleration, such as ENMO (van Hees et al., 2013), activity index, open-source ActiGraph counts, MIMS units (John et al., 2019), mean amplitude deviation (Vähä-Ypyä et al., 2015), and possibly others. Ideally, a consensus metric could be determined through comparative analyses and adopted for use across PA research. Most physiologic measures do not have an array of competing metrics; they have widely accepted measures. This should also be a goal of PA research. However, I fear that the reward system of research publication and promotion makes this a challenging proposition.
- •ActiGraph’s new open-source accelerometer counts have advantages over their original counts, as well as newer types of counts like MIMS or reverse-engineered counts (Brønd et al., 2017). Open-source counts can be used with traditional cut-point methods for the respective age groups and attachment sites; it is just that the traditional methods are no longer device-specific. In this sense, the new open-source counts are advantageous if other manufacturers adopt them and make it easy for end users (who are usually not familiar with Python code) to use them.
- •Advantages include (a) more intuitive/easier to interpret counts than raw accelerations or other metrics, (b) the ability to compare across different generations of ActiGraph and different devices, (c) opportunities for meta-analyses at a more granular level (e.g., bouts) due to ease of combining data from large cohorts, (d) allows for reanalysis/comparisons from older data sets collected with other devices (backward compatibility), and (e) permits comparisons with other data from other countries that have used devices other than ActiGraph. Disadvantages may include the potential to steer investigators toward regression-based, “cut-point” approaches, rather than more sophisticated analyses (e.g., machine learning).
- •Having multiple ways of translating accelerometer data into the desired physical behavior outputs is important, as being able to use both new and old methods allows researchers the opportunity to test the new approaches against the older approaches to determine accuracy and reliability, feasibility for use, and so on.
Question 4: What Do You See as Future Directions in the Field of Accelerometer Use, Data Analysis, and Interpretation? How, If At All, Do You See Open-Source ActiGraph Counts Fitting Into These Future Directions?
- •Future directions in the field of PA accelerometer research include the use of raw acceleration data, analyzed by artificial intelligence. I must admit that I do not understand these advanced methods, which include Hidden Markov models, pattern recognition, machine learning, deep learning, and artificial neural networks. Another direction in the field of PA research using wearable devices is to supplement accelerometer signals by adding in other raw signals. For example, recent research has sought to determine whether inclusion of the ActiGraph’s gyroscope and magnetometer signals can lead to more accurate assessment of PA (Hibbing et al., 2018). Another new direction in the field of PA accelerometer research is to move away from “time spent in different PA intensity categories” (e.g., sedentary behavior, light-intensity activity, moderate-intensity activity, and vigorous-intensity activity). For instance, many researchers today are trying to discern types (i.e., modes) of PA. A few of them are even trying to discern the best method of counting the total number of steps taken in a day! Rather than time interval-based methods of analyzing data (i.e., epochs), newer methods may move to event-based methodologies.
- •Future directions may include combining accelerometer data with other biosensor data and granular integration (e.g., patterns of accumulation and weekend vs. weekday). Additionally, there may be efforts to look backward to enhance ability to look to the future (e.g., gleaning info from older studies to direct future research and research methods).
Question 5: Are There Other Comments You Would Like to Make Regarding ActiGraph’s Release of An Open-Source Method to Generate Their Activity Counts?
- •Over the past couple of decades, it has become clear that PA measurement researchers cannot exert direct control over the companies that manufacture wearable, accelerometer-based devices for assessing PA. Researchers cannot, for instance, say that they desire device harmonization, and therefore, all manufacturers must agree to use the same microelectrical mechanical system accelerometer in their devices, or that they must all process their data in a certain way. However, I believe that it is possible to insist that key output variables are measured with a prescribed level of accuracy. Medically based devices (e.g., automated blood pressure monitors, cholesterol analyzers, and pulse oximeters) have standardized output variables, and the manufacturers must demonstrate that their devices are within predetermined limits when compared to a gold standard. I recognize that consumer-based “fitness trackers” do not fall under the purview of the Food and Drug Administration, but the adoption of wearable devices for assessing PA, sleep, cardiac arrhythmias, arterial blood oxygen saturation, and so on by the medical community is coming, so there is a vital need for accuracy and reliability.
- •Researchers in the field have spent considerable time trying to reverse engineer ActiGraph’s counts because ActiGraph traditionally had a business model to not share their algorithms with the research community. There have been requests to the company in the past to publicize their algorithm, and at those times, they refused to share it. An apology from ActiGraph for causing the research community so much extra work would have been appreciated. Other companies which produce accelerometer-based physical behavior monitoring devices, such as GENEActiv (ActivInsights Ltd.) and Axivity (Axivity Ltd.), have made transparency and raw data availability central components of their business model since their inception. Therefore, ActiGraph should not necessarily be celebrated for finally releasing their counts algorithm when other companies have practiced data transparency for much longer and when many researchers have long shared their methods for translating raw data into meaningful physical behavior outputs using transparent and reproducible methods.
Interpretation and Conclusions
The experts who provided perspectives to the introduction of open-source ActiGraph counts were universally in favor of the adoption of transparent, open-source methods which enhance comparability across device brands and allow for more harmonization of methods for measuring, analyzing, and interpreting accelerometer data collected for physical behavior assessments. However, there was some nuance among experts in their perceptions of the value in open-source ActiGraph counts, specifically. Several were in favor of increased transparency in generating counts, especially considering the widespread use of ActiGraph counts in past and current research. Conversely, some experts felt that other open-source metrics such as ENMO and MIMS share the transparency advantages of open-source counts while also having unique, additional advantages which would make them preferable to open-source counts, depending on the application. For example, the measurement bias with running at speeds above 10 km/hr pointed out by Rowlands et al. (2007) in addition to the removal of digital noise using a dead-band threshold of 66.6 mg (Brønd et al., 2017; Neishabouri et al., 2022; Tryon & Williams, 1996), which is not in line with the reduced noise observed with today’s modern microelectrical mechanical system accelerometers, suggests that there are intrinsic elements of ActiGraph counts which are not optimal.
Relatedly, some experts noted that making counts open-source may be long overdue, as researchers have reverse-engineered many aspects of ActiGraph counts over the years, and other companies have practiced data transparency since inception. Furthermore, ActiGraph counts are only usable with other device brands because such brands have long made raw data easily accessible, a limitation which ActiGraph has only more recently addressed. On a more positive note, because ActiGraph counts are widely used and are now available to utilize across device brands, researchers may be able to select devices with better form or function (e.g., device size, battery life, cost, and integration with other sensors) for their specific needs.
Additionally, transparency in generation of activity counts does not solve many of the challenges in the field of physical behavior measurement, notably the adoption of standards for wear location (e.g., waist vs. wrist vs. thigh), method of determining physical behavior outputs (e.g., cut-points and machine learning), and data analysis considerations (e.g., determination and handling of nonwear), just to name a few. Therefore, while the push for open-source data processing is important, the new availability of open-source counts may be viewed as an important step but not an endpoint in the continued efforts aimed to harmonize and better assess physical behaviors with accelerometer-based wearable devices.
The comment made by one expert that the field is lacking consensus on the most appropriate metric, and, therefore, is at risk of running into an issue similar to the “cut-point conundrum” is informative. While the currently available metrics such as counts may have been developed with consideration of which intrinsic properties are best, for the field to arrive at some consensus on an optimal metric, there needs to be an in-depth discussion on what intrinsic accelerometer and analytic properties would be essential and required; head-to-head comparisons across metrics using various protocols would also yield helpful insights. As an example, in a study in this special issue by Brønd, Møller, and Grøntved (2024), 12 intrinsic properties were compared across six accelerometer metrics (e.g., counts and ENMO), revealing that (a) MIMS was the metric based on the strongest theoretical background, (b) elements of eliminating digital noise were not included with mean amplitude deviation or ENMO which might essentially introduce noise with recordings, and (c) there was large overall diversity in what intrinsic properties (e.g., noise removal) were included in the different metrics. Thus, obtaining consensus on what intrinsic properties should be included could help provide the community with a single optimal accelerometer metric and, importantly, simplify and harmonize analysis and interpretation across studies. Also as noted by the experts, future directions aimed at potentially combining accelerometer data with other sensors are promising, as is the adoption of some standards for assessing device validity and reliability to better determine how, and how well, devices and analysis approaches work for assessing physical behaviors.
In closing, this Expert Question and Answer paper, and the JMPB special issue more broadly, provide important commentary related to the recent availability of open-source counts. Additionally, included studies using open-source counts provide examples of how researchers seeking to utilize ActiGraph counts can implement them in conventional and new ways. Finally, this JMPB special issue provides a forum for answering several important questions that arose with the availability of open-source ActiGraph count generation, but it should be noted that more research is needed to understand the implications, advantages, and disadvantages of having open-source counts available as a way of processing raw accelerometry data.
Acknowledgments
The authors thank Drs. David R. Bassett Jr., I-Min Lee, Edward L. Melanson, Alex V. Rowlands, John R. Sirard, Richard P. Troiano, and anonymous expert(s) who provided their insight.
References
Belcher, B.R., Wolff-Hughes, D.L., Dooley, E.E., Staudenmayer, J., Berrigan, D., Eberhardt, M.S., & Troiano, R.P. (2021). US population-referenced percentiles for wrist-worn accelerometer-derived activity. Medicine & Science in Sports & Exercise, 53(11), 2455–2464.
Brage, S., Wedderkopp, N., Andersen, L.B., & Froberg, K. (2003). Influence of step frequency on movement intensity predictions with the CSA accelerometer: A field validation study in children. Pediatric Exercise Science, 15(3), 277–287.
Brage, S., Wedderkopp, N., Franks, P.W., Andersen, L.B., & Froberg, K. (2003). Reexamination of validity and reliability of the CSA monitor in walking and running. Medicine & Science in Sports & Exercise, 35(8), 1447–1454.
Brønd, J.C., Møller, N.C., & Grøntved, A. (2024). The intrinsic properties of ActiGraph counts and alternatives. Journal for the Measurement of Physical Behaviour, 7(1), jmpb.2023-0049.
Brønd, J.C., Andersen, L.B., & Arvidsson, D. (2017). Generating ActiGraph counts from raw acceleration recorded by an alternative monitor. Medicine & Science in Sports & Exercise, 49(11), 2351–2360.
Doherty, A., Jackson, D., Hammerla, N., Plotz, T., Olivier, P., Granat, M.H., White, T., van Hees, V.T., Trenell, M.I., Owen, C.G., Preece, S.J., Gillions, R., Sheard, S., Peakman, T., Brage, S., & Wareham, N.J. (2017). Large scale population assessment of physical activity using wrist worn accelerometers: The UK Biobank Study. PLoS One, 12(2), e0169649.
Freedson, P., Bowles, H.R., Troiano, R., & Haskell, W. (2012). Assessment of physical activity using wearable monitors: Recommendations for monitor calibration and use in the field. Medicine & Science in Sports & Exercise, 44(1, Suppl. 1), S1–S4.
Freedson, P.S., Melanson, E., & Sirard, J. (1998). Calibration of the Computer Science and Applications, Inc. Accelerometer. Medicine & Science in Sports & Exercise, 30(5), 777–781.
Hibbing, P.R., Lamunion, S.R., Kaplan, A.S., & Crouter, S.E. (2018). Estimating energy expenditure with ActiGraph GT9X inertial measurement unit. Medicine & Science in Sports & Exercise, 50(5), 1093–1102.
John, D., Tang, Q., Albinali, F., & Intille, S. (2019). An open-source monitor-independent movement summary for accelerometer data processing. Journal for the Measurement of Physical Behaviour, 2(4), 268–281.
Montoye, A.H.K., Clevenger, K.A., Boudreaux, B.D., & Schmidt, M.D. (2024). Comparability of 24-hour activity cycle outputs from ActiGraph counts generated in ActiLife and RStudio. Journal for the Measurement of Physical Behaviour, 7(1), jmpb.2023-0047.
Neishabouri, A., Nguyen, J., Samuelsson, J., Guthrie, T., Biggs, M., Wyatt, J., Cross, D., Karas, M., Migueles, J.H., Khan, S., & Guo, C.C. (2022). Quantification of acceleration as activity counts in ActiGraph wearable. Scientific Reports, 12(1), Article 1.
Rowlands, A.V., Stone, M.R., & Eston, R.G. (2007). Influence of speed and step frequency during walking and running on motion sensor output. Medicine & Science in Sports & Exercise, 39(4), 716–727.
Stevens, M.L., Gupta, N., Inan Eroglu, E., Crowley, P.J., Eroglu, B., Bauman, A., Granat, M., Straker, L., Palm, P., Stenholm, S., Aadahl, M., Mork, P., Chastin, S., Rangul, V., Hamer, M., Koster, A., Holtermann, A., & Stamatakis, E. (2020). Thigh-worn accelerometry for measuring movement and posture across the 24-hour cycle: A scoping review and expert statement. BMJ Open Sport & Exercise Medicine, 6(1), e000874.
Troiano, R.P., Berrigan, D., Dodd, K.W., Masse, L.C., Tilert, T., & McDowell, M. (2008). Physical activity in the United States measured by accelerometer. Medicine & Science in Sports & Exercise, 40(1), 181–188.
Tryon, W.W., & Williams, R. (1996). Fully proportional actigraphy: A new instrument. Behavior Research Methods, Instruments, & Computers, 28(3), 392–403.
Vähä-Ypyä, H., Vasankari, T., Husu, P., Suni, J., & Sievänen, H. (2015). A universal, accurate intensity-based classification of different physical activities using raw data of accelerometer. Clinical Physiology and Functional Imaging, 35(1), 64–70.
van Hees, V.T., Gorzelniak, L., Dean Leon, E.C., Eder, M., Pias, M., Taherian, S., Ekelund, U., Renstrom, F., Franks, P.W., Horsch, A., & Brage, S. (2013). Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity. PLoS One, 8(4), e61691.
van Hees, V.T., Pias, M., Taherian, S., Ekelund, U., & Brage, S. (2010). A method to compare new and traditional accelerometry data in physical activity monitoring [Conference session]. 2010 IEEE International Symposium on “A World of Wireless, Mobile and Multimedia Networks” (WoWMoM), Montreal, Quebec, Canada. 1–6.