The representation of women in high-performance sport has increased in recent decades. Indeed, the Tokyo 2020 Olympic Games was the first to achieve near parity in medal opportunities for women’s and men’s events, with female representation rate at 49% of total competitors, an increase from 45% in Rio 2016 and 38% in Sydney 2000 (Houghton et al., 2017; International Olympic Committee, 2021). Meanwhile, the Paralympic Games lags behind with fewer medal opportunities for women, and a representation of 42% of athletes in Tokyo 2020 (Tokyo 2020, 2021). Across the globe, professional female teams are becoming firmly established, demands for equal prize money normalized, and increases in female participation rates across all levels are driving the increase in the popularity of women’s sport (Claus, 2020; Douglas, 2018; Oxley, 2021; Townes, 2019). These trends justify the support for female-specific sports science and sports medicine (SSSM) research to investigate specific needs of this athletic population and their events. Numerous anatomical and biological differences exist between the sexes, which in turn can influence performance, fundamental biomechanics, and physiological responses to exercise (Devries, 2016; Green et al., 2016; McNulty et al., 2020). Women’s sports may also differ from those of their male counterparts, according to the event demands or characteristics of the typical playing styles, including shorter distances and lighter equipment for women (Kovalchik & Reid, 2017; Sanders et al., 2019). It is, therefore, problematic to apply conclusions drawn from male athletes directly to women without considering any influence of these sexual dimorphisms and event-specific demands.
Unfortunately, the conspicuous imbalance of female-specific SSSM research is well known. For example, Costello et al. (2014) surveyed nearly 1,400 studies involving more than 6 million participants published in three principal SSSM journals across a 2-year period. Overall, women contributed just 39% of the total accumulated participant pool, with 4%–13% of studies (depending on the journal) exclusively investigating female participants, compared with 18%–34% focusing entirely on men (Costello et al., 2014). A smaller audit of three journals over a 5-month period reporting SSSM studies found a similar proportion (42%) of women across the total accumulated participant count, with just 4% of studies investigating only women, while 27% involved male-only participant cohorts (Brookshire, 2016). Across SSSM topics, the bias against female participants was most striking in studies investigating strategies to enhance athletic performance; only 3% of all participants were women (Brookshire, 2016). Female-specific SSSM research shows little evidence of mirroring the recent increase in the popularity of women’s sport. Cowley et al. (2021) examined 5,261 studies across six SSSM journals including 12.5 million participants and reported similar findings to Costello et al. (2014) 7 years earlier. Women accounted for 34% of overall participants, with 6% of studies focusing exclusively on women compared with 31% studying men in isolation (Cowley et al., 2021). Interestingly, both Cowley et al. (2021) and Costello et al. (2014) reported that on average 63% of studies included both men and women. Thus, given the substantially greater number of male participants in both audits (61%–66% of total participant pool), this suggests an influence of volunteer (or self-selection) bias on participation rates; whereby, women are perhaps less willing or available to volunteer for certain investigations despite being eligible to participate (Nuzzo, 2021).
Traditionally, the “typical 70 kg man” reference participant has been considered a sufficient proxy for women across SSSM research (Marts & Keitt, 2004; Miller, 2005). Involving female participants necessitates additional methodological considerations, including controlling for sex differences in concentrations of the reproductive hormones, as well as intrafemale fluctuations in estrogen and progesterone due to menstrual cycle (MC) phase, the use of hormonal contraceptives (HC) or impaired menstrual function (Elliott-Sale et al., 2021). Ultimately, this results in more expensive, labor-intensive, and time-consuming study designs, which has traditionally been viewed as an inconvenience (Bruinvels et al., 2017). Moreover, female athletes are more likely to experience nutritional issues, such as iron deficiency or low energy availability (Areta et al., 2021; Coad & Conlon, 2011; Logue et al., 2020), adding to the screening burden or the risk of interference in study outcomes. The availability and recruitment of female athletes can also be challenging due to the tendency for smaller team sizes and the disproportionately low number of professional female athletes with opportunities to participate in research projects (Emmonds et al., 2019). Collectively, these factors have created a tendency to exclude women from SSSM research, with findings in men extrapolated to recommendations for female athletes with minimal consideration of sexual dimorphisms. While understandable in terms of the burden of cost, convenience, and complexity of research, this bias is unacceptable and has likely hampered the opportunities for women to maximize their athletic potential.
The urgency of the need to address the sex disparity across SSSM research is receiving substantial publicity within scientific literature (Elliott-Sale et al., 2021; Hutchins et al., 2021; Martínez-Rosales et al., 2021) and popular/social media (Yu, 2018, 2021). It is therefore important to efficiently direct the current enthusiasm for research on the female athlete into high-quality and meaningful outputs. Already, meta-analyses of selected SSSM topics have been undertaken to determine how female athletes respond to a particular intervention or stressor (Delextrat et al., 2018; Gomez-Bruton et al., 2021; Saunders et al., 2021). However, such technically well-conducted meta-analyses are challenged by substantial limitations in the quantity and quality of the original literature. Moving forward, a comprehensive understanding of the quantity and quality of research pertaining to female athletes is needed before time and resources are invested into further original research or undertaking meta-analyses of topics where women are substantially underrepresented or the female-specific methodological considerations do not align with current recommendations (Elliott-Sale et al., 2021). This can be achieved through “auditing” the literature in specific subdisciplines of SSSM research to determine how well female athletes are currently addressed. In the context of this paper, we define an “audit” as a systematic analysis of the literature in specific subdisciplines of SSSM to help to create a gap analysis of areas in which there is little information/representation of women alongside research areas with the greatest scope for development or impact. Conducting such audits across SSSM topics may produce information to help direct future research activities in a systematic and expedient manner. Examples of such audits are beginning to appear (Brookshire, 2016; Costello et al., 2014; Cowley et al., 2021; Hutchins et al., 2021). However, implementing a more standardized audit protocol will optimize the outputs and interpretation of such audits, ensuring that comparisons can be made across time and across SSSM topics. In particular, it is important to assess key quantitative and qualitative aspects, such as the performance/fitness level of female athletes who are absent or represented in the literature, or the quality of methods used to tackle the challenges of menstrual function. Our proposed audit protocol utilizes new frameworks that have been proposed as best practice to assess menstrual status (Elliott-Sale et al., 2021) and athletic caliber (McKay et al., 2022). Therefore, this paper proposes a standardized protocol to facilitate the efficient and effective conduct of audits of the representation of female athletes in SSSM research, detailing both our proposed methods of conducting the audit (guided by crowd sourced ideas with experts in the field) and a standardized process of reporting the results. Our aim is to encourage a collaborative and harmonized effort that can expedite a systematic correction to the current sex-based biases in SSSM research.
Methods
Proposed Standardized Protocol for Auditing the Representation of Women in SSSM Research
This standardized protocol and spreadsheet tool (provided in the Supplementary Material [available online]) are designed to facilitate a uniform and coordinated audit of the representation of female participants within a topic or subdiscipline of SSSM research (e.g., nutritional pre-event strategies, use of performance supplements, recovery strategies, altitude training). The topic of a specific audit can be defined and justified by the authors. In addition to standardizing the quantification of female participation, the audit methodology assesses key qualitative features of the study design. These include the characterization of the caliber of the athletic populations that have been included in the available research, alongside arguably the most crucial qualitative aspect of research on female athletes: characterization of menstrual status or HC use. Collation of this qualitative information facilitates a comparison of SSSM research conducted on men compared with women, as well as between SSSM subdisciplines and across time. The concept of the proposed auditing process is illustrated in Figure 1.
—Concept flowchart illustrating how our “standardized auditing protocol” will be utilized and the associated benefits.
Citation: International Journal of Sport Nutrition and Exercise Metabolism 32, 2; 10.1123/ijsnem.2021-0257
The spreadsheet tool (Supplementary Material [available online]) is provided to aid researchers in conducting such audits through providing a template to extract metrics A–F. The tool contains three tabs: “flowchart” (a copy of Figure 2; to be populated upon the completion of preliminary study screening as explained below), “study screening” (basic study information is first inputted into the “general information” category; whereby information to complete metrics A–F are then populated in the subsequent sections as denoted by subheadings), and “MC control” (where the information necessary to grade studies on the basis of menstrual status classification and control according to our tiering system is inputted). Example data are also provided in the spreadsheet to guide researchers as to how we propose each tab is completed.
—Flowchart to be used in reporting results. Bold indicates papers, nonbold is used for studies. From Section B onwards, all counts and percentages refer to individual studies (not papers). Symbols should be used after the reported percentage in Section B to denote studies directly comparing different athletic tiers. Men = menstruating; HC = hormonal contraception.
Citation: International Journal of Sport Nutrition and Exercise Metabolism 32, 2; 10.1123/ijsnem.2021-0257
Data Selection/Search Strategy
An electronic literature search is first conducted to identify relevant papers using the proposed search terms: “(athlete OR sport OR healthy) AND (**discipline-specific terms**) AND (exercise OR performance OR endurance OR aerobic OR strength OR power OR anaerobic OR speed OR skill OR tactics) NOT animal NOT rodent).” It is recommended that the search is exclusive to original research papers among human participants, with no date restriction. Following the literature search, papers are then initially screened for separate review papers and to remove duplicates or papers meeting any of the following exclusion criteria: (a) populations with lifestyle diseases, such as obesity or hypertension, (b) outcomes irrelevant to areas of interest (i.e., not related to performance, health, or indirect associations with performance/health), (c) the subdiscipline/area of interest is not investigated as the independent variable or the primary outcome of interest, (d) failure to explicitly state the sex of participants, even if this could be inferred from data (e.g., anthropometric information), or not specifying the male to female participant ratio in mixed sex studies. Inferring participant sex from the use of pronouns within a report may be acceptable but requires explanation in the methodology of the audit outcomes, due to potential differences between gender and sex. Audits may include additional search/exclusion criteria specific to the subdiscipline of SSSM, alongside the population of interest (e.g., age, sport, pregnancy). We advise that papers whose full text cannot be obtained are also excluded. Following the initial search, we suggest that review papers are screened for additional relevant papers that were not detected in the primary search.
The use of specialist software which support systematic reviews, such as Covidence (Covidence, 2021) or Rayyan (Ouzzani et al., 2016) is encouraged to aid the initial screening process. The selected papers are then exported from the chosen software (including title, date, journal, authors, and abstract) to a spreadsheet program, such as Microsoft Excel (template provided in the Supplementary Material [available online]) for more detailed analysis. Single papers that report multiple separate studies should be identified, and each discrete study analyzed individually. Hereafter, the term “paper” refers to the entire single publication; whereas, “study” is used to describe separate investigations within a single paper or across several papers. After preliminary screening, the following metrics are extracted: (a) population, (b) athletic caliber, (c) menstrual status, (d) research theme, (e) study impact, and (f) sample size. A Microsoft Excel template has been provided in the Supplementary Material (available online) to support this process as explained above. The template provides a table to record extracted metrics (“study screening” tab), alongside a flowchart to report the results. The flowchart pictured in Figure 2 provides an illustration of the requisite information that we recommend is extracted in Sections A–F to ensure consistent reporting across audits. Additional discipline-specific sections may be added to the flowchart as appropriate. It is suggested that both the absolute and relative (in percentage) values, where applicable, are reported as denoted on the flowchart (Figure 2). If for any reason, it is not possible to complete the flowchart in full or a section is intentionally excluded, a clear explanation is needed in the study text. However, we recommend that incomplete or omitted sections are avoided if possible.
Population
It is valuable to describe the potential of the study to contribute to knowledge of sex-based differences in SSSM or to directly inform evidence-based guidelines for female athletes. Therefore, we recommend that studies be separated into five population categories (a) males only, (b) females only, (c) mixed-sex cohort, (d) male versus female subanalysis, and (e) male versus female design features. Male versus female design features (e) describes studies that have been purposely designed to investigate differences in the intervention response between sexes and include a statement in the aims as well as clear features in the study design. Meanwhile, male versus female subanalysis (d) describes studies in which sex-based comparisons were completed within the statistical procedures; although, this was not a primary aim of the study or the study was not specifically controlled or designed for best practice comparisons. In addition to quantifying the representation of female participants within the SSSM discipline, this approach will differentiate research that excludes female participants (Category A, males only), simply includes women without methodological consideration of sexual dimorphisms (Category C, mixed-sex cohort), evaluates the response of women to an intervention or topic (Category B, females only), or specifically targets sexual dimorphisms within SSSM disciplines (Category D, male vs. female subanalysis or Category E, male vs. female design features).
Athletic Caliber
A general principle of research is that the results of a study apply to appropriately defined participant populations and scenarios that are similar to those included in its design (Schünemann et al., 2021). Therefore, it is important to provide a clear and standardized description of participants of SSSM research in terms of their caliber of athletic ability and/or level of competition. This information can determine the ecological validity of the application of the findings to high-performance female athletes and their direct inclusion in evidence-based recommendations/guidelines. A recently developed framework for classifying athletic caliber (McKay et al., 2022) allows participants to be ranked on a six-tiered classification system, ranging from sedentary/healthy participants (Tier 0) up to World-Class athletes (Tier 5). Classification is made from information around performance indicators and training status that is easily accessed and commonly reported in most papers. Importantly, classification is determined, wherever possible, according to objective quantitative data (i.e., personal best performances or world rankings) rather than subjective statements, such as “elite” or “trained” (McKay et al., 2022). Studies that provide insufficient information to be robustly classified into a single tier are graded as “unclassified.” For studies in which a range of athletic calibers are included, a classification noting the majority of participants is stated. If such a determination is not possible (i.e., participant numbers in each tier are not reported), we recommend that the mean tier is taken. However, studies comparing distinct athletic calibers through purposeful methodological design should be recognized for their superior study design, and therefore we suggest that each tier is reported individually. For example, a paper comparing Tier 3 versus Tier 4 athletes would contribute to the census of each tier, with a symbol in the flowchart to note that multiple cohorts were counted in a study and; thus, there is a difference between the total number of athlete cohorts and the number of studies. Importantly, this applies only to a priori and not retrospective comparison between athletic calibers. Two authors should independently classify studies, with discrepancies resolved through discussion, adjudication by a third author, or, if possible, contact validation with the author(s) of the original study. Table 1 provides examples of how the participant classification system would be implemented. An additional benefit of conducting audits in this manner is that it may provide the first systematic quantification of the caliber of male participants in SSSM research, as well as allowing the comparison with female athletes.
Examples Across a Range of Studies Demonstrating How the Participant Classification Framework (Mckay et al., 2022) Is Specifically Applied Retrospectively for the Purpose of an Audit
| Study | Description of participant characteristics | Reported participant characteristics | Premise for classification | Classification |
|---|---|---|---|---|
| Influence of caffeine and sodium citrate ingestion on 1,500-m exercise performance in elite wheelchair athletes: A pilot study (Flueck et al., 2014) | 1. Elite wheelchair-racing athletes, including several Paralympic Games, World and European Championship medalists 2. Competed in the category T53/54 and were national team members | None provided | Participants were Paralympic/World medalists | Tier 5 World class |
| The impact of individualizing sodium bicarbonate supplementation strategies on world class rowing performance (Boegman et al., 2020) | 1. A total of 23 elite male rowers. Recruited across two research centers (Canadian Sport Institute Pacific and the New South Wales Institute of Sport, Australia) 2. A total of 13 Olympic/World Champs team members as well as one rowing ergometer world record holder | The 2,000-m ergometer time-trial personal bests ranged from 5 min 39 s (open weight) to 6 min 14 s (lightweight) | The majority (13 out of 23) of participants competed at international (Olympic/World championship level). The 2,000 m ergometer times are between 1% and 5% of world record | Tier 4 Elite/international level |
| Combined creatine and sodium bicarbonate supplementation enhances interval swimming (Mero et al., 2004) | 1. Competitive national-level male and female swimmers 2. All subjects had a minimum of 4 years of experience in competitive swimming training | The 100-m freestyle personal best: 57.9 ± 1.5 s (males), 67.1 ± 1.7 s (females) | Participants compete at the national level, with personal best times within 20%–26% of world leading times in the year of publication | Tier 3 Highly trained/ national level |
| The effects of serial and acute NaHCO3 loading in well-trained cyclists (Driller et al., 2012) | 1. Eight well-trained male cyclists 2. All the cyclists were competing at the state level, with some competing at national level (n = 5) | VO2max = 66.8 ± 8.4 ml·kg−1·min−1 | Participants identify with a specific sport. The majority (five out of eight) of participants compete at the national level, which justifies a Tier 3 classification | Tier 3 Highly trained/national level |
| The effect of caffeine ingestion during evening exercise on subsequent sleep quality in females (Ali et al., 2015) | 1. Participants from a range of team sports (soccer, hockey, and netball), at various competitive levels (recreational to international) 2. Trained two to six times per week | None provided | Participants all identify with a specific sport. Cohort includes individuals from multiple tiers: regional (Tier 2), national (Tier 3), and international (Tier 4). As the number of participants in each tier is not reported the “median” classification is taken | Tier 3 Highly trained/ national level |
| Glycerol hyperhydration improves cycle time trial performance in hot humid conditions (Hitchins et al., 1999) | 1. Trained male cyclists were chosen on the basis of their success in recent, regional cycling race 2. Subjects had at least 3 years of competition experience | VO2max = 65.0 ± 3.9 ml·kg−1·min−1 Wattmax = 376 ± 24 W | Participants meet physical activity guidelines (Bull et al., 2020), identify with a sport and are competing regionally. No further information to justify “sub-elite” terminology | Tier 2 Trained/ developmental level |
| High-velocity intermittent running: Effects of beta-alanine supplementation (Smith-Ryan et al., 2012) | 1. All participants were moderately trained, engaging in 3–7 days/week of aerobic, resistance, or recreational activities | Recreationally active (1–5 hr/week) | Participants meet physical activity guidelines (Bull et al., 2020). No information to suggest participants are competitive or identify with a specific sport to justify a Tier 2 classification | Tier 1 Recreationally active |
| No thermoregulatory or ergogenic effect of dietary nitrate among physically inactive males, exercising above gas exchange threshold in hot and dry conditions (Fowler et al., 2021) | 1. Healthy, none of the participants trained for endurance exercise on a regular basis and were deemed to be physically inactive based on exercising <30 min of moderate exercise per week | VO2max = 41.1 ± 3.6 ml·kg−1·min−1 | Participants do not meet physical activity guidelines (Bull et al., 2020) | Tier 0 Sedentary |
| Factors influencing serum caffeine concentrations following caffeine ingestion (Skinner et al., 2014) | 1. Trained male cyclists/triathletes and active males 2. “Trained” group cycled competitively for >1 season; and consistently trained at high volume and intensity for >6 months 3. “Active” group completed >150 min physical activity per week but not currently nor previously involved in regular, high volume, and/or intensity endurance training | “Trained” group had a VO2max >60 ml·kg−1·min−1 | “Trained” group meet physical activity guidelines (Bull et al., 2020), identify with a sport and are competing which justifies a Tier 2 classification “Active” group also meet physical activity guidelines (Bull et al., 2020), but there is no information to suggest participants are competitive or identify with a specific sport and are therefore classified as Tier 1 | Comparison—Tier 1 vs. Tier 2 (n = 1 study are assigned to both Tiers 1 and 2) |
Menstrual Status
Because the ovarian hormones estrogen and progesterone influence multiple biological systems (Ansdell et al., 2020; Devries, 2016; Green et al., 2016; Laurent et al., 2014; Wohlgemuth et al., 2021), they have the potential to cause downstream effects on the nature and heterogeneity of the findings of various study topics. This is of relevance to high-performance athletes (specifically) whose fluctuations in ovarian hormone concentrations, and any resulting influence on study outcomes, may mask marginal (i.e., <1%–2%) changes in performance that are of significance within real-world elite sporting scenarios. There are myriad of fluctuations in ovarian hormone concentrations across a MC acutely over a given month and chronically over an athlete’s training blocks (MC status over months to years) and over a lifespan (e.g., prepuberty versus menopause) (Elliott-Sale et al., 2021). This highlights the importance of robust characterization of overall menstrual status and the specific phase of the cycle in female participants and/or the alignment of experimental design with specific hormonal milieu. Table 2 provides a detailed guide to our proposed tiered system to grade the standard of methodological control, underpinned by a new framework promoted as best practice for this theme (Elliott-Sale et al., 2021). Because the extent to which ovarian hormones influence particular areas of SSSM and athletic performance is currently unclear, failure to control for hormonal profiles within female participants in a research project introduces uncertainty in the application of the project’s results to the larger female population and sacrifices an opportunity to investigate the extent of sexual dimorphisms.
Tiered Ranking System to Assess Studies With Female Participants; Studies Are Assessed on the Basis of Participant Classification and Methodological Control
| Hormonal contraceptive studies | ||||
|---|---|---|---|---|
| Tier | MC studies | Oral contraceptive pill | Other | Menstrual irregularities studies |
| Gold | Participants are eumenorrheic: 1. have MC lengths ≥21 days and ≤35 days resulting in nine or more consecutive periods per year 2. evidence of LH surge 3. correct hormonal profile (from blood sample analysis) 4. no HC use 3 months prior to recruitment MC characteristics are tracked for ≥2 months prior to testing Outcome measures are repeated in a second cycle | OCP use ≥3 months prior to recruitment (i.e., length of usage), with the type (e.g., mono, bi, or triphasic; combined or progesterone only and formulation [name and concentration of exogenous hormones]) stated Stipulate and consider OCP taking (i.e., active OCP) days and OCP free (i.e., inactive/placebo OCP) days One brand/type of OCP per group of participants | HC use ≥3 months prior to recruitment (i.e., length of usage), with the type (e.g., implants, injections, intrauterine devices/coils that are hormone releasing and NOT copper-based, vaginal rings, contraceptive transdermal patches), and formulation (e.g., combined or progesterone only; names and concentration of exogenous hormones) stated One type of HC per group of participants | Condition diagnosed by medical professional as part of the study Length of condition stated |
| Silver | Participants are naturally menstruating with ovulatory cycles: 1. they experience menstruation, with MC lengths ≥ 21 days and ≤ 35 days 2. confirmed ovulation (LH) but without correct hormonal profile 3. prior HC use not stated or less than 3 months prior to recruitment MC characteristics are tracked for 1 month prior to testing Outcome measures not repeated in a second cycle | Two of three stated: OCP length of usage, type, and formulation Do/do not stipulate and consider OCP-taking (i.e., active OCP) days and OCP-free (i.e., inactive/placebo OCP) days One or more than one brand/type of OCP per group of participants | Two of three stated: HC length of usage, type, and formulation One or more than one type of HC per group of participants | Condition diagnosed by medical professional not as part of the study—self-reported or via medical records Length of condition stated/not stated |
| Bronze | Participants are naturally menstruating: 1. they experience menstruation, with MC lengths ≥21 days and ≤35 days 2. without confirmed ovulation and correct hormonal profile 3. prior HC use not stated or <3 months prior to recruitment No tracking of MC characteristics prior to testing | One of three stated: OCP length of usage, type, and formulation Do not stipulate and consider OCP taking (i.e., active OCP) days and OCP free (i.e., inactive/placebo OCP) days More than one brand/type of OCP per group of participants | One of three stated: HC length of usage, type, formulation More than one type of HC per group of participants | Self-reported condition without medical diagnosis OR not specified if/how the condition was diagnosed Length of condition not stated |
| Ungraded | Insufficient detail to award a gold, silver, or bronze | Insufficient detail to award a gold, silver, or bronze | Insufficient detail to award a gold, silver, or bronze | Insufficient detail to award a gold, silver, or bronze |
Note. HC = hormonal contraception; LH = luteinizing hormone; MC = menstrual cycle; OCP = oral contraceptive pill.
Step 1. Study populations are initially characterized according to four categories of menstrual status: (i) MC (i.e., including eumenorrheic and naturally menstruating women); (ii) HC; (iii) menstrual irregularities; and (iv) mixed menstrual status (i.e., including women from more than one of the aforementioned categories, with each group being distinguishable). If there is insufficient information to provide a robust classification of participants, or a mixed female cohort in which individual menstrual status cannot be discerned, it is reported as an “unclassified” cohort. Studies that state that HC use was not excluded, but do not provide further information as to the number of HC users in the cohort and/or identify results from this group distinguishably, are also graded as “unclassified.” In the case of studies that have excluded HC users, it cannot be assumed that participants were naturally menstruating, unless this is stated and described, and hence these cohorts are also graded as “unclassified.” Explicit information about the MC should describe participants as either eumenorrheic or naturally menstruating based on five criteria; MC length, number of consecutive menses per year, evidence of LH surge, hormonal profiles and absence of HC use (Elliott-Sale et al., 2021). Participants achieving all five criteria are described as eumenorrheic, whereas those fulfilling fewer than five are described as naturally menstruating (Elliott-Sale et al., 2021).
Step 2. Following the classification of participants, a further assessment of the standard of methodological control regarding ovarian hormonal profiles of each category is completed. This assessment places the methodological control into one of four tiers (Gold, Silver, Bronze or Ungraded). It is recommended that mixed participant studies assess each participant classification separately. For example, if a study includes naturally menstruating women alongside those using HC, these are assessed as two separate groups, with 0.5 assigned to each participant classification. Separating the participant classifications in mixed studies allows the overall standard of studies to be identified, while considering the methodological adjustments for ovarian hormone concentration specific to the menstrual status of each female population in the cohort. Studies using “unclassified” participants (as determined in step one) are not assessed for methodological control. Judgment should be used for those cases which are not explicitly covered by the categories (i.e., implicit judgment).
Examples Demonstrating How Different Studies Would Be Assessed Using Our Classification System
| Study | Information provided relating to menstrual status | Female participant classification | Standard of methodological control |
|---|---|---|---|
| The influence of caffeine ingestion on strength and power performance in female team-sport players (Ali et al., 2016) | 1. Monophasic OCP 2. Same hormonal composition (30-μg ethinyl estradiol and 150-μg levonorgestrel) 3. Used HC for >3 months prior to recruitment 4. All testing was performed during Days 5–8 and 18–22 of one pill cycle | OCP | Gold |
| Acute caffeine intake increases performance in the 15-s Wingate test during the menstrual cycle (Lara et al., 2020) | 1. The MC length ranged from 24 to 31 days 2. No information regarding number of consecutive periods per year 3. Urinary measurements of LH 4. Hormonal profile was not retrospectively confirmed by blood sampling 5. No HC use 1 month prior to testing 6. Blood sampling was not used to retrospectively confirm MC phase 7. Cycles were tracked for 4 months prior to testing 8. Outcomes were not repeated in a second cycle | MC Score 2/5 = naturally menstruating | Silver |
| Pre-exercise hyperhydration delays dehydration and improves endurance capacity during 2 hr of cycling in a temperate climate (Goulet et al., 2008) | 1. Self-reported amenorrhea 2. Length of condition not stated | Menstrual irregularities | Bronze |
| Effect of creatine loading on anaerobic performance and skeletal muscle volume in NCAA Division I athletes (Ziegenfuss et al., 2002) | 1. No information on MC length 2. No information regarding number of consecutive periods per year 3. No measurements of LH concentration 4. Hormonal profile was not retrospectively confirmed by blood sampling 5. The HC users were excluded, but no time frame for minimum abstinence from HC prior to testing 6. Blood sampling was not used to retrospectively confirm MC phase 7. Cycles were not tracked at all prior to testing 8. Outcomes were not repeated in a second cycle | MC Score 0/5 = naturally menstruating | Ungraded |
| Effects of 3-day serial sodium bicarbonate loading on performance and physiological parameters during a simulated basketball test in female university players (Delextrat et al., 2018) | 1. No information provided regarding the menstrual status of participants | Unclassified | N/A |
Note. HC = hormonal contraception; LH = luteinizing hormone; MC = menstrual cycle; NCAA = National Collegiate Athletic Association; OCP = oral contraceptive pill.
Research Theme
We recommend that the distribution of research in female athletes across different themes of SSSM is evaluated to assess how this distribution compares to research conducted in men; whereby, male-oriented research provides control/normative values. Even within a given SSSM topic (e.g., caffeine supplementation or altitude exposure), we recommend that studies are classified into three key research themes as follows: (a) performance focus/outcome, (b) clinically established health focus/outcome, or (c) indirect or emerging associations with performance or health, including underlying mechanisms. Table 4 outlines these three categories, with a nonexclusive list of examples to illustrate the definitions, while Table 5 provides an illustration of how themes might be decided for a selected range of studies. It is noted that specific allocation of these themes will likely change according to the topic or subdiscipline of SSSM being assessed and may involve some subjectivity. Direct measures of performance or health have the most relevance or practical application to high-performance athlete populations (Schünemann et al., 2021). Meanwhile, indirect associations with performance or health are one step removed but may provide important insight regarding potential performance/health outcomes if interpreted carefully. With this in mind, in scenarios where the findings of an investigation could be classified into multiple categories, we suggest that the study is assigned according to the following priority scale: performance or health, and then indirect associations with performance or health. Two authors should independently classify studies, with discrepancies resolved through discussion or involving an additional author where necessary.
Definitions and Examples Across the Three Themes (Performance, Health, or Indirect Associations With Performance or Health) of Sports Science and Sports Medicine Research
| Themes | Definition | Examples |
|---|---|---|
| Performance | Studies measuring a performance outcome following an intervention or in association with a topic of interest. Note: Although this might not be the primary outcome, a direct measurement of performance is a high-priority theme of sports and exercise science/medicine. Ideally, the measured performance outcome is validated within the sport of application. | 1. Time trial, time to exhaustion, and one-repetition maximum for whole-body movements that are directly relevant to sporting actions (i.e., squats, deadlifts, bench press, power cleans), repeated sprints, and Wingate tests 2. Competition outcomes, tests of skill or tactics, movement patterns within a competition, sport-specific tests (e.g., serving accuracy, intermittent exercise protocols designed to replicate physiological, tactical, and/or skill demands of team sports) |
| Health | Studies measuring outcomes related to health status or condition. Note: Studies considered to have a health focus should attempt to measure a validated clinical or functional outcome. | 1. Illness (type, prevalence, and recovery) 2. Injury (type, prevalence, success of rehabilitation, or recurrence) 3. Side effects/safety outcomes (functional outcomes) associated with an intervention or practice 4. Disease or medical/clinical condition arising from, or associated with, sports involvement (e.g., asthma, diabetes, disordered eating) 5. Doping outcomes (e.g., where effects on urine or blood markers used to detect anti-doping rule violations are measured) |
| Indirect associations with performance or health | Studies measuring a physiological/ psychological adaptation or response that may subsequently transfer to athletic performance or health, but performance or health was not directly measured. Note: Studies that measure changes in body systems that suggest alterations or associations (correlations), but are not necessarily validated, to health status but preclude a clinical diagnosis or functional manifestation of a health issue may belong in this classification. | 1. Exercise economy, oxygen cost of exercise (e.g., 2. Body composition, resting metabolic rate, and gastrointestinal symptoms/discomfort 3. Muscular adaptations (isometric strength, torque, electromyography action, handgrip strength, and one-repetition maximum for isolated joint movements, where the rest of the body is stationary and has minimal direct transfer to sports performance) 4. Cognitive performance, visual attention/pupil dilation, and neurological pathways 5. Muscle damage, inflammation, oxidative stress, cortisol, mitochondrial respiration/biogenesis, and pharmacokinetics 6. Measures of fluid balance, such as diuresis or fluid retention 7. Changes in markers of organ or system metabolism that suggest perturbations or acute/subclinical alterations rather than chronic or functional changes (e.g., changes in blood biochemistry and markers of immunology, hormonal, and metabolic profiles) |
Examples Demonstrating How Different Studies Would be Categorized Into Our Proposed SSSM Research Themes
| Study | Key relevant information provided | Theme | Explanation |
|---|---|---|---|
| Acute ketogenic diet and ketone ester supplementation impairs race walk performance (Whitfield et al., 2021) | Elite race walkers undertook a four-stage exercise economy test and real-life 10,000 m race before and after a 5-day isoenergetic high- or low-carbohydrate, high-fat diet | Performance | A 10,000-m race is a performance measure |
| B-Alanine supplementation’s improvement of high-intensity game activities in Water Polo (Brisola et al., 2018) | The participants performed a simulated water polo game before and after the supplementation period | Performance | A simulated match scenario is a performance measure |
| Low bone mineral density is two to three times more prevalent in nonathletic premenopausal women than in elite athletes: A comprehensive controlled study (Torstveit & Sundgot-Borgen, 2005) | The study included a questionnaire (Part I), measurement of bone mineral density (Part II), and a clinical interview (Part III). Bone mineral density was measured with dual-energy X-ray absorptiometry. All scanning and analyses were conducted by the same operator | Health | A z score ≤2 indicates a bone density below the expected range for age (Lewiecki et al., 2004), while a z score ≤1 is lower than expected for a weight bearing athlete and warrants further investigation (Nattiv et al., 2007) Low bone mineral density may also be a key indicator of low energy availability in the correct clinical context |
| Effect of sodium bicarbonate on (HCO3–), pH, and gastrointestinal symptoms (Carr et al., 2011) | Physically active subjects undertook eight sodium bicarbonate experimental ingestion protocols and one placebo protocol. Capillary blood was taken every 30 min and analyzed for pH and bicarbonate. Gastrointestinal symptoms were quantified every 30 min via questionnaire | Health | Gastrointestinal symptoms are a functional side effect of sodium bicarbonate ingestion. Although blood pH and bicarbonate concentration (indirect measures) were also measured. The health measure ranks higher in priority on our scale |
| Gastrointestinal function during exercise: Comparison of water, sports drink, and sports drink with caffeine (Van Nieuwenhoven et al., 2000) | Ten well-trained subjects underwent a rest–cycling–rest protocol three times. Measurements taken were esophageal motility, gastroesophageal reflux, intragastric pH, orocecal transit time, intestinal permeability, glucose absorption, and gastric emptying | Indirect associations with performance or health | Multiple measures of gastrointestinal function, but without any measurements of functional gastrointestinal symptoms |
| Neither beetroot juice supplementation nor increased carbohydrate oxidation enhance economy of prolonged exercise in elite race walkers (Burke et al., 2021) | Measured carbohydrate and fat oxidation, plasma nitrate, and nitrate concentrations, alongside oxygen uptake across a 26-km race walking protocol | Indirect associations with performance or health | Measured substrate oxidation and exercise economy, with no measure of performance |
| A short-term ketogenic diet impairs markers of bone health in response to exercise (Heikura et al., 2019) | Measured markers of bone modeling/remodeling after short-term ketogenic low-carbohydrate, high-fat diet Markers included serum markers of bone breakdown (cross-linked C-terminal telopeptide of Type I collagen), formation (procollagen 1 N-terminal propeptide), and metabolism (osteocalcin) | Indirect associations with performance or health | Only measured acute bone markers with no information on bone density over a chronic period |
Study Impact
The trajectory of interest in a particular SSSM research topic can be summarized by plotting the frequency of publications over time. Whether this trajectory is mirrored for studies involving female athletes is also of interest. Consideration of the impact factor (IF) of the journals that publish SSSM outputs involving male and/or female participants may illuminate the actual, or perceived interest in, and value of, the work (Scully & Lodge, 2005; Waltman & Traag, 2020). For example, substantial differences in the IF of the journals that support different types of studies may highlight a sex bias in the perception or reality of study quality or, possibly, publication bias. Therefore, it is of interest to extract the IF of the journal in which the study is published; we recommend that this is the most recent 4-year IF, regardless of publication date, to standardize changes in journal IF over time. Moreover, for the purpose of regulating IF calculation method, we strongly recommend that IF is extracted from the International Scientific Indexing IF, or an index reporting this calculation method, rather than the IF directly reported by the journal. It is noted that when a single paper contains several discrete studies, we suggest that the IF of the journal is counted against each study included in the audit. Such information may be able to highlight disparities to journal editors or provide an incentive for researchers to conduct more challenging study designs if there is an apparent reward in terms of publication in higher IF journals. When interpreting the results of the audit regarding IF, the limitations of this metric, including the propensity for manipulation to a journal’s advantage, should be considered (Kaldas et al., 2020). Although this assessment is not essential to achieve the primary aims regarding directing new/targeted research and is therefore not a compulsory section of an audit, it is strongly recommended.
A more recent system to quantify study impact involves Altmetrics (alternative metrics). Altmetrics involve bibliometrics that measure the attention, dissemination, influence, and impact of scientific papers beyond their traditional academic outputs. The concept was first promoted by Priem et al. (2010) and is now run by the “Altmetric.com” company, which collects data on the online exposure to, and engagement with, scientific publications within news, social media, article pageviews and downloads, article repositories, expert commentaries, and public policy documents (Altmetric, 2016). Although there are limitations to the gathering and interpretation of this metric, the advantages of including it within an audit are that it provides a standardized characteristic of the wider community interest in a scientific output and is accumulated more quickly than scientific citations (Sutton, 2014; Thelwall, 2020). Therefore, there is less disadvantage to including it for recently published papers that have had less time to create traditional academic impact (Sutton, 2014; Thelwall, 2020). It is suggested that Altmetrics and journal IF are considered in combination when interpreting findings, rather than in isolation, to build a more holistic view of study impact. We suggest that the Altmetric Score is collected for all papers published from 2012 (the date of the initiation of the Altmetric company); this can be derived using the company bookmarklet (https://www.altmetric.com/products/free-tools/bookmarklet/). Again, if multiple studies are included in the same paper, we advise that these are counted separately in the audit tally.
Sample Size
A difference in sample size between papers focused on men, women, and/or both sexes, may suggest differences in the ease of recruitment or the requirements for study power (dependent on the research topic). Furthermore, changes in the male to female participant ratio in mixed sex studies over time or across research themes (Metric D) may help to track whether there is greater interest in female study involvement by participants or researchers, or greater awareness of opportunities to increase sample sizes to enable a priori comparisons between sexes. We therefore recommend that sample size is extracted, separating between male and female participants where applicable.
Summary
This audit protocol proposes a standardized approach to conducting literature audits on the representation of female athletes in topics and subdisciplines of SSSM (Figure 1). By identifying strengths and gaps in the quantity and quality of the existing literature, sports practitioners and academics can uncover underdeveloped topics or potential problems with the quality of research in relation to female athletes. The standardization of the process of conducting and reporting the results of such audits will facilitate an understanding of how well female athletes are addressed by currently available research, and how these might differ over time or between subdisciplines of interest. This will help to direct the priority for new research in a systematic way. In addition, these guidelines will contribute to best practices for future studies by highlighting the key elements of appropriate scientific design in female athletes. Of critical importance to the first auditing process is the assessment of how well studies classify participant menstrual status, alongside the implementation of appropriate methodological controls for ovarian hormonal profiles (Elliott-Sale et al., 2021). An additional bonus of conducting the proposed audits is that information on the SSSM literature involving male athletes will also be collected; therefore, capturing aspects that have not been previously described in a systematic process in either men or women (e.g., the caliber of athletes, sample sizes used to investigate various research themes). Although further analysis of the outcomes of the studies included in individual audits, alongside specific aspects relating to the quality of research methodology, is needed. The broader audits that we propose (undertaken using a standardized format) may provide a greater understanding regarding the state of the general SSSM literature, before resources are expended in futile analysis of research areas where females are not well represented, or sex-specific methodological considerations do not align with current recommendations (Elliott-Sale et al., 2021). The completion of audits across a range of SSSM topics may therefore help to identify and support topics that merit a meta-analysis involving further and detailed screening of the research outcomes. Furthermore, audits may identify resources that could be developed to guide new research, areas in which specific investigation of female athletes or comparisons between the responses of male and female athletes should be undertaken as a priority, or the presence and absence of athletes from a specific demographic of athletic caliber/achievement. Ultimately, this working guide provides a crucial step toward accelerating sex parity in SSSM research, with the eventual goal of delivering evidence-based recommendations specific to the female athlete. The potential benefits of greater inclusion of women in SSSM research and more targeted research on female athletes include better athletic performance, safer athlete practices, greater parity in access to resources and opportunities, and an increase in general SSSM knowledge and expertise.
Acknowledgments
The authors acknowledge support of this work by the Wu Tsai Human Performance Alliance and the Joe and Clara Tsai Foundation. L.M. Burke, E.S. Smith, and A.K.A. McKay formulated the concept. E.S. Smith wrote the manuscript with input from A.K.A. McKay, K.E. Ackerman, R. Harris, K.J. Elliott-Sale, T. Stellingwerff, and L.M. Burke. All authors revised and approved the final version of the manuscript.
References
Ali, A., O’Donnell, J., Foskett, A., & Rutherfurd-Markwick, K. (2016). The influence of caffeine ingestion on strength and power performance in female team-sport players. Journal of the International Society of Sports Nutrition, 13(1), 46. https://doi.org/10.1186/s12970-016-0157-4
Ali, A., O’Donnell, J.M., Starck, C., & Rutherfurd-Markwick, K.J. (2015). The effect of caffeine ingestion during evening exercise on subsequent sleep quality in females. International Journal of Sports Medicine, 36(6), 433–439. https://doi.org/10.1055/s-0034-1398580
Altmetric. (2016). What are Altmetrics? https://www.altmetric.com/about-altmetrics/what-are-altmetrics/
Ansdell, P., Thomas, K., Hicks, K.M., Hunter, S.K., Howatson, G., & Goodall, S. (2020). Physiological sex differences affect the integrative response to exercise: Acute and chronic implications. Experimental Physiology, 105(12), 2007–2021.
Areta, J.L., Taylor, H.L., & Koehler, K. (2021). Low energy availability: History, definition and evidence of its endocrine, metabolic and physiological effects in prospective studies in females and males. European Journal of Applied Physiology, 121(1), 1–21. https://doi.org/10.1007/s00421-020-04516-0
Boegman, S., Stellingwerff, T., Shaw, G., Clarke, N., Graham, K., Cross, R., & Siegler, J.C. (2020). The impact of individualizing sodium bicarbonate supplementation strategies on world-class rowing performance. Frontiers in Nutrition, 7, Article 138. https://doi.org/10.3389/fnut.2020.00138
Brisola, G.M.P., de Souza Malta, E., Santiago, P.R.P., Vieira, L.H.P., & Zagatto, A.M. (2018). Beta-alanine supplementation’s improvement of high-intensity game activities in water polo. International Journal of Sports Physiology and Performance, 13(9), 1208–1214. https://doi.org/10.1123/ijspp.2017-0636
Brookshire, B. (2016). Women in sports are often underrepresented in science. Science News, 847–851.
Bruinvels, G., Burden, R., McGregor, A., Ackerman, K., Dooley, M., Richards, T., & Pedlar, C. (2017). Sport, exercise and the menstrual cycle: Where is the research? Sports Medicine, 51(6), 487–488.
Bull, F.C., Al-Ansari, S.S., Biddle, S., Borodulin, K., Buman, M.P., Cardon, G., Carty, C., Chaput, J.-P., Chastin, S., Chou, R., Dempsey, P.C., DiPietro, L., Ekelund, U., Firth, J., Friedenreich, C.M., Garcia, L., Gichu, M., Jago, R., Katzmarzyk, P.T., Lambert, E., . . . Willumsen, J.F. (2020). World Health Organization 2020 guidelines on physical activity and sedentary behaviour. British Journal of Sports Medicine, 54(24), 1451–1462. https://doi.org/10.1136/bjsports-2020-102955
Burke, L.M., Hall, R., Heikura, I.A., Ross, M.L., Tee, N., Kent, G.L., Whitfield, J., Forbes, S.F., Sharma, A.P., Jones, A.M., Peeling, P., Blackwell, J.R., Mujika, I., Mackay, K., Kozior, M., Vallance, B., & McKay, A.K.A. (2021). Neither beetroot juice supplementation nor increased carbohydrate oxidation enhance economy of prolonged exercise in elite race walkers. Nutrients, 13(8), Article 2767. https://doi.org/10.3390/nu13082767
Carr, A.J., Slater, G.J., Gore, C.J., Dawson, B., & Burke, L.M. (2011). Effect of sodium bicarbonate on [HCO3–], pH, and gastrointestinal symptoms. International Journal of Sport Nutrition and Exercise Metabolism, 21(3), 189–194. https://doi.org/10.1123/ijsnem.21.3.189
Claus, L. (2020). Why gender equal pay is so hard to achieve in sport. https://www.mgmt.ucl.ac.uk/news/gender-equality-sports-%E2%80%93-why-gender-equal-pay-so-hard-achieve-sport
Coad, J., & Conlon, C. (2011). Iron deficiency in women: Assessment, causes and consequences. Current Opinion in Clinical Nutrition and Metabolic Care, 14(6), 625–634. https://doi.org/10.1097/MCO.0b013e32834be6fd
Costello, J.T., Bieuzen, F., & Bleakley, C.M. (2014). Where are all the female participants in sports and exercise medicine research? European Journal of Sport Science, 14(8), 847–851. https://doi.org/10.1080/17461391.2014.911354
Covidence. (2021). Covidence systematic review software, Veritas Health Innovation. https://www.covidence.org
Cowley, E.S., Olenick, A.A., McNulty, K.L., & Ross, E.Z. (2021). “Invisible sportswomen”: The sex data gap in sport and exercise science research. Women in Sport and Physical Activity Journal, 29, 146–151. https://doi.org/10.1123/wspaj.2021-0028
Delextrat, A., Mackessy, S., Arceo-Rendon, L., Scanlan, A., Ramsbottom, R., & Calleja-Gonzalez, J. (2018). Effects of three-day serial sodium bicarbonate loading on performance and physiological parameters during a simulated basketball test in female university players. International Journal of Sport Nutrition and Exercise Metabolism, 28(5), 547–552. https://doi.org/10.1123/ijsnem.2017-0353
Devries, M.C. (2016). Sex-based differences in endurance exercise muscle metabolism: Impact on exercise and nutritional strategies to optimize health and performance in women. Experimental Physiology, 101(2), 243–249. https://doi.org/10.1113/EP085369
Douglas, L. (2018). The rise of women’s sports. https://www.nielsen.com/wp-content/uploads/sites/3/2019/04/the-rise-of-womens-sports.pdf
Driller, M.W., Gregory, J.R., Williams, A.D., & Fell, J.W. (2012). The effects of serial and acute NaHCO3 loading in well-trained cyclists. Journal of Strength and Conditioning Research, 26(10), 2791–2797. https://doi.org/10.1519/JSC.0b013e318241e18a
Elliott-Sale, K.J., Minahan, C.L., de Jonge, X.A.J., Ackerman, K.E., Sipilä, S., Constantini, N.W., Lebrun, C.M., & Hackney, A.C. (2021). Methodological considerations for studies in sport and exercise science with women as participants: A working guide for standards of practice for research on women. Sports Medicine, 51(5), 843–861.
Emmonds, S., Heyward, O., & Jones, B. (2019). The challenge of applying and undertaking research in female sport. Sports Medicine–Open, 5(1), 1–4. https://doi.org/10.1186/s40798-019-0224-x
Flueck, J.L., Mettler, S., & Perret, C. (2014). Influence of caffeine and sodium citrate ingestion on 1,500-m exercise performance in elite wheelchair athletes: A pilot study. International Journal of Sport Nutrition and Exercise Metabolism, 24(3), 296–304. https://doi.org/10.1123/ijsnem.2013-0127
Fowler, R., Jeffries, O., Tallent, J., Theis, N., Heffernan, S.M., McNarry, M.A., Kilduff, L., & Waldron, M. (2021). No thermoregulatory or ergogenic effect of dietary nitrate among physically inactive males, exercising above gas exchange threshold in hot and dry conditions. European Journal of Sport Science, 21(3), 370–378. https://doi.org/10.1080/17461391.2020.1739144
Gomez-Bruton, A., Marin-Puyalto, J., Muñiz-Pardos, B., Matute-Llorente, A., Del Coso, J., Gomez-Cabello, A., Vicente-Rodriguez, G., Casajus, J.A., & Lozano-Berges, G. (2021). Does acute caffeine supplementation improve physical performance in female team-sport athletes? Evidence from a systematic review and meta-analysis. Nutrients, 13(10), Article 3663. https://doi.org/10.3390/nu13103663
Goulet, E.D., Rousseau, S.F., Lamboley, C.R., Plante, G.E., & Dionne, I.J. (2008). Pre-exercise hyperhydration delays dehydration and improves endurance capacity during 2 h of cycling in a temperate climate. Journal of Physiological Anthropology, 27(5), 263–271. https://doi.org/10.2114/jpa2.27.263
Green, D.J., Hopkins, N.D., Jones, H., Thijssen, D.H., Eijsvogels, T.M., & Yeap, B.B. (2016). Sex differences in vascular endothelial function and health in humans: Impacts of exercise. Experimental Physiology, 101(2), 230–242. https://doi.org/10.1113/EP085367
Heikura, I.A., Burke, L.M., Hawley, J.A., Ross, M.L., Garvican-Lewis, L., Sharma, A.P., McKay, A.K.A., Leckey, J.J., Welvaert, M., McCall, L., & Ackerman, K.E. (2019). A short-term ketogenic diet impairs markers of bone health in response to exercise. Frontiers in Endocrinology, 10, Article 880. https://doi.org/10.3389/fendo.2019.00880
Hitchins, S., Martin, D.T., Burke, L., Yates, K., Fallon, K., Hahn, A., & Dobson, G.P. (1999). Glycerol hyperhydration improves cycle time trial performance in hot humid conditions. European Journal of Applied Physiology and Occupational Physiology, 80(5), 494–501. https://doi.org/10.1007/s004210050623
Houghton, E.J., Pieper, L., & Smith, M. (2017). Women in the 2016 Olympic and Paralympic Games: An analysis of participation, leadership, and media coverage. Women’s Sports Foundation.
Hutchins, K.P., Borg, D.N., Bach, A.J.E., Bon, J.J., Minett, G.M., & Stewart, I.B. (2021). Female (under) representation in exercise thermoregulation research. Sports Medicine–Open, 7(1), 43. https://doi.org/10.1186/s40798-021-00334-6
International Olympic Committee. (2021). https://olympics.com/en/
ISI Impact Factor. (2017) ISI impact factor. http://isi-impactfactor.com/
Kaldas, M., Michael, S., Hanna, J., & Yousef, G.M. (2020). Journal impact factor: A bumpy ride in an open space. Journal of Investigative Medicine, 68(1), 83–87. https://doi.org/10.1136/jim-2019-001009
Kovalchik, S.A., & Reid, M. (2017). Comparing matchplay characteristics and physical demands of junior and professional tennis athletes in the era of big data. Journal of Sports Science & Medicine, 16(4), 489–497. https://www.ncbi.nlm.nih.gov/pubmed/29238248.
Lara, B., Gutierrez Hellin, J., Ruiz-Moreno, C., Romero-Moraleda, B., & Del Coso, J. (2020). Acute caffeine intake increases performance in the 15-s Wingate test during the menstrual cycle. British Journal of Clinical Pharmacology, 86(4), 745–752. https://doi.org/10.1111/bcp.14175
Laurent, C.M., Vervaecke, L.S., Kutz, M.R., & Green, J.M. (2014). Sex-specific responses to self-paced, high-intensity interval training with variable recovery periods. Journal of Strength and Conditioning Research, 28(4), 920–927. https://doi.org/10.1519/JSC.0b013e3182a1f574
Lewiecki, E.M., Watts, N.B., McClung, M.R., Petak, S.M., Bachrach, L.K., Shepherd, J.A., Downs, R.W., Jr., & International Society for Clinical Densitometry. (2004). Official positions of the international society for clinical densitometry. The Journal of Clinical Endocrinology & Metabolism, 89(8), 3651–3655. https://doi.org/10.1210/jc.2004-0124
Logue, D.M., Madigan, S.M., Melin, A., Delahunt, E., Heinen, M., Donnell, S.M., & Corish, C.A. (2020). Low energy availability in athletes 2020: An updated narrative review of prevalence, risk, within-day energy balance, knowledge, and impact on sports performance. Nutrients, 12(3), Article 835. https://doi.org/10.3390/nu12030835
Martínez-Rosales, E., Hernández-Martínez, A., Sola-Rodríguez, S., Esteban-Cornejo, I., & Soriano-Maldonado, A. (2021). Representation of women in sport sciences research, publications, and editorial leadership positions: Are we moving forward? Journal of Science and Medicine in Sport, 24(11), 1093–1097. https://doi.org/10.1016/j.jsams.2021.04.010
Marts, S.A., & Keitt, S. (2004). Foreword: A historical overview of advocacy for research in sex-based biology. In E. Bittar (Ed.), Advances in molecular and cell biology (Vol. 34, pp. v–xiii): Elsevier.
McKay, A.K., Stellingwerff, T., Smith, E.S., Martin, D.T., Mujika, I., Goosey-Tolfrey, V.L., Sheppard, J., & Burke, L.M. (2022). Defining training and performance calibre: A participant classification framework. International Journal of Sports Physiology and Performance. Advance online publication. https://doi.org/10.1123/ijspp.2021-0451
McNulty, K.L., Elliott-Sale, K.J., Dolan, E., Swinton, P.A., Ansdell, P., Goodall, S., Thomas, K., & Hicks, K.M. (2020). The effects of menstrual cycle phase on exercise performance in eumenorrheic women: A systematic review and meta-analysis. Sports Medicine, 50(10), 1813–1827. https://doi.org/10.1007/s40279-020-01319-3
Mero, A.A., Keskinen, K.L., Malvela, M.T., & Sallinen, J.M. (2004). Combined creatine and sodium bicarbonate supplementation enhances interval swimming. Journal of Strength and Conditioning Research, 18(2), 306–310. https://doi.org/10.1519/R-12912.1
Miller, V.M. (2005). Sex-based physiology prior to political correctness. American Journal of Physiology—Endocrinology and Metabolism, 289(3), E359–360. https://doi.org/10.1152/classicessays.00035.2005
Nattiv, A., Loucks, A.B., Manore, M.M., Sanborn, C.F., Sundgot-Borgen, J., Warren, M.P., & American College of Sports Medicine. (2007). American College of Sports Medicine position stand. The female athlete triad. Medicine & Science in Sports & Exercise, 39(10), 1867–1882. https://doi.org/10.1249/mss.0b013e318149f111
Nuzzo, J. (2021). Volunteer bias and female participation in exercise and sports science research. Quest, 73(1), 82–101. https://doi.org/10.1080/00336297.2021.1875248
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan—A web and mobile app for systematic reviews. Systematic Reviews, 5, 210. https://doi.org/10.1186/s13643-016-0384-4
Oxley, S. (2021). Prize money in sport: What can football learn from cricket? https://www.bbc.co.uk/sport/56240686
Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. http://altmetrics.org/manifesto/
Sanders, D., van Erp, T., & de Koning, J.J. (2019). Intensity and load characteristics of professional road cycling: Differences between men’s and women’s races. International Journal of Sports Physiology and Performance, 14(3), 296–302.
Saunders, B., Oliveira, L.F., Dolan, E., Durkalec-Michalski, K., McNaughton, L., Artioli, G.G., & Swinton, P.A. (2021). Sodium bicarbonate supplementation and the female athlete: A brief commentary with small scale systematic review and meta-analysis. European Journal of Sport Science, 21, 1–10. https://doi.org/10.1080/17461391.2021.1880649
Schünemann, H.J., Vist, G.E., Higgins, J.P., Santesso, N., Deeks, J.J., Glasziou, P., Akl, E.A., Guyatt, G.H., & Cochrane GRADEing Methods Group. (2021). Chapter 15: Interpreting results and drawing conclusions. Cochrane Handbook for Systematic Reviews of Interventions, 6(2), 403–431. https://doi.org/10.1002/9781119536604.ch15
Scully, C., & Lodge, H. (2005). Impact factors and their significance; overrated or misused? British Dental Journal, 198(7), 391–393. https://doi.org/10.1038/sj.bdj.4812185
Skinner, T.L., Jenkins, D.G., Leveritt, M.D., McGorm, A., Bolam, K.A., Coombes, J.S., & Taaffe, D.R. (2014). Factors influencing serum caffeine concentrations following caffeine ingestion. Journal of Science and Medicine in Sport, 17(5), 516–520. https://doi.org/10.1016/j.jsams.2013.07.006
Smith-Ryan, A.E., Fukuda, D.H., Stout, J.R., & Kendall, K.L. (2012). High-velocity intermittent running: Effects of beta-alanine supplementation. Journal of Strength and Conditioning Research, 26(10), 2798–2805. https://doi.org/10.1519/JSC.0b013e318267922b
Sutton, S. (2014). Altmetrics: What good are they to academic libraries. Kansas Library Association College and University Libraries Section Proceedings, 4(2), 1–7. https://doi.org/10.4148/2160-942X.1041
Thelwall, M. (2020). The pros and cons of the use of altmetrics in research assessment. Scholarly Assessment Reports, 2(1), 2.
Tokyo 2020. (2021). Tokyo 2020 sets the record for most athletes and women at a Paralympic games. https://www.paralympic.org/news/tokyo-2020-sets-record-most-athletes-and-women-paralympic-games
Torstveit, M., & Sundgot-Borgen, J. (2005). Low bone mineral density is two to three times more prevalent in non-athletic premenopausal women than in elite athletes: A comprehensive controlled study. British Journal of Sports Medicine, 39(5), 282–287.
Townes, C. (2019). A decade of fighting for equal pay in sports. https://www.forbes.com/sites/ceceliatownes/2020/12/31/a-decade-of-fighting-for-equal-pay/?sh=6557a29a1376
Van Nieuwenhoven, M.A., Brummer, R.M., & Brouns, F. (2000). Gastrointestinal function during exercise: Comparison of water, sports drink, and sports drink with caffeine. Journal of Applied Physiology, 89(3), 1079–1085. https://doi.org/10.1152/jappl.2000.89.3.1079
Waltman, L., & Traag, V.A. (2020). Use of the journal impact factor for assessing individual articles: Statistically flawed or not? F1000Research, 9, 366. https://doi.org/10.12688/f1000research.23418.2
Whitfield, J., Burke, L.M., McKay, A.K.A., Heikura, I.A., Hall, R., Fensham, N., & Sharma, A.P. (2021). Acute ketogenic diet and ketone ester supplementation impairs race walk performance. Medicine & Science in Sports & Exercise, 53(4), 776–784. https://doi.org/10.1249/MSS.0000000000002517
Wohlgemuth, K.J., Arieta, L.R., Brewer, G.J., Hoselton, A.L., Gould, L.M., & Smith-Ryan, A.E. (2021). Sex differences and considerations for female specific nutritional strategies: A narrative review. Journal of the International Society of Sports Nutrition, 18(1), 27. https://doi.org/10.1186/s12970-021-00422-8
Yu, C. (2018). Where are the women in sports science research? https://www.outsideonline.com/health/training-performance/where-are-women-sports-science-research/
Ziegenfuss, T.N., Rogers, M., Lowery, L., Mullins, N., Mendel, R., Antonio, J., & Lemon, P. (2002). Effect of creatine loading on anaerobic performance and skeletal muscle volume in NCAA Division I athletes. Nutrition, 18(5), 397–402. https://doi.org/10.1016/s0899-9007(01)00802-4
