The Integration of Sex and Gender Considerations Into Biomedical Research: Lessons From International Funding Agencies

1In the past 40 years, paradigms in health research have shifted from studying mechanisms and treatments in predominantly male participants to investigating complex systems in heterogeneous populations. Concurrently, research funding agencies have questioned the overreliance on male patients, animals, and cells; the lack of transparency and reproducibility of studies; inattention to sex effects; and inconsistent reporting of sex-specific findings, all of which have compromised the rigor, reproducibility, and generalizability of basic science and clinical studies. In addition, the notions that females are just smaller males and that their hormonal levels interfere with the interpretation of data because of greater variability due to sex have been disproven. When results from males are improperly generalized to females, the missing results for females and inattention to sex and gender lead to errors and false conclusions. Sex differences in physiology, pharmacology, and disease progression form the basis of understanding and developing appropriate treatments for many endocrine disorders and conditions. Studies with irreproducible results, biases, or unforeseen toxicities put people at risk of being harmed, contributing to the erosion of public trust in medicine and science.

2A hallmark of 21st century biomedical research has been the integration of sex- and gender-based analysis (SGBA) into study design. And now with the globalization of science, it is crucial that researchers consider, collect, characterize, and communicate sex and gender influences on health within biomedical research. The current pandemic has showcased the importance of SGBA to health outcomes. Presently, evidence shows that factors related both to sex and gender, along with sex and gender interactions, play into the risk of mortality and morbidity of COVID-19 patients. Results also show the interplay of endocrinological and biological factors, such as sex hormone levels and immune responses, as well as societal factors, on diseases. Factors related to sex, age, and chronic metabolic disease can produce a deadly confluence of dysregulated immunometabolism and inflammation in patients who have contracted COVID-19. The disaggregation of data relevant to those factors is imperative to understand how they influence health outcomes and the underlying causes and effects of disease.

3It is important to note that, although the terms sex and gender are often used interchangeably, “sex” refers to biological attributes, such as chromosomes, gonadal organs, and endogenous hormonal profiles that distinguish organisms as male, female, intersex, and hermaphrodite, whereas “gender” refers to socially and culturally constructed factors that shape behaviors, stereotypes, and attitudes across societies over time. Sex, gender, and their interactions influence health.

4Several government-based research funding agencies developed policies that integrate sex as a biological variable (SABV) and gender as a sociocultural determinant of health into medical research. We review the historical development of research-related policies of countries around the world, with a focus on the United States, Canada, and the European Union (EU) (Table 1).

5At the US National Institutes of Health, the Office of Research on Women’s Health (ORWH) implemented the 2019 to 2023 Trans-NIH Strategic Plan for Women’s Health Research: Advancing Science for the Health of Women to ensure that sex and gender influences are integrated into biomedical research to enhance research relevant to the health of women; that every woman receives evidence-based care tailored to her own needs and circumstances through the appropriate representation of all women in biomedical research; and that women advance in biomedical careers. At the Canadian Institutes of Health Research (CIHR), the Institute of Gender and Health (IGH) plays a leadership role in advancing knowledge and building capacity in sex and gender science across disciplines and career stages and accelerates the uptake of evidence in practice and policy. At the European Commission (EC), Horizon Europe seeks to integrate sex, gender, and intersectional analysis into research and innovation to improve the scientific quality and societal relevance of science and technology. All agencies strive to incorporate sex and gender considerations into each stage of the research process, from study design through data collection and analysis to reporting, resulting in scientific outputs that are more rigorous, reproducible, inclusive, and applicable.

6In 1993, the NIH Revitalization Act mandated the inclusion of women and minorities in clinical research (Table 1). In 2015, the NIH introduced an initiative to enhance reproducibility through rigor and transparency by clarifying expectations for experimental designs and by requiring researchers to account for relevant biological variables (such as sex) and to authenticate key biologic agents.

7In 2016, the US Congress passed the 21st Century Cures Act to accelerate the discovery and delivery of cures. The NIH also issued “Consideration of SABV in NIH-funded Research,” a new policy requiring awardees to account for SABV in research design, analysis, and reporting in vertebrate animal and human studies. It does not require investigators to look specifically for sex differences but, rather, to rigorously account for sex and justify single sex studies, to enhance the knowledge base and inform subsequent investigations. NIH Policy and Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research now require recipients conducting applicable NIH-defined phase 3 clinical trials to ensure that results of valid analyses by sex/gender, race, and/or ethnicity are submitted to ClinicalTrials.gov.

8The government of Canada requires all departments to consider how government policies and programs affect people of different genders. CIHR is a signatory on the Government of Canada’s Health Portfolio’s SGBA policy, which differentiates the biological factors associated with sex from the sociocultural factors associated with gender. SGBA has 2 facets: the first pertains to participation, that is, who gets funding and whether those decisions are made in an equitable way, and the second pertains to research, that is, what or who is studied and will those studies generalize across sexes and genders?

9The EC first attempted to integrate the gender dimension into Framework Programme 6 (2002-2006; see Table 1). The EC’s “gender dimension” is the third objective of its Gender Equality Policy. Whereas CIHR and NIH policies apply only to health and medicine, the EC policy applies across all domains of priority-driven science and technology. In its current program, Horizon Europe (2021-2027), sex and gender analysis applies across its major research clusters: Health; Culture, Creativity, and Inclusive Society; Civil Security for Society; Digital, Industry, and Space; Climate, Energy, and Mobility; and Food, Bioeconomy, Natural Resources, Agriculture, and Environment; and across its major research mission areas: Cancer; Adaptation to Climate Change, including Societal Transformation; Healthy Oceans, Seas, Coastal and Inland Waters; Climate-Neutral and Smart Cities; and Soil Health and Food.

10Several European national funding agencies have implemented policies for SGBA; for a full list of policies, see reference. An Australian group recently called for its country’s policies to align with those of North American and European countries.

11Accounting for SABV begins when developing research questions, and consideration of sex is critical to the interpretation, validation, and generalizability of research findings. The NIH’s SABV policy is incorporated into peer review processes, governance, programming, and oversight. Before the policy was developed, the NIH offered supplements to existing funded studies to investigate sex and gender; the additional funds were for animals/subjects of the opposite sex/gender, to increase the sample size, and/or to perform secondary data analyses. Revised grant application instructions and forms and reviewer guidelines provide information on how and where to consider SABV. Assessment of how well SABV is accounted for in the research strategy contributes to an application’s score. NIH institutes and centers also incorporate topic-related SABV language into funding opportunities and strategic planning processes.

12The NIH has a longstanding commitment to SABV through funding. In 2003, the NIH began offering $1 million grants to establish Specialized Centers of Research (SCOR) on sex differences (see Tables 1 and 2). In 2013, the ORWH began offering supplements to NIH grantees to catalyze new investigations of sex/gender influences across many fields. Through 2019, ORWH invested more than $37 million, supporting 375 principal investigators from every eligible institute and center. In 2014, the NIH Common Fund provided more than $5 million in similar supplements to foundational disease-agnostic platforms.

13The NIH conducted its first SABV workshop in October 2017, featuring NIH investigators who presented their findings garnered from applying SABV perspectives to their research. To facilitate application of the policy in multiple contexts, other institutes and centers conducted SABV workshops, formed area-specific working groups, and published review papers. ORWH convenes and disseminates information to key stakeholders by hosting seminars, workshops, and training sessions and by publishing blog posts, a newsletter, articles, and commentaries. ORWH developed free interactive e-learning courses (see Table 3)—the SABV Primer, on how to consider SABV in all stages of the biomedical research spectrum and Bench to Bedside: Integrating Sex and Gender to Improve Human Health, which helps users understand sex and gender influences on health and diseases. The module on endocrinology presents information on sex and gender differences in 3 conditions—osteoporosis, type 2 diabetes, and differentiated thyroid cancers—and explains the impact of sex differences on the clinical presentation and treatment of endocrine and metabolic disorders. It also details knowledge gaps in endocrine and metabolic research to help develop ideas for future preclinical and clinical research studies. Lastly, in 2019, ORWH issued its first Funding Opportunity Announcement for R01 grants that support disease-agnostic research on the intersection of sex and gender across scientific disciplines.

14During the COVID-19 pandemic, ORWH expanded the scope of its sex and gender research programs to encompass SARS-CoV-2 infection, developed guiding principles on how SABV and inclusion policies could inform COVID-19 studies to improve outcomes for everyone, and developed web content and seminars outlining the influences of sex, gender, and intersectionality on outcomes for diverse women.

15When the SGBA policy was introduced in 2010, applicants for health-research funding were asked to explain how their study design accounts for sex and/or gender (see Table 2). By adding that question, the number and proportion of applicants answering “yes” across all research areas nearly doubled. In the first year, the percentage of applicants who accounted for sex or gender was 20% to 50%, with the lowest proportion in basic science and higher proportions in clinical, health systems and policy research, and population health. Now, the percentage exceeds 75% in basic sciences and 80% to 90% in the clinical sector and population health. Since 2000, the IGH has invested $8 million per year from the CIHR’s priority-driven budget to promote SGBA in research. Incentives include funding opportunities with SGBA as the primary objective, such as catalyst grants for basic scientists who have not yet factored sex into their experiments or who require funds to include female cells or animals. Team grants explore the added value of accounting for gender in human health interventions. SGBA requirements are embedded at various points within the request for applications, as a principle of funding, and in evaluation criteria. IGH funding investments are leveraged by other institutes and external partners. The IGH also holds workshops for new investigators to coach them on integrating sex and gender into their research programs.

16The IGH launched 2 capacity-building mechanisms to augment researchers’ ability to integrate and critically appraise sex and gender in research studies. In 2014, the IGH began requiring the inclusion of a sex and gender champion on the research team, platform, or consortium for select large grant competitions. The champion ensures that sex and/or gender considerations are integrated into every step of the research project, including project rationale, experimental design, methods, analysis, and knowledge translation and dissemination. Champions also act as coaches, set best practices for sex and gender science, and participate as members of a community of practice.

17To increase the capacity of peer reviewers to assess the quality of integration of sex and gender analyses, the CIHR’s SGBA Research Action Plan changed the peer reviewer assessment form for investigator-initiated project competitions to require peer reviewers to rate the integration of sex and gender as a strength, weakness, or not applicable to the application, and to recommend improvements. The plan also requires peer-review committee chairs and scientific officers for the project competitions to complete one of the IGH’s online training modules, and for each committee to undergo an in-person coaching session. A sex-and-gender expert is also assigned to each peer-review committee to ensure the discussions around the appropriate inclusion or omission of sex and gender meet the highest scientific standards. CIHR applicants for specific grant competitions must provide proof of completion of at least one online training module to be eligible to submit an application. The modules effectively increased researcher and peer-reviewer knowledge and skills around sex and gender integration and assessment. In its new strategic plan for 2021 to 2031, the CIHR amplifies the focus on equity in research with a commitment to further refine concepts of research excellence, improve the diversity of research teams, and codify consideration of sex and gender in research.

18During the COVID-19 pandemic, the CIHR implemented new gender policies to support female scientists and to improve the integration of sex and gender in COVID-19–related research. The changes included extending the application time window, reducing administrative burden, doubling parental leave credits, providing supplements for factoring in biological sex, providing specific guidance to applicants and reviewers on how to integrate sex and gender in COVID -19 research, and cracking down on evaluation criteria. Following these changes, the proportion of female applicants increased by 29% to 39%, and the proportion of successful applications led by a female investigator doubled from 22% to 45%. Consideration of biological sex increased from 55% to 94% in funded applications, and applications indicating that sex was considered in the proposed work were more likely to be funded (odds ratio 3.13, 95% CI, 1.57-6.23).

19The EC relaunched its “gender dimension in research” policy with Horizon 2020 (2014-2020; see Table 1). Requirements and guidance for integrating sex and/or gender analysis into research were summarized in a vademecum that describes the integration process for each stage of the research cycle—from programming through implementation, monitoring, and evaluation. It became apparent that the research community required methodological tools for integrating SGBA into basic and applied research. To address these needs, the EC convened an expert group entitled Innovation through Gender that joined the Gendered Innovations project to produce a peer-reviewed website and a report, “Gendered Innovations: How Gender Analysis Contributes to Research”. More than 60 basic scientists, engineers, and gender experts developed methods of sex and gender analysis and contributed to a set of case studies that demonstrate how sex and gender analyses add value to research and innovation in terms of excellence, creativity, societal relevance, and business opportunities. Additional tools and guidance to assist researchers preparing proposals for Horizon 2020 were developed by, among others, the GENDER-NET ERA-NET project.

20In the Horizon 2020 work programmes, topics with an explicit gender dimension were “flagged” as fields where research could benefit from integrating SGBA. These fields included computing, nanotechnology, oceanography, geosciences, organic chemistry, aeronautics, space medicine, biodiversity, ecology, and biophysics, among others. For gender-flagged topics, applicants were required to integrate sex and gender analysis into their research design. In 2018, the proportion of flagged topics, by field of study, was 8% of energy topics, 34% of climate topics, 60% of health topics, and 78% of social science studies.

21To support implementation, Horizon 2020 funded the Advisory Group on Gender, consisting of gender experts from each research domain. This group produced position papers that suggested ways to better integrate the gender dimension into each research domain and societal challenge. The advisory group also addressed the issue of gender bias in proposal evaluations and advised EC staff on how to develop briefings for evaluators on the gender dimension relevant to their particular domain.

22In the EU, an interim evaluation of the gender dimension as a cross-cutting issue revealed that the proportion of gender-flagged topics in the work programmes increased from 16% (99 of 610 topics) in 2014 to 2015, to 19% (108 of 568 topics) in 2016 to 2017, and to 23% (110 of 473 topics) in 2018 to 2019. Among individual projects, 32.4% were identified by project officers as including a gender dimension. Further, a qualitative analysis of a subset of 111 projects from gender-flagged topics showed that 53% included the gender dimension either wholly or in part. Under Horizon 2020, research applications could include training on the gender dimension as an eligible cost. Since none of the 111 projects mentioned here included gender training, more action is needed to make this funding better known. To address that, an expert group developed metrics to grade proposals on how well the gender dimension is incorporated into the whole research process.

23The EC just released its new funding framework: Horizon Europe (2021-2027), the largest EC research and innovation programme to date, with a budget of nearly €95.5 billion. To further the gender dimension, the EC convened an expert group again in 2018 to 2020 and published a report, “Gendered Innovations 2: How Inclusive Analysis Contributes to Research and Innovation”. Although the policy is not finalized, it will likely require all proposals (not simply flagged topics) to include sex and gender analysis or to explain why such analysis is not relevant. This is a major step forward, but more is needed. To create a multiplier effect, the EU established a new eligibility requirement: Applicants who wish to apply for Horizon Europe funding must implement a gender equality plan. This new requirement will be put in place gradually, by first addressing public bodies, research organizations, and institutions of higher education, with a transition period of 1 year before it is fully enforced.

24Much progress has been made in terms of integrating sex and gender considerations into research. Awareness of this issue within the scientific community has increased by requiring the inclusion of both sexes in clinical and preclinical studies. In Table 2, we highlight individual policies, mechanisms, and lessons learned by each agency that could be considered by other entities when trying to promote the integration of sex and gender into research. Our agencies implemented policies and requirements; provided incentives, resources (Table 3), and guidance; performed outreach; and developed training.

25To ensure accountability, it is essential to train evaluators to critically appraise the appropriate integration of sex and gender in proposals. Once applicants realize that high-quality integration is associated both with improved research quality and funding success, as occurred with the CIHR’s new COVID-19 research policies and the NIH’s SABV policy, the practice will be institutionalized. A key lesson that each agency learned was that such a multipronged approach using both incentives and requirements is needed to change minds, attitudes, behaviors, and research strategies across the scientific community. In addition, each country has its unique culture and limits, which led to certain policy adjustments to fit the context of those barriers or opportunities. A universal lesson—among researchers least likely to change—was that money talks. To successfully identify which carrots and/or sticks are most effective and the context in which they work, it will be crucial to ensure uptake of, accountability for, and critical appraisal of policies and standards that integrate sex and gender considerations across the research continuum.

26Globally, the Gendered Innovations project, launched at Stanford University in 2009, works with researchers and funders to develop methods of sex and gender analysis for the natural sciences, medicine, engineering, and environment. Gendered Innovations collaborated with the EC in 2011 to provide the intellectual foundations for the Horizon 2020 policy and again in 2018 to develop new materials for Horizon Europe. Gendered Innovations also recently partnered with the Wellcome Trust to evaluate funding agencies globally on the effectiveness of their policies for sex, gender, and diversity analysis, in efforts to share best practices. In the Netherlands, the Ministry of Health, Wellbeing and Sports funded a Gender & Health programme (2016-2020) with a budget of €12 million aimed at sex and gender analysis in research. In Asia, South Korea founded a Centre for Gendered Innovations in Science and Technology Research and is currently implementing policies for sex and gender analysis across its research funding agencies. The Science Council of Japan and the Japanese Science and Technology Agency recently held meetings on sex and gender analysis in research and innovation. For a full list of funding agency policies on sex and gender analysis, see reference.

27Another lesson learned by these agencies is that they cannot go at this alone. Next steps require funding agencies to collaborate with other stakeholders, including peer-reviewed journals, researchers, universities, biomedical companies, pharmaceutical companies, institutional review boards, philanthropic groups and professional societies to adopt similar standards for considering sex and gender. Editors of peer-reviewed journals should ensure scientific excellence when selecting papers for publication. To this end, the CIHR, NIH, and EC work with organizations, such as the European Association of Science Editors, to encourage journals to adopt standards, such as the Sex and Gender Equity in Research guidelines. It is important that granting agencies and journals send a consistent message that publishing data disaggregated by sex/gender is necessary to ensure scientific excellence. Yet, journal guidelines are not always followed.

28Funding agencies must also continue to support universities’ efforts to integrate these tools into education at undergraduate, graduate, and interprofessional levels. To train the next generation, it is crucial to integrate knowledge of sex and gender analysis into core medical, natural science, and engineering courses. Yet, at present, even most medical schools do not include SABV and SGBA in their curriculum. To encourage the consideration of sex by animal care committees and sex and gender by human research ethics boards, work is ongoing with the Secretariat on Responsible Conduct of Research (in Canada).

29Globally, gross domestic expenditures on research and development (R&D) are driven mainly by business and higher education, with government expenditures on R&D since 2016 starkly below spending based on sector. Currently, few pharmaceutical companies integrate SGBA into their research. In addition, globally there is underinvestment in companies and technologies relating to women’s health and in R&D efforts regarding sex differences, such as sex and gender-aware clinical trials. We suggest that government agencies partner with industry to share lessons learned, collaborate, and promote systemwide solutions for integrating sex and gender considerations into research and clinical studies.

30Last, government agencies need to engage with professional societies, such as the Endocrine Society, to ensure lasting cultural shifts in the scientific community with regard to considering sex and gender. Recently, the Endocrine Society released a scientific statement on sex differences in research that calls for females and males to be studied thoroughly to improve public health. The statement assists clinicians and researchers by providing guidance and recommendations for areas of practice.

31Furthermore, it is necessary to develop new quantitative tools to measure gender. One such tool was recently published. There is also a recognized need for methods to understand the interactions of sex and gender and the intersections of sex and/or gender with other factors that can influence research results, such as ethnicity/race, age, socioeconomic status, geographic location, and sexual orientation, along with the intersectionality of all these variables.

32Herein, we have sounded a call to action by researchers and key stakeholders globally: Government funding agencies, including the NIH, CIHR, and the EC, need your help to actualize the promise of sex and gender integration into biomedical research. Our agencies hope not only that others can learn from the policies and mechanisms that have been implemented but also that together we can promote systemwide solutions that integrate sex and gender considerations throughout the biomedical research enterprise. The ultimate goal is to create rigorous, responsible, and generalizable science worldwide for all.