Strengthening the Reporting of Observational Studies in Epidemiology for respondent-driven sampling studies: “STROBE-RDS” statement

1Strengthening the Reporting of Observational Studies in Epidemiology for respondent-driven sampling studies (STROBE-RDS) was developed based on STROBE statement following published Guidance for Developers of Health Research Reporting Guidelines. The STROBE-RDS checklist includes modifications to 12 of the 22 items on the STROBE checklist. The two key areas that required modification concerned the selection of participants and the statistical analysis of the sample.Respondent-driven sampling (RDS) is a new data collection methodology used to estimate characteristics of hard-to-reach groups, such as the HIV prevalence in drug users. Many national public health systems and international organizations rely on RDS data. RDS reporting quality and available reporting guidelines are inadequate. STROBE-RDS is a checklist of essential items to present in RDS publications.The STROBE-RDS statement seeks to enhance the transparency and utility of research using RDS. If widely adopted, the STROBE-RDS checklist should improve public health decision making in infectious diseases.

2Strengthening the Reporting of Observational Studies in Epidemiology for respondent-driven sampling studies (STROBE-RDS) was developed based on STROBE statement following published Guidance for Developers of Health Research Reporting Guidelines. The STROBE-RDS checklist includes modifications to 12 of the 22 items on the STROBE checklist. The two key areas that required modification concerned the selection of participants and the statistical analysis of the sample.Respondent-driven sampling (RDS) is a new data collection methodology used to estimate characteristics of hard-to-reach groups, such as the HIV prevalence in drug users. Many national public health systems and international organizations rely on RDS data. RDS reporting quality and available reporting guidelines are inadequate. STROBE-RDS is a checklist of essential items to present in RDS publications.The STROBE-RDS statement seeks to enhance the transparency and utility of research using RDS. If widely adopted, the STROBE-RDS checklist should improve public health decision making in infectious diseases.

3Strengthening the Reporting of Observational Studies in Epidemiology for respondent-driven sampling studies (STROBE-RDS) was developed based on STROBE statement following published Guidance for Developers of Health Research Reporting Guidelines. The STROBE-RDS checklist includes modifications to 12 of the 22 items on the STROBE checklist. The two key areas that required modification concerned the selection of participants and the statistical analysis of the sample.

4Respondent-driven sampling (RDS) is a new data collection methodology used to estimate characteristics of hard-to-reach groups, such as the HIV prevalence in drug users. Many national public health systems and international organizations rely on RDS data. RDS reporting quality and available reporting guidelines are inadequate. STROBE-RDS is a checklist of essential items to present in RDS publications.

5The STROBE-RDS statement seeks to enhance the transparency and utility of research using RDS. If widely adopted, the STROBE-RDS checklist should improve public health decision making in infectious diseases.

6Hidden or hard-to-reach population subgroups are often key to the maintenance of infectious diseases in human populations. However, it is often difficult to investigate the factors that drive transmission in these groups by using commonly used epidemiologic methods of data collection because of the lack of an adequate sampling frame. Researchers have therefore typically resorted to various types of convenience sampling to gather data on hidden populations. Although convenience sampling has its advantages, this approach is unable to generate unbiased population-based estimates of infection prevalence and risk factors. In an attempt to address these limitations, respondent-driven sampling (RDS), a variant of a link-tracing design, was proposed in 1997.

7RDS studies are characterized by both a specific data collection method and specific statistical analysis methods. Key features of the data collection include: (1) a small proportion of the sample is recruited by the researcher (i.e., the “seeds”) and a large proportion of the sample is recruited (in recruitment “waves”) by other members of the target population to whom they have a social relationship; (2) recording of recruitment connections between respondents (e.g., who recruited whom); (3) the maximum number of people that each participant can recruit is determined by the researcher by giving out a limited number of recruitment “coupons”; (4) respondents are compensated for participating in the study and recruiting others into the study. Collectively, these features often make RDS an efficient data collection method. Although often efficient, the RDS data collection method produces multiple challenges for the analysis. First, because most of the sampling is conducted by respondents, assumptions about the sampling process are needed. Second, under a variety of assumptions, not all members of the target population will have the same probability of selection, so this probability is typically estimated using a combination of modeling assumptions and study data. Finally, because sampling happens through pre-existing relationships, the observations are not independent. These challenges make point and variance estimation from RDS data more complex than from other forms of sampling. Numerous approaches have been developed, and more are currently under development,,,,,,.

8Since its introduction in 1997, there has been a rapid increase in the number of surveys of hidden or hard-to-reach populations using the RDS methodology, primarily of individuals at risk of sexually or parenterally transmitted infections including HIV,, but also on topics as diverse interpersonal violence and strategies for improving cancer screening recruitment. Many countries including the United States, Ukraine, Vietnam, Mauritius, Morocco, and Brazil use RDS as part of their national public health systems, and data from RDS studies are used by major public health organizations including the US Centers for Disease Control and Prevention, the Joint United Nations Programme on HIV/AIDSs, and the Global Fund to Fight AIDS, Tuberculosis and Malaria. The National Institutes of Health alone has awarded around $100 million in funding to projects using RDS and its variants.

9Making sense of the rapidly increasing amount of data collected using RDS, is crucial to the integration of this information into the practice of medicine and public health. However, the assessment of the strengths and weaknesses of RDS data and methods has been limited by the inadequate reporting of RDS studies. An assessment of 22 randomly selected RDS studies has recently been carried out. The assessment found that overall only around one-third of items sought were reported. Key details of the sampling and statistical methods were particularly poorly reported, including the methods of seed selection (reported in 45% of studies), the number of recruits from each seed and number of recruits in each recruitment wave (33%), the details of the recruitment venues (33%), eligibility criteria for seeds (<20%), wording of network size questions (<20%), if seeds were included in the analysis (<20%), how participants were trained to recruit others (0%), and an explanation for differences between unadjusted and adjusted estimates (0%).

10To improve the quality of reporting of observational epidemiologic studies, the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) statement,,, and extensions, were developed. However, the STROBE statement is inadequate for reporting RDS studies because of the major differences in the RDS sampling and estimation procedures. Our aims are to present a systematic review of the number of RDS studies and present the “STROBE-RDS” statement, a checklist of essential items to present in RDS publications, justified and supported by a stand-alone explanation and elaboration document.

11A systematic literature review was carried out purely to assess the number of published RDS studies and summarize their geographical distribution. Briefly, we searched the MEDLINE (1970–2013), EMBASE (1974–2013), and Global Health (1910–2013) databases and asked experts for their collections of relevant articles. Studies conducted in any country, in any language, among any study population were included; reviews, editorials, commentaries, and methodological articles were excluded. A previous assessment had identified the inadequacy of previous RDS reporting, and therefore, further details on RDS reporting were not collected. Full details are shown in the Supplementary Material (Document 1 at www.jclinepi.com).

12The STROBE-RDS statement was developed after the Guidance on Health Research Guidelines of Moher et al. The initial need for the RDS reporting guidelines was identified in RDS expert and stakeholder discussions at an RDS symposium in 2011. This was followed by a systematic evaluation of the reporting of RDS studies that concluded that the reporting of RDS studies was inadequate. Existing guidelines were then reviewed, and the most suitable was the STROBE statement,, but this was assessed to be inadequate for reporting RDS studies because of the major differences in the RDS sampling and estimation procedures. The vast majority of existing RDS studies use a cross-sectional study design. Therefore, this statement is an extension of the STROBE guidelines, the STROBE explanation and elaboration document, and the STROBE checklist for cross-sectional studies. Version 1 of the STROBE-RDS checklist was distributed for consultation by posting on the Equator Network Web site and the RDS listserv and by sending to known experts. Themes emerging from the feedback included: strong support for the initiative and a request to restrict the scope of the guidelines to cross-sectional epidemiologic studies that seek to generate representative estimates for the target population. The checklist was revised based on this feedback, and version 2 was published in early October 2012. Version 2 of the checklist was piloted during October 2012 by using it to guide manuscript drafting and was sent to the STROBE group for feedback. Researchers (n = 5) piloting the checklist provided much useful feedback and requested a stand-alone checklist and supporting document. This and other feedback was used to develop version 3 of the checklist, which was discussed at a 2-day face-to-face meeting in New Orleans in October 2012. A list of potential meeting invitees was drawn up by a subset of coauthors (R.G.W., A.J.H., and W.H.) after consultation with STROBE initiative members, statisticians, epidemiologists, and empirical RDS researchers. Potential invitees were categorized into three groups: statisticians/survey methodologists, epidemiologists/empirical RDS researchers, and journal editors. Fifty percent (2 of 4), 46% (6 of 13), and 0% (0 of 2), respectively, of meeting invitees in these three groups participated (11 meeting attendees in total). Participants were sent the draft checklist, a summary of the previous RDS reporting and the guidelines by Moher et al.. At the meeting, each draft checklist item was presented and discussed in turn and edited on screen in real time until agreement on the final version of the checklist was reached by consensus and is presented in this manuscript. Consensus was defined as asking verbally all participants if they agreed to the written text for each item. There were no items on which consensus was not reached during the meeting. After the face-to-face meeting, this summary manuscript was drafted by the authors, and to accompany the checklist, an “Explanation and Elaboration” document was developed and revised based on feedback from experts and the STROBE group and is presented in the Supplementary Material (Document 2 at www.jclinepi.com).

13The scope of the “STROBE-RDS” statement is limited to (1) epidemiologic studies (the scope of the original STROBE guidelines), (2) cross-sectional studies (the most common RDS study design to date), and (3) RDS studies that seek to generate representative estimates for the target population (currently the most contentious and potentially most policy-relevant use of RDS). Furthermore, as RDS is both a sampling and a data analysis method, guidelines for reporting on both aspects of RDS are provided. Finally, in response to feedback from researchers piloting the STROBE-RDS checklist, we aim to provide a self-contained statement that minimizes the need to refer to other documents when reporting on an RDS study.

14The systematic literature review and input from experts identified that globally over 460 peer-reviewed publications have reported using RDS since the mid-1990s, with most published since 2006 (Fig. 1A). Fig. 1B shows the global distribution of study locations. RDS studies have been conducted in 69 countries, including the United States of America (151 studies), China (70), India (32), Mexico (22), and South Africa (16). The articles came from 141 different journals. Most journals (91) had published either one or two articles included in the review. Nine journals had published 10 or more articles (or conference abstracts). Supplementary Material (Document 1 at www.jclinepi.com) provides more details of review methods, results, and included and excluded articles.

15In Table 1, we present the proposed STROBE-RDS statement checklist, a modification of the STROBE statement checklist for cross-sectional studies. The left column lists the original STROBE checklist for cross-sectional studies. The right column summarizes the reporting recommendation. A three-column version of the checklist, highlighting the changes from the original STROBE checklist, is shown in the supplementary information, along with an explanation and elaboration document [Supplementary Material (Document 2 at www.jclinepi.com)]. As there is considerable variation in the use of terms and definitions across the disciplines in which RDS is used, we also present a list of suggested RDS terms and definitions to be used when reporting RDS studies (Box 1).

16The STROBE-RDS checklist provides modifications to 12 of the 22 items on the STROBE checklist. The two key areas requiring modification concerned the selection of participants and the statistical analysis of the sample. These modifications are summarized below.

17As members of the target population, not researchers, recruit most study participants in RDS studies, details of the formative research conducted before the study (T5b) and of how participants were trained to recruit others (6a) should be reported. Key details of the recruitment process should also be reported, including the number of coupons issued per person, any time limits for referral (6a), procedures of seed selection (6b), the exact wording of personal network size question(s) (6d), the incentives for participation and recruitment (6e), and how the recruiter–recruit relationship was tracked (7b). Variation in study procedures during data collection should also be reported (6c). Methods to assess eligibility and reduce repeat enrollment should be described (8b) so that other researchers can understand the data collection process and any biases it might introduce. Authors should report reasons for nonparticipation at each stage (e.g., not eligible, does not consent, decline to recruit others) (13b), the number of coupons issued and returned (13d), the number of recruits by seed and number of recruitment waves for each seed (13e), any recruitment challenges (e.g., commercial exchange of coupons, imposters, duplicate recruits), and how they were addressed (13f) (Table 1, items 5b, 6a,b,c,d,e, 7b, 8b, 13b,d,e,f).

18Several different estimators exist for estimating the prevalence of a specific trait (e.g., HIV prevalence) from RDS data,,,,,,. There are also a number of different methods for producing confidence intervals around these estimates,,,. Evaluations of these methods have been equivocal,,, and the best estimator may depend on specific features of a study. At this time, there is no consensus that one estimator should be universally used. As such, we recommend authors clearly describe the statistical methods used, including those to adjust for sample design, both when making estimates (Table 1, 12a,b) and when quantifying the uncertainty in those estimates (16a,c). As the utility of the various RDS estimators is unknown, we recommend reporting unadjusted and study design–adjusted estimates and, if applicable, confounder-adjusted estimates and their precision (16a). If adjustment of the primary outcome leads to marked changes, information on factors causing the changes (e.g., personal network sizes, recruitment patterns by group, key confounders) should be reported (16c) (Table 1, items 12a,b, 16a,c).

19The STROBE-RDS statement is a checklist of essential items that should be reported in RDS studies. The statement has several strengths. It is based on existing guidance for reporting observational studies,,, was developed by an interdisciplinary group that included epidemiologists, statisticians, and empirical RDS researchers, and explicitly justifies changes from the original STROBE statement.

20The decisions made for the conduct and data analysis of RDS studies will influence the representativeness of a study's results. The empirical evidence on how representative the study results are is limited, and improving the methodology is an active research area. Transparent reporting is essential for developing a better evidence base and improving RDS methods.

21Based on feedback we received from researchers writing up RDS studies during the piloting of earlier versions of this checklist, we decided to provide a stand-alone STROBE-RDS checklist with full supporting documentation, rather than only providing a modified STROBE checklist. This should mean that researchers writing up RDS studies will be more likely to use these reporting guidelines. We encourage readers to read the accompanying explanation and elaboration document in full (supporting material) before embarking on writing up an RDS study.

22The statement can be used by authors, peer reviewers, and editors to improve the reporting of RDS studies. We invite journals to endorse STROBE-RDS, and although STROBE-RDS will be published in English only once, we invite others to translate STROBE-RDS and to submit commentaries, editorials, or use other means to raise the awareness of the STROBE-RDS publication. The ability to provide information in Web supplements should alleviate concerns about the increased length of manuscripts resulting from following the guidelines. We welcome comments directed to the corresponding author or via the journal or Equator Network Web sites where the guidelines are also deposited. These will be used to update this STROBE-RDS statement as RDS methods develop.

23The STROBE-RDS statement does not prescribe or dictate how an RDS study should be designed or analyzed. Rather, it seeks to enhance the transparency of research using RDS to increase the understanding of individual studies and enable comparisons between studies. If widely adopted, the STROBE-RDS checklist should improve global public health decision making for infectious diseases. Further studies could assess the impact of STROBE-RDS on the transparency of RDS research and on global public health decision making.

24We searched published (physically published or online), peer-reviewed literature accessible through July 2013 that reported using RDS. Studies from all countries were included. We conducted searches using MEDLINE (1970–2013), EMBASE (1974–2013), and Global Health (1910–2013). Search terms used included “respondent driven” or “respondent-driven” or “RDS.”