The art and science of study identification: a comparative analysis of two systematic reviews
Laura Rosen, Ruth Suhami
BMC Medical Research Methodology · 2016
doi:10.1186/s12874-016-0118-2 (opens in a new tab)
We email you when a new notice or a question about this paper appears.
Abstract
Abstract Background Systematic reviews (SRs) form the foundation for guidelines and evidence-based policy in medicine and public health. Although similar systematic reviews may include non-identical sets of studies, and it is recognized that different sets of studies may lead to different conclusions, little work has been published on why SR study cohorts differ. Methods We took advantage of concurrent publication of two SRs on the same topic – prevention of child exposure to tobacco smoke - to understand why study cohorts differed in the two reviews. We identified all studies included in just one review, investigated validity of specified reasons for exclusions, and, using database records, explored reasons for study non-identification. We assessed review methods and discordancy, and attempted to assess whether changes in study cohorts would have changed conclusions. Results Sixty-one studies were included in the two reviews. Thirty-five studies were present in just one review; of these, twenty were identified and excluded by the parallel review. Omissions were due to: review scope (9 studies, 26 %), outcomes of interest not measured (8 studies, 23 %), exclusion of reports with inadequate reporting (6 studies, 17 %), mixed or unclear reasons (3 studies, 8 %), search strategies concerning filters, tagging, and keywords (3 studies, 8 %), search strategies regarding sources (PUBMED not searched) (2 studies, 6 %); discordant interpretation of same eligibility criteria (2 studies, 6 %), and non-identification due to non-specific study topic (2 studies, 6 %). Review conclusions differed, but were likely due to differences in synthesis methods, not differences in study cohorts. Conclusions The process of study identification for SRs is part art and part science. While some differences are due to differences in review scope, outcomes measured, or reporting practices, others are caused by search methods or discrepancies in reviewer interpretations. Different study cohorts may or may not be a cause of differing SR results. Completeness of SR study cohorts could be enhanced by 1 – independent identification of studies by at least two reviewers, as recommended by recent guidelines, 2 - searching PUBMED with free-text keywords in addition to MEDLINE to identify recent studies, and 3 - Using validated search filters.
Abstract licence: CC BY-NC-ND
References77
- 2.
- 40.
38 more references, whose wording the paper’s licence keeps private.
All 77 referencesReads
Read by people on this page
None yet.
Checked or read by AI through naulon
None yet.
A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.
Are you an author?
Is this your paper?
Claim it to add your note and be paid when agents cite it. Free.