A systematic review of adaptations of evidence-based public health interventions globally

1Emphasis on evidence-based practice in medicine, public health, and the social services has led to a prominence of the application of practice guidelines and evidence-based interventions or EBIs. When situating an EBI in a new context, public health professionals, or health practitioners who work in community settings, sometimes adapt the EBI during the process of replication. However, in planning and implementing these interventions, there may be mismatches between the original EBI and the characteristics of the population of interest, implementing agency, and/or community. In addition, agencies may lack the resources, funding, or expertise to deliver the EBI as it was originally intended. Consequently, public health professionals often make both intended and unplanned program adaptations to the EBI to better fit the new audience or context.

2The concept of program adaptation was originally introduced by Rogers when he defined adaptation as the degree to which an innovation is modified in the process of its adoption and implementation. Other definitions have evolved in the era of translation of EBIs and the emergence of adaptation frameworks (Table 1). These definitions share similar characteristics, including modifying a program to meet the needs of the target population, local circumstances, or new contexts. Some definitions focus on the need to retain the core components or logic of the program. The adaptations could be deletions, additions, or modifications. Some posit that adaptations should be systematic or planned to involve stakeholder input and to have a more rigorous process in program planning, while the CSAP (Center for Substance Abuse Prevention) framework notes that adaptations could be accidental modifications. Moore and colleagues proposed the schema of timing of adaptation of proactive (planned) vs. reactive. Furthermore, three definitions included modifications related to matching the culture for the new population, “cultural adaptation”. A few definitions specify elements that could be changed such as program components, content, provider, and delivery. Of these definitions, CSAP’s Guidelines for Adaptation, Map of Adaptation Process, ADAPT-ITT, and Research-based Program Adaptation are cited most frequently in the published literature. In summary, although many adaptation definitions share similar characteristics, the most frequently cited ones do not emphasize the same concepts. Thus, it is important to discern how professionals in the field describe their adaptations, why they make modifications and the types of changes that they make, and how they use frameworks to conduct adaptations.

3Previous reviews have found that modifications to original EBIs often occur spontaneously when they are adopted into other practice settings. Common reasons for adaptations include responding to participants’ attributes, needs or culture, constraints such as limited time or resources, issues related to participant recruitment or retention, and accommodating practice or setting circumstances/context.

4Increased development of models and frameworks to guide the adaptation of EBIs began with national EBI dissemination efforts related to disease prevention areas in substance abuse and HIV/AIDS or cultural adaptations to existing programs; these frameworks provide approaches to facilitate adaptation. Escoffery and associates recently conducted a scoping study that found 13 adaptation frameworks. They reported 11 common steps including assess the community, understand the intervention, select intervention, consult with experts, consult with stakeholders, decide what needs adaptation, adapt the original program, train staff, test the adapted materials, implement, and evaluate. These frameworks enhance the translation of evidence-based practices. As Wandersman’s Interactive Systems Framework suggests, supports are necessary to guide the public health system or agencies to adopt and implement new public health interventions. These frameworks assist public health professionals as capacity building tools for a structured adaptation process.

5Limited research has explored how adaptation occurs in practice. Little is understood about who is involved in adaptation processes, what common types of changes are made to the original program, and what mechanisms are used. This review advances the concept of adaptation and elucidates common adaptation processes in real-world community settings as reported in the published literature. Community settings are defined as various organizations or places in communities such as schools, faith-based organizations, social services or public health agencies, households, or worksites. The research questions for the review were as follows:What are the reasons for and common types of adaptations being made public health EBIs in community settings as reported in the published literature?What steps are described to making adaptations to EBIs?What outcomes are assessed in evaluations of adapted EBIs?

6What are the reasons for and common types of adaptations being made public health EBIs in community settings as reported in the published literature?

7What steps are described to making adaptations to EBIs?

8What outcomes are assessed in evaluations of adapted EBIs?

9We followed procedures for systematic reviews based on the Cochrane Handbook of Systematic Reviews of Public Health Interventions and the reporting guideline, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).

10We searched Ovid PubMed, PsycINFO, PsycNET, and CINAHL with the assistance of an experienced health sciences librarian. The date of the last search was December 2015. Concepts for the search included adaptation, evidence-based interventions and practice, health behavior, and quality of healthcare. Combinations of the associated MeSH terms were used to develop the initial PubMed search and then adapted to search other databases. The search strategies can be found in Additional file 1. We also manually cross-referenced reference lists of included studies. We downloaded relevant citations into a reference manager software program, EndNote, which facilitated removing duplicate citations identified in the multiple databases. We exported the resulting composite library into an Excel file for documentation of the title and abstract review process.

11The project team created an Eligibility Assessment Checklist restricting included articles to those reporting primary studies published in English after 1995 and that examined the adaptation process or outcomes of an adapted evidence-based intervention (public health program or policy). Programs are defined as a combination of strategies designed to create behavior change or improve health status and policies are rules, regulations, or actions related to a health goal or service. These adaptations reported could be proactive (purposeful) or reactive. Articles were excluded if they did not describe the adaptation methods or if the full-text article was unable to be located after an exhaustive search. We combined articles reporting different aspects of the same EBI, e.g., the evaluation findings and the adaptation process.

12Two trained reviewers (CE, HU) independently screened titles and abstracts after duplicates were removed, using the Eligibility Assessment Checklist. We selected potentially relevant abstracts for a full-text review conducted independently by the two trained reviewers. The first author resolved any disagreement between the reviewers.

13We assessed study quality of the articles based on their use of a theory or adaptation framework, and in the case of those that included an evaluation, we assessed the rigor of the design. We used these variables descriptively, however, and did not differentiate studies based on these variables.

14We reviewed the articles of EBI adaptations for six categories of variables: (1) characteristics of the original and adapted EBI (name, disease/topic, population and setting), (2) reason(s) for adaptation, (3) type(s) of modifications, (4) steps (tasks) in adaptation described by the authors, (5) reference to an adaptation framework, and (6) measures of implementation and intervention outcomes (see definitions in Additional file 2). In addition, we described how they presented the EBI adaptations made in the articles. We used a structured data abstraction form designed in Excel 2016 to record the extracted information. We used Stirman and associates’ typology of modifications, the adaptation steps identified in the scoping study of adaptation frameworks, and implementation outcomes defined by Proctor and colleagues. Context modifications were defined as changes to format, location, or personnel delivering the intervention, while content modifications were changes to the intervention materials, procedures, or delivery. In coding adaptation steps, we combined consulting with stakeholders and experts and had an “other” option, resulting in nine named steps. For each study, we examined bias in the study through documentation of participants (e.g., selection, generalizability), study design, and inadequate results reporting. Two trained reviewers (CE, HE, RW, ME, PDM, EL) independently coded the included articles. Discrepancies were discussed and adjudicated by the larger team.

15We presented summaries of study-specific adaptation reasons, steps, types of adaptations, and outcome measures with descriptive statistics across studies. We described the original and adapted EBI, the study population, reasons for adaptations, the name of adaptation frameworks, and examples of adaptations qualitatively.

16We found 543 unique citations that yielded 45 articles reporting 42 distinct program adaptations after the two levels of screening (Fig. 1). Main reasons for exclusion were a lack of description of the adaptation process or methods, not being a public health program or policy, and not being a primary study (e.g., protocol, review).

17Publication years of the primary citations are from 2003 to 2014, and common disease topics included HIV/AIDS, mental health, substance abuse, and chronic illnesses (Table 2). Many of these studies had non-experimental designs (k = 27, 64.3%), and the remainder had experimental (k = 12, 28.6%) or quasi-experimental (k = 3, 7.1%) designs. Thirty-six adaptations included an evaluation. Most (k = 26, 61.9%) reported on adaptations that took place in the USA, and one EBI was adapted in three locations (USA, Africa, and Asia). Other EBIs were adapted in Africa (k = 4), Asia (k = 5), Europe (k = 3), Canada (k = 1), the Caribbean (k = 1), and Australia (k = 1).

18The most common reasons for adaptation included the need for a culturally appropriate program (k = 27; 64.3%), a new target population (k = 25; 59.5%), and a new community setting (k = 24; 57.1%) (Table 2). Less common reasons for adaptation were the desire to improve ease and feasibility of implementation (k = 6; 14.3%), attempting to make the program more widely accessible (k = 1; 2.4%), and trying to condense the original intervention (k = 1; 2.4%).

19Authors reported making an average of 3.4 (SD = 0.90, range 2–5) different types of adaptations with a mode of 3 (Table 3). All 42 (100%) reported some modification of the EBI content. The form this took usually included tailoring (k = 39; 92.9%) or adding elements (k = 30; 71.4%). For example, Cornelius and associates modified HIV prevention videos originally tested with young pregnant women to be relevant to older African American women. In the adaptation reported by Steiker, the study team added four new videos to accompany the curriculum and rewrote scenarios used in the workbooks to incorporate local culture. In the EBI adapted by Gitlin and associates, a moment of silence was added at the beginning of each session to recognize spiritual practices and their importance to participants [content modification-adding elements]. More than half of the authors reported shortening the original EBI as one of the adaptations made. For the 42 programs, some teams described adapting the intervention content by shortening it (k = 13; 31.0%), removing elements (k = 12; 28.6%), loosening the structure (k = 10; 23.8%), lengthening the program (k = 9; 21.4%), substituting modules or activities (k = 7; 16.7%), or integrating other approaches to the intervention (k = 5; 11.9%).

20Nearly all of the adaptations (k = 40; 95.2%) described modifying context, and 31 (73.8%) included cultural modifications. Most context modifications included making changes to the original EBI by adapting it to fit with the new intervention population (k = 33; 78.6%) and setting (k = 29; 69.0%). Mueller and colleagues, for example, adapted a curriculum originally delivered in community agencies and after-school programs to a school setting. Over half of the adaptations included changes to the delivery of the original intervention (k = 26; 61.9%), either by modifying the role of the personnel delivering the intervention (k = 16; 38.1%) or by adapting the format or channel of delivery (k = 4; 33.3%). Masters-level research interventionists, for instance, delivered the family-based behavioral pediatric obesity treatment rather than medical staff in the intervention reported by Riggs and colleagues. In the EBI adapted by Danielson and team, a web-based delivery platform was used instead of small group sessions. Fewer authors reported modifying procedures for training personnel (k = 16; 38.1%) or for evaluating the program (k = 19; 45.2%). Four (9.5%) studies described changing what they regarded as core elements of the original EBI.

21The most common combinations were content, context, and delivery (k = 9), and content and context (k = 8) (Fig. 2). Content and context were part of all other combinations; four other combinations only had one study each.

22For content adaptations, the classifications reported varied greatly. However, some patterns emerged with certain content combinations, including tailoring, adding elements, and cultural modifications (k = 5); tailoring, adding elements, loosening structure, and cultural modifications (k = 4); tailoring, adding elements, lengthening, and cultural modifications (k = 3); and tailoring and cultural modifications.

23Each of the nine steps derived from the scoping review of evaluation frameworks is represented in most of the adaptations (combining consulting with experts and stakeholders), with fewer reporting selecting the EBI (k = 23; 54.8%) and pilot testing (k = 24; 57.1%) (Table 3); 37 (88.1%) conducting a community assessment; 37 (88.1%) preparing new materials; 35 (83.3%) implementing the adapted intervention; 32 (76.2%) evaluating the adapted intervention; 31 (73.8%) determining needed changes based on action step assessments; 31 (73.8%) training staff members; and 30 (71.4%) consulting stakeholders or experts before adapting the materials (Fig. 3). Overall, the average number of steps was 6.7 (range 3–9, mode = 7). Of the 37 authors who reported conducting community assessments, 21 (56.8%) held focus groups with community members, 12 (32.4%) conducted interviews with key informants and stakeholders, five (13.5%) formed and consulted with community advisory boards or steering committees, and two (5.4%) administered a survey to get community feedback. Ten of these (27.0%) used a combination of methods to collect community input and assess need.

24The authors of less than half of the reports named a pre-existing adaptation framework as guiding the adaptation process (k = 15; 35.7%) (Table 2). Most frameworks were mentioned once; the Ecological Validity Model, Map of the Adaptation Process, and Cultural Adaptation Framework were referenced twice.

25Of the 36 reports that included an evaluation, most authors reported measuring program acceptability (k = 28; 66.7%), fidelity (k = 22; 52.4%), and feasibility (k = 22; 52.4%) (Table 3). With respect to implementation outcomes, most authors reported evaluating program acceptability (k = 28; 66.7%), fidelity (k = 22; 52.4%), and feasibility (k = 22; 52.4%). Several studies also reported assessing the adoption/implementation (k = 20; 47.6%) and sustainability of the program (k = 11; 26.2%). Numerous authors also reported measuring behavioral and program outcomes. The majority reported measuring behavioral outcomes (k = 30; 71.4%), while a smaller number measured changes in practice (k = 9; 21.4%), knowledge (k = 7; 16.7%), self-efficacy (k = 5; 11.9%), or environment (k = 4; 9.5%). Only a few studies included assessments of changes in well-being (k = 3; 7.1%), attitudes (k = 3; 7.1%), skills (k = 3; 7.1%), or communication (n = 2; 4.8%). Lastly, 11 (26.2%) of the evaluations included satisfaction with the adapted intervention as an outcome (Table 4).

26The authors used a variety of formats to present their adaptation processes. All 42 adaptations were described in the article’s narrative, while others also used tables and figures to present certain elements. Seventeen (40.5%) included a table of the adaptations or modifications made. Three adaptations (7.1%) illustrated the adaptation process with a figure, and two (4.8%) included a side-by-side comparison of the adapted and original EBIs.

27This study presents findings based on a systematic review of published reports of adaptations of 42 EBIs. We present a systematic characterization of reasons for adapting EBIs, types of modifications made, steps taken during adaptation, reference to existing adaptation frameworks, and the constructs measured in evaluations of the adapted EBIs. In our review, the most common reasons for adaptation were to be relevant to a particular culture or new population, and to implement a program in a new setting. A previous study by Moore also found cultural adaptation to be a common reason for adaptations among evidence-based grantees, although less frequently (43% compared to our 64%). Higher frequency reasons in Moore’s study were related to resource constraints or logistics: lack of time (80%), limited resources (72%), difficulty retaining participants (71%), and resistance from implementers (64%).

28Among our included reports, all adaptation teams, or individuals involved in the research or adaptation, conducted content modifications, most commonly tailoring, adding or removing elements, and shortening. In their review of 32 published descriptions of interventions implemented in routine care or community settings, Stirman and colleagues also found the same four content modifications most frequently reported. Consistent with the Stirman review, we found that context modifications were the next most frequently mentioned type of adaptation for either the program population or setting. Stirman, however, also found that format changes were frequently described. In our review, delivery modifications were described in the majority of the studies, with training and evaluation modifications much less common. It is unclear whether these did not occur or were less often reported. Moore’s review found slightly different frequencies of modifications, with more reports of changes related to logistics such as changes in delivery and dose, and much less frequent content changes.

29Like Krivitsky, we also found that 29% reported removing elements. This type of adaptation should be explored more because of its implications for reducing the fidelity to program core elements and potentially reducing the EBI’s effects. Additionally, four studies explicitly described changing the core elements of the original EBI. This is an area of concern because the integrity of the original program could have been jeopardized. More research is needed to understand why the elements were deleted and if the program implementers (i.e., researchers, community planners) consulted others before undertaking this change. The low reporting of changes to core elements may be because it is difficult to identify what the core elements are in an EBI. They may include elements that are readily adapted such as delivery or content. However, unless the original developers of the program or health-related online clearinghouses or resources where they are housed clearly describe them, it is often difficult for planners to identify them. Therefore, considerations of fidelity are critical when making decisions about what to adapt. In a systematic review by Gearing and colleagues of 24 meta-analyses and review articles focusing on fidelity over the past 30 years, the authors identified core components of fidelity including design, training, and monitoring of intervention receipt and suggested that greater attention is needed to document threats to fidelity that remain underreported. While this is true for any implementation effort, it is even more important to consider when making and reporting adaptations. Of particular note in our findings, while many authors reported changes to the delivery of the EBI, including who delivered it, there were fewer who reported adapting training or monitoring of that delivery.

30Although cultural modification is not part of Stirman’s taxonomy of adaptation modifications, we found that almost 75% of the authors described their adaptation in this way. Because cultural adaptations would almost always require some adaptation related to population and context, it is likely that authors in the Stirman review reporting adaptations to content, context, and new populations were, at least in some cases, making cultural adaptations. More clarity in definitions of what is meant by each type of adaptations is needed.

31Our review uses a new taxonomy of steps or tasks for adaptation derived from a scoping study of existing frameworks. We looked for nine steps or tasks and found that two adaptations reported all of the steps, with the mean number being seven. Thus, most adaptation teams completed the majority of the steps. Overall, we found that most reported community assessment, preparation of materials, implementation, evaluation, and engaging stakeholders/experts as part of a program planning process. The least common step was selection of the EBI. This may be because some program staff may have already decided on the EBI a priori and did not undertake a process to review candidate EBIs and select one.

32The Escoffery classification from a review of adaptation frameworks seems applicable to real-world adaptation and could be used by others as a taxonomy for describing adaptation steps. However, there are details that may be nuanced that are important to understand for the field both in describing adaptation steps and for informing future adaptations of the same EBI. For example, some reports include information about which components of the intervention were pilot tested and what decisions were made based on assessment finding, who the stakeholders were, and how they were engaged. Additionally, specific details about who is involved in the adaptation process (stakeholders, target population, program deliverers, health promotion, and behavioral scientists) and who makes the final decisions on what changes to make are critical to document. This information could be very important in interpreting reasons for specific adaptations and informing subsequent ones for future EBI implementation.

33Capacity building efforts can assist practitioners to document the process in more detail and be deliberative or proactive with adaptations. The Cancer Prevention and Control Research Network (CPCRN) has modules on program selection of EBIs and adaptation with tools that help practitioners to document the discussion and decisions related to those processes in their Putting Public Health Prevention into Practice training. In addition, the new online decision support tool, IM ADAPT, walks public health professionals through a systematic process to create a logic model for the adapted EBI and a selection adaptation, implementation, and evaluation plan based on intervention mapping.

34Among those reports that included an evaluation, the most common outcomes were acceptability, fidelity, and feasibility. This is not surprising since acceptability and feasibility of an intervention is often associated with program adoption. Only one third reported the use of an adaptation framework to inform their process. This number is surprisingly low. Adaptation frameworks would provide guidance and rationale for this process and should be used. Many frameworks exist, but perhaps program planners may not be aware of them or may not know how to follow them without training or technical assistance. Due to the limited research on program adaptation, there also may not be the emphasis on adaptation models and frameworks. There needs to be increased dissemination and education on these frameworks to offer assistance with recommended steps in program adaptation.

35Through a search of the published literature, this is the first systematic review of adapted evidence-based public health interventions internationally. Findings from the present study lead to important implications for the field of implementation science. First, many of the reasons for adaptation focus on either a change of population or setting, while the most common modifications were related to content, context, and delivery. Program developers of EBIs could anticipate program adaptation, instead of only adoption, and provide technical guidance in making modifications in their implementation protocols (or facilitator’s manual) or program website. Recognizing that it is likely that successful programs will be adapted, program developers should also provide guidance about the theory and mechanisms of change that were used in the intervention and where possible design flexibility to match various contexts and populations. They also can serve as expert consultants to help in the adaptation process as recommended as part of adaptation steps in adaptation frameworks or our scooping study. In addition, they could support a community of their EBI adopters by making adapted versions available or offering contacts for practitioners implementing the same program. Due to the low reporting of use of adaptation frameworks, the frameworks could be more widely disseminated to inform future adaptation efforts.

36Research on best methods to document program adaptation is warranted to better understand whether it is best to describe and code adaptations based on document reviews of adaptation plans, published articles or reports, interviews of the adaptation team, or all of the above. Each of these methods has limitations, but implementing them all may not be practical for research studies. Finally, we found a variety of styles in reporting the reasons, modifications made, and process of adaptation. Standardization of reporting elements on program adaptation would guide professionals in describing their changes to EBIs and advance the field. Through this process of better reporting on adaptations, practitioners and program planners can better understand the reasons for adaptation, the adaptation process, and results to inform their own practice. Currently, TREND and Standards for Reporting Implementation Studies (StaRI) statement only ask researchers to report on adaptation in general or adaptation results. Other critical elements of adaptations that we have identified (i.e., reasons, types of adaptations, steps taken) are not mentioned or delineated. There is a growing body of literature of adaptation taxonomies that could be recommended for some of these elements, including types of modifications, reasons, timing and valence, frameworks employed, and steps taken. Creation of detailed reporting standards for program adaptation will result in commonalties for describing adaptations in the published and gray literature and will advance the field of implementation science in terms of producing adaptation data for further analysis.

37Future research could explore planned versus unplanned adaptations and patterns of program modifications and the reasons for that happening. We present some early findings of patterns of modifications made to public health EBIs, but there is scarce understanding of them. In addition, further evaluations of adapted interventions are required to determine if adapted versions are as effective as the original program or other adapted versions. In this study, over 60% of the adaptation reports were non-experimental (i.e., observational, pilot program, post-test only) and less than one third were experimental. It is important for the field to have more rigorous evaluations of adapted programs to understand their outcomes and if their effects are comparable, better, or worse than the original EBI. Some preliminary research suggests that adapted versions of interventions are not associated with worst outcomes. The evaluations also could inform if different types or combinations of modifications (e.g., content and context) impact effectiveness as well. Researchers also should determine critical adaptation elements to record and standardize across studies. Finally, while there have been repositories of evidence-based programs for public health practice such as the National Registry for Effective Programs and Practices and Research Tested Intervention Programs (RTIPs), there is no clearinghouse for adapted programs for the field to understand the issues around external validity of EBIs. Chambers recommends the creation of an adaptome to catalog adapted programs and their results to share with the field to potentially address this gap.

38Several limitations exist for this study. Although we searched for relevant articles of adapted EBIs, it is likely that some articles were overlooked based on our search strategy. For example, we did not review gray literature for adapted EBIs. In addition, we limited our studies to those that focus on public health interventions and excluded clinical interventions. Additionally, our data on adaptations made and outcomes were limited to the authors’ description in the article and were not augmented with other data (e.g., surveys of authors). Although we had two raters to increase the reliability of the data abstracted, some of the modifications may have been underreported if the authors fail to report on that aspect (i.e., training) or may not have fully implemented the program yet (i.e., evaluation). Finally, while our review included adapted interventions globally, we did not review articles in languages other than English. However, we were able to find 16 studies in international settings. A limitation of this study is that we did not confirm with the authors that all of the adaptations made were reported; for example, some could have not reported on unplanned adaptation since some were not yet implemented. It is helpful for planners to document all adaptation, both planned and unplanned, for other practitioners to learn from this process. Finally, this review is becoming dated, especially in an area with a much active research and reporting.

39This review offers a practical examination of adaptation across multiple programs and program types that were implemented in community settings. It reports systematically on reasons for adaptation, types of modifications, and steps of adaptations for public health EBIs in public health practice. Adaptations are occurring in natural settings for a variety of reasons, and commonly, adaptations are made to intervention content or context. A few steps were used across adaptation teams in the process of adaptations, but the science of adaptation is still an emerging area of study in implementation science. More critical appraisal of intervention adaptations and their outcomes could assist with EBI transferability to increase the scale up and spread of EBI to increase population health impact.