Six steps in quality intervention development (6SQuID)

1Improving the effectiveness of public health interventions depends as much on improving their design as their evaluation. Yet, compared to the vast literature on intervention evaluation, there is little to guide researchers or practitioners on developing interventions in logical, evidence-based ways to maximise effectiveness. Poor intervention design can waste public resources through expensive evaluation or, worse, the implementation of ineffective interventions unevaluated.

2Existing frameworks and guidance for the development of interventions are summarised in table 1. These tend to be orientated towards social-psychological, individual behaviour change and either provide little specific detail on intervention development or require great technical skills and resources. Drawing on the strengths of these existing frameworks and our own experiences, this article outlines a pragmatic six-step guide to the essential stages of intervention development to assist public health practitioners and researchers. The focus is on public health interventions, although the model should have wider applicability.

3A public health intervention is defined as planned actions to prevent or reduce a particular health problem, or the determinants of the problem, in a defined population. Most require some level of social interaction. They are rarely simple, singular actions that can be easily replicated but more often complicated (multicomponent) or complex programmes (with feedback loops and emergent outcomes) that are designed to affect change at several levels of the socioecological model (table 2). By and large, ‘upstream’ interventions ‘require less individual effort (from recipients) and have the greatest population impact’, whereas interventions requiring voluntary uptake are more likely to exacerbate health inequalities.

4Interventions are best developed through collaborations between interdisciplinary teams of practitioners, researchers, the effected population and policymakers. Such coproduction maximises the likelihood of intervention effectiveness by improving: the fit with the target group's perceived needs and thus acceptability; practicality; evaluability, including the theorising of causal pathways; and uptake by practitioners and policymakers.

5This paper sets out six crucial steps in the development of public health interventions (box 1). For an illustrative case study we use the early prevention of gender-based violence (GBV) in Uganda through a parenting intervention that addresses familial predictors of GBV. However, we also draw on other relevant examples to illustrate key points. Define and understand the problem and its causes.Clarify which causal or contextual factors are malleable and have greatest scope for change.Identify how to bring about change: the change mechanism.Identify how to deliver the change mechanism.Test and refine on small scale.Collect sufficient evidence of effectiveness to justify rigorous evaluation/implementation.

6Define and understand the problem and its causes.

7Clarify which causal or contextual factors are malleable and have greatest scope for change.

8Identify how to bring about change: the change mechanism.

9Identify how to deliver the change mechanism.

10Test and refine on small scale.

11Collect sufficient evidence of effectiveness to justify rigorous evaluation/implementation.

12Our starting point is that a public health problem has already been identified as requiring intervention. Often this results from a needs assessment, for which there are several practical guides, or from a political process such as a manifesto commitment.

13Clarifying the problem with stakeholders, using the existing research evidence, is the first step in intervention development. Some health problems are relatively easily defined and measured, such as the prevalence of a readily diagnosed disease, but others have several dimensions and may be perceived differently by different groups. For instance, ‘unhealthy housing’ could be attributed to poor construction, antisocial behaviour, overcrowding or lack of amenities. Definitions therefore need to be sufficiently clear and detailed to avoid ambiguity or confusion. Is ‘the problem’ a risk factor for a disease/condition (eg, smoking) or the disease/condition itself (eg, lung cancer)? If the former, it is important to be aware of the factor's importance relative to other risk factors. If this is modest even a successful intervention to change it might be insufficient to change the ultimate outcome.

14Once defined one should try to establish how the problem is socially and spatially distributed, including who is currently most/least likely to benefit from an intervention. It is also important to consider what interventions or policies currently exist and why they are not deemed adequate.

15Having defined the problem, one needs to understand, as far as possible, what are the immediate (proximal) and underlying (distal) influences that give rise to it. These are often suggested by the distribution of the problem, its history and relationship to the life course. It is only by understanding what shapes and perpetuates the problem (the causal pathways) that one can identify possible ways to intervene. Case study step 1 applies Funnell and Rogers’ useful questions for problem analysis to GBV (ref., p. 160). The main influences on the problem can also be classified according to the socioecological model. It can often be helpful to present the various causal pathways affecting the problem diagrammatically: figure 1 attempts to do this for GBV, distinguishing different levels of the socioecological model.

16The next step is to identify which of the immediate or underlying factors that shape a problem have the greatest scope to be changed. These might be at any point along the causal chain. For example, is it more promising to act on the factors that encourage children to start smoking (primary prevention), or to target existing smokers through smoking cessation interventions (secondary prevention)? In general, ‘upstream’ structural factors take longer and are more challenging, to modify than ‘downstream’ proximal factors, but if achieved structural changes have greatest population impact, as noted above.

17With complex problems the causal pathways can be very diverse and interwoven. If they have been described diagrammatically in step 1, it will be easier to identify where one might intervene and, critically, whether it is necessary to intervene at more than one point, or on more than one level, to interrupt the most important causal pathways. One must also assess which changes would have most effect. Most interventions take place within systems (eg, healthcare, education, criminal justice) and exert their influence by changing relationships, displacing existing activities and redistributing and transforming resources. It is necessary, therefore, to consider which system an intervention would operate in, how the system is likely to interact with the intervention and whether that system needs to be/can be modified as well. For example, a school-based intervention to improve pupils’ social and emotional well-being is likely to be affected by existing school structures, relationships and timetables and might require their modification. Interventions that address complex problems through multilevel actions are more likely to maximise synergy and long-term success. The potential different levels of intervention are shown in table 2.

18In the case study of GBV, it was decided to focus on early prevention in families since: this has the potential for widespread, long-term change; it may improve other outcomes; and more proximal factors were already being addressed.

19Having identified the most promising modifiable causal factors to address, the next step is to think through how to achieve that change. All interventions have an implicit or explicit programme theory about how they are intended to bring about the desired outcomes. Central to this is the ‘change mechanism’ or ‘active ingredient’, the critical process that triggers change for individuals, groups or communities (see table 2).

20It is usually helpful to depict the programme theory diagrammatically (see http://www.theoryofchange.org/). Many interventions are not intended to achieve the final goal directly, but have short-term and intermediate outcomes that are expected to lead to the long-term outcomes. Ideally a range of stakeholders are involved in formulating the programme theory A common pitfall is that it is wildly optimistic, with little empirical evidence to support each link in the causal chain. For instance, a short-term change in health-related knowledge may be necessary, but it is rarely sufficient to achieve behaviour change let alone prevent the disease in question.

21The best developed programme theories are based on formalised theories of behaviour change (eg, Social Cognition Theory or the Theory of Reasoned Action). This is not essential, but can be very helpful if the theory has strong predictive and explanatory power. However, not many do, perhaps because they often only address one causal strand (cognitions or motivation) and not socioenvironmental determinants. Furthermore, few interventions said to be based on such formalised theories clarify how the theory has been operationalised. What is crucial in intervention development is that the change mechanisms in the programme theory are clearly articulated.

22The interpersonal change mechanisms for the GBV case study are shown below (there is not space to show those at the community level). Critical to their effectiveness is who delivers the intervention and their relationship with the target group.

23Having identified the change mechanisms, step 4 requires working out how best to deliver them. As with other steps, it is helpful to involve stakeholders with the relevant practical expertise to develop the implementation plan. Sometimes change mechanisms can only be brought about through a very limited range of activities, for instance legal change is achieved through legislation. However, other change mechanisms might have several delivery options; for instance modelling new behaviours could be performed by teachers, peers or actors in TV/radio soap operas. The choice is likely to be target group-specific and context-specific.

24The implementation plan requires clarifying the conditions and resources necessary for successful implementation and the related risks and assumptions. For example, if an intervention is to be delivered by health visitors, are they available everywhere and will their senior managers allow time for training and delivery? In low income countries resource constraints can seriously restrict options for delivery, for instance the existence of suitably skilled facilitators, or an ethos of voluntarism. In step 4 one should also anticipate possible unintended effects of the intervention and minimise any that might be harmful. These have been categorised by Lorenc and Oliver as fivefold: direct, psychological, equity, group/social and opportunity. ‘Equity harms’ are currently of particular policy concern, the greatest beneficiaries of many behaviour change interventions being the higher educated or more affluent, thereby exacerbating inequalities in health outcomes.

25The delivery of the change mechanisms for our case study of GBV is set out below.

26Once the initial intervention design has been resolved, in most cases its feasibility needs to be tested and adaptations made. This varies considerably according to the type of intervention. For instance, national legislation or large-scale health protection measures, such as water fluoridation, are difficult to pilot before full implementation. Phased region by region implementation might allow incremental adjustments, but the scope for adaptation is primarily around implementation rather than the mechanism of change. With individual or community level interventions a long process of repeated testing and adaptation is often required, often called ‘formative evaluation’, especially if the intervention is novel or highly innovative.

27Testing the intervention can clarify fundamental issues such as: acceptability to the target group, practitioners and delivery organisations; optimum content (eg, how participatory), structure and duration; who should deliver it and where; what training is required; and how to maximise population reach.

28Frequently this is the most hurried stage of intervention development, due to lack of resources and time, but this often compromises subsequent effectiveness. Ideally incremental adaptations would each be tested separately, but in practice adaptations can be made simultaneously if sufficiently rich data are collected to enable judgements about which are helpful and which not. Practical constraints eventually force the decision that the intervention is ‘good enough’ to go to the next step. The testing and adapting of the GBV programme is set out below.

29Before committing resources to a large scale rigorous evaluation (typically a ‘phase III’ RCT), the final step is to establish sufficient evidence of effectiveness to warrant such investment. Beyond the research world, especially in third sector organisations, inadequate resources often mean practitioners move to wide scale implementation without such rigorous evaluation. This makes step 6 all the more critical.

30What is being sought at this stage is some evidence that the intervention is working as intended, it is achieving at least some short-term outcomes, and it is not having any serious unintended effects, for instance exacerbating social inequalities. It is unlikely that the evaluation design will seek to prove causality so theory-based evaluation approaches are likely to be most appropriate. There are numerous guides on evaluation that adequately cover step 6, but it is worth re-stating that often the most practical way to collect evidence of effectiveness with limited resources is through a before and after survey, or by using routinely collected data. If possible, a control group greatly increases the strength of evidence. If a phase III RCT is planned, an ‘exploratory trial’ can provide valuable information about the acceptability of evaluation designs, appropriate measures and likely effect sizes to inform subsequent trials. Plans for this final step with our case study are set out below.

31In order to improve the effectiveness of public health interventions, a systematic approach to intervention development is required, as well as rigorous evaluation. However, little practical guidance exists for public health practitioners and researchers that explains the essential stages of intervention development. We argue that this process can be broken down into six key steps: defining and understanding the problem; identifying modifiable determinants; deciding on the mechanisms of change; clarifying how these will be delivered; testing and adapting the intervention; and collecting initial evidence of effectiveness. This model imposes somewhat arbitrary cut-offs in the process of intervention development and suggests a linear progression. In practice developers often return to an earlier step in the sequence before reaching step 6, and subsequently ‘definitive trials’ can lead to further revisions of the intervention. However, we hope that if each of these six steps is carefully addressed in the design of interventions better use will be made of scarce public resources by avoiding the costly evaluation, or implementation, of unpromising interventions. There is little practical guidance for researchers or practitioners on how best to develop public health interventions. Existing models are generally orientated towards individual behaviour change and some are highly technical and take years to implement. This paper provides a pragmatic six-step guide to develop interventions in a logical, evidence-based way to maximise likely effectiveness. If each step is carefully addressed, better use will be made of scarce public resources by avoiding the costly evaluation, or implementation, of unpromising interventions.

32There is little practical guidance for researchers or practitioners on how best to develop public health interventions. Existing models are generally orientated towards individual behaviour change and some are highly technical and take years to implement.

33This paper provides a pragmatic six-step guide to develop interventions in a logical, evidence-based way to maximise likely effectiveness. If each step is carefully addressed, better use will be made of scarce public resources by avoiding the costly evaluation, or implementation, of unpromising interventions.