Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)?

1Randomised trials are a central component of all evidence-informed health care systems and they form a body of evidence that can help health care users, health professionals, policy-makers and others make informed choices about the effectiveness of treatments and therapies that they use and provide. The same is not true for trial design, conduct and reporting decisions, which are generally uninformed by evidence because there is little relevant evidence to turn to.

2Trial Forge (www.trialforge.org) is an initiative that aims to increase the evidence base for trial decision-making and, in doing so, to improve trial efficiency. One way to fill gaps in evidence is to run Studies Within A Trial, or SWATs. Descriptions of SWATs have been published but here we provide some guidance that provides a brief definition of a SWAT, an explanation of why they are important and some practical ‘top tips’ that come from existing experience of doing SWATs. We hope the guidance will be useful to trialists, methodologists, funders, approvals agencies and others in making clear what a SWAT is, as well as what is involved in doing one. We encourage them to use the text freely on their own websites and materials, with appropriate acknowledgement.

3The text is based on discussions held during and after a 1-day meeting in Aberdeen, UK on 23 March 2017 as part of the Trial Forge initiative. This paper is the first Trial Forge Guidance document and there will be more guidance documents in the future, each providing what we hope is clear help and guidance around an issue relevant to improving the evidence base for trial decision-making. Trial methodologists and other stakeholders will be consulted to determine the topic areas and scope for future guidance.

4A SWAT is a self-contained research study that has been embedded within a host trial with the aim of evaluating or exploring alternative ways of delivering or organising a particular trial process.

5The need for randomised trials to evaluate the effects of health care interventions, such as new drugs and other treatments, is a familiar concept to people working in health and health research. The result of the trial provides evidence on how effective (or ineffective) the intervention is, helping both practitioners and health care users to make well-informed decisions about using it. These trials are central to improvements in health and social care.

6Therefore, it is essential that the trials themselves are done in the most effective ways and one way to do this is to use the same types of evaluation to investigate and improve the processes of how we do randomised trials.

7Unfortunately, only a small number of such studies have been done and there is very little evidence to allow researchers to make well-informed decisions about how to do their trials. This means that researchers doing trials, funders paying for them and patients taking part in them cannot always be sure that the way the trial is being done is as effective and efficient as it could be. The most obvious example of this is that the evidence available to support trial teams to recruit patients to their trials is very thin, despite recruitment being a recognised problem for many trials and being identified as the top priority for research into trial methods.

8One way of increasing this evidence base is to do a Study Within A Trial (SWAT). SWATs evaluate alternative ways of doing a trial process (e.g. recruiting patients, helping them to stay in the study, or reporting the findings) to provide evidence about how to improve the process.

9It seeks to resolve important uncertainties about the processes used in trialsIt is embedded within a host trialIt must not affect the scientific integrity of the host trial, its rationale or outcome measuresIt should have a formal protocol, just like the host trialIt can be evaluated in a single trial but is well-suited for running across more than one host trial, either at the same time or sequentiallyIt will provide data to inform the design and conduct of future trials but might also provide data to inform decisions about the ongoing host trial

10It seeks to resolve important uncertainties about the processes used in trials

11It is embedded within a host trial

12It must not affect the scientific integrity of the host trial, its rationale or outcome measures

13It should have a formal protocol, just like the host trial

14It can be evaluated in a single trial but is well-suited for running across more than one host trial, either at the same time or sequentially

15It will provide data to inform the design and conduct of future trials but might also provide data to inform decisions about the ongoing host trial

16For some practical considerations regarding SWATs, see Table 1. The information in the table comes mainly from experience with SWATs in the UK and Ireland but is likely to be useful for SWATs planned in other countries too.

17Most trials have a Participant Information Leaflet (PIL), which tells a potential participant about the trial. The trial team uses this to offer information to potential trial participants in a way that it hopes will also help recruitment (and perhaps retention) whilet adhering to ethical standards.

18The Systematic Techniques for Assisting Recruitment to Trials (START) programme (http://research.bmh.manchester.ac.uk/mrcstart/) developed a SWAT to evaluate the effect of a bespoke, tailored and user-tested PIL on recruitment compared with a standard PIL. The bespoke method of developing a PIL is expensive so it is important to know how much, if any, difference it makes to the trial. For instance, if the aim is to increase recruitment, it is essential to know that recruitment is indeed increased with the bespoke PIL compared with a standard PIL before using it in a future trial. The SWAT has already been evaluated in several trials (see, for example) and the emerging results are shown in Fig. 1. This meta-analysis shows that the current estimate for the effect on recruitment is small and not statistically significant: 1% improvement (95% confidence interval, − 1–2%).

19In other words, the bespoke PIL had little or no effect on recruitment compared with a standard PIL. By approaching investigators, encouraging them to embed an evaluation of the two types of PIL into their trials and then coordinating the analysis of data from those trials that did, the START programme’s coordinated, collaborative approach of embedding a SWAT evaluation in trials involving over 6600 people now provides an evidence base for researchers trying to decide on whether to develop a bespoke PIL for their trial.

20Other examples of questions that could be addressed in SWATs include:Comparing the effect of different financial incentives to encourage patients to complete a questionnaire used to collect trial outcomesDetermining whether recruitment is boosted if non-responders to postal invitations to join a trial are reminded by telephoneEvaluating the effect on recruitment and retention of a two-stage Participant Information Leaflet (i.e. the leaflet is delivered to participants in two parts: a short ‘key points’ version together with a longer version containing more detail) compared with a standard, single-stage leafletEvaluating the effect on data quality of providing site staff with face-to-face data entry training compared with Skype or video-conference trainingExploring which type of information participants think would best recognise the value of their contribution to the host trial results

21Comparing the effect of different financial incentives to encourage patients to complete a questionnaire used to collect trial outcomes

22Determining whether recruitment is boosted if non-responders to postal invitations to join a trial are reminded by telephone

23Evaluating the effect on recruitment and retention of a two-stage Participant Information Leaflet (i.e. the leaflet is delivered to participants in two parts: a short ‘key points’ version together with a longer version containing more detail) compared with a standard, single-stage leaflet

24Evaluating the effect on data quality of providing site staff with face-to-face data entry training compared with Skype or video-conference training

25Exploring which type of information participants think would best recognise the value of their contribution to the host trial results

26There are plenty of uncertainties around how we should do trials, so it is highly likely that a trial team can find something that is interesting to them and worth investigating in a SWAT. For example, the Prioritising Recruitment in Randomised Trials (PRioRiTy) project (http://priorityresearch.ie/) generated a list of priority areas for recruitment research and many of these could be addressed by SWATs.

27Just as health researchers have a responsibility to make the findings of their clinical trials available, the findings from SWATs should be made public, so that the evidence base available for future decisions can increase. The findings can then be picked up by systematic reviewers and others who synthesise research evidence. The person doing the SWAT can facilitate this by, for example, contacting those who have done a relevant Cochrane Methodology Review (who will be updating it), to let them know about the SWAT. This means that even if the SWAT is not published separately itself, its results can be incorporated into the review.

28SWAT results can also directly inform decisions within the host trial where uncertainty exists as to the best method to use for a particular process. The BEEP trial (https://www.journalslibrary.nihr.ac.uk/programmes/hta/126712/#/) is using a SWAT and an interim analysis to help make a decision about the retention strategies to be used in the trial. A web-based trial linked to antibiotic prescribing also used a SWAT to make a decision about the best way to invite participants to take part in the second stage of the trial. Although both of these SWATs provide useful information for other trials, they were designed to directly inform process decisions taken within the host trial.

29Queen’s University Belfast in Northern Ireland hosts a SWAT repository (go.qub.ac.uk/SWAT-SWAR), which contains a list of prepared SWAT outlines. A form to register a new SWAT is also available online at http://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/ApplicationForms/SWATApplication/. Registering SWATs on the repository helps to avoid unnecessary duplication of effort and provides other researchers with ideas for how they might test the processes they will use in their own clinical trial.

30If your SWAT addresses one of the top 20 PRioRiTy research questions it can also be added to the PRioRiTy online repository (http://priorityresearch.ie/) which is a collection of ongoing research specific to recruitment to trials. This repository is hosted by the Health Research Board – Trials Methodology Research Network in Ireland. Ideally, these methodology studies should be included in both the PRioRiTy and the SWAT repository to help people to find them.