Core outcomes in neonatology: development of a core outcome set for neonatal research

1Inconsistent reporting of outcomes of limited relevance to former patients, parents and healthcare professionals is an important cause of research waste.

2There is a lack of evidence to guide many neonatal practices, leading to variation in both the care provided and outcomes for patients.

3Core outcome sets (agreed, standardised outcomes to be reported by all trials) have been developed in other fields to improve outcome selection and facilitate meta-analysis.

4Former patients, parents, doctors, nurses and researchers show differences in how they prioritise neonatal care outcomes.

5We have identified 12 outcomes that are important to these stakeholders.

6If these outcomes are reported in a standardised manner by all neonatal research, this will enhance future evidence synthesis.

7The neonatal period is crucial to long-term health, and neonatal conditions are the leading cause of disability-adjusted life-year loss. Preterm birth is a major cause of childhood morbidity and implicated in the pathogenesis of adult non-communicable diseases. Neonatal care is common; in high-resource settings one in ten babies are admitted to a neonatal unit, a proportion that is increasing.

8Unfortunately there is a paucity of high-quality evidence to guide much neonatal practice, leading to variation in clinical care and outcomes. One reason research fails to guide practice is because neonatal meta-analyses rarely provide conclusive recommendations, commonly because trials have used heterogeneous, non-comparable outcomes. A further limitation of neonatal and paediatric research is that the outcomes reported are frequently not meaningful to patients and parents.

9One solution is the development of a core outcome set: important outcomes identified by key stakeholders using robust consensus methods. A core outcome set could ensure all future research in a field reports a common subset of clinically meaningful outcomes and reduces research waste by facilitating meta-analysis. A core outcome set is a minimum set and does not preclude researchers reporting other outcomes where relevant. The use of core outcome sets for trials is promoted by journals, Cochrane Review Group editors and research funders. Relevant, standardised outcomes are also crucial for observational research, benchmarking, clinical audit and quality improvement studies.

10The objective was to develop a core outcome set for research in neonatology.

11The core outcome set has been developed to apply to all research involving babies receiving care on any designation of neonatal unit in a high-income setting, with no limitation by gestational age at birth, birth weight or illness severity. It is intended to apply regardless of the specific population of babies, clinical setting or clinical condition that a particular study addresses. The scope was established at the initial steering group meeting following direction from former patients and parents. The parents and former patients all strongly expressed the view that ‘a sick baby is a sick baby’. They were also clear that while it is possible to separate babies on a neonatal unit by gestation, weight or underlying diagnoses, the outcomes that are most important are universal to all these groups. Research involving babies cared for exclusively on labour or postnatal wards or in the community will be excluded as the majority are healthy needing limited medical input.

12We prospectively registered the study with the Core Outcome Measures in Effectiveness Trials (COMET) initiative (Registration number 842) and published the study protocol. Research ethics approval was not required; the project involved consenting adults completing surveys (online supplementary eFigure 1). We formed a steering group to guide the core outcome set development comprising different disciplines, perspectives and expertise (online supplementary eText 1).

13We followed COMET initiative methodology with reference to previous core outcome set development work. We identified outcomes reported in neonatal trials and qualitative research and then used these to determine a core outcome set using a consensus process (online supplementary eFigure 2).

14We undertook a prospectively registered systematic review to identify outcomes reported in neonatal clinical trials. Randomised controlled trials are widely considered to be the most rigorous method to determine how a treatment affects patients. We searched Cochrane Controlled Trials Register, Cumulative Index to Nursing and Allied Health Literature (CINAHL), EMBASE and Medline from July 2012 to July 2017. Three researchers (SA, SS, JWHW) independently double-screened potentially relevant records based on titles and abstracts and reviewed the full text of selected studies to assess eligibility. Due to the large number of trials identified, only those with over 100 infants in each arm were included. As many trials lead to more than one publication reporting outcomes at different time points, we reviewed all linked publications. Outcomes were extracted and categorised by physiological system.

15We undertook a second, prospectively registered review to identify outcomes from qualitative research. We searched Applied Social Sciences Index and Abstracts, CINAHL, EMBASE, Medline and PsycINFO from 1997 to 2017. Publications describing qualitative data relating to neonatal care outcomes, reported by former patients, parents or healthcare professionals, were included. Narrative text and grouped outcomes were thematically analysed by physiological system.

16The steering group assessed outcomes identified in the two reviews to produce a final inventory in which duplicated or closely related outcomes were grouped. The inventory informed a three-round, online eDelphi survey which was followed by a consensus meeting.

17We recruited participants for the eDelphi from the following groups:

18Former patients cared for on a neonatal unit, and parents of neonatal patients, recruited through neonatal charities and social media.

19Nurses and allied health professionals (including neonatal nurses, midwives, speech and language therapists, occupational therapists, and physiotherapists), recruited through professional journals and associations.

20Doctors (including neonatologists, obstetricians, paediatric surgeons, general paediatricians, community paediatricians and general practitioners), recruited through the Royal College of Paediatrics and Child Health and professional organisations.

21Academics and researchers in the neonatal field, recruited through meetings, academic publications and organisations.

22Recruitment was international; participants had to have personal experience of neonatal care or research in a high-income setting. We aimed for 30 participants in each group to achieve a total of 120 participants. The sample size followed guidance and previous core outcome set development.

23Participants completed a three-round eDelphi survey to establish consensus. We ran the eDelphi using DelphiManager software. To maximise response rates, the survey was kept as short as possible and extensive demographic data were not collected. In each round we asked participants to rank outcomes between 1 and 9 (with 1–3 meaning ‘limited importance for decision making’ and 7–9 meaning ‘critical for decision making’) following the Grading of Recommendations Assessment, Development and Evaluation guidelines (figure 1). In round 1, participants could suggest outcomes not identified in the reviews which they felt were important; these outcomes were included in rounds 2 and 3. After each round we collated the results. Before participants reviewed and rescored outcomes in rounds 2 and 3, we presented them with a bar chart showing how each outcome had been scored previously. Each graph combined the scores from all stakeholder groups. We applied predefined consensus criteria to round 3 results. Provisional core outcomes were those over 70% of participants in each group scored as ‘critical’ and less than 15% of each group scored as ‘limited importance’. Conversely, if over 70% of participants in each group scored an outcome ‘limited importance’ and less than 15% in each group scored it ‘critical’, it was not included. If neither criterion was met, an outcome was classified as ‘no consensus’.

24We compared scoring patterns using the first round results to assess agreement between stakeholder groups. The mean scores for each outcome were calculated for each group, and pairwise comparisons were then made between groups. Pearson’s correlation coefficient was calculated for each comparison; differences between coefficients were tested using Fisher’s r-to-z transformation.

25We undertook an attrition analysis to ensure the eDelphi results had not been distorted by differences in opinion between those who dropped out and those who completed all surveys. We compared two groups: participants who only took part in round 1 (including those who dropped out during this round) and participants who contributed in all rounds. We compared how these groups scored outcomes in round 1. We used Mann-Whitney U to test for differences in scoring with Bonferroni correction for multiple comparisons (corrected to 5% significance). For outcomes where a difference in scoring was identified, we also tested if the different scoring patterns observed would have changed whether the outcome was considered ‘core’ in round 1 (according to the predefined consensus definition), suggesting attrition affected whether the outcome met the criteria for inclusion in the final core outcome set.

26The final prespecified phase was a face-to-face meeting to confirm the final core outcome set based on the eDelphi results. We only invited steering group members and eDelphi participants with additional expertise; the meeting was limited to 16 participants to facilitate discussion. The consensus meeting remit was limited to refining the final survey results, no new outcomes were considered, and the eDelphi results were paramount. The consensus group were presented the results of the eDelphi and the attrition analysis. They considered whether the identified core outcomes covered all necessary domains, whether there was overlap between outcomes and whether it would be feasible to expect all trials to record each outcome. They discussed the following outcomes in depth: outcomes that met the consensus definition, ‘borderline’ outcomes that narrowly missed the consensus definition (defined as 70% of at least one stakeholder group scored the outcome as ‘critical’) and any outcomes identified during the attrition analysis. Meeting attendees discussed each outcome, then an anonymous vote was held on the question ‘should the outcome be included in the core outcome set?’ For inclusion in the final set, 70% of attendees had to vote ‘Yes’. We have published the meeting minutes online.

27This study was completed according to the study protocol. The only deviation occurred during the review of trials: due to the large number of studies identified, only trials with over 100 neonates in each arm were included. The results of this core outcome set development are reported using COS-STAR (Core Outcome Set–STAndards for Reporting) reporting guidelines.

28In the review of clinical trials we identified 76 large neonatal trials reporting 216 outcomes, and in the qualitative literature review we identified 62 publications with 146 outcomes. The steering group reviewed these 362 outcomes, identified 19 duplicates and grouped 239 closely related outcomes. This resulted in a final inventory of 104 outcomes, which were entered into the eDelphi (figure 2) (full list in online supplementary eTable 1). Participants added 10 additional outcomes following the first round (online supplementary eTable 2).

29We recruited a total of 414 participants from 25 countries across 5 continents (online supplementary eFigure 3). The distribution of participants in different stakeholder groups and their participation during the eDelphi are presented in table 1. Participation in all rounds exceeded our target of 120 participants.

30Two hundred and sixty participants completed the first round. The mean scores for parents and patients correlated with the scores of nurses and therapists more closely (r=0.83) than with the scores of doctors (r=0.51). The mean scores from doctors correlated most closely with those of researchers (r=0.96). The differences between these correlations were statistically significant (p<0.01). Pairwise comparisons are shown in figure 3.

31The final round was completed by 173 participants. The highest scoring outcomes from each stakeholder group are shown in table 2.

32The prespecified consensus definition was met for 15 outcomes; these were discussed at the consensus meeting along with 21 outcomes ranked as ‘borderline’. The attrition analysis identified a statistically significant difference between scoring for 19 outcomes (online supplementary eTable 3); for 17 there was no difference in whether the outcome would have been included in the core outcome set. The remaining two outcomes were discussed at the consensus meeting to ensure attrition had not distorted the consensus process.

33At the consensus meeting 16 participants representing all stakeholder groups (5 former patients/parents, 3 nurses/therapists, 5 doctors and 3 researchers) discussed and voted on each of the 38 outcomes identified from the eDelphi results. Twelve outcomes were identified for inclusion in the final core outcome set. During discussion the outcome ‘Harm from medical treatment’ was defined as ‘Adverse events’ to allow better alignment with existing classifications of iatrogenic harm. Two outcomes (‘Retinopathy of prematurity’ and ‘Chronic lung disease/bronchopulmonary dysplasia’) relate only to preterm infants and should only be reported by trials involving this group. Meeting minutes and voting results are provided in online supplementary eText 2.

34The final core outcome set comprises the following:

35Survival.

36Sepsis.

37Necrotising enterocolitis.

38Brain injury on imaging.

39Retinopathy of prematurity (preterm only).

40General gross motor ability.

41General cognitive ability.

42Quality of life.

43Adverse events.

44Visual impairment or blindness.

45Hearing impairment or deafness.

46Chronic lung disease/bronchopulmonary dysplasia (preterm only).

47(Outcomes were ranked by percentage of round 3 participants who scored each outcome ‘critical for decision making’.)

48Using robust, preregistered consensus methodology, we identified 12 outcomes to be reported in all future trials involving infants receiving care on a neonatal unit in a high-income setting. We hope use of this core outcome set will improve research quality and reduce waste. The core outcome set is a minimum set of outcomes that are so important to all stakeholders that failing to report them will mean that important clinical uncertainties cannot be addressed, both at the level of individual studies and in subsequent meta-analyses.

49This core outcome set complements the work by van’t Hooft et al in which a core outcome set for interventions to prevent preterm birth was identified. This contained maternal and neonatal outcomes, but the scope was limited to antenatal interventions. A number of core outcome sets have been developed in women’s health; in the newborn period these exist only for gastroschisis and Hirschsprung’s disease, with work under way for neonatal abstinence syndrome. In rheumatology widespread adoption has led to full reporting of the rheumatoid arthritis core outcome set in 80% of relevant trials. Similar uptake in neonatal research would reduce barriers to meta-analysis and aid translation of research findings into clinical practice.

50A strength of our project was the number of parents and former patients who took part. Our review of trials found no reported involvement of parents or former patients in outcome selection; it is therefore unsurprising that these groups report dissatisfaction with outcomes currently reported in neonatal research. In our work former patients and parents scored outcomes by importance and could suggest additional important items. Their priorities differed from other stakeholder groups, emphasising the importance of wide involvement in outcome selection.

51A limitation of our work was attrition during the eDelphi, which occurred despite efforts to optimise response rates. The attrition rates in this study are comparable with similar projects. Explanations for the attrition include the wide range of outcomes (each survey took 20 min) and that participation was voluntary. Former patients and parents were most likely to drop out, perhaps due to their caring commitments. The attrition analysis identified outcomes where dropout could have skewed scoring patterns and distorted results; those identified were discussed further at the consensus meeting. Participant attrition is common during Delphi surveys; steps to minimise attrition are evolving. Another limitation is that potential stakeholder groups were not represented (eg, hospital administrators/policy makers). No guidance mandates which groups should be involved in core outcome set development; our project included all groups included in most core outcome set development.

52Future work will standardise outcome measures and measurement time points for the outcomes identified. While our review found outcome domains were similar across large neonatal trials, disparate measures and time points meant results were not comparable. Heterogeneity of measures and time points is a known barrier to evidence synthesis. Defining outcomes like ‘Adverse events’ or ‘Quality of life’, endpoints we have demonstrated to be important to all stakeholder groups, will allow research to report them consistently. Further input from former patients and parents is needed to ensure that outcome measures reflect their lived experiences. Starting in 2020 we will define measures and time points following OMERACT (Outcome Measures in Rheumatology) 2.0 methodology, in collaboration with other international efforts. Other core outcome sets have also been developed or are in development in the field of neonatology; it is important that overlapping core outcome sets are harmonised to avoid the multitude of incomparable outcomes being replaced by multiple incompatible core outcome sets. The aim is that future research will report the core outcome set alongside trial-specific outcomes; trial-specific outcomes will address a particular research question and core outcomes will provide data for meta-analyses, particularly for prospectively planned meta-analyses.

53While core outcome sets are associated with clinical trials, integration with routine data collection will reduce the burden on researchers, facilitate efficient research and improve quality. This will ensure future audit, benchmarking and quality improvement projects are focused on outcomes important to all.

54We have identified a core outcome set for neonatal research. Adoption of this set will standardise outcome selection and ensure these are relevant to those most affected by neonatal care. This will help research translate into improved clinical practice, optimising outcomes for neonatal patients.