Reporting of complex interventions in clinical trials: development of a taxonomy to classify and describe fall-prevention interventions

1Reduction of falls in older people is an important health care target in many countries. However, the aetiology of falls in older people is not fully understood. Although some falls result from a single factor, many are caused by an interaction between multiple factors. As a consequence, a wide variety of approaches to prevention and management have emerged, most of which are "complex interventions". Complex interventions are usually described as interventions that contain several interacting components. Despite many clinical trials which have identified effective interventions, translation into clinical practice has been slow. There are many potential barriers to implementation of new interventions. Poor reporting of interventions in clinical trials can result in practitioners being unable to replicate the intervention. Researchers evaluating interventions in meta-analysis lack an internationally agreed framework for conceptualizing the main forms and important components of fall-prevention techniques, limiting the ability to draw meaningful comparisons and identify factors associated with success.

2Several sets of international guidance have highlighted the importance of documenting and conceptualizing the influential components of an intervention, recognizing the importance of these steps in evaluation and understanding the potential for generalization of an intervention. However, this documentation and conceptualization is not simple, particularly when trials may be from countries with different health and social care services.

3The aim was to develop a classification system to characterize the major influential components of fall-prevention interventions and promote consistency of reporting across international boundaries. We wished to encourage investigators to report all fall prevention interventions in a standardised and comprehensive manner, and to produce a taxonomy that could be used to classify existing and future interventions. We report the process of development, involving repeated testing and refinement, from the initial stages of design through to the final version of the taxonomy.

4The study was developed through the Prevention of Falls Network Europe (ProFANE) project, a collaborative project to reduce the burden of fall injury in older people through excellence in research and promotion of best practice. The network activities were funded by the European Commission, but link clinicians, members of the public, and researchers worldwide.

5Classification systems are of two major types-typologies and taxonomies. Typologies are theoretically driven classification schemes. A taxonomy necessitates identifying characteristics that can be objectively measured in real-world circumstances, and have sufficient variance to demonstrate meaningful differences. We aimed to develop an internationally accepted taxonomy. Sartorius provides useful guidance on the development of taxonomies, highlighting the importance of broad agreement on the content and framework to ensure compliance with the intent and method of classification, and the need for an accompanying glossary defining operational terms. Our method was an iterative process, comprising a phase of development, followed by a period of refinement and finalisation. The process was informed and approved by international expert consensus throughout its span, the nature of the consensus building techniques varied during the project, but formal nominal group methods were implemented at key stages. The consensus panel included 33 representatives of the academic, policy, practice, and user communities drawn from a range of disciplines and countries in Australasia, Europe and North America (detailed at the end of the manuscript). Disciplines included were medicine, physiotherapy, occupational therapy, nursing, psychology, public health, exercise physiology, statistics and epidemiology.

6The process for developing the taxonomy is summarized in Figure 1. In the initial stages, a draft hierarchical framework of classification that identified the major domains was developed by a small group of investigators (SEL, EJS, KP, CB). This draft framework included the domain headings: "Approach", "Base", "Components" and "Descriptors". These domains provide the overall structure and framework to the taxonomy. The proposed framework was discussed and approved by a meeting of 50 academics and practitioners engaged in research in falls prevention in Manchester in 2004. Definitions were developed for each domain, along with sub-domains. The intention was that sub-domains would represent meaningful differences in the ways in which interventions had or could be delivered. Within each sub-domain the taxonomy was further characterized to provide more detailed levels of information on the service delivery model which we labeled as categories. The development was informed by close reference to the published literature to ensure the taxonomy was sufficiently comprehensive to describe the variety of interventions, settings and service delivery models reported in the research literature. Firstly, we identified all trials published in full and in English included in Gillespie et al, and used a random sampling method to select half of these (N = 28,). We retained the remaining papers (N = 27) for validation studies in later components of the study. Second, we undertook a broader search of the non-randomised and grey literature search strategy reported in to identify new or untested approaches to fall prevention. Where possible, we utilized existing international classification systems to provide category definitions and headings, and sourced additional definitions from the Medical Subjects Headings (MeSH) browser of the US National Library of Medicine. In some domains of the taxonomy, there were no pre-existing classification systems, and these were developed by consulting members of the expert panel, matching expertise to the areas requiring development. At each stage of the process, the rationale, definitions and instructions for completion of the various components of the taxonomy was documented, and turned into a draft instruction manual.

7The first version of the taxonomy and manual was circulated prior to a formal two-day consensus meeting of 33 international experts. We utilized a modified nominal group technique, facilitated by experienced health services researchers, in which panel members were asked to consider eight questions relating to the proposed taxonomy (shown in Table 1). The relevance of the domains, sub-domains and categories were agreed and prioritized by consensus panel. Only the most important categories were retained in the taxonomy, recognizing the need to balance detail against practical issues of completion and ease of communication. The panel also considered the future use of the taxonomy and its capacity to assimilate new and emerging areas of practice.

8The taxonomy and manual were then tested using the remaining 27 articles retained after our search of trials published in the Cochrane review. Data on the intervention characteristics from these 27 papers were extracted by five academic health practitioners who had not been involved in development stages of the taxonomy (the test panel). These practitioners included disciplines of medicine, physiotherapy, occupational therapy, psychology and nursing. Each rater was provided with a copy of the taxonomy manual, and a recording form. In addition, a number of papers were replicated across the raters to enable estimation of the test re-test reliability of the taxonomy. A cut-off of poor agreement between raters (a cut of kappa <0.41 as per Landis and Koch) was used to indicate areas of the taxonomy and manual requiring revision. We also worked with individual raters to determine areas of the manual and taxonomy that lacked clarity, were difficult to complete, or were incomplete. Results of the panel test were reported back to a group of 12 of the original members of the expert panel to agree the final format and manual for the taxonomy.

9Finally, two independent reviewers completed a taxonomy for the interventions reported in ten randomised controlled trials selected at random from the systematic review of Gates et al, to determine the agreement and chance corrected agreement (kappa) for each sub-domain. Agreement was calculated over all categories included in each sub-domain, according to Landis and Koch. Chance corrected agreement (kappa) was then established using SPSS (version 17) for each sub-domain for all ten papers. For each paper we then determined the frequency of items (kappa values) contained within each kappa banding (poor, fair, moderate, good or very good) as described by Altman. The frequencies calculated were then added together across all ten papers and converted into a percentage. These results therefore show the percentage of items (kappa values) across all ten papers within each kappa banding.

10The full taxonomy including the recording form and instruction manual are provided in Additional Files, 1 and 2. A description of the domains, sub-domains, and categories in each sub-domain are given in Table 2, along with a brief synopsis of the consensus justification for inclusion within the taxonomy. Agreement between the two independent raters for extraction for the final version of the taxonomy was good to excellent for all sub-domains. In over 90% of cases chance corrected agreement was very strong between the two independent raters (kappa = 0.81-1.00) (Table 3).

11The evaluation of complex interventions is an infant science. An inability to identify, define and communicate the potentially important components of falls prevention interventions has the potential to hinder development of research and uptake into clinical practice. We have identified key characteristics of interventions that we believe should be described alongside the publication of a randomised controlled trial of fall-prevention interventions. Many of the domains, sub-domains and categories are transferable to other similar interventions.

12The taxonomy represents judgments in determining essential from non-essential information, and balances the need for brevity and simplicity against complexity and detail. There is no single accepted method to develop a taxonomy. Empirically based classification requires data, and in this case, data sources were publications of randomised controlled trials of fall-prevention interventions, meta-analyses and qualitative studies. This was supplemented by extensive involvement of experts, which is acknowledged as essential in developing taxonomies. Elicitation of expert knowledge from consensus methods can be problematic. We protected against predomination of individual's opinion by using a modified nominal group technique, guided by pre-specified questions and independent facilitation. The modified nominal group technique was selected in preference to the Delphi method, as it allows discussion between experts to gain consensus.

13The taxonomy went through several iterations, being refined by a diverse group of international experts at different time points. There were a number of internationally agreed classification systems already in existence, and we used these wherever possible. These included the controlled vocabulary (Medical Subject Headings (MeSH)) used to index MEDLINE. The performance of the draft taxonomy was assessed using retrospective data extraction from published papers. The validation sample contained a variety of intervention types. Whilst we have allowed for the possibility of new areas emerging within the taxonomy, further refinements will be required. For example, further refinement of the selection criteria section might include data on screening tools used, for example whether the tool intends to capture single or repeat fallers, or aims to identify osteoporotic fracture risk. We also accept that the sub-domains of the taxonomy may not be entirely distinctive in all situations, but that for the majority of situations it is fit for purpose. We intend to review the taxonomy in five years time and welcome feedback on the structure and content.

14The taxonomy is intended for a number of purposes, including to assist with data extraction alongside meta-analysis of research data, reporting of interventions tested in research studies, and in the process of development of interventions. The list is purposefully not exhaustive. The most recent update of the CONSORT guidelines for reporting clinical trials necessitates reporting on the main elements of complex interventions. The challenges of reporting complex interventions have been recognised for sometime. The Medical Research Council (MRC) first published a framework for reporting and evaluating complex interventions in 2000. Although the MRC framework stresses the importance of accurate reporting, little operational guidance is provided. A recent suggestion has been to use graphical techniques to present key features of the timeline and content of interventions, a technique which can be expanded to a number of non-pharmacological interventions. In comparison, our method, which has more limited application as it focuses on fall-prevention interventions, is structured to ensure ease of complete reporting (by way of yes/no answers), uses standard internationally transferable set of definitions to describe the intervention components, settings and populations tested; and provides a more detailed description of intervention. Replication of interventions should be easier to achieve, although this assumption requires testing. We anticipate that similar methods could be developed in other fields. The taxonomy is complementary to but does not replace the Consort Guidance on the reporting of complex interventions.

15Subsequently, the taxonomy has proved useful in a number of Cochrane and other high quality systematic reviews, and has enabled identification and pre-specification of important aspects of service configuration and intervention delivery. We have also used the taxonomy as the framework for a UK national survey of falls services. Authors of trials are encouraged to register details of their interventions using the taxonomy in a prospective format (open registration is available at http://www.warwick.ac.uk/go/fallstaxonomy), and to utilize the taxonomy to ensure accurate, complete and useful intervention reporting.

16We have developed a taxonomy to describe and classify fall-prevention interventions, with sufficient breadth to capture elements of the intervention that are thought influential in determining effectiveness. This was achieved through a mix of methods, including expert consensus, literature reviews, and validation by test panels.

17The authors declare that they have no competing interests.

18SL and CB had the concept for the original study, study design and analysis. SL wrote the first draft of the paper and is the guarantor. LG contributed search strategies and to the study design, analysis and write up. JS and SF were independent reviewers in final testing process and performed the statistical analysis. KP developed and edited the different versions of the taxonomy and manual, and contributed to the drafting of the manuscript. RP participated in developing the taxonomy. All authors read and approved the final version of the manuscript.