APOSTEL 2.0 Recommendations for Reporting Quantitative Optical Coherence Tomography Studies

1Increases in the numbers of quantitative optical coherence tomography (OCT) studies have raised the need for consistent and coherent standardized reporting recommendations. In 2016, the Advised Protocol for OCT Study Terminology and Elements (APOSTEL) recommendations were published to provide a 9-point checklist of relevant aspects for reporting quantitative retinal OCT studies. The original APOSTEL recommendations were conceived as expert opinion (level D evidence according to the Grading of Recommendations Assessment, Development and Evaluation [GRADE] working group criteria; gradeworkinggroup.org) from discussions among the authors, the IMSVISUAL consortium (imsvisual.org), and consideration of the literature. Without a formal consensus-building approach, and without involving a broader audience, further validation was warranted. We aimed to revise and achieve consensus on these recommendations by using a modified Delphi method, including a larger group of OCT scientists and clinicians, in a formal procedure to review the consensus and develop level C evidence-based guidelines (GRADE criteria). The long-term goal was to improve the reproducibility and interoperability of OCT studies for retinal and neuro-ophthalmology diseases.

2In order to identify experts in the field while minimizing the risk of selection bias, we chose to contact corresponding authors of studies reporting quantitative retinal OCT results published within 24 months prior to our initial survey by email. A total of 892 authors of 1,189 publications were identified by a PubMed search (performed 3 July 2017) using the search terms “quantitative” and “optical coherence tomography” for 2015 to 2017. The Delphi method is a systematic, multistage survey to obtain consensus on a specified question. The process involves multiple rounds of questionnaires presented to participants. The responses are analyzed by a panel of experts and fed back to participants and assessed for consensus. Most of the members of the panel of experts were also corresponding authors of quantitative retinal OCT studies and were therefore also invited to participate in the survey. Following the consensus-building procedure of a modified Delphi method (figure 1), we conducted the following steps:We contacted all corresponding authors of the identified publications and asked them to evaluate and give feedback on the initial APOSTEL recommendations. The participants were asked about their agreement on each item of the recommendations, rating from 1 (full disagreement) to 4 (full approval). Participants were given the opportunity to provide comments. In a blinded fashion, we collected feedback and suggestions using a free online survey via Google Forms (initial questionnaire; raw data of survey results can be obtained from the corresponding author upon qualified request).We then formed a panel of 54 international experts who gathered at congress meetings and during 4 rounds of telephone conferences. The aggregated results of the initial questionnaire were reviewed online through a second questionnaire by the panel, who also revised the original APOSTEL recommendations and proposed a list of changes.This list was then reviewed in a second Delphi round by the original group of corresponding authors through a third online questionnaire (Google Forms). In this last Delphi round, the participants were given the opportunity to approve or reject the final list of suggestions of the panel of experts by majority vote.

3We contacted all corresponding authors of the identified publications and asked them to evaluate and give feedback on the initial APOSTEL recommendations. The participants were asked about their agreement on each item of the recommendations, rating from 1 (full disagreement) to 4 (full approval). Participants were given the opportunity to provide comments. In a blinded fashion, we collected feedback and suggestions using a free online survey via Google Forms (initial questionnaire; raw data of survey results can be obtained from the corresponding author upon qualified request).

4We then formed a panel of 54 international experts who gathered at congress meetings and during 4 rounds of telephone conferences. The aggregated results of the initial questionnaire were reviewed online through a second questionnaire by the panel, who also revised the original APOSTEL recommendations and proposed a list of changes.

5This list was then reviewed in a second Delphi round by the original group of corresponding authors through a third online questionnaire (Google Forms). In this last Delphi round, the participants were given the opportunity to approve or reject the final list of suggestions of the panel of experts by majority vote.

6Seventy-three (8%) of the 892 contacted corresponding authors of quantitative OCT studies completed the first online questionnaire and provided feedback, the majority of these being ophthalmologists (71%), followed by neurologists (10%) and neuro-ophthalmologists (10%). Eighty percent of participants agreed with the recommendations as they were published and 95% planned to adhere to the recommendations in future publications. At the same time, 64% stated having reported their previous research with less detail than suggested.

7Based on the feedback obtained during the first survey, the panel of 54 experts drafted a list of 15 suggested changes to the original APOSTEL recommendations. Twelve (80%) of these suggestions (see below) were accepted through the second questionnaire, while proposals already covered in the original recommendations or to include OCT angiography (OCT-A) were rejected. With this feedback, we generated a revised version of the APOSTEL recommendations with an updated 9-point checklist.

8A total of 116 (13%) of the 892 corresponding authors responded to the third survey. Among them, 53% were ophthalmologists, 35% neurologists, and 12% non-MD researchers. The overall acceptance of the proposed changes was over 95%, with the only exception of the recommendation to report the pixel to millimeter ratio and the image format if the images are exported from the device for analysis, which was accepted by 84% of the authors.

9After the modified Delphi process for consensus building, we decided to maintain the initial recommendations of stating the acquisition protocol and imaging modalities and addressing concomitant eye pathologies with the exact scanning protocol. The changes made to the original APOSTEL recommendations checklist are highlighted in the table and summarized below:As already addressed in correspondence to the initial recommendations, we harmonized the nomenclature of the outer retinal layers to match the 2014 consensus article by Staurenghi et al. (figure 2).We removed references to specific reporting guidelines to avoid favoring any guidelines or omitting relevant recommendations.When utilized, we suggest reporting device-specific features (e.g., enhanced depth imaging, swept-source OCT, adaptive optics).We added the exact area of measurement (e.g., analysis grids) to the description of volume scans.We added a commentary regarding the importance of addressing potential bias in manual segmentation or manual correction of segmentation errors (masking). In several comments, concerns were raised regarding the length of the methodology section of articles that fully adhered to the APOSTEL recommendations. In case of limited word count availability, we now advise submitting the exact OCT methodology as supplementary material, if permitted.Another issue raised by several comments was concerning the relevance of some of the details to be reported regarding the acquisition setting, namely the room lighting conditions and whether pupils were dilated. The panel of experts agreed that reporting the ambient lighting condition is likely to be of low clinical importance, although shaded room lighting is suggested. However, off-axis beam placement could affect the results of OCT imaging studies, and the risk for this phenomenon increases with pupil dilation and is greater for the outer retinal layers (outer plexiform layer/outer nuclear layer) compared to the inner retinal layers (peripapillary retinal nerve fiber layer to inner nuclear layer). Oberwahrenbrock and colleagues showed that the greatest error is for the outer retinal layers. Therefore, pupil dilation is relevant because it can directly affect quantitative OCT measures. We thus omitted room light conditions but retained pupil dilation.

10As already addressed in correspondence to the initial recommendations, we harmonized the nomenclature of the outer retinal layers to match the 2014 consensus article by Staurenghi et al. (figure 2).

11We removed references to specific reporting guidelines to avoid favoring any guidelines or omitting relevant recommendations.

12When utilized, we suggest reporting device-specific features (e.g., enhanced depth imaging, swept-source OCT, adaptive optics).

13We added the exact area of measurement (e.g., analysis grids) to the description of volume scans.

14We added a commentary regarding the importance of addressing potential bias in manual segmentation or manual correction of segmentation errors (masking). In several comments, concerns were raised regarding the length of the methodology section of articles that fully adhered to the APOSTEL recommendations. In case of limited word count availability, we now advise submitting the exact OCT methodology as supplementary material, if permitted.

15Another issue raised by several comments was concerning the relevance of some of the details to be reported regarding the acquisition setting, namely the room lighting conditions and whether pupils were dilated. The panel of experts agreed that reporting the ambient lighting condition is likely to be of low clinical importance, although shaded room lighting is suggested. However, off-axis beam placement could affect the results of OCT imaging studies, and the risk for this phenomenon increases with pupil dilation and is greater for the outer retinal layers (outer plexiform layer/outer nuclear layer) compared to the inner retinal layers (peripapillary retinal nerve fiber layer to inner nuclear layer). Oberwahrenbrock and colleagues showed that the greatest error is for the outer retinal layers. Therefore, pupil dilation is relevant because it can directly affect quantitative OCT measures. We thus omitted room light conditions but retained pupil dilation.

16The formal consensus-building approach of a modified Delphi method was used to revise the APOSTEL recommendations for the reporting of quantitative OCT studies.

17We observed a high consensus of the participants already with the initial APOSTEL recommendations in the first survey. The majority of the participants acknowledged the need for guidance.

18Whereas the original APOSTEL recommendations were conceived by a panel dominated by neurologists, a more heterogeneous mix of specialties, with broader expertise, contributed to this new version, the majority being ophthalmologists. Ninety-seven percent of all participants agreed that that the APOSTEL 2.0 guidelines should apply to all studies reporting on quantitative retinal OCT research and not be restrained to certain disorders or disciplines. Furthermore, choosing to identify the experts to be addressed by the survey as the corresponding authors of relevant research articles based on a PubMed search assured a broad consensus-building approach, eliminating the selection bias typically immanent to expert consortia. However, there was a low response rate: 8% of the contacted corresponding authors responded to the first round of the survey and 13% to the second round. Possible explanations for this limitation may include the fact that corresponding authors are senior supervisors or principal investigators and are not necessarily as involved in the technical details and specifications addressed by the APOSTEL recommendations. Likewise, there are time constraints to consider. This can be viewed as a limitation of the study but we have to assume that those who participated in the survey were knowledgeable about the matter and contact details for the first authors or technicians involved in these studies were not available.

19The modified Delphi method tends to eliminate extreme (but possibly relevant) positions and steers a middle-course consensus. However, all survey participants were given the opportunity to provide feedback in free text and all comments were critically discussed among the panel of experts. The achieved consensus is based on the opinion of the participants and the panel of experts and therefore it should be regularly counterchecked and revised along with evolving scientific evidence.

20These recommendations do not cover all aspects and techniques possibly amenable to OCT research and are based on expert opinion and a single consensus finding investigation rather than on a systematic review of a large body of literature. Therefore, they are not intended as an indispensable premise for all experimental OCT research. The APOSTEL recommendations are intended for clinical OCT studies using established techniques and help to provide the necessary comparability between studies.

21Some additions suggested during the revision process were not included in the final version as consensus was not reached. One of these suggestions was to incorporate a section on OCT-A. However, the inclusion of details pertaining to OCT-A in the APOSTEL 2.0 recommendations would be premature. The field of OCT-A, both clinically and academically, is in a phase of rapid evolution and essentially in its infancy. Its use is not well established in routine clinical care in either the fields of ophthalmology or neurology. Interpretation of OCT-A scans across devices is challenging and standardized quantitative OCT-A metrics are lacking or vary across OCT platforms. Moreover, there is a lack of consensus regarding quality control criteria for image acquisition and the implementation of such standards as they pertain to OCT-A. These limitations are likely to change in the future. For these reasons, the evidence and corresponding investigative and clinical recommendations for OCT and OCT-A should remain on separate tracks.

22A future revision of the APOSTEL criteria likely will also need to consider the role of artificial intelligence–based data from image analyses.

23We present revised APOSTEL recommendations based on this investigation using a modified Delphi process that involves a broad group of experts. Therefore, the resulting APOSTEL 2.0 can be considered an expert-led guideline (evidence class C, GRADE criteria) covering all relevant aspects of quantitative retinal OCT research. It will be necessary to update these recommendations to new research and practices regularly.