Internet and Computer-Based Cognitive Behavioral Therapy for Anxiety and Depression in Youth: A Meta-Analysis of Randomized Controlled Outcome Trials

1Depression and anxiety are common in children and adolescents (in the following referred to collectively as youth) and often co-occur. Both disorders are associated with substantial burdens an increased risk for other mental disorders and they often tend to persist in adulthood.

2In the past decades, a number of psychological treatments for anxiety and depression in youth have been developed with demonstrated efficacy in a substantial number of clinical trials. Cognitive behavioral therapy (CBT) is generally regarded as the treatment of choice for depression and anxiety in youth.

3However, up to 80% of children and adolescents with mental health needs receive no treatment. The reasons include not only a lack of treatment availability, but also reticence to seek help because of perceived stigma associated with mental illness, discomfort discussing mental health problems, and / or a preference for self-help.

4Using the computer or the Internet to provide CBT may overcome some of the limitations of traditional treatment services. Advantages of computer and Internet-based CBT (cCBT) include (1) availability (2) anonymity, (3) accessibility at any time and place, (4) flexibility in self-direction and self-pacing, (5) reduced travel time and costs for both participants and clinicians, and (6) the appeal of interactivity and visual attractiveness of Internet-based programs. Given the Internet savviness of the younger generations, these advantages might be even more relevant in youths than in adults.

5cCBT is effective in the treatment of depressive and anxiety disorders in adults. For example, a systematic review of 19 randomized controlled trials evaluating Internet-based and other computerized interventions for adult depression symptoms in N = 2996 individuals found a mean effect size of d = 0.56. In another meta-analyses on self-help interventions for anxiety disorders, a pooled mean effect size of d =.90 was found across 27 randomized controlled trials evaluating Internet-based treatments for anxiety.

6Less is known, however, about the effectiveness of cCBT for anxiety and depression in youth. In a recent systematic review, only three randomized controlled trials were included. Hence, the authors could not calculate pooled effect sizes using meta-analytic techniques. With the recent addition of several new published trials, we performed a meta-analysis to evaluate the efficacy of cCBT for anxiety and depression in youth (up to the age of 25) in comparison to a non-active control condition within randomized controlled trials.

7Several strategies were used to identify relevant studies. Two independent assessors (Ebert & Zarski) searched three major bibliographical databases (PubMed, PsycInfo, Cochrane) by combining terms indicative of each of the disorders with terms indicative of psychological treatment and randomized controlled trials. The searches were performed December 04, 2013. We also checked the references of the identified studies for earlier publications. Details of the searches are provided in Fig. 1.

8We included (a) randomized controlled trials in which (b) a computer or Internet- or mobile-based (c) cognitive behavioural intervention (d) targeting either depression, anxiety or both (e) in children or adolescents up to the age of 25 (f) were compared to a non-active control condition (wait-list; placebo). We only included studies (g) that used an anxiety or depression measure as the primary outcome measure and (h) in which participants were selected because of elevated levels of depressive/anxiety symptoms based on either a standardized self-report measure or a formal diagnosis of major depressive or any anxiety disorder (including OCD), and (i) effect sizes could be calculated from the information provided in the paper. If effect sizes could not be calculated, authors were contacted to retrieve the necessary information. Studies were not included when they had only an active treatment comparison (e.g., face-to-face CBT) or only included participants older than 18 years. Comorbid general medical and psychiatric or substance/alcohol use disorders were not used as an exclusion criterion. No language restrictions were applied. Studies that evaluated interventions in which the Internet/computer was not the primary treatment modality (e.g., blended treatments with face-to-face psychotherapy and some online elements) were excluded.

9Risk of bias assessment and data extraction was conducted independently by two assessors (Ebert & Zarski). We assessed the validity of included studies using four criteria of the Risk of Bias assessment tool, developed by the Cochrane Collaboration (adequate generation of allocation sequence, the concealment of allocation to conditions, masking of assessors, and the missing data handling (the assessment was positive when intention-to-treat analyses were conducted)). We examined the relationship between risk of bias and the effect size by performing meta-regression techniques. In these analyses, the total bias score was entered as the dependent variable. We compared effect sizes of studies rated as low-risk of bias (fulfilling all four assessed criteria) compared to studies with some risk of bias (fulfilling less than four assessed criteria). We also identified and coded additional aspects of the included studies, relating to the study design, participants and interventions.

10Variables related to the study design included the target condition (anxiety vs. depression vs. transdiagnostic) and the comparison condition (no treatment vs. placebo). The comparison group was coded as placebo control group if cCBT was compared to an active condition with no intended therapeutic properties that would be expected to produce change in the target diagnosis.

11We also coded the outcome informant (youth, parent, teacher), that is, whether the information about symptoms were derived from the youth report, parent report, or teacher report. We further coded the type of outcome assessment (self-report vs. observer based) and whether the studies included a follow-up (yes/no; duration).

12In line with previous meta-analyses on psychotherapy in children and adolescents, we classified studies as child, adolescents or mixed studies. Studies in which all participants were 13 years-old or younger were classified as child, studies in which all participants were over 13 years-old were classified as adolescent, studies that included both children and adolescents were classified as mixed. The mean age was also coded, and the percentage of boys was included. We further coded whether studies included only participants with a confirmed diagnosis of depression or an anxiety disorder using an established diagnostic interview (yes/no).

13Interventions were classified as with or without parental involvement (yes/no) and with or without professional guidance (therapeutic or simply administrative) (yes/no). We also coded the technology on which the intervention was delivered (computer: cCBT; Internet: iCBT; mobile: mCBT), the setting in which the intervention was conducted (at home/not at home, e.g., clinic, school, university), as well as the number of treatment sessions.

14For each comparison between cCBT for anxiety or depression or both and a control group, we calculated the effect size indicating the difference between the two groups in change from baseline to post-test (Hedges’ g). To calculate pooled mean effect sizes, we used the Comprehensive Meta-Analysis (CMA) software (Biostat, Inc.). In the main outcome analysis, we examined the overall mean effects of cCBT on symptoms of depression or anxiety. To generate a mean effect size, we only used instruments that related to the principal measure of the disorder. If interventions were transdiagnostic, targeting both anxiety and depression, we used the pooled effect size of these outcomes. If more than one measure per outcome was used, the mean of the effect sizes was calculated, so that each study provided only one effect size. Given that youth depression and anxiety symptoms highly correlate and in light of the current debate over whether depression and anxiety require separate treatments or can be treated by combined interventions, we also examined the overall effects of all interventions on symptoms of anxiety and depression in a separate analysis.

15We expected considerable heterogeneity among the studies, hence we employed a random-effects pooling model (DerSimonian-Laird) in all analyses. Given the difficulty of interpreting the standardized mean difference (Hedges’ g) from a clinical perspective, we transformed these values into the number-needed-to-treated (NNT) using the formula provided by Kraemer and Kupfer. The NNT indicates the number of patients that must be treated to generate one additional positive outcome.

16As a test of homogeneity of effect sizes, we calculated the I 2 statistic as an indicator of heterogeneity in percentages. We calculated 95% confidence intervals around I² using the non-central chi-squared-based approach within the heterogi module for Stata. We also calculated the Q statistic but only reported whether the result was significant.

17Publication bias was tested by visually inspecting the funnel plot on primary outcome measures. We also conducted Egger’s test to quantify the bias captured by the funnel plot and to test whether it was significant. The Duval and Tweedie trim-and-fill analysis was performed to further verify an unbiased estimate of the pooled effect size. This method calculates an estimation of the number of missing studies and the effect that these studies might have had on its outcome.

18We further conducted a series of subgroup-analyses, according to the mixed-effect model. In this model, studies within subgroups are pooled with the random-effects model, while tests for significant differences between subgroups are conducted with the fixed-effects model.

19Thirteen studies met inclusion criteria for the meta-analysis of cCBT for depression and anxiety in youth. Fig. 1 presents a PRISMA flowchart describing the inclusion process. Two reviewers (Ebert & Zarski) independently reviewed studies for inclusion or exclusion. Agreement between raters on inclusion was 100%.

20The 13 studies on cCBT for symptoms of anxiety and depression in youth included 796 participants, 420 in the treatment groups and 376 in the control groups. Selected characteristics of the included studies are displayed in Table 1. Seven studies were directed at treating anxiety, four at depression, two were transdiagnostic and targeted both depression and anxiety. Two studies were aimed at children (< 13 years), six studies at adolescents (≥ 13 years), five studies had a mixed sample. About half of the studies (n = 6) included only participants with a diagnosis, confirmed by an established diagnostic interview. The intervention in the majority of studies could be completed in the home of participants (n = 9).

21A non-treatment comparison was used in 10 studies, the other three studies applied a placebo control. The majority relied on a multi-method outcome assessment approach (i.e., observer-based outcome interview and self-report measure, n = 8). Only five studies used only self-report measures. Outcome informants were both youths and parents in seven studies and youth-only in six studies. Studies using youth-only as outcome informant were mainly directed at adolescents (n = 4), two studies on mixed-age range samples. Many studies included a follow-up assessment. However, only two studies assessed a follow-up in both the intervention and control group. Thus, we were not able to examine long-term effects of treatments.

22In most studies, the intervention was delivered via the Internet (n = 8), followed by computer (n = 5). The number of treatment sessions ranged from 6 to 14. Six studies were classified as brief treatment (6–7 sessions) and seven as long treatment (≥ 8 sessions). Parents were involved in the intervention within six studies and the majority of interventions included guidance by a professional (n = 11). Four Studies were conducted in the USA and Australia, respectively, two studies were conducted in New Zealand, two studies in Sweden and one study was conducted in the UK and the Netherlands, respectively. Thus, all studies were conducted in high-income countries. Cohen's kappa for agreement between independent raters was very good (κ =.84).

23Overall risk of bias was low. Eleven studies reported an adequate sequence generation. Twelve studies reported blinding of outcome assessors or used only self-report outcomes, whereas five did not report blinding. Only eight studies reported the information needed to clarify whether allocation to conditions was performed by an independent (third) party. Ten studies reported adequate handling of missing data, using intention-to-treat principles. Seven studies met all four quality criteria, four studies met three of four criteria, and two studies met only one criterion. Inter-rater reliability between independent raters on the risk of bias was very good (κ = 0. 94).

24The overall mean effect size of cCBT on symptoms of anxiety or depression when compared to a control group at post-test was g =.72 (95% CI: 0.55 to 0.90; p <. 001; Table 2). This value corresponded to the number needed to be treated to achieve one additional positive outcome of 2.56. Heterogeneity was low (I² = 20.14%, 95% CI: 0% to 58%). Fig. 2 provides a forest plot of the effect sizes per study and the corresponding 95% CI.

25Excluding studies that also assessed participants above 18 years of age resulted in an effect size of 0.61 (95% CI: 0.55 to 0.90). Excluding the study with the lowest effect size further reduced heterogeneity (I² = 0, 95% CI: 0% to 58%), while excluding the study with the highest effect size did not change heterogeneity substantially (I² = 15.11%, 95% CI: 0% to 55%).

26Including also anxiety outcomes of interventions targeting depression, the effect size on anxiety was g = 0.65 (95% CI 0.40 to 0.90) with an NNT of 2.82. Including also depression outcomes of interventions targeting anxiety resulted in a pooled effect size of g =.56 (95% CI 0.31 to 0.82), NNT = 3.25.

27We conducted a series of subgroup analyses (Table 2). Effects in subgroups were all in favor of cCBT and all significantly different from zero. Moderator analyses revealed that age group significantly moderated treatment outcome, with studies aimed at adolescent achieving better results (g = 0.95, 95% CI 0.76 to 1.17) compared to studies aimed at children (g = 0.51, 95% CI: 0.11 to 0.92) and mixed samples (g = 0.48, 95% CI 0.25 to 0.71). Given that some meta-analyses on treatments of anxiety disorders in youth categorized children up to 13 years of age as children, we repeated the analysis classifiyng the study of Khanna as directed at children. This result did not produce a different outcome pattern, with slighty higher effects for children (g = 0.56, 95% CI 0.21 to 0.91, NNT = 3.25) but still a significant difference in effectiveness between age groups (p =.007). We found that target condition, confirmation of disorder using a diagnostic interview, parental involvement, or risk of bias was not significantly associated with the size of the effect (Table 2).

28The inspection of the funnel plot and Egger’s test indicated some possible publication bias. However, after adjustment for missing studies using the Duval-Tweedie trim and fill procedure (four imputed studies) Hedge’s g for the main outcome analyses was 0.64 (95% CI 0.46 to 0.82), corresponding to an NNT of 2.86.

29We found cCBT for youth was associated with significant and moderate to large effects on symptoms of anxiety and depression, with an NNT of 2.56. Overall, risk of bias was low and the seven studies that met all quality criteria showed a significant effect comparable with the overall effect size. Heterogeneity was low in most analyses, suggesting that most studies pointed in the same direction, with no major outliers. We found some indications for publication bias, but adjusting effect sizes resulted in no major changes.

30Effects sizes were slightly lower than those found for cCBT for anxiety and depressive disorders in adults (g = 0.88), were comparable to those found in recent meta-analyses on traditional face-to-face CBT for anxiety disorders in youth (0.66) and somewhat higher as those found for CBT for depression in youth (0.35). However, we also included studies with mixed samples that also included participants above the age of 18. Removing these studies resulted in an overall ES of 0.61 and NNT of about 2.99, still in range of ESs of face-to-face CBT. The comparable effects to face-to-face psychotherapy are consistent with research on cCBT for psychiatric and somatic disorders in adults, showing no differences between internet-based CBT and face-to-face treatments.

31We also found significant, moderate but somewhat smaller effects compared to the main-outcome analysis on symptoms of depression when including also outcomes of interventions targeting anxiety. Likewise, we also found significant and moderate effects on symptoms of anxiety when also including outcomes of interventions targeting depression. These findings are consistent with growing evidence indicating that (a) youth depression and anxiety are closely associated, (b) psychotherapeutic treatment for youth can have significant effects on comorbid problems, and (c) may be treated simultaneously. In fact our results support such an assumption, with significant and large effects (g = 0.94, NNT = 2.02) for those treatments targeting both disorders at the same time.

32Participant age was the only significant moderator of treatment effects. The finding was consistent also when using different classifications of children (< 13 vs. ≤ 13 years of age). In line with the results of the most recent meta-analyses on psychotherapy for anxiety disorders in youth, we found that treatments for children were significant but with smaller effects compared to treatments for adolescents. However, meta-analyses on psychotherapy for depression in youth have not found such an association.

33We found no association between the inclusion of parents and better treatment outcomes. Although it is often argued that parental involvement is important in the treatment of anxiety and depressive disorders in youth, our results are consistent with the meta-analyses of Reynolds and colleagues, showing no additional benefit of treatments with parental involvement.

34When interpreting results of this study, several limitations should be considered.

35First, 11 of 13 studies reviewed reported no follow-up assessment with treatment versus control comparison. Hence, we could not examine long-term effects of treatments. This shortcoming points to the need for future studies with follow-up assessments. Second, the interventions evaluated in the studies were very heterogeneous regarding format of treatment, ranging from therapist focused group-based chat interventions based on face-to-face treatment manuals to unguided more serious gaming-based approaches. Given the large variability and the limited amount of studies available, we could not examine the differential effectiveness of treatment formats. Third, also because of the limited number of studies, we were not able to perform subgroup analyses with adequate power separately for depression and anxiety or other potential subgroup analyses of interest. Fourth, most studies targeting depression excluded participants with severe depression. Consequently, our results should not be generalized to youths with severe depression. Fifths, the present study did not search for unpublished studies. Thus it may be possible that there are unpublished studies that are not included here, which may lead to an over- or under-estimation of the true intervention effect sizes. Sixths, true levels of heterogeneity in meta-analyses cannot be detected with currently available statistical methods. Although we applied a number of state-of-the art methods to estimate heterogeneity (i.e. I² including 95%-CIs, Q statistics, sensitivity analysis excluding studies with the lowest/highest effect size), we cannot rule out that heterogeneity exits and the reported effect sizes are over- or underestimated. Finally, all studies were conducted in high-income countries. The generalizability of the study findings for low- and middle income countries is therefore limited.

36Findings from the present study provide evidence for the efficacy of cCBT in the treatment of anxiety and depressive symptoms in youth. Hence, such intervention might be a promising treatment alternative when evidence based face-to-face treatment is not available or simply not wished. Applying cCBT at large scale may thus serve as one (out of several) strategies to bridge the enormous gap between the need and provision of evidence-based treatments for anxiety and depression in children and adolescents. Given that access to evidence-based psychotherapeutic treatment is not only a problem in high, but even more in low- and middle-income countries, future studies should evaluate the potential of cCBT effectiveness for youths in non-high income countries. Subsequent studies should also examine the long-term effectiveness of cCBT and evaluate potential negative effects of cCBT. Findings also indicate that more research is necessary to draw clear conclusions with regard to moderators. Finally, this meta-analysis was limited to studies including participants with elevated symptoms, excluding studies directed at the prevention rather than treatment of symptoms (e.g.). Given the potential scalability of Internet-based treatments, there might be a large potential for their use in the prevention of common mental health disorders, which should also be examined in future studies.