What Works and What Doesn’t Work? A Systematic Review of Digital Mental Health Interventions for Depression and Anxiety in Young People

1In Australia, approximately 8% of young people between 11–17 years of age meet the DSM criteria for major depressive disorder (MDD), while about 20% report high levels of psychological distress. The rates of MDD may be as high as 11% in youths in the U.S.. In fact, suicide is the second leading cause of death among 15–29-year-olds globally. In addition, depression is highly under-diagnosed and thousands who fall outside these statistics experience its debilitating effects on functioning at an important developmental stage. Depression and other mental illnesses affect the social and intellectual development of young people, reducing engagement with education, and if untreated, can become lifelong disabilities.

2Despite the importance of addressing mental illness early only 20–40% of youths in need in Australia and 25% of youths in the U.K. receive professional help. This low engagement with mental health services appears to occur for a variety of reasons: the lack of motivation inherent in conditions such as depression, low rates of mental health literacy, and the stigma, discrimination and embarrassment surrounding mental illness. Young people are also still developing skills in executive functioning such as self-monitoring and organization, which are necessary to identify a mental health problem and obtain support.

3Although few young people seek professional help, during episodes of depression consumption of media such as music, internet, and television increases. Thus Digital Mental Health Interventions (DMHIs) are increasingly of interest as a solution to the low help-seeking and uptake rates of professional mental health services. Studies tend to support the effectiveness of self-help mental health programs whether digital or otherwise, which can be as effective if not more effective than face-to-face delivery. Numerous studies have demonstrated the usefulness of web-based programs. Young people report feeling more comfortable discussing sensitive and personal issues in the relative anonymity of an online context and use the internet as a major source of mental health information. Mobile “apps” are proving particularly useful for administering DMHIs because of the widespread ownership of mobile phones, with the majority of young people in the U.S. reporting almost constant usage of smartphones. Several reviews of smartphone applications for mental health across age groups have reported positive benefits.

4Notably, however, many apps for depression and anxiety that are currently available are not evidence-based and may thus actually be harmful to people with mental illness. Even among those purporting to be drawn from evidence-based therapies such as cognitive behavioural therapy, only a small percentage actually contain the core principles of those therapeutic traditions. Furthermore, Hollis and colleagues in their meta-review reported that while there is some evidence in support of the effectiveness of DMHIs for depression and anxiety in young people, studies are methodologically limited making it difficult to draw clear conclusions. Furthermore, they suggest the need for identification of the components that make DMHIs effective such as human interaction.

5In fact, human interaction has been identified as an important factor influencing effectiveness and engagement with DMHIs, but it may also detract from the cost-effectiveness of DMHIs in comparison to face-to-face treatment. Furthermore, other disadvantages to the inclusion of social features exist, such as unhelpful advice from peers, and the possibility that some youths may feel afraid to share personal problems even anonymously.

6The aim of the current review, therefore, is to examine the literature about DMHIs to address mental health in young people. We have focused on depression and anxiety as these are among the most prevalent mental health conditions experienced by young people and often co-occur with many other disorders. Specifically we aimed to investigate:

7Do DMHIs reduce anxiety and depression in young people aged 12–25 compared to no intervention or an active control group?

8How effective are DMHIs in reducing anxiety and depression in young people when interaction with the intervention is unsupervised?

9What features and components of DMHIs are most liked or disliked by young users?

10Given the focus in the current review on both effectiveness and engagement, it was expected that the literature reviewed could include both quantitative and qualitative data. Therefore, a mixed methods approach was selected. Mixed methods reviews attempt to combine looking at ‘what works’ with ‘how and why it works’, combining varied research methods in the analysis or in the types of studies reviewed. Mixed method reviews can provide a more holistic understanding than a meta-analysis alone since they are able to integrate a wider variety of studies and provide insights into mechanisms and processes.

11A systematic search was conducted in PsychInfo, PubMed, Proquest, and Web of Science using the following terms in the title, abstract and subject descriptors: “mobile”, “application”, “smartphone”, “mobile phone”, “cell phone”, “text message”, “internet-administered therapy”, “computer-aided therapy”, “online” AND “depression”, “anxiety” AND “youth”, “young person”, “adolescent”. The initial search returned 4,828 articles. With duplicates deleted this number was reduced to 3,352.

12Identified references were screened according to the following inclusion criteria: (1) participants aged 12 to 25, (ii) interventions targeting depression or anxiety, (iii) interventions delivered by computer, on smartphones, or online, (iv) studies published between 2007 and 2017. We also excluded reviews, opinion, or discussion pieces and unpublished works. While quality assessment was part of the review process, studies were not excluded on the basis of study type or quality since the mixed methods approach taken allowed the inclusion of varied methodologies.

13After title screening by 1 researcher 184 abstracts were uploaded to Covidence, an online platform for conducting systematic reviews (http://www.covidence.org). These were scrutinised by 2 researchers. Once agreement was reached on eligibility, 68 articles remained. Full text appraisals were then conducted by 2 researchers and a further 27 articles not meeting the inclusion criteria were excluded, leaving 41 in the review (Figure 1 and Supplementary Materials).

14Two researchers independently conducted the data extraction. Information about the characteristics of the studies, participants, interventions, and final outcomes were entered into a pre-established template in Covidence. Qualitative data from the studies relating to usability and appeal of the interventions was also extracted. The quality of studies was assessed using the Joanna Briggs Institute’s critical appraisal tools and the CONSORT-EHEALTH Checklist (V.1.6.1). Studies were considered methodologically sound if they had a matched control group, pre-post data, and randomization, and were only included in the meta-analyses if they met these criteria. However, since this is a mixed-methods review, studies not meeting the criteria for inclusion in the meta-analysis were retained in the study to form part of the narrative synthesis. Risk of bias was assessed using the standard Cochrane Risk of Bias tool in Covidence, which poses questions about aspects of trial design, conduct, and reporting which the user rates as ‘Low’ or ‘High’ risk of bias for each study. Single cohort studies and RCTs were assessed using different indicators of bias under the 5 categories of possible bias as outlined in the Cochrane Handbook of Systematic Reviews, and a overall assessment of bias as ‘Low’ or ‘High’ given for each study. Where not enough details were reported in the study to assess risk of bias, this was labeled ‘Unclear’. Again, however, studies with some risk of bias were not excluded from the analyses but sub-group comparisons were conducted to assess whether results differed depending on bias.

15Due to a lack of comparable randomized controlled trials (RCTs) for anxiety, the meta-analysis focused on DMHIs targeting depression. For each RCT of a DMHI for depression the effect size indicating the difference between the intervention group and the control group at post-test was calculated in OpenMeta Analyst. Effect size calculations (Cohen’s d or standardized mean differences) were conducted using the means and standard deviations of post-test scores on instruments measuring symptoms of depression such as the depression subscale of the Depression, Anxiety, Stress, Scale (DASS, ) or the Centre for Epidemiological Studies Depression Scale (CES-D, ). Scores on these measures immediately after the study were used rather than longitudinal follow-up scores due to a lack of consistent reporting across the studies. Sample sizes used were the number of participants with complete data in each group rather than intention-to-treat numbers. If effect sizes could not be calculated due to a lack of information, the study was excluded from the effectiveness analysis. Pooled mean effect sizes were calculated using the random effects model as considerable heterogeneity between studies was expected. These were calculated separately for studies using a no-intervention control group and those using an active control group. Subgroup analyses were conducted to investigate the impact of other variables on effectiveness by testing for significant differences between subgroups using a fixed effects model. In particular a sub-group analysis according to risk of bias was conducted to assess how study design contributed to outcomes. Since one area of interest in this review was to investigate whether DMHIs are useful for the high percentages of young people not obtaining professional help, we also conducted a sub-group analysis according to the level of supervision or interaction involved in the treatment condition.

16All studies including those not in the meta-analysis were evaluated using a narrative synthesis model. Qualitative data included both results of interviews with participants, and descriptions of survey results relating to the appeal of DMHIs. For example, some studies reported direct comments from participants about their experiences using the DMHI, while others reported levels of agreement with statements about appeal and quality. These data were coded by 2 researchers using standard techniques for thematic analysis to generate an understanding of features that are appealing and aspects that promoted engagement and adherence. Codes were grouped to detect patterns, and themes were identified and defined. Once consensus was reached the lead author prepared a narrative analysis, which was checked independently by the other authors.

17The 41 studies included 11 from Australia, 10 from the U.S., 12 from other English speaking countries, 5 from Northern Europe, 2 from Asia, and one from South America. The majority of studies were RCTs (n = 27) (Table 1), while 13 were single cohort studies (including 4 with pre-post designs), and one used a case study methodology. Most of the studies used participants recruited from educational institutions (n = 20), 9 were conducted in mental health care settings, 4 with the general community, 4 in primary care settings, 2 in hospitals, and one in a youth organisation. One study did not report recruitment methods or participant information in enough detail to determine the setting.

18Studies included participants with no specific mental health symptoms at baseline (n = 12), some with varying levels of depression (mild to moderate n = 7, moderate to severe n = 3, severe = 1, all levels n = 6), others with diagnosed MDD (n = 2) or suicidal risk (n = 5). Participants with varying levels of anxiety were also a focus in some studies (mild n = 1, mild-mod n = 2, mod to severe n = 1, severe n = 1, all levels n = 1). Two studies looked at people with a variety of mental illnesses, and another focused on mood issues in people with traumatic brain injury (Table 1).

19Overall, 32 different DMHIs were investigated across the 41 papers (Table 2). Several DMHIs were evaluated in multiple studies including Bite Back (n = 2), CATCH-IT (n = 2), Master Your Mood (n = 2), MoodGym (n = 3), Reframe-IT (n = 3), SPARX (n = 2), and MEMO (n = 2). Several of these papers reported results from the same data sets, but reported on different aspects of the study and therefore both papers were included in the review. Most of the DMHIs drew on established therapeutic models, primarily Cognitive Behavioural Therapy (CBT) (n = 28) or a combination of CBT with other models.

20The technologies utilized in the various DMHIs included some phone-based interventions such as text-messages (n = 4) and smartphone applications containing assessment tools and/or psychoeducational materials (n = 3). The majority of DMHIs were web-based (n = 30), including many with online modules, learning materials or activities (n = 24), group chats or courses (n = 2), online forums (n = 2), and online chat facilities with a mental health professional (n = 2). Others were computer-based but not online, including games (n = 5) and psychoeducational computer programs (n = 2).

21Many DMHIs included learning modules (n = 18), interactive learning activities (n = 6), psychoeducational materials in a variety of formats including text and video (n = 7), or game-based learning activities (n = 4). Additional features included regular inspiring messages (n = 1), challenges (n = 3), mood tracking (n = 3), or diary/journals (n = 2). Four studies included DMHIs with an accompanying workbook for participants or their parents. Only 11 of the studies reviewed included DMHIs that were entirely self-help and were completed in the participant’s own time. The rest of the studies involved interaction with a mental health professional or completion of the intervention in some kind of supervised setting. Two of these were completed in hospitals, one with some minimal supervision from a health professional. Others were completed in a school setting (n = 10). Some of the studies completed at school involved a high level of supervision (n = 5) such as in studies where the intervention was presented by the school wellbeing staff, the classroom teacher, in the presence of the research team, or in class groups. Other school-based studies involved lower levels of interaction with a therapist or the research team (n = 3) such as moderation of an online discussion board by a therapist, initial training completed at school but the intervention otherwise used in the students’ own time, and where the intervention was accessed at school with minimal supervision by the school counselor. Studies outside of school settings included DMHIs that could be completed at home in the participant’s own time, but included interactions with a therapist such as sending reminders or text messages (n = 4), or participating in online group courses or chats (n = 9).

22The effectiveness of various DMHIs in treating symptoms of depression was compared to a control group in 15 studies (Table 3). Nine of these studies compared DMHIs to no intervention (a waitlist control group), while five of the studies compared DMHIs to an active control in which some alternative online materials were used, including one with some psycho-educational content, and one contained a Treatment As Usual (TAU) comparison group in which face-to-face counseling was offered. Since the TAU group in this case included active treatment it was combined in analysis with the active control groups. The pooled effect size of studies comparing the intervention group to a no-intervention group (n = 9) was 0.33 (95% CI 0.11 to 0.55) (Figure 2), suggesting that DMHIs have a small effect size when compared to a no intervention control group, while the pooled effect size of studies comparing the intervention group to an active control group (n = 6) was 0.14 (95% CI -0.04 to 0.31). Thus this review did not find a difference in outcomes between DMHIs and active controls, including a mixture of usual care for depression and non-depression specific interventions (Figure 3). Heterogeneity was relatively high (I2 = 70%) and statistically significant (p <.001). Two studies had negative effect sizes indicating that the control group had lower depression scores at post-test than the intervention group. Reid and colleagues evaluated the effectiveness in comparison to a waitlist control group of a smartphone application that allowed self-assessment on 8 domains of mood and functioning, referring this information to general practitioners for medical review. No significant effect on depression was found at post-test, but increased emotional self-awareness was reported. Whittaker and colleagues similarly used a phone-based approach, delivering multimedia messages based on CBT and comparing this to use of similar multimedia messages with no focus on depression. The authors concluded that there was no evidence of benefits superior to the active comparison program with content about healthy behaviours.

23Sub-group analyses were conducted to investigate the effect of therapist interactions and study completion settings on the outcomes in the 9 studies that compared the intervention to no intervention (Table 3), and the 6 studies comparing the intervention to an active control group. Studies were categorised as having High levels of human interaction (H) if they involved direct contact with a therapist or were completed in supervised settings such as a lab, clinic or school. They were categorised as Low interaction (L) if they had some limited interaction such as regular emails, text messages, or optional opportunities to contact a therapist, or No interaction (N) if they did not involve any interaction with a mental health professional and were completed unsupervised in personal time.

24For studies comparing the DMHI to no intervention, the pooled effect size was smallest in the No interaction group (d = 0.04), and also small in the Low interaction group (d = 0.16), while the High interaction group returned a medium effect size (d = 0.52) (Figure 4). This indicates that DMHIs were mostly effective when they involved high levels of human interaction. The DMHIs in the No interaction group that did have a positive effect size were highly interactive, containing multimedia lessons, interactive online exercises, and game-based challenges and puzzles aiming to improve mental health literacy. Levin and colleagues did not include direct conversations with mental health professionals. However, the system did send automatically generated emails that were customized based on participants’ earlier input. This could have given the illusion of human interaction, thereby increasing the effectiveness of the program despite there being no direct personal communication. No significant differences were found according to intervention type or severity of symptoms at baseline.

25Similarly, for the studies with active comparison groups, the pooled effect size was again smallest in the No interaction group (d = 0.08), and also small in the Low interaction group (d = 0.21), while the High interaction group returned a medium effect size (d = 0.49) (Figure 5). Both of the studies in the High interaction group were completed in school classroom settings. Thus, across both active and no intervention control groups, effect sizes reached a moderate size only when there was a high level of therapist interaction or supervision in the study design.

26The proportion of studies at high/unclear risk of bias was: 34% selection bias (e.g. randomization or allocation concealment), 63% detection bias (e.g. blinding of outcome assessment), 41% attrition bias, and 31% selective reporting (Table 4). Overall 76% of the studies included in the review as a whole, and all but 4 studies included in the meta-analysis had some risk of bias. Sub-group analyses were performed on the 15 studies included in the meta-analyses according to type of bias. Significant differences were found for selection bias with the low risk of bias group having a lower pooled effect size (d = 0.01, 95% CI -0.05 to 0.24) than the unclear or high-risk group (d = 0.44, 95% CI 0.22 to 0.69). Studies with low risk of detection bias also had a lower pooled effect size (d =.09, 95% CI -0.08 to 0.26) than the unclear or high-risk group (d =.40, 95% CI 0.18 to 0.62).

27Attrition rates were defined as the number of participants who completed the study as a percentage of the participants who commenced the intervention (Table 5).Information about adherence and engagement (how much those who completed the study engaged with the intervention), tended to be reported differently across papers. For example, some papers reported module completion rates. Others reported time spent on a website. Overall 16 (39%) of the studies had attrition rates over 20%, the level broadly considered indicative of possible attrition bias (Table 5). In several of these studies, while attrition rates were high, they were equal between groups (for e.g. ) suggesting that drop out rates related more to recruitment methods than to non-engagement. However, this was not always the case and in many studies, even among those with low study attrition, engagement tended to be low, with participants completing less than half of the intervention components. The majority of these studies were again ones that involved completion in one’s own time. For example, Ip and colleagues (2016) reported low drop out rates and a small effect size. However, participants only completed roughly three of 10 modules and spent about 39 minutes on the website over 4 months, suggesting relatively low engagement.

28Qualitative data from all papers were categorized according to features of the DMHIs that were liked by participants, features that were disliked (Table 6), and features predicting adherence. There were four key categories of data in relation to liked features. The first category related to social support. Several studies reported that participants had found it useful to be in contact with professionals. Participants in one study who had access to a trained supporter in addition to regular text messages reported that they “liked talking to someone who was friendly” (, p. 101). Participants in one study of a game-based CBT intervention called Pesky gNATs reported liking the fact that doing it on a computer was “not as full on as face-to-face” (p. 15). However this was not perceived by all to be preferable to in-person contact. For example, two studies that evaluated an interactive CBT-based fantasy game called SPARX, both found that some participants preferred face-to-face support from a therapist. Similarly, some participants in an online CBT-based group course reported a preference for a face-to-face version of the course.

29For other participants, it was the opportunity to connect with peers who were experiencing similar difficulties that was helpful. Gerrits and colleagues reported that participants “found chatting to be a pleasant and positive way to talk about being down and their feelings of depression” (p. 6). Similarly, Horgan and colleagues studied the impact of an online forum and reported that participants found it “good to say what was going on aloud (albeit in writing)” (p. 87). One participant in the same study stated: “Its about empathy and the realization that you’re not alone. That others are feeling the same way you do and are having trouble coping” (p. 87).

30For some participants the primary attraction of DMHIs were their online or computer-based nature. In contrast to users noted above who reported a preference for face-to-face contact, for many users a key benefit of DMHIs was the privacy they afforded. One participant in an online self-help program, Crystal, stated: “You can kind of do it in a secluded area where nobody is watching you … the privacy is kind of like a really big appeal” (, p. 28). In one study of an online peer-support website, anonymity allowed participants to share details that they had never shared before, and in fact had “put a lot of effort into hiding” (, p. 87). DMHIs also had the advantage of fitting into the daily routines of users, connecting with current interests, and helping “to bring back a sense of normality” (, p. 290). A participant named Rob stated: “Most teens are always on the internet … while you’re on say Facebook or something, you can just open up another tab” (, p. 28) It can also be accessed from a variety of locations such as school, home or in a clinic and participants could “learn by myself and at my own pace” (, p. 7). Participants reported that it was “fun to be able to do it on a computer” (, p. 13).

31Other participants commented on particularly useful content. Participants reported that the DMHIs “showed me things I didn’t know” (, p. 7), and helped them learn more about mental health. Appreciation was expressed for content that helped participants to learn specific techniques such as problem solving and anger control, or challenging negative thoughts.

32Another major category of the data related to the look and feel of the DMHIs. Participants preferred situations, characters, or avatars that were relatable. For example, participants reported that it was helpful when the focus was “situations any teenager goes through” such as school and interpersonal relationships (, p. 27). Conversely, several studies reported that drop-outs occurred when the content “did not seem relevant for them” (, p. 7). Other features that participants reported liking included interactive activities, and video components. Similar comments were made in relation to DMHIs with a game-like feel. Participants stated that this made engaging with the DMHIs fun. It was also important to the users that DMHIs were aesthetically appealing and easy to use and navigate.

33One of the most prominent features that participants reported disliking was the educational content of many DMHIs. For example in their case study of use of a smartphone app for anxiety, Whiteside and colleagues reported that the participant “appeared less engaged and interested in the background educational content” (p. 86). Multiple other studies reported similar comments by participants, particularly non-completers. Educational modules were viewed as too long, “tough and sometimes quite tiring” (, p. 6), “tedious and laborious” (, p. 6). Some participants argued that it would be more convenient to be able to tailor modules to one’s own needs: “I didn’t like that you couldn’t skip out of something if you already understood the concept”(, p. 163). Burckhardt and colleagues (2015) suggested that more structured settings and dose effects may have contributed to their negative results, since other studies have found that the number of activities participants are required to do can reach a saturation point.

34A noteworthy point was that numerous participants reported that the DMHIs often felt too juvenile or patronising. Participants did not enjoy using DMHIs that seemed like they were designed for younger children. One participant suggested: “make it more grown up” (, p. 14).

35Technical glitches and difficulties navigating sites were also frequently cited as reasons for low adherence and engagement. Participants stated that DMHIs should be improved to make them “comparable to commercially available games” (, p. 290). Others reported disliking particular aesthetic features such as the colour scheme and a lack of variety of icons, cartoons, and diagrams.

36Factors predicting adherence. Only four studies reported predictors of adherence to the DMHIs. Neil and colleagues compared a school-based completion setting for MoodGYm to a community setting, finding that a school-based setting predicted greater adherence. Gender was also a consistent predictor of adherence, with females being more likely to complete compared to males. Mental health also played a role, with higher pre-test scores in depression, a longer history of mood disorders, or low scores in anxiety at pre-test predicting greater adherence.

37This review aimed to determine the types of DMHIs that are effective in treating depression and anxiety in young people and the components of these interventions most associated with positive outcomes and engagement. Overall, studies in relation to depression demonstrated a small effect size in favour of DMHIs when interventions were compared to no intervention. While this might not always reflect a clinically significant level of change, it suggests that such DMHIs may be of value in the context of public health and preventative interventions. On the other hand, studies comparing DMHIs to active control conditions were not effective. In fact, in two studies the control group actually had lower depression levels at post-test than the intervention group. Both studies included phone based interventions: an app that referred users for medical review, and a program of multimedia mobile phone messages. However, given that there was a risk of bias in many studies included in the meta-analyses, these results should be interpreted with caution. In fact, the two studies in the meta-analysis that reported negative effect sizes were two of the only three studies assessed as having low risk of bias. Studies which did not involve blinding of either group allocation or of outcome assessment tended to have higher effect sizes than studies with low risk of bias, indicating that methodological limitations of the studies reviewed likely inflated the larger effect sizes.

38Of further importance in our findings was the fact that only DMHIs involving regular interactions with a therapist or that were completed in a supervised setting reached a moderate effect size in comparison to a no-intervention control group, while DMHIs that involved educational programs completed in the participant’s own time were not found to be effective in this study. This suggests that currently available DMHIs may not be effective in causing clinically detectable levels of change unless they involve a high level of supervised use or therapist involvement. These results reflect the overarching significance of human interaction in psychological interventions. However, the preference among some participants for human contact revealed in this review existed in tension with the need for privacy and anonymity, suggesting that there is a need for more effective design of DMHIs to fill a gap that traditional face-to-face therapies do not.

39Despite this, adherence and engagement rates tended to be low in many studies particularly those where interventions were completed in their own time. This reinforces the idea that many DMHIs are most likely to be useful for people already receiving mental health support or at least those not averse to doing so. However, DMHIs completed in settings such as schools, labs, or clinics cannot reliably indicate their effectiveness in reaching young people outside of these settings. Where DMHIs were completed in their own time or did not include interaction with a mental health professional, effectiveness was much reduced. Nevertheless, for many users it is the anonymity and privacy afforded by the online context that holds the greatest appeal. Therefore, there is a need to balance the competing advantages of anonymity and social support. Other studies have similarly concluded that social networking features in DMHIs are a “gamble” due to the potential for both negative and positive effects.

40These results indicate two distinct needs in DMHI development. Firstly, a dire need exists to increase the appeal of DMHIs so as to reach the 80% of young people who are not already obtaining professional help. These young people may not understand that their symptoms indicate the need for mental health assistance. Other barriers such as a lack of energy or motivation to engage with complex tasks, or a fear of the stigma of mental illness may prevent them from accessing even DMHIs where such are overtly about mental health or are educational in nature. Research indicates that young men are particularly unlikely to receive professional help, which makes the findings in this review that males are less likely to engage with DMHIs than females particularly disturbing. The development of DMHIs that build on the existing interests of young people in non-confronting ways may be of more appeal to this group. DMHIs that particularly cater to the interests of young men are especially needed. Only highly interactive DMHIs involving multimedia materials or game-based activities were successful in studies with low levels of human interaction, suggesting that these types of features are highly appealing.

41Secondly, for DMHIs designed to help young people who are already seeking or willing to seek professional assistance but prefer to do so in the relative anonymity of digital settings, it is clear that the help of schools and mental health professionals is a crucial part of the roll out of such interventions. Similarly, some scholars have recommended a model of ‘supportive accountability’, in which accountability to a supporter or coach can enhance adherence to eHealth interventions.

42The need for further refinement to available DMHIs was confirmed by our qualitative analysis which revealed factors involved in high drop out rates and low engagement rates in numerous studies. In general, participants liked the online or computer based formats and game-like feel of some DMHIs, particularly when the content was interactive, had relatable situations or characters, and had appealing aesthetic features. However, feedback by study participants or people who withdrew from the studies referred to the boring nature and hard work involved in the online learning modules. Participants found DMHIs with non-appealing interfaces, frequent technical glitches, or material that seemed too juvenile to be off-putting. Other studies have similarly found that making DMHIs easy to use and to navigate is important to users, and in fact criteria like this are commonly used to evaluate usability and appeal.

43Although some participants seemed to appreciate the opportunity to learn various psychological skills to improve their wellbeing, many disliked the high educational focus of the interventions, the fact that they seemed designed for children much younger, and that they did not match commercially available programs in quality. This highlights the need for future DMHIs to consider the opinions of young people closely in their design. There is a need for feedback from young people and co-design methods to ensure that both content and aesthetics are appealing to the target audience. There is also a need for DMHI developers to ensure that materials are not ‘dumbed down’ and that they are presented in a way that does not feel like hard work but that builds on the natural interests of young people. This again presents a challenge for developers, to balance the need for simplicity of use with age-appropriate content. This is especially important for DMHIs designed to address depression and anxiety since these conditions are associated with both a lack of motivation and impaired concentration, making such users a particular challenge to engage.

44The current study was limited by the search terms used. Future reviews should also include search terms such as “internet-delivered”, “computer”, or “computerised”, since this could have picked up a broader range of studies in the current review. Nevertheless, this review demonstrates that while somewhat effective for those who use them, DMHIs fail to appeal to a large proportion of young people. In fact, when compared to active comparison groups including online materials with no psycho-educational content, DMHIs had only minimally better effects. As yet there seems to be a dearth of DMHIs that are likely to attract the large numbers of young people with mental illness who are not already open to receiving professional help. There is thus a need for urgent attention to developing high quality DMHIs that address the weaknesses and focus on the strengths identified, to help young people currently in the shadows to access appealing and accessible tools for managing their mental health. There is also a need for methodologically robust double-blinded RCTs to be designed to provide more stringent testing of the effectiveness of such interventions.

45The datasets generated for this study are available on request to the corresponding author.

46All authors contributed to the conception and design of the review. SG, CM, and DC were involved in searches, quality assessment and data extraction. SG and CM conducted the analyses. All authors contributed to manuscript revision and read and approved the submitted version.

47The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.