Actissist: Proof-of-Concept Trial of a Theory-Driven Digital Intervention for Psychosis

1Psychosis onset typically occurs in early adulthood, a critical period for psychosocial development. Despite initial response to intervention, the early course of psychosis is characterized by repeated relapse, compromising functional and social development, service engagement, and the resilience of carers and services. Meta-analyses have shown that discontinuation of antipsychotic medication, substance misuse, family criticism, poorer premorbid functioning, and social isolation are firmly associated with relapse following a first episode of psychosis (FEP). Early intervention for psychosis services (EIS) exist worldwide and aim to provide both pharmacological and psychosocial interventions. Despite mandates published to address the treatment gap, timely access to these services is problematic.

2Health care providers worldwide are forming digital strategies for addressing mental health challenges, and self-management in long-term conditions is now a cornerstone of many national health policies. Smartphones are commonplace technology that can deliver unconstrained, real-time packages of care, extending the reach of health care delivery. Smartphone-extended care could drive improvements in quality, efficiency, cost, and access to treatment, while enhancing patient experience by: providing more choices over how health care is delivered; facilitating self-management; and assisting clinicians to gain a richer understanding of an individual’s day-to-day experiences by receiving real-time data that can be used to deliver ecologically valid treatment. Smartphone ownership rates in psychosis are comparable to the general population and use of smartphones for health care appears acceptable to people with severe mental health problems. Large-scale meta-analyses of randomized controlled trials (RCTs) have demonstrated that digital health interventions (DHIs) can provide effective treatment for mental health problems such as depression and anxiety. While a number of smartphone-delivered open trials have shown promising effects in reducing hospital admissions, improving positive psychotic symptoms, socialization, social connectedness, depression, and medication adherence (see review), a user-informed, theory-driven app tested in early psychosis has not been reported in the literature. As such, the lack of controlled trials in the field precludes firm conclusions regards feasibility, acceptability, and intervention effects in this group.

3We report here on Actissist, a DHI that is broadly designed to speed up recovery and improve quality of care. The Actissist system is unconstrained by traditional service settings and can be used for active self-management and facilitate shared-decision making about treatment. Grounded in the cognitive model of psychosis, and following an extensive period of co-design with patients and stakeholders, Actissist is more specifically a theory-informed smartphone app targeting key early psychosis domains. The system swiftly identifies and challenges unhelpful appraisals of psychosis-related experiences and provides alternative, more helpful coping strategies in the real-time context of one’s daily life. Actissist was informed by content described in various published academic works. We draw on experience sampling methodology to prompt participants to engage with the app and build on the clinical protocols described by Granholm et al and Ben-Zeev et al. Using an agile, iterative process of development, beta-testing and with user experience design (UX) in mind, Actissist targets 5 domains associated with early psychosis relapse: auditory verbal hallucinations; paranoia; perceived criticism; socialization; and cannabis use.

4The overarching aim of this Medical Research Council (MRC)-funded trial was to establish proof-of-concept evidence that the Actissist intervention is feasible and acceptable in early psychosis compared with a symptom-monitoring app as an active control condition. In line with MRC guidelines for developing complex interventions, the a priori focus of the trial was estimation of treatment effects. The study had 2 aims: (1) test the safety, feasibility, and acceptability of the Actissist intervention; (2) provide preliminary evidence of intervention effects on clinical and functional outcomes. This is the first RCT of a DHI targeting putative mechanisms for early psychosis against an active control condition.

5A single blind, proof-of-concept, pilot RCT of 36 early psychosis patients with random allocation using a 2:1 ratio to receive either Actissist plus treatment as usual (TAU; n = 24) or ClinTouch (a symptom monitoring app) plus TAU (n = 12) over 12 weeks. Trained researchers blind to treatment allocation completed study assessments at time 1 (baseline), time 2 (12-weeks, post-treatment), and time 3 (22 weeks). Eligibility criteria were: (1) in current contact with an EIS in the North West of England; (2) capacity to provide informed consent; and (3) English language proficient. EIS are multidisciplinary community mental health services that provide psychosocial and pharmacological treatment and support to people in their first 3 years of their initial episode of psychosis. Exclusion criteria were: (1) aged less than 16 years at point of recruitment; (2) not capable of giving informed consent; (3) non-English proficient; and (4) inpatient at point of recruitment. Inclusion criteria were as broad as possible to improve the external validity of the trial. The trial was prospectively registered (ISRCTN34966555) and received ethical approval from the National Research Ethics Committee West Midlands—South Birmingham (14/WM/0118).

6Participants were recruited over 7 months from several NHS Trusts in the North West of England. Health professionals working within EIS identified eligible participants and passed on the contact details of those who consented to be contacted to either a trained researcher or a clinical studies officer (CSO) from the UK Clinical Research Network who supported recruitment to the trial. Following consent to contact, the researcher or CSO invited and consented participants into the trial. Following baseline assessment, participants were randomized in a 2:1 random allocation designed to maximize information about the Actissist intervention. Since hypothesis testing was not the objective of this study, a sample size of 36 was chosen a priori to assess feasibility, conduct preliminary statistical analyses, and obtain parameters to inform a robust power calculation for a fully powered efficacy trial.

7Where possible, randomization occurred within 2 working days of baseline assessment. Receipt of the Actissist or control app typically commenced within 2 working weeks of randomization. The study statistician produced a randomization list using random permuted blocks of size 3 and 6. Notification of group allocation occurred using an independent tool (eLabs; NWEH), an online research platform that concealed group allocation. The study coordinator was unblinded to treatment allocation and participants were informed about the outcome of randomization from a researcher after the baseline assessment. Many strategies were used to protect blinding, including researchers working on different days to minimize overlap, researchers not being involved in the randomization process, considering room use and diary arrangements, and reminding participants not to disclose group allocation. Group allocation was revealed only to the participant, responsible clinician, baseline research assessor, and project officer. Overall, there was only one blind break in the treatment group; another rater masked to group allocation completed each respective assessment when unblinding occurred. Accordingly, all ratings used for analysis were masked.

8The primary outcome was feasibility, which was assessed in terms of uptake (the proportion of eligible participants consenting to the study), attrition, the proportion of participants completing user, and alert-initiated data entries across participants (>33% data points), and the proportion continuing for 12 weeks (both arms). Acceptability of the Actissist intervention was assessed via participant feedback. Once written informed consent had been obtained, trained researchers administered a battery of secondary outcome measures (details of measures reported elsewhere). Demographic information was collected as well as measures of frequency, intensity, and distress of psychotic symptoms (Positive and Negative Syndrome Scale, PANSS; PSYRATS, depression (Calgary Depression Scale for Schizophrenia, CDSS), functioning (Global Assessment of Functioning scale, GAF, APA, 1994; Personal and Social Performance Scale, PSP), empowerment (Empowerment Rating Scale, ERS), health status and health-related quality of life (EQ-5D-5L). Frequency and quantity of alcohol and cannabis use (Timeline Follow Back, TLFB), perceived criticism from significant others (perceived criticism scale), medication adherence and attitudes to medication (Medication Adherence Rating Scale, MARS), and satisfaction with technology was also measured.

9All measures were administered at each assessment time-point. Participants were reimbursed £20 for completing assessment time-points. Assessors underwent a rigorous training process and received weekly supervision by the chief investigator. Researchers also attended monthly PANSS supervision groups run by experienced senior clinical academics for the duration of the study period. All assessors met departmental reliability standards after pretrial training (mean ICC = 0.89 PANSS total score across the 4 raters), followed by regular supervision in administration, scoring procedures and inter-rater reliability checks over the course of the trial period.

10Full details of the procedure are reported elsewhere. Briefly, participants in both conditions received a 45-min phone set-up training session focused on basic use of the smartphone (eg, charging the phone; on/off), demonstration of the app (both conditions), setting a passcode, and navigating participants through the app domains and settings. For Actissist, participants watched an in-built video explaining the basic principles of cognitive therapy, the theoretical orientation upon which the Actissist app is based. For ClinTouch, participants were provided a rationale for symptom monitoring. In both apps, participants could also view written and visual “in-app” instructions. Participants had the opportunity to use the app and ask questions. They were instructed to charge the phone regularly, to carry the phone at all times, and to go about their daily life as usual. No restrictions were placed on smartphone use.

11Following smartphone demonstration, participants were instructed to use the app for 12 weeks. Participants were instructed to respond to alerts wherever possible and were encouraged to use the on-demand features as and when needed. All participants received a weekly phone call from the project manager to troubleshoot equipment functions. Software was either preloaded on a loaned smartphone with £10 and topped up remotely each month to support data connectivity over the trial period or at their request downloaded on to the participant’s own smartphone. Participants using their own handset were given £10/month to cover data usage costs. Engagement with the apps was incentivized; a £10 shopping voucher was given to participants on a fortnightly basis over the intervention period who completed at least one-third of data entry points. A criterion was applied to determine whether an entry contributed to the overall app engagement algorithm. Specifically, there was a maximum of 3 valid entries per day. To prevent participants from artificially inflating their app usage to achieve the app usage incentives, where a participant self-initiated use (or where there was a combination of self- and prompted- initiation) which exceeded 3 entries per day, only the first 3 entries contributed to the engagement figure. Serious adverse events (SAEs) were strictly monitored, reviewed and documented by the team and discussed with a nominated senior clinical academic and trialist independent to the team.

12Actissist is a DHI that the user can engage with spontaneously or in response to being prompted. It then collects responses from the user and wirelessly uploads user responses to a server. Actissist is divided in 2 parts, although presented as a single app. Firstly, at 3 pseudo-randomized time points per day, 6 days a week between 10.00 and 22.00, an auditory alert followed by a visual prompt is emitted from the app inviting participants to access the app. The notifications persist on the handset (ie, no time out) until such point as they are accepted, dismissed, “snoozed” (up to 15 min), or another notification is received. The notifications serve merely as a reminder; the app also allows self-initiated use at any point. If a user accepts a notification or initiates use they are invited to select an intervention domain(s) and then complete a series of self-assessment questions structured as question-answer exchanges that focus on cognitive appraisals, belief conviction, emotions and associated behaviours. Depending on the appraisal selected, the exchange is followed by normalizing messages and cognitive or behavioral strategies aimed at suggesting ways of coping with distressing experiences. Multiple messages and images associated with each exchange minimize boredom and repetition within the app. Alternatively, participants can report that they have had “no problems like this” since their last notification or (self-initiated) interaction. Part 2 includes a menu of multi-media options that act in a stand-alone fashion designed to complement and support the feedback from the intervention domains. This supplementary content contains information and activities including relaxation and mindfulness exercises, recovery stories (videos), a range of fact sheets (eg, low mood, anxiety, self-esteem), external links to web-related content (eg, TED talks), daily diary, and emergency contacts resources. Furthermore, a graphical summary of data points entered over the previous 7 days allows users to track distressing experiences to support active self-management of symptoms and shared decision making about treatment with clinicians. Users can customize the aesthetics of the Actissist interface; for example, personally meaningful images from the smartphone’ s local storage can be set as wallpaper to facilitate positive mood induction. Figure 1 displays a visual schematic of the system, including screenshots of the Actissist app.

13The ClinTouch app is a symptom-monitoring app that triggers, collects, and wirelessly uploads symptom data to a server. As in the treatment condition, the app emits an alarm prompting participants to access the app at 3 pseudo-randomized time points per day, 6 days a week between 10.00 and 22.00 for 12 weeks alongside usual treatment. The ClinTouch protocol is outlined in detail in Palmier-Claus et al; although, the number of prompts was altered for parity with Actissist alerts, such that participants submit one-and-a-half data points daily with 10 branching items covering positive psychotic symptoms, anxiety, and mood. As each full data point was collected over 2 separate alerts, this equates to having received 3 alerts every day. The alert invites participants to use a touchscreen slider to rate the severity of 12 individual symptoms on a 1–7 scale. This takes an average of 70 s and the data are wirelessly uploaded to a secure server. Symptom items have been validated against corresponding items on the PANSS. The aesthetics and interface mirror the Actissist interface. However, unlike the Actissist app, ClinTouch does not facilitate self-initiated access; data entries must be in response to a notification. The notifications time out 30 min after receipt whereby the notification is no longer visible and the ClinTouch items are no longer accessible.

14To minimize risk, we did not store identifying data on either the app or the server. Participants set a passcode to access the smartphone. Three general principles of information security (confidentiality, integrity, and availability) were followed in the design and implementation of the trial. All data transmitted to and from the servers was encrypted over https with strong ciphers as detailed in the Approved Cryptographic Algorithms Good Practice Guidelines.

15TAU involved regular clinician meetings, medication, risk monitoring, and psychosocial interventions. Actissist and ClinTouch are standalone apps that do not link with services.

16Analyses follow the CONSORT 2010 Statement, showing referral and attrition (ie, participant flow) and an a priori analysis plan was published. Analyses were undertaken in Stata (version 14.1) after completion of the endpoint assessment. The primary outcome (feasibility) was assessed in terms of uptake (the proportion of eligible participants consenting to join the study), attrition, proportion of participants completing user and alert-initiated data entries across participants, and the proportion continuing for 12 weeks (both arms). Demographic data were presented using descriptive statistics. Ecological momentary interventions typically operationalize >33% data points completed as evidence of compliance,, which was the “accept” criterion for compliance that we adopted. The “target” criterion was 50% of participants submitting 50% of data entries. Linear regression was used to examine the effect of random allocation on the secondary outcomes at 12 and 22 weeks separately, adjusting for outcome measures at baseline. We report adjusted mean differences and their standard errors, Cohen’s D standardized effect sizes and their corresponding bootstrapped 95% confidence intervals (CIs), based on adjusted mean differences and the pooled standard deviation at baseline. Acceptability was assessed by participant feedback using a semi-structured interview. Feedback was thematically organized.

17A summary of demographic and clinical information for all participants is displayed in tables 1 and 2. There were very few drug users in the sample (2 cannabis, 2 cocaine, 1 mephodrone, 1 gogaine); 14/36 participants smoked cigarettes regularly. As can be seen in figure 2, uptake into the trial was high: 38/59 people (64.4%) referred participated to the full trial. Reasons for declining participation included medication side effects, concentrating on studies/employment, and not wishing to focus on mental health at the current time. Retention in the Actissist arm and tolerance for the Actissist app was excellent, as evidenced by no participant withdrawals. The “accept” criterion for data points completed was met in both trial arms (75% and 50% participants, respectively, submitting >33% data entries as per our pre-specified criteria). In other words, 75% of Actissist participants used the app on average at least once a day over the 12-week intervention period, suggesting excellent engagement and acceptability of the Actissist app. The “target” criterion was achieved in the Actissist arm only (63% vs 42% in ClinTouch). All participants except one (97%) remained in the trial (both arms) until the end. The participant who withdrew from the study returned the phone and withdrew from EIS all together (nonresearch related incident). No research-related SAEs were recorded for any participants during the study period, suggesting that both apps are safe. Completion of assessments was also high: 72% (26/36) and 83% (30/36) participants successfully followed up at post-treatment and 22 weeks, respectively.

18Participants mostly interacted with the app in the latter part of the day and earlier in course of the 12-week intervention (although the target engagement rates were still achieved for the intervention period). The most popular prompted domain was voices (481 entries), followed by suspicious thoughts (404 entries), socializing (220 entries), criticism (157 entries), and cannabis (31 entries). The most popular unprompted domain was suspicious thoughts (97 entries), followed by socializing (65 entries), voices (64 entries), criticism (39 entries), and cannabis (14 entries). Considering both prompted and unprompted entries, voices overall was the most frequently accessed domain overall (545 entries), followed by suspicious thoughts (501 entries), socializing (285 entries), criticism (196 entries), and cannabis (45 entries). Expressed as an average per participants across each domain over the intervention period, participants clicked on average 1.85 times (range: 0–11) on the cannabis domain (2 outliers noted, both clicking 11 times), 1.88 times (range: 1–50) on the socializing domain (2 outliers noted, clicking 40 and 50 times), 20.88 times (range: 0–94) times on the suspicious thoughts domain (1 outlier noted, clicking 94 times), 8.17 (range: 1–25) times on the criticism domain (1 outlier noted, clicking 31 times), and 22.71 times (range: 0–174) times on the voices domain (5 outliers noted, clicking 51, 54,, 58, 74, and 174 times). These findings show that while voices was the most frequently accessed domain overall, this seemed to be influenced by a few participants who interacted with this domain quite intensively. The remaining domains appeared less influenced by outlier responses. Participants lost 2 phones during the trial. The majority of participants used a study phone (n = 31). There was no difference in those who used their own phone (12.5% Actissist; 8.3% ClinTouch) versus those who were loaned a study phone (83.3% Actissist; 91.7% ClinTouch).

19Summary statistics for all secondary outcomes across time points and conditions are shown in table 3. Inspection of the effect sizes and confidence intervals suggest that there were improvements in key outcome measures, including PANSS negative score, general and total scores, and Calgary (mild, moderate and total) scores in the Actissist group relative to the control group post-treatment. The regression coefficients (ie, adjusted mean differences) and standardized effect sizes (Cohen’s D) are numerically higher on these variables at the post-treatment Actissist assessment. This suggests that participants understood the content of the app and learnt new skills, improving general psychotic symptoms and mood in the short-term. Effects were not fully sustained at 22-week follow-up; although, there was no decline on any of the clinical outcomes measured.

20The Actissist system was acceptable, enjoyable, beneficial, and easy to use (see supplementary table 1); 90% participants said they would recommend Actissist to others in a similar position. Illustrative feedback quotations are provided in table 4.

21This is the first study to evaluate the feasibility, acceptability, and safety of a theory-driven smartphone app in a randomized controlled design compared against an active control condition in early psychosis. Benefits reported for psychosis outcomes were observed in comparison to a similar smartphone app platform for tracking mental health in psychosis. Specifically, we showed that Actissist was feasible, acceptable, and safe, with high levels of satisfaction and indications of a beneficial effect, even over an active control group (symptom monitoring). Retention in the study was excellent (35/36 participants; 97%), which reflects other DHI retention rates of around 92% psychosis patients remaining in the trial until the end. Assessment completion was high and reflects similar DHI trials in psychosis. Engagement with the system was high (75% participants used the app at least once/day), which reflects engagement with other alert-based DHIs in psychosis (average response rate to alerts across trials of 71.9%, with 86.5%–94% participants interacting with the app on the predefined study days). Feedback from participants was overwhelmingly positive, suggesting that participants enjoyed using the app, understood the content of the app, with some suggestion that participants implemented new skills in the course of their day-to-day lives, evidenced by the promising treatment effect estimates. Findings sit alongside those emerging in the psychosis literature, which shows that digital symptom monitoring systems are feasible and acceptable to patients.,,,

22The treatment effects immediately post-treatment favoured the Actissist group, and were large. This suggests that the Actissist app conferred added benefit over and above routine symptom monitoring in terms of negative, general and total psychotic symptom scores and mood in the short term. Although there was no decline on any of the clinical outcomes measured at 22-week follow-up, treatment estimates were not maintained at this time-point, suggesting that further testing of sustained effects over time is needed. Furthermore, participants tended to use the app later in the day, reflecting the importance of using personalized alerts rather than pseudo-random alerts over a prespecified time period within days.

23Further research should be hastened in light of digital health care initiatives that lack an evidence base, and future consideration evaluating fast-paced technological innovations outside an RCT context is needed. There is a need to ensure parity and to limit the exclusion of low-income individuals who cannot afford smartphones and their associated cost. This study has notable strengths. We used an active control symptom-monitoring app condition, which matched the overall look-and-feel and functionality of the Actissist app, thus accounting for the nonspecifics of smartphone use. The apps were different, however, in that ClinTouch is designed to be completed in-the-moment and cannot be left to complete at a chosen time, whereas Actissist acts as more of an active and responsive self-management tool and is therefore available whenever the user requires (either prompted or self-initiated). We implemented a rigorous reporting SAE procedure for both groups. Trials without an active control condition have problems gathering a true measure of SAEs as researchers typically have less contact with the TAU group than the treatment arm. Finally, raters were blind to group allocation and stakeholders were co-designers of the system. There are some limitations. Participants were incentivized to use both apps, which may have increased usage beyond what would be observed in a real-world setting. Second, we operationalized engagement according to experience sampling methodology criteria (completion of 33% data entries over the intervention period). As this study was incentivized, we needed to apply a criterion to prevent participants from artificially inflating app usage to obtain incentives. In future DHIs, we suggest researchers report app usage more descriptively rather than prespecify completion rates., Finally, whilst the small sample could impact generalizability of findings, participants were representative of a help-seeking early psychosis group.

24Our findings suggest that the Actissist system may confer additional benefits over routine mobile symptom monitoring in the short term. Participants were engaged, active, and adherent with the system; therefore, findings justify proceeding to a fully powered trial. This study represents an important and significant step toward developing a technology platform for delivering a range of psychosocial interventions for psychosis. Indeed, this study is the first to show that an active self-management app can potentially improve outcomes in psychosis, even beyond a passive symptom-monitoring app. It shows how proof-of-concept trials can underpin digital experimental health care with empirically derived theoretical frameworks. If trials such as Actissist are effective, a major challenge is for mental health services to recognize and incorporate DHIs into the health care setting.

25Supplementary data are available at Schizophrenia Bulletin online.

26The work was supported by the Medical Research Council Developmental Pathway Funding Scheme (grant number MR/L005301/1) and was supported by the University of Manchester and Greater Manchester Mental Health NHS Foundation Trust.