Multimodal Machine Learning Workflows for Prediction of Psychosis in Patients With Clinical High-Risk Syndromes and Recent-Onset Depression

1The clinical high-risk (CHR) criteria for psychosis have been established to detect vulnerable individuals as early as possible to intercept disease development. These criteria identify a patient population with increased incidence compared with the general population, yet only 22% of patients with CHR as detected by ultra–high-risk criteria show a psychosis transition during a 3-year period. The clinical utility of the CHR designation may be further limited because its ascertainment is laborious and confined to specialized, well-equipped health care services that do not sufficiently cover the vulnerable population., Hence, improved prognostic accuracy and clinical scalability are needed to accurately identify patients truly at risk for psychosis.

2Prognostic accuracy may be increased using psychosis risk calculators for populations with CHR based on conventional statistics,,, or machine learning,,,,,, with studies finding that a first episode can be predicted with clinical data,, combinations of clinical and cognitive data,, neuroimaging, and, recently, with polygenic risk scores (PRS) for schizophrenia, among other measures. However, reviews,, have also highlighted methodological shortcomings, such as inadequate sample sizes and model validation strategies,, that may have inflated accuracy. Moreover, studies suggested that psychosis does not only emerge from CHR states, but occurs and can be predicted across a broader spectrum of comorbid conditions commencing in late adolescence and early adulthood., Hence, generalizable risk prediction models may require transdiagnostic discovery and validation populations, encompassing patients with CHR states and early-onset affective syndromes that share environmental, clinical, and neurobiological features.,,,,, In addition, the growing diversity of risk prediction models originating from different data modalities has led to uncertainty about the minimum number of modalities needed to increase prognostic accuracy to a level justifying clinical implementation., Finally, algorithms should be compared and integrated with clinicians’ predictions of psychosis transition to determine their potential utility from public health and service provision perspectives.,

3Addressing these challenges, the European Union–funded PRONIA study (Personalised Prognostic Tools for Early Psychosis Management [https://www.pronia.eu]) collected multimodal longitudinal data from adolescents and young adults in CHR states, those with recent-onset depression (ROD), and healthy control individuals. We evaluated clinical, neuroanatomical, and genetic machine learning models trained to identify patients with CHR syndromes and ROD who undergo psychosis transition. We compared our models’ performance with our clinical raters’ ability to predict psychosis transition and explored whether the sequential integration of algorithmic and expert-based prognoses produces clinically efficient cybernetic workflows,, that is, structured interactions between humans and machines that maximize prognostic accuracy while minimizing the examination burden in the given patient. We assessed potential confounders and moderators of prognostic performance and tested whether our models’ and raters’ estimates predicted not only psychosis transition but also distinct CHR syndromes and nonpsychotic disease trajectories. Finally, we explored whether a considerably condensed, and hence less burdensome, clinical model could be generalized to 3 independent patient cohorts with CHR syndromes and other mental conditions.,, This external validation step also benchmarked the neuroanatomical and combined models derived from the PRONIA cohort.

4The eMethods section in the Supplement details all of the methods for this prognostic study, which followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guideline. In summary, our analysis included 334 patients with CHR states (n = 167) or ROD (n = 167) recruited across 7 sites in Finland, Germany, Italy, Switzerland, and the United Kingdom from February 1, 2014, to May 31, 2017, using standardized inclusion and exclusion criteria (eTables 1 and 2 and CONSORT diagram in eFigure 1 in the Supplement). Follow-up for all patients ranged from 9 to 36 months, with visits every 3 months to the 18-month point and every 9 months thereafter. Furthermore, 334 healthy controls matched for age, sex, and site were included to evaluate prognostic patterns. Adult participants gave informed consent before study inclusion. Participants younger than 18 years and their guardians provided their written informed assent and consent. The PRONIA observational study was registered at the German Clinical Trials Register (DRKS00005042) and approved by all local research ethics committees.

5Sociodemographic and clinical variables were compared between diagnostic groups (eTable 3 in the Supplement), patients with psychosis transition and nontransition, and patients with 18-month or later follow-up data (discovery sample [n = 246]; PRONIA plus 18M) or earlier attrition (validation sample [n = 88]; PRONIA minus 18M) (Table 1). Psychosis transition was defined when at least 1 of the 5 positive symptom items in the Structured Interview for Psychosis–Risk Syndromes reached psychotic intensity daily for at least 7 days. Diagnoses of cases with psychosis transition are listed in eTable 4 in the Supplement.

6Data were analyzed from January 1, 2019, to March 31, 2020. Using the machine learning software NeuroMiner, version 1.05 (GitHub [https://github.com/neurominer-git/NeuroMiner-1]), we constructed and tested unimodal, multimodal, and clinically scalable sequential risk calculators for transition prediction in the PRONIA plus 18M cohort using leave-one-site-out cross-validation (LOSOCV), (eMethods and eFigures 2-4 in the Supplement). We evaluated the risk calculators using baseline and longitudinal data and validated their specificity in the PRONIA minus 18M sample, which did not include cases with psychosis transition but provided all data modalities. In addition, 3 external data sets consisting of cases with psychosis transition and nontransition were available to test selected models.,,

7First, unimodal risk calculators were trained with literature-based baseline predictors of psychosis transition,,, including prodromal symptoms,, functioning,, childhood adversity,, and neurocognitive measures, in the clinical-neurocognitive domain (eTable 5 in the Supplement); PRS for schizophrenia in the genetic domain; and gray matter volume maps in the structural magnetic resonance imaging (sMRI) domain, (eTable 6 in the Supplement). In addition, we evaluated our raters’ predictions, which, at the conclusion of baseline assessments, were provided as yes or no replies to the question, “Do you think the patient will likely transition to psychosis?” Then we assessed whether combining unimodal algorithms using stacked generalization improved prognostic accuracy (eFigures 2 and 3 in the Supplement and Table 2). Following the concept of expert-based machine learning, we integrated our raters’ estimates as additional predictors to produce a cybernetic model (eFigure 2 in the Supplement). Models’ predictive signatures were visualized in Figures 1 and 2 and eFigure 5 in the Supplement using measures of pattern element stability (cross-validation ratio) and pattern element significance (sign-based consistency; eMethods in the Supplement). In addition, the prognostic models were assessed using random-label permutations (Table 2). Raters’ and models’ performances were compared statistically at the omnibus level using the Quade test, an extension of the nonparametric Wilcoxon signed rank test, followed by post hoc pairwise mean differences tests using the t distribution. Statistical significance was determined at α =.05. Obtained P values were 2-sided; P values computed in the pairwise classifier comparisons were corrected using the false discovery rate (FDR) (Figure 3). Classifiers were visually compared in eFigure 6 in the Supplement).,

8We tested differences between the prognostic assignment groups and the matched healthy control group to determine whether the models’ predictive patterns represented a deviation from normality (eFigures 7-9 in the Supplement). Potential model confounders and moderators were systematically tested, including image quality (eFigure 10 in the Supplement), treatments (eTable 7 in the Supplement), follow-up frequency and duration (eFigure 11 and eTable 8 in the Supplement), site-related effects (eTables 9-11 in the Supplement), baseline study group membership (eTables 12 and 13 in the Supplement), and, specifically, the inclusion of patients with ROD (eTable 14 and eFigure 12 in the Supplement). Using the patients’ longitudinal data, we evaluated whether model predictions were not specific for the binary transition vs nontransition outcome, but we also separated transitions from nonremitting/de novo CHR symptom courses (P-CHR) and asymptomatic/nonpersisting trajectories (NP-CHR) (eTable 15 in the Supplement). To further explore a prognostic generalization effect, we used unsupervised machine learning (eMethods in the Supplement) to construct fine-grained CHR syndrome trajectories (eFigure 13A in the Supplement) and used linear mixed-effects modeling to compare trajectories between predicted and observed outcome groups (eFigure 13B [part 1] in the Supplement). Finally, we investigated whether assignments generalized to the prediction of nonpsychotic outcomes (eTable 16 in the Supplement).

9To facilitate clinical implementation, we developed a sequential prediction method that optimizes the ordering and number of data modalities as well as the prognostic uncertainty thresholds to decide whether a patient needs further testing (eTable 17 and eMethods in the Supplement). We analyzed the identified optimal prognostic workflow (eFigure 14 in the Supplement) and tested whether it achieved similar performance as the fully stacked models at lower diagnostic burden for the patients. To further enhance clinical scalability, we condensed the clinical-neurocognitive model, which was the workflow’s entry point, from 141 to 7 (5.0%) variables (Figure 2) using sign-based consistency mapping (eMethods in the Supplement). We tested the condensed model and the respective workflow’s specificity in the PRONIA minus 18M sample (Table 2). Finally, we explored whether the prognostic sequence could be further trimmed using diagnostic parsimony regularization (eMethods in the Supplement and Table 2). Nonregularized and regularized workflows including the full or condensed clinical-neurocognitive models were compared in eFigure 15 in the Supplement.

10We validated the condensed clinical-neurocognitive model in 2 external cohorts: 146 patients with CHR (aged 15-35 years; 16 [11.0%] transitions) provided by the Zurich Early Recognition Program (ZInEP) (eTable 18 in the Supplement) and 462 patients with diverse mental conditions (aged 8-17 years; 13 [2.8%] transitions) drawn from the Bi-national Evaluation of At-Risk Symptoms in Children and Adolescents study (eTable 19 in the Supplement). Second, we validated the sMRI-based model in ZInEP and in 37 patients with CHR (16 [43.2%] transitions) from the Früherkennung von Psychosen study (Table 2)., To validate the increased performance of multimodal risk calculators, we trained a stacked model using the condensed clinical-neurocognitive and sMRI-based models, tested it in the ZInEP data (Table 2), and used the Quade test to compare the 2 unimodal prediction models with the stacked classifier (Figure 3). Finally, we made our models available in the NeuroMiner Model Library (http://www.proniapredictors.eu) to facilitate their independent external validation.

11A total of 668 patients and controls were included in the analysis (mean [SD] age, 25.1 [5.8] years; 354 [53.0%] female and 314 [47.0%] male). Patients in the PRONIA plus 18M and PRONIA minus 18M groups were followed up for a mean (SD) of 842.7 (272.3) and 390.6 (99.6) days, respectively (Table 1). They did not differ in any examined variable (Table 1). Psychosis transition occurred after a mean (SD) of 246.9 (244.5) days in 26 cases and developed into schizophrenia in 8 (30.8%) (eTable 4 in the Supplement). Follow-up durations differed between sites but not time to psychosis transition (eTable 9 in the Supplement). Compared with nontransition, individuals with psychosis transition had more repeated school years (mean [SD], 0.67 [0.88] vs 0.26 [0.61] years) and more prevalent attenuated positive symptoms (APSs) at baseline (18 of 26 [69.2%] vs 88 of 308 [28.6%]) (Table 1). Of 167 patients with CHR, 23 (13.8%) developed psychosis, whereas 53 (31.7%) had remitted from CHR criteria at the 9-month visit. Major depression affected 103 patients with CHR syndromes (61.7%) but was not differentially associated with psychosis transition (Table 1). Nonremitting mood and anxiety disorders were present during follow-up in cases with psychosis transition and nontransition (eTable 16 in the Supplement). Compared with healthy controls, ROD was associated with low but significantly elevated CHR symptom scores (eg, mean [SD] Structured Interview for Psychosis–Risk Syndromes positive symptoms, 0.43 [0.44] vs 0.10 [0.21]; P <.001; mean [SD] Schizophrenia Proneness Instrument: Cognitive Disturbances symptoms, 0.24 [0.32] vs 0.02 [0.08]; P <.001) (eTable 3 in the Supplement). Functional-cognitive and interpersonal abnormalities were comparable between ROD and CHR groups. Of 167 patients with ROD, 32 (19.2%) developed psychosis-related outcomes, including CHR states in 29 (17.4%) and psychosis transitions in 3 (1.8%).

12The full clinical-neurocognitive model predicted psychosis transition with a balanced accuracy (BAC-LOSOCV) of 75.7% (sensitivity, 84.6%; specificity, 66.8%; P <.001) (Table 2). Significant predictors as determined by sign-based consistency mapping (z>3.28; P <.05 for FDR) were APS and motor disturbances, a nonsupportive family environment during childhood, and reduced facial emotion recognition (Figure 1B). Compared with healthy controls, those assigned to psychosis transition had elevated abnormality scores in these variables, whereas those with nontransition assignments showed an abnormality pattern focused on unusual thought content, suspiciousness, perceptual abnormalities, and childhood adversity, with higher visual working memory and semantic verbal fluency performance (P <.05 for FDR) (eFigure 7 in the Supplement).

13The PRS-based model achieved a BAC-LOSOCV of 66.1% (sensitivity, 76.0%; specificity, 56.2%; P <.001). Among the 10 tested genome-wide significance thresholds, only P = 1.0 reached significance (Figure 2). Compared with healthy controls, those with psychosis transition assignments had elevated PRS across all whole-genome P thresholds, whereas those with nontransition assignments expressed reduced PRS at P ≥ 5.7 × 10−4 (eFigure 8A in the Supplement). Patients with observed nontransition and those with ROD did not show reduced PRS (eFigure 8B-C in the Supplement).

14The sMRI-based model attained a BAC-LOSOCV of 70.7% (sensitivity, 88.0%; specificity, 53.5%; P <.001). At a stability threshold (cross-validation ratio) of at least 3, the brain pattern predicting psychosis transition involved reduced gray matter volume in the superior temporal, supramarginal, angular, orbitofrontal, inferior frontal, dorsomedial prefrontal, and occipital cortices. The predictive pattern also included areas of increased gray matter volume covering the dorsolateral prefrontal, precuneal, insular, hippocampal, and cerebellar brain regions (eFigure 5 in the Supplement). This brain signature differentiated psychosis transition-assigned patients from healthy controls, whereas nontransition-assigned patients showed a partial pattern inversion with increased temporo-occipital gray matter volume compared with healthy controls (see threshold-free cluster enhancement statistics thresholded at P <.05 for FDR) (eFigure 9 in the Supplement).

15Clinical raters achieved a BAC of 73.2% (sensitivity, 61.5%; specificity, 84.9%), which was independent of the length of their early recognition experience (mean [SD] length for correct predictions: 31.8 [46.8] months; mean [SD] length for wrong predictions, 29.4 [38.1] months; unpaired 2-tailed t326 = 0.35; P =.72). The stacked model combining unimodal algorithms produced a BAC-LOSOCV of 82.9% (sensitivity, 80.8%; specificity, 85.0%). Integration of raters’ prognoses into the stacked model increased BAC-LOSOCV to 85.5% (sensitivity, 84.6%; specificity, 86.4%), and they were the cybernetic model’s most relevant predictor (Figure 2C).

16Image quality (eFigure 10 in the Supplement), baseline treatments or previous hospitalizations (eTable 7 in the Supplement), follow-up duration and frequency (eTable 8 and eFigure 11 in the Supplement), and site effects (eTable 10 in the Supplement) did not influence model performance. Study group (CHR vs ROD) could be classified with a BAC-LOSOCV of 82.3% (sensitivity, 73.7%; specificity, 91.0%) using clinical-neurocognitive data and with a BAC of 55.7% (sensitivity, 49.7%; specificity, 61.7%) using PRS (eTable 12 in the Supplement). These diagnostic classifiers explained 49.7% and 18.3%, respectively, of the variance of the respective prognostic counterparts (both P <.001 for FDR) (eTable 13 in the Supplement). Raters’ prognoses were also significantly informed by baseline study group (BAC, 62.7%; sensitivity, 30.9%; specificity, 94.5%) (eTable 12 in the Supplement). The removal of the patients with ROD from the training samples or their substitution with healthy controls significantly reduced the balanced accuracy of all risk calculators by −2.8% to −11.7% in the CHR group (eTable 14 and eFigure 12 in the Supplement).

17Prognostic assignments also delineated psychosis transition and P-CHR and NP-CHR courses irrespective of model type (eTable 15 in the Supplement). The separability of P-CHR from NP-CHR courses was lower and only significant for clinical-neurocognitive models and raters. The nonnegative matrix factorization and linear mixed-model analysis showed that CHR syndrome trajectories were stratified by the predictions of the clinical-neurocognitive classifier (factor F1 paranoid-perceptual disturbances, F1,832 = 136.35 [P <.001 for FDR]; factor F2 disturbances of volition and affect, F1,832 = 12.76 [P =.001 for FDR]; factor F3 functional disturbances, F1,832 = 24.34 [P <.001 for FDR]; factor F4 cognitive disturbances, F1,832 = 44.05 [P =.007 for FDR]) as well as by raters' outcome estimates (factor F1 paranoid-perceptual disturbances, F1,825 = 30.64 [P <.001 for FDR]; factor F2 disturbances of volition and affect, F1,825 = 6.15 [P =.03 for FDR]; factor F3 functional disturbances, F1,825 = 5.80 [P <.03 for FDR]; factor F4 cognitive disturbances, F1,825 = 9.00 [P =.007 for FDR]) (eFigure 13 in the Supplement). Nonpsychotic disease courses were not associated with prognostic assignments (eTable 16 in the Supplement).

18We identified a prognostic sequence that produced a BAC-LOSOCV of 85.9% (sensitivity, 84.6%; specificity, 87.3%) (Table 2) and started with the clinical-neurocognitive model, added raters, and finally integrated PRS- and sMRI-based models (eFigure 14 in the Supplement). Across this sequence, the positive likelihood ratio increased to 6.6, whereas the population requiring all prognostic assessments decreased to 41.1%. Regularization for diagnostic parsimony further reduced this population to 23.2% (regularization strength Γ = 0.5) and 0 (regularization strength Γ = 1.0) (PRONIA plus 18M sample, left panel of eFigure 15B in the Supplement), with the latter parsimony level significantly reducing BAC by −6.4% (PRONIA plus 18M sample, left panel of eFigure 15C, part 2 in the Supplement). Highly similar findings were obtained when analyzing the complete PRONIA cohort (right panels in eFigure 15B and C in the Supplement).

19Further clinical scalability experiments showed that the condensed clinical-neurocognitive model matched the full model in correctly predicting PRONIA minus 18M cases with nontransition (specificity, 65.9%). Furthermore, its performance in the external ZInEP-CHR cohort (BAC, 65.3%; sensitivity, 87.5%; specificity, 43.1%) and the Bi-national Evaluation of At-Risk Symptoms in Children and Adolescents sample (BAC, 70.4%; sensitivity, 76.9%; specificity, 63.9%) was similar to the full model in the PRONIA-CHR sample (BAC, 63.3%; sensitivity, 87.0%; specificity, 39.6%) (eTable 14 in the Supplement) and the complete PRONIA cohort (BAC, 72.8%; sensitivity, 88.0%; specificity, 57.5%) (Table 2). Replacing the full model with its condensed counterpart in the nonregularized or regularized (Γ = 0.5) workflows did not increase the false-positive rate in the PRONIA minus 18M sample or the complete PRONIA cohort (eFigure 14C in the Supplement).

20The validation of the unimodal sMRI-based workflow component in the ZInEP (BAC, 67.9%; sensitivity, 75.0%; specificity, 60.8%) and Früherkennung von Psychosen (BAC, 68.5%; sensitivity, 75.0%; specificity, 61.9%) samples approximated the PRONIA-CHR results (BAC, 70.8%; sensitivity, 86.4%; specificity, 55.3%) (eTable 14 in the Supplement). Finally, the stacked risk calculator composed of the condensed clinical-neurocognitive and sMRI-based models significantly outperformed these models in the ZInEP data (BAC, 71.3%; sensitivity, 75.0%; specificity, 67.7%) (Figure 3 and Table 2).

21All risk calculators were significant in the permutation analysis (mean [SD] BAC, 77.3 [4.8]; mean [SD] sensitivity, 80.2% [6.5%]; mean [SD] specificity, 74.5% [13.8%]; P <.001 for FDR for all models) Table 2), but differences in BAC emerged (Figure 3). Multimodal risk calculators outperformed all unimodal counterparts (t range, 3.14-6.20; P < 3.19 × 10−8 to P =.002 for FDR), whereas the nonregularized prognostic sequence did not differ from the cybernetic model (mean [SD] BAC for nonregularized sequence, 83.7% [9.6%]; mean [SD] BAC for cybernetic model, 83.4% [9.6%]; t = 0.21; P =.44 for FDR). In addition, the stacked model (mean [SD] BAC, 79.7% [7.9%]) was outperformed by both the cybernetic model (mean [SD] BAC, 81.5% [9.6%]; t = 3.18; P =.001 for FDR) and the nonregularized sequential model (mean [SD] BAC, 82.0% [9.6%]; t = 3.82; P <.001 for FDR) in the complete PRONIA sample. Raters were comparable to unimodal risk calculators (mean [SD] BAC for raters, 68.2% [12.6%]; mean [SD] BAC range for unimodal predictors, 67.5% [16.4%] to 74.8% [6.8%]; t range, 0.002-1.56; P =.50 to P =.07 for FDR) but were outperformed by multimodal prediction algorithms in terms of higher BAC and reduced prognostic variability (stacked model vs raters, t = 4.64 [P <.001 for FDR]; cybernetic model vs raters, t = 7.82 [P <.001 for FDR]; sequence model vs raters, t = 8.46 [P <.001 for FDR]). Finally, the nonregularized sequential model reduced raters’ false-negative rate from 38.5% to 15.4% (PRONIA plus 18M sample) or 19.2% (complete PRONIA cohort) of cases.

22Using a thorough model discovery and validation approach, our study demonstrated geographic transportability of expert-based clinical and biological psychosis transition prediction approaches across a transdiagnostic, multinational risk population. We found that combined risk calculators outperformed all unimodal counterparts and clinical raters in terms of prognostic accuracy and cross-site stability. Importantly, our study revealed that the increased diagnostic burden arising from data fusion could be mitigated through optimized sequential testing that arranges clinicians and risk calculators into clinically scalable prognostic workflows. Based on this form of deferral learning, we showed that the complete assessment battery is only needed in 23.2% of the initial population (eFigure 15B in the Supplement). This subgroup was enriched for patients who received a prediction of psychosis transition in the initial clinical-neurocognitive examination, suggesting that biological markers of psychosis transition are useful for delineating true-positive from false-positive findings at the later steps of a multistep prognostic assessment.

23Examining the baseline heterogeneity of our transdiagnostic population, we found functional-neurocognitive impairments in the ROD group akin to the CHR group and low between-group neuroanatomical and genetic separability (eTable 12 in the Supplement), supporting the neurobiological proximity between early-onset affective and psychotic disorders.,, Although CHR syndromes expectedly separated CHR and ROD groups at the cross-sectional level, we observed that these syndromes emerged in 19.2% of patients with ROD during the follow-up period, which led to psychosis transition in 1.8% of cases. Strikingly, our analyses also showed that the models’ prognostic accuracy, particularly the sensitivity for psychosis transition in patients with CHR, depended on patients with ROD being part of the model discovery process, which further supports the pooling of both groups into a broader risk population (eTable 14 and eFigure 12 in the Supplement). Our finding of a transdiagnostic predictability of psychosis was corroborated by the generalizability of the clinical-neurocognitive and neuroanatomical models to external samples, which showed markedly different risk levels, age distributions, and diagnostic compositions.

24The in-depth analysis of the clinical-neurocognitive domain revealed that the presence of APSs facilitated a good baseline separability of CHR syndromes vs ROD and substantially informed the prediction of psychosis transition (eTable 13 in the Supplement). However, measures of childhood adversity, motor disturbances, and facial affect recognition did not overlap between diagnostic and prognostic models (eFigure 16 in the Supplement) and thus could be regarded as transdiagnostic markers,,, of poor psychosis-related outcomes, including transition to psychosis. This interpretation was supported by the prognostic generalization of the clinical-neurocognitive model to the clinically relevant separation of patients with (1) nonremitting/de novo and nonsymptomatic CHR syndrome courses (eTable 15 in the Supplement) or (2) unfavorable perceptual, affective, functional, and basic symptom trajectories (eFigure 13 in the Supplement)., Importantly, the model’s prognostic generalization capacity did not encompass nonpsychotic diagnoses (eTable 16 in the Supplement), and, thus, its prognostic pluripotency was confined to diverse CHR-specific symptom courses.

25Furthermore, we confirmed the prognostic value of PRS for schizophenia, as reported recently, and extended those findings by showing that genetic information augments the performance of clinical-neurocognitive models and prognostic workflows in a broader risk population (Table 2 and eTable 20 in the Supplement). Within this transdiagnostic setting, we replicated group-level differences among patients with psychosis transition, patients with nontransition, and healthy controls but also found that PRS-based prognostic assignments specifically differentiated APS-related trajectories (eFigure 13 in the Supplement). They also delineated patient groups with abnormally high and low genetic risk compared with healthy controls (eFigure 8 in the Supplement)—a finding that may point to distinct environmental and/or neurobiological pathways conferring risk and resilience to psychosis.

26The analysis of the structural neuroimaging data revealed a psychosis-predictive brain signature that generalized well across 3 independent cohorts. This signature overlapped with brain alterations previously reported to correlate with perceptual abnormalities, disorganization of speech and thought, and poor insight in early, subsyndromal, or prodromal stages of psychosis.,,, Interestingly, nontransition-assigned patients showed reversed temporo-occipital volume reductions, which differentiated them from healthy controls (eFigure 9 in the Supplement). These findings may point to ongoing compensatory mechanisms of resilience to psychosis, as reported previously in a longitudinal sMRI study of adolescents with CHR. In this regard, our sMRI-based risk calculator may serve as a useful tool for enriching future observational studies and clinical trials with at-risk patients who express potential brain mechanisms of resilience to psychosis transition.

27We observed that our raters matched unimodal risk calculators in predicting psychosis as measured by their BAC. However, raters also showed a pronounced optimism bias (low sensitivity and high specificity) toward the true risk of poor clinical outcome. It is noteworthy that their prognoses were based on all information collected in an extended study-related assessment and likely would be less accurate in routine, time-restricted diagnostic settings. Because the algorithmic counterparts showed exactly the inverted bias (high sensitivity and low specificity), the integration of clinicians and risk calculators into the cybernetic model produced a superior predictive system. Furthermore, our prognostic workflows demonstrated that similar levels of prognostic accuracy can be achieved by reducing the false-positive rate through sequential model application in patients with an estimated higher risk for psychosis transition (eFigure 14 in the Supplement). In this subgroup, the removal of the final sMRI-based assessment step increased false-positive findings (eFigure 15 in the Supplement), suggesting that the cost-benefit ratio of expensive neuromarkers needs to be individually adjusted according to the patient’s predicted risk.

28The finding that prognostic workflows always started with the clinical-neurocognitive model places the recognition of the clinical gestalt of emerging psychosis at the gateway of more precise early detection techniques. Our scalability experiments suggest that the laborious recognition of this pattern currently practiced in early recognition services could be effectively condensed to a few clinical-neurocognitive variables, thus enhancing the clinical utility of the proposed workflow. Nonetheless, future studies should revisit the validity of the selected 7 variables because they have been taken out of their original assessment context. Further studies also need to quantitatively explore the information patterns guiding clinicians’ gut-feeling estimates of psychosis transition and in turn foster more effective clinical early recognition strategies that integrate with cybernetic systems.

29Psychosis transitions were limited to 26 individuals in the PRONIA discovery sample. This sample size increased the risk of producing overly optimistic prediction results owing to an accidental collection of well-classifiable cases. We implemented a multistep model validation procedure to guard against this possibility, including label permutation testing, strict nested cross-validation of all processing steps, in-depth model analysis to assess possible prognostic confounds and moderators, specificity testing of all models in a completely held-back portion of the PRONIA sample, and model benchmarking in 3 independent data sets, which provided a further 45 individuals with psychosis transition and 600 with nontransition for external validation. Owing to limited data availability in these samples, only the condensed clinical-neurocognitive, sMRI-based, and a specific stacked risk calculator trained on the outputs of the former 2 models could be externally validated. However, our internal-external validation approach followed established guidelines for model construction and validation. In keeping with this literature, the similar performance levels observed in our LOSOCV and independent validation experiments support the validity of the models not tested in external samples.

30In this prognostic study, we identified generalizable risk assessment tools that can be arranged into a multimodal prognostic workflow for a clinically viable, individualized prediction of psychosis in patients with CHR states and ROD. Our study showed for the first time, to our knowledge, that the augmentation of human prognostic abilities with algorithmic pattern recognition improves prognostic accuracy to margins that likely justify the clinical implementation of cybernetic decision-support tools. New international collaborations, such as the HARMONY (Harmonization of At Risk Multisite Observational Networks for Youth) initiative, may help to propel a reciprocal and iterative process of clinical validation and refinement of these prognostic tools in real-world early recognition services.