A systematic review of the applications of artificial intelligence and machine learning in autoimmune diseases

1Three elements contribute to autoimmune disease development: genetic predisposition, environmental factors and immune system dysregulation (Fig. 1). Due to the heterogeneity of onset and progression, diagnosis and prognosis for autoimmune disease is unpredictable.

2A predisposition to autoimmunity is strongly linked to genetics, and caused by defects in mechanisms that result in loss of self-tolerance. Autoimmune disease develops after further immune system dysregulation, in both the innate and adaptive immune system. Microbial antigens, foreign antigens and cytokine dysregulation, can cause induction of self-reactive lymphocytes. Moreover, hyper-activation of T and B cells may occur, along with a change in the duration and quality of their response, which further disrupts the homeostasis of the immune system.

3The prevalence of autoimmune disease is difficult to estimate; diseases are variably represented across different studies and no definitive list exists.– There is a reported prevalence rate of between 4.5% and 9.4%, across all autoimmune diseases.

4Personalised care is valuable for autoimmune disease, with variability within the disorders, and presence of autoimmune comorbidities for 15–29% of patients.– Arguably, patients with multiple autoimmune comorbidities would particularly benefit from personalised healthcare for the causal molecular mechanism as opposed to specialist treatment of symptoms.

5Standard patient care generates diverse clinical data types. Examples of such data include laboratory test results from blood or urinary samples, symptoms at diagnosis and images obtained using colonoscopies and magnetic resonance imaging (MRI). The majority of these data are reproduced longitudinally over a chronic disease course.

6In addition to this wealth of clinical data, ‘omic data—such as patients’ genomic, transcriptomic and proteomic profiles—are now increasingly available. ‘Omic data are large, as molecular measurements are made on a genome-wide scale, and high throughput omics technologies have allowed fast analysis of these data. The inclusion of multiple types of ‘omic data into machine learning models may give a more complete picture of autoimmune disease, leading to novel insights.

7Combined clinical and ‘omic data have limited utility without methods for interpretation. Artificial intelligence and machine learning techniques have the capacity to identify clinically relevant patterns amongst an abundance of information, fulfilling an unmet need. The ability to stratify patient’s using these data has implications for their care, from estimation of autoimmune disease risk, diagnosis, initial and ongoing management, monitoring, treatment response and outcome.

8The terms “machine learning” (ML) and “artificial intelligence” (AI) are often conflated. Artificial intelligence is the study of methods to imitate intelligent human behaviour (for example to make decisions under conditions of uncertainty). Machine learning is a subset of AI that focuses on the study of algorithms that enable a computer to perform specific tasks (typically classification or regression) without specific instructions, but instead inferring patterns from data. Both AI and ML differ from traditional statistical methods as they focus on prediction and classification from high-dimensional data, rather than inference. Successful ML requires robust data from which it can learn. These data must be sufficiently abundant to enable the model to be robust and generalisable to unseen data.

9Two types of ML are discussed here: supervised and unsupervised learning. During supervised learning, an algorithm is trained on a “training dataset” to recognise the patterns that are associated with specific “labels” (for example, healthy or diseased). Once predictive patterns have been learned from training data, the ML algorithm is then able to assign labels to unseen “test data”. In a well-trained model, the patterns identified in the training data will generalise to the test data. Brief descriptions of some of the most common supervised ML techniques referred to in this review are summarised in Box 1.

10For unsupervised learning, training data are unlabelled, and the algorithm instead attempts to find and represent patterns within the data, for example by identifying clusters based upon the similarity of the examples. Other types of ML exist, but are reviewed elsewhere. Some of the more common unsupervised methods discussed in this review include hierarchical clustering and self-organising maps.

11Neural networks: outputs are learned from inputs via a series of nested nonlinear functions, encoded in a network of “neurons”, which may vary in its topology.

12Decision trees: outputs are learned from inputs via a series of yes/no questions that successively divide the predictor space into discrete piece.

13Random forest: a simple ensemble method that grows a large number of decision trees, each of which see only a subset of the data, and learns output from input by combining the predictions.

14k nearest neighbours: learns output from input by comparing the identity of each data point to its (k) nearest neighbours.

15Support vector machine: a binary classification method that can be adapted to multiclass classification or regression. They seek to partition the predictor space into two, such that data points from each class are concentrated on one side of the decision boundary.

16Natural language processing: a set of advanced ML methods that seek to extract sentiment from text.

17Recommendations cannot be made on the best model to use in general, as this is always situation specific and dependent on data type, size and dimensionality. Decision trees are simple and highly interpretable, but they rarely achieve performance accuracies higher than other algorithms. Using the random forest method can improve performance, at the cost of losing some interpretability. K nearest neighbours is a non-parametric method, and copes well when complex boundaries separate classes, but this flexibility can lead to poor classification results due to overfitting. Neural networks and support vector machines have similar strengths and weakness: they achieve high accuracies, and can extract linear combinations of features, but interpretability is poor, scaling to very large data can be difficult, and they are not robust to outliers.

18Technical aspects regarding the operation and fitting of machine learning algorithms are outside the scope of this systematic review, but comprehensively discussed elsewhere.,

19Machine learning models are often complicated, and can involve optimizing many free parameters. For this reason, they are prone to overfitting. Overfitting is the process by which the algorithm learns patterns that are specific to the training data but do not generalise to test data. For example, there may be some random technical error in the training data that is not of clinical relevance, yet is learned by the algorithm. Training any model accurately while avoiding overfitting is a central part of an ML pipeline. If data are abundant and/or the ML model is computationally expensive to train then the standard strategy is to remove a portion of the data for training, optimize the model on the remaining portion, and finally determine the model performance by comparison with the unseen test portion. If data are not abundant, a process known as cross-validation is typically employed (Fig. 2). There are many variations of cross-validation but they are all essentially generalisations of the training/test splitting process described above. For example, in k-fold cross validation, the data are randomly split into k subsets, with all but one subset used to train an ML model, and the remaining subset used to test the model. This process leads to the generation of k ML models. Each subset of the data is used only once as a test set, and overall model performance is determined by averaging the performance of the k models (Box 2 describes model evaluation metrics).

20Accuracy: percentage of correct predictions.

21Area under the receiver-operator curve (AUC): appropriate for binary classification problems, this method uses a plot of sensitivity versus specificity to determine model performance.

22Balanced accuracy: measure of the total number of correct predictions in either class, therefore taking into account an unbalanced dataset.

23F-score: an accuracy measure calculated using precision and recall.

24Out-of-bag error: this metric applies to tree-based ensemble methods, and measures the test error by comparing predictions with true labels for samples that were not used in the construction of a particular decision tree.

25Precision: equivalent to positive predictive value.

26Recall: another term for sensitivity.

27R2: measures the amount of variation explained by the model regression.

28Sensitivity: correctly identified true positives.

29This systematic review aims to inform on the current status of the application of artificial intelligence and machine learning methods to autoimmune disease to improve patient care. To the best knowledge of the researchers, this is the first study on this topic. The review identifies the most common methods, data and applications, the issues surrounding this exciting interdisciplinary approach, and promising future possibilities.

30Of 702 papers identified in database searches, 169 were selected for inclusion in the analysis, 227 duplicates were removed, 273 records were excluded based on the abstract and 33 were excluded after reading the full article (Fig. 3) using the criteria described above. A summary and detailed information for qualifying studies are described in Table 1 and Supplementary Table 1, respectively. Six diseases included in the database search returned no studies that met the inclusion and exclusion criteria (Addison disease, myasthenia gravis, polymyalgia rheumatica, Sjӧgren syndrome, systemic vasculitis and uveitis).

31Machine learning and artificial intelligence are most commonly applied to multiple sclerosis (MS), rheumatoid arthritis (RA) and inflammatory bowel disease (IBD). MS, IBD and RA models used the most types of data, including 13 studies generating models using two data types (always including clinical data). Random forests and support vector machines were the most commonly used methods throughout diseases and applications. Clinical data were used in models for every type of autoimmune disease, and models using genetic data were created for the majority of disorders. The variety in methodological approaches, applications and data, as well as use of validation methods (Supplementary Table 1) renders meta-analysis of these methods inappropriate.

32The applications for ML can be categorised into six broad topics: patient identification, risk prediction, diagnosis, disease subtype classification, disease progression and outcome and monitoring and management.

33Studies utilised ML methods to identify patients with autoimmune diseases from electronic medical records,– and employed natural language processing. Gronsbell et al. worked to improve the efficiency of algorithms for this purpose., These algorithms are intended to replace International Classification of Diseases billing codes, which have error rates of between 17.1–76.9% due to inconsistent terminology. Electronic medical records also identified comorbidities associated with alopecia and vitiligo using natural language processing. This identified similar autoimmune comorbidities for both diseases.,

34Prediction of disease risk– and identification of novel risk factors through feature selection– was documented for IBD, type 1 diabetes (T1D), RA, systemic lupus erythematosus (SLE) and MS. Fifteen studies employed genetic data, using either sequencing arrays (GWAS) or exome data (nine studies), individual SNPs within in the HLA regions, or pre-selected genes, or gene expression data., Only one study employed clinical data, and two others combined clinical and genomic data., Popular models included random forest, support vector machine and logistic regression.

35Patient diagnosis was the most frequent ML application, and this approach was used for all autoimmune diseases. Distinguishing cases from healthy controls was an aim for 27 studies. Diagnostic classification models used patients with other autoimmune diseases as controls,– differentiated between diseases with overlapping or similar symptoms or phenotype, for example stratifying coeliac disease and irritable bowel syndrome,– or examined classification of multiple autoimmune diseases., ML specifically for early diagnosis was specified by seven studies for the later onset degenerative conditions MS and RA.,– Other diagnostic applications included distinguishing coeliac disease from an at-risk group, and differentiating those who have complications in T1D., Random forests and support vector machine most frequently utilised.

36Disease subtypes in one RA, two IBD, and six MS studies were classified by ML. Three types of unsupervised clustering were used by these studies: hierarchical clustering for identifying novel IBD subtypes; consensus clustering to identify high, low and mixed levels of inflammation in RA; and agglomerative hierarchical clustering to cluster MS by genetic signature. Two of these studies employed support vector machine,, which is a popular supervised method in general, as well as random forest. There was wide variation in data types used. These included clinical (in particular MRI), genetic, RNA sequencing and gene expression data.

37Disease progression and outcome was a focus for 27 studies. Other considered issues were disease severity– in psoriasis, RA, IBD and coeliac disease; treatment response– in IBD, RA and primary biliary cirrhosis (PBC); and survival prediction– in PBC, RA and SLE. Other models focused on improved image segmentation to aid prognoses– for IBD and MS. Disease progression and outcome was the second-most prevalent area for model development. Throughout, the most common models were support vector machines, random forest and neural networks. The majority of data used was clinical, with very few papers utilising ‘omic data.,–

38Ten different studies of type 1 diabetes (T1D) used ML for monitoring and management: four predicted blood glucose level, four identified or predicted hypoglycaemic events, and two supported decision making using case-based reasoning or decision support systems. The majority of models used clinical data. Three models were developed using activity measurements for monitoring movement in MS, and one in RA. Support vector regression was used most frequently.–

39Eighteen studies only used hold-out validation, not including studies with random forest models, where cross validation is unnecessary, or neural networks, where this process can be too computationally intensive. Eleven studies did not use any validation method, and so model integrity and applicability is unconfirmed. Methods that use hold-out validation have the potential to provide useful information, but it is accepted that unless the dataset is very large, these models are not as robustly validated as those that have used k-fold or leave-one-out cross validation, or a combination of cross-validation and testing on an independent dataset.

40Only 14 of 169 studies combined cross-validation with independent test data for evaluating their models. These studies did not have any model types or applications in common. Clinical and genomic data were most common inputs for these studies. Models that used cross-validation and independent test data were applied to a number of the autoimmune diseases.

41The research reviewed here demonstrates that, much like the disease studied, the ML models and methods used are heterogeneous. It can be difficult then, to determine which methods should be taken forward to clinical application. Alternatively, models from existing studies could be combined. Models have utilised different types of ‘omic data, including proteomic, metagenomic and exome data. More popular has been sequencing array (SNP/GWAS) data, particularly when predicting autoimmune disease risk. By far the most prevalent type of data is the use of clinical and laboratory data.

42To optimise the use of these data types, accessibility is key, and EMRs allow easy extraction of these data. Some researchers have moved beyond only storing medical data in these systems. The eMERGE (electronic medical records and genomics) network combines the genomic and EMR repositories to further genomic medicine research. Other studies such as SPOKE (Scalable Precision Medicine Oriented Knowledge Engine), wish to integrate these data within the storage platform, by building a knowledge network using unsupervised machine learning that informs on how data types such as GWAS, gene ontology, pathways and drug data are connected to EMRs. Improving knowledge of how these data are related is a key step towards implementing precision medicine.

43Many models were created for autoimmune disease diagnosis, more specifically classifying those with disease and controls. The majority achieved high classifier performance (where any combination of the following metrics are over these thresholds: accuracy > 81%, AUC > 0.95, Sensitivity > 82, Specificity > 84), and provided evidence of machine learning’s utility in diagnostics.

44Identifying the molecular diagnosis to inform tailored treatment strategies has revolutionised cancer prognoses, improving patient outcomes and quality of life, along with economic benefits to the treatment provider. Targeted therapies such as monoclonal antibodies and small molecule inhibitors transformed treatment of some cancers, or improved patient survival times. Key to precision treatment has been the identification of the driver mutations specific to the cancer type. Machine learning has been utilised for cancer classification, and discovery of relevant pathways. Across the spectrum of autoimmune diseases, there has traditionally been a one-size-fits-all approach to patient therapeutics. The expectation is that machine learning represents a necessary key tool that will use ‘big’ data to stratify patients and move towards personalised treatment approaches that have proven so effective in cancer. Proof of this concept has already been demonstrated through machine learning to stratify patient’s inflammation status in RA, and further investigate IBD subtypes.

45Six models from the evaluated studies returned more than one of the following measures as either 1 or 100%: AUC, accuracy, precision and recall, sensitivity and specificity.,,,– This perfect performance indicates that a model may not be required, as there exists data that classifies the groups without error. An alternative explanation of apparently optimal performance may reside in poor implementation of cross-validation strategies.

46Common metrics reported are accuracy, AUC, and sensitivity and specificity. However, accuracy is inferior to AUC, particularly when imbalanced datasets are used. The AUC measure is unaffected by imbalanced data, but precision-recall curves may reflect model performance more accurately. Dataset rebalancing methods should potentially be utilised more for a thorough review of model performance.

47When creating and evaluating a model, increasing focus could be placed on which measure is more important, sensitivity or specificity. Scully et al. demonstrated this, where a lesion segmentation model could achieve high specificity (99.9%) through labelling all tissue as non-lesion.

48An ML model by Ahmed et al. provides evidence for using an additional independent test dataset subsequent to cross validation. In their study, the AUC dropped by 0.25, indicating decreased model performance on new data.

49Studies included in this systematic review have shown that artificial intelligence and machine learning models provide useful insight, despite the heterogeneity of presentation, diagnosis, disease course and patient outcome. However, the heterogeneity in data used, models and model evaluation cause difficulties in obtaining consensus. Furthermore, the number of autoimmune diseases this literature search focussed on was restricted, and may have resulted in an incomplete picture of ML applied to autoimmune diseases.

50From this analysis, it seems appropriate to advocate for standardised methods of model evaluation, by utilising a combination of cross validation and independent test data for model validation. Increased confidence in model results allows for more complex model creation, by layering data types or combining classifiers. These models could be applied to more difficult tasks that reflect the complexity of autoimmune disease. With these advances, AI and ML have the potential to bring personalised medicine closer for patients with complex and chronic disease.

51Autoimmune diseases selected for the systematic review are based on prevalence estimates and include Addison disease, alopecia, Coeliac disease, Crohn’s disease, ulcerative colitis, type 1 diabetes, autoimmune liver diseases, hyper- and hypo-thyroidism, multiple sclerosis, myasthenia gravis, polymyalgia rheumatica, psoriasis, psoriatic arthritis, rheumatoid arthritis, Sjӧgren syndrome, systemic sclerosis, systemic lupus erythematosus, systemic vasculitis, uveitis and vitiligo.

52The literature search was performed electronically with OvidSP using MEDLINE from 1946, and EMBASE from 1974. A search was also performed on the Computers & Applied Sciences Complete database available on EBSCO. The literature search was completed in December 2018. All searches conformed to the same structure: the words “machine learning” or “artificial intelligence” combined with the chosen search term(s) for each autoimmune disease (see Table 2). Boolean operators OR and AND (for combining search terms) were used in order to streamline the procedure. In both databases, the title, abstract and subject terms/keyword headings assigned by authors were searched (last search 17/12/2018).

53Studies that applied ML methods to autoimmune diseases listed above, or to complications that arise from autoimmune diseases were included. Studies not written in English, published prior to 2001, that did not use real human patient data, were not peer reviewed, or were review papers were also excluded. This systematic review conforms to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards.

54Further information on research design is available in the Nature Research Reporting Summary linked to this article.