The removal of multiplicative, systematic bias allows integration of breast cancer gene expression datasets – improving meta-analysis and prediction of prognosis

1Successful microarray experiments are reliant on sufficient care being taken to minimize and account for experimental variability. Formalization and control of all stages of the experimental pipeline is now routine, and the need to associate experiments with detailed descriptions of protocols and techniques is now widely accepted. However, despite these efforts, it is still not possible to account for all potential sources of variation, and even identical experiments performed at different sites have been shown to produce significantly different results. This makes it difficult to routinely compare gene expression data generated from different experiments, even when using samples from comparable sources that have been processed on the same microarray platform using similar protocols.

2Issues of experimental reproducibility have become increasingly important with the advent of microarray databases and repositories (e.g. ArrayExpress, GEO), given the potential they offer for cross-experimental comparison and data mining. Even if it is possible to successfully control inter-experiment variation to a point where this might be possible, rapid developments in both the hybridization protocols and in the arrays themselves have also led to major improvements in the technology. Lower requirements for the amount of starting RNA have enabled gene expression profiling to be combined with cell sorting methods or laser capture microdissection, while increases in the number of features represented on the arrays have resulted in progressively more detailed coverage of the transcriptome. Since each advance in technology leads to genuine improvements, there is a strong incentive to use the latest arrays and protocols whenever possible. This is however, tempered by a lack of backward compatibility between datasets produced using different array and protocol versions, and any decision to move to a newer (better) iteration of the technology must be made with an appreciation of the difficulty in maintaining compatibility with previous studies.

3In this study we first demonstrate, using an extended series of validation data, that Affymetrix datasets cannot in general be compared at the raw expression level due to systematic, multiplicative biases. Secondly, we show that simple batch mean-centering can significantly reduce the level of inter-experimental variation and that this allows raw transcript levels to be compared across datasets. The approach is then applied to a series of published breast cancer studies and we show that the integrated datasets possess increased statistical power and improved prognostic ability, compared to the individual datasets alone.

4All validation datasets (consisting of six GeneChips each) were produced by hybridizing triplicate RNA samples from a breast cancer cell line (MCF7) and an immortalised normal breast cell line (MCF10A) using a variety of different array types and sample preparation protocols. In all cases, the aim of the validation study was to assess the correspondence between the sets of differentially expressed probesets (or transcripts) identified when using different versions of the same underlying technology. In the hypothetical situation when all datasets yield identical results, the same set of differentially expressed probesets would be identified, irrespective of the array type or protocol used to process the data; we considered how close the data approaches this ideal. For example, a comparison of the fold changes between datasets (MCF7-amplified/MCF10A-amplified vs. MCF7-unamplified/MCF10A-unamplified) generated using Affymetrix's small sample protocol and Affymetrix's standard protocol yields good, but not perfect correspondence (Fig 1A, Table 1). However, when fold changes are calculated across datasets (MCF7-amplified/MCF10A-unamplified vs. MCF7-unamplified/MCF10A-amplified) correspondence falls dramatically (Fig. 1B grey dots, Table 1).

5Batch mean-centering (see methods) of the amplified and unamplified datasets was found to dramatically increase the correspondence comparison across the datasets, with 100% of probesets having fold changes within two-fold between experimental branches (Fig. 1B black dots, Table 1). Similarly, when significance analysis of microarrays (SAM) was used to identify lists of probesets with statistically significant changes between the same replicate groupings used to generate Fig. 1, the intersection between sets was also found to be greater following mean batch correction (Table 1) and Pearson correlation of raw intensities was also found to increase (Figs. 1C, 1D and Additional File 1). It is notable that while correction improves fold-change correspondence across protocols, fold changes between datasets are preserved (Fig. 1E, Table 1).

6The same approach was applied to a variety of other validation datasets that were designed to investigate the effect of using different generations of Affymetrix GeneChips, alternative protocols and scanning hardware (Table 1, Additional File 1). A systematic bias was found to be present in all datasets, and correspondence improved following mean-centering in all cases (median centering also performed similarly; data not shown). These results demonstrate that systematic, multiplicative bias is a widespread property within Affymetrix array data, and mean-centering was found to lead to improvements irrespective of the summarisation method used (RMA or MAS5) to process the data or when using alternative Chip Description Files (CDFs) to group probes according to Unigene cluster (Additional File 2).

7Breast tumours have been classified into five molecular subtypes; basal, luminal A, luminal B, ERBB2 and normal-like by identifying a set of genes with significantly greater variation in expression between different tumours than between paired samples from the same tumour. Since members of this set appear to define properties 'intrinsic' to each subtype, the authors referred to the genes as an 'intrinsic gene set'. 640 Affymetrix probesets representing the 534 'intrinsic gene set' from were identified using NetAffx. These probesets were used to cluster MAS5 normalised expression data from two published Affymetrix gene expression studies with similar numbers and composition of tumours. Despite using similar starting material (primary tumours) and the same microarray platform, when combined the two datasets formed two distinct, independent clusters representing the two datasets (Fig. 2A), suggesting a dataset-specific systematic bias as observed with the validation datasets described above. Although clusters of known luminal and basal-specific genes show similar patterns of differential expression in each of the two datasets (Figure 2A iii, iv), the majority of the probesets representing the full 'intrinsic gene set' show greater differences in expression between the two studies than between the different classes of tumours. Following mean-centering as before, the 'basal-like tumours' from the Richardson et al. dataset were found to cluster alongside the 'basal tumours' from Farmer et al., and the 'non-basal tumours' with the 'luminal tumours' (Fig. 2B). A third cluster of tumours was identified with high expression of the ERBB2 cluster of probesets. This cluster contained all of the molecular apocrine tumours, plus a mixture of basal/basal-like and luminal/non-basal-like tumours. Using centroid prediction as described previously, the tumours from both datasets were assigned to the five Norway/Stanford subtypes (basal, luminal A, luminal B, ERBB2 and normal-like), one of the tumours in this third cluster was assigned to the luminal B subtype, thirteen to the ERBB2 subtype and 9 could not clearly be assigned to a subtype (Fig 2B v).

8The greatest single difference between molecular subtypes has repeatedly been demonstrated to be between estrogen receptor-positive (ER+) luminal tumours and ER-negative basal tumours. SAM analysis was used to identify probesets differentially expressed between the basal/basal-like and non basal-like/luminal subtypes using the combined data from both sets of samples. It was only following mean-centering that probesets were identified that represent genes that are known to characterize the differences between these subtypes (Additional File 3), including the fundamental estrogen receptor alpha and GATA binding protein 3, which maintains differentiation into the luminal cell fate in the mammary gland. In addition, following mean-centering, a greater number of statistically significant probesets were found to be differentially expressed between the tumour subtypes than were found between the two initial sets of samples (Additional File 4).

9These results encouraged us to apply the approach to integrate six previously published datasets (Table 2) of primary breast cancer tumours processed on Affymetrix U133A, U133AA or plus 2.0 GeneChips. Multidimensional scaling of 1107 tumours based upon the expression of all common probesets between the three arrays (22,215) demonstrated that global gene expression is highly influenced by dataset, with tumours clustering by study (Fig. 3A), again suggesting a systematic, dataset-specific bias. However, following mean-centering, the tumours appear to cluster by breast cancer subtype (assigned using centroid prediction), regardless of the dataset from which they were generated (Fig. 3B).

10A growing body of evidence has accumulated, supporting the notion that gene expression profiling of primary breast tumours can be used to stratify patients by subtype and the likelihood of disease progression (reviewed in). The approaches have included both unsupervised (intrinsic gene set) and supervised (genes associated with patient follow up) methods, along with studies of cancer-associated pathways or tumor characteristics, in all cases these signatures appear to predict recurrence, despite the lack of overlap in their respective profiles or signatures. In order to establish whether integrating multiple datasets can improve prognostic prediction, the six published datasets (described above) were used individually or in combination as 'training sets' for supervised principal components analysis. This method has been shown to produce more accurate predictions than several competing methods on both simulated and real microarray datasets. Using the Superpc package for R, a Cox proportional hazards model was fitted to each predictor (generated for all combinations of one to five datasets used as the 'training set') and cross validation curves were plotted to determine the optimum threshold for the predictor of survival as described previously. The 1st supervised principal component was found to be the most significant in the vast majority of cases, which is consistent with the hypothesis that recurrence is an inherent property of primary tumour gene expression (examples of cross-validation and survival curves are shown in Additional File 5). The remaining dataset(s) were used as a test set for each predictor and an R2 statistic was computed to assess the performance. Combining greater numbers of datasets or tumours significantly improves prediction of prognosis based upon gene expression data (Fig. 4). Mean-centering of the datasets significantly increased the correlation between the supervised principal components and clinical follow up, therefore improving prognostic performance. It is clear that the predictive power of some combinations of training and test sets is more reliable than others. Although only a limited number of patient and tumour characteristics were available (Table 2), it seems that the most accurate predictions are achieved for test datasets that have characteristics most similar to those of the individual or combined training dataset. R2 statistic and p-values (log rank) for all possible combinations of training datasets and test datasets are given in Additional File 6.

11We investigated the effect of altering the composition of luminal (ER+) and basal (ER-) tumours from the two published datasets of Farmer et al. and Richardson et al. compared above. Unbalancing the composition of the datasets from a 1:1 ratio of basal to luminal tumours to a 2:1 or 5:1 ratio of tumours reduced correspondence between datasets and caused a distortion across datasets (Additional File 7). Similar results were also observed with the MCF7/MCF10A datasets described above (Fig 1F, Table 3). Weighted-mean-centering for ER status removed the distortion but also reduced correspondence for the 2:1 ratio of luminal tumours, and increased correspondence in the 5:1 ratio luminal to basal comparison, at the expense of high false discovery rates (Table 3). An extreme example of the effect of dataset composition was seen when looking at the expression level of the estrogen receptor in homogeneous datasets from Loi et al. and Minn et al. composed wholly of ER+ or ER- tumours. Following mean-centering it appears that these datasets contain a mixture of ER+ and ER- tumours (Additional File 8A). Replacing any of the six heterogenous datasets above (containing 67–85% ER+ tumours) with homogeneous datasets (containing only ER+ or ER- tumours) showed a dramatic reduction in the correlation between dataset or tumour number and prediction of recurrence (Additional File 8B). Using weighted mean-centering to account for the differences in the composition of ER+ tumours in five out of the six datasets (individual ER status by immunohistochemistry for tumours in the Pawitan et al. dataset was not available) did not significantly improve prognostic performance over mean-centering alone (Additional File 9).

12The mean-centering approach was compared with a previously described method for integrating breast cancer tumour microarray data generated on different platforms, using a distance weighted discrimination (DWD) method to adjust for systematic microarray data biases. For both the validation datasets and the published datasets, mean-centering out-performed DWD (Table 3 and Additional File 10).

13An alternative approach to assess whether mean-centering improves comparisons across published datasets was to identify lists of the five hundred most highly differentially expressed probesets across each dataset (those with the highest variance) and compare these gene lists with the most differentially expressed probesets from other single or combined datasets. Bringing together greater numbers of datasets or tumours increased the overlap in differentially expressed probesets (Figure 5). The number of probesets in common was significantly greater with mean-centering (p = 3.6 × 10-107) or weighted mean-centering (p = 9.2 × 10-106) over uncorrected data, although there was no significant improvement between the two methods (p = 0.5). The mean number of genes in common was higher following weighted mean-centering than mean-centering when the dataset was made up of less ER-positive tumours (Chin et al., Farmer et al. and Richardson et al.) and lower when the dataset was made up of more ER-positive tumours (Ivshina et al., Sotoriou et al., Wang et al.).

14Mean-centering has been widely used in the past to compare relative gene expression of high and lowly expressed genes together within a single dataset, particularly for heatmaps and clustering programs. However, this is the first study to assess the utility of mean-centering for minimising the effects of dataset-specific bias and integration of multiple datasets. An unknown, systematic, multiplicative bias associated with each group of arrays processed together ('dataset') is simply removed when the GeneChips are considered relative to each other. The approach clearly shows significant improvements in the degree of correspondence found across datasets, without any loss of internal coherence within each of the initial datasets from which the integrated dataset is assembled. Relative intensities within each individual dataset are left unchanged (Figure 1D), with the consequence that both fold-changes and p-values produced by techniques such as SAM, remain identical to those found prior to correction (Table 1). Therefore, balanced corrected datasets can be treated with at least as much confidence as the initial uncorrected data. We have also demonstrated that combining greater numbers of datasets or tumours increases the overlap in differentially expressed probesets between studies and that this is further improved with mean-centering.

15A number of previous studies have also investigated the level of consensus found between different experimental datasets. The mean-centering approach out-performed a distance weighted discrimination method that attempted to adjust for systematic microarray data biases for integrating breast cancer tumour microarray data generated on different platforms. This group stated that they had also applied this technique to 'merge two distinct Affymetrix breast tumor datasets together' and 'saw similar, but less dramatic results due to fewer systematic biases present in datasets performed on the same Affymetrix microarrays'. Our results suggest that there are many sources of systematic biases in Affymetrix data, which are highly significant and multiplicative, but that these can be largely corrected for, allowing the integration of datasets. An empirical Bayes method to adjust for batch effects (ComBat; ) has also been used to integrate published datasets for meta-analysis. This approach generated plots analogous to those in Additional File 8 for mean-centering and weighted mean-centering when ER status was used as a covariate (data not shown). The mean-centering method described in this study was used in a recent meta-analysis whilst our manuscript was under review, although no attempt was made to account for differences in dataset composition.

16Combining two published studies without mean-centering, clearly demonstrated how dataset-specific biases can mask the biological differences between breast cancer tumour subtypes (Figure 2). The Farmer et al. dataset was generated from trucut tumour biopsies (4 × 2.5 μm sections), necessitating RNA amplification prior to hybridization to U133A GeneChips. By contrast, RNA in the Richardson et al. 2006 study was derived from tumours following surgical removal, so amplification was not required prior to hybridisation to U133 plus 2.0 GeneChips. Despite the experimental differences between the studies, both of which have been shown above to lead to significant deviations in measured raw intensities in our validation datasets, mean-centering appears to reconcile the data and leads to the identification of biologically plausible relationships not found when combining uncorrected data.

17The gold standard for demonstrating the power of a gene expression classifier is to test it against independent datasets. However, if the molecular profile of a set of tumours is representative of its patient characteristics, then any prognostic signature will be dependant upon the composition of the patient cohort and therefore be dataset-specific. Thus in order to generate accurate prognostic predictions, the characteristics of this second 'test' dataset must have similar characteristics to the first 'training' set. Recently, strong time dependence was identified for a prognostic signature when comparing an independent validation dataset with a longer median follow-up time (14 years) compared to the original study (8 years). A number of recent microarray studies have been performed after increasing the size of a dataset with additional samples, however it is unclear whether subsequent changes in the results are due to changes in the sample composition of the extended dataset or simply to technical effects arising from the microarrays being processed in different batches. Some studies have also based their findings upon combined data from more than one type of Affymetrix GeneChip without evaluating any GeneChip-specific effects.

18By integrating six published datasets with patient follow-up information we have demonstrated that combining breast cancer datasets can increase the accuracy of prognosis prediction and that this can be improved by removing systematic, multiplicative bias. The most accurate prognosis predictions are generated when the test-sets closely share the patient and tumour characteristics of the training-sets. An alternative approach to building ever larger combined datasets representing the whole breast cancer population, would be to concentrate on generating gene expression classifiers for clearly defined groups of patients (e.g. node-negative, ER-positive from patients aged 50–60, with 10 years of follow-up). Strict entry criterion would severely restrict the number of tumours eligible for inclusion, whilst taking no account of possible unknown confounding factors. In clinical practice, we urgently need single sample predictors, applicable to all patients and our work strongly suggests that these will be best generated from the largest possible cohorts (or integrated datasets) representing the wider population, which will involve large international collaborations and public sharing of data. The current consensus for best practices for breast cancer treatment are based upon bringing together data for hundreds of trials representing thousands of women within the Early Breast Cancer Trialists' Collaborative Group (EBCTCG). If we can begin to bring together many large datasets of gene expression data free from dataset-specific bias the opportunity exists to create a highly valuable resource. A possible 'new' breast cancer subtype characterized by the high expression of interferon regulated genes was identified by cluster analysis of a combined (non-Affymetrix) dataset of 315 breast tumours, which is consistent with the notion that rare molecular subtypes will only be detected with larger datasets. Our findings are in agreement with the conclusions of a recent study that integrated breast tumour datasets generated on two different microarray platforms; they also showed that the gene expression profile generated by integrated analysis of multiple datasets achieves better prediction of breast cancer recurrence, and that the performance of profiles is confounded by the known and unknown clinical background of patients. In the current study however, we demonstrate improved prediction of prognosis in datasets derived from the same platform integrated using a simpler scaling method of the raw data rather than a normalisation method reliant on fold changes. One limitation of our study is that it was not possible to use a single definition of follow-up endpoint across the published datasets, in each case we used the most conservative indicator of relapse available (recurrence-free survival, disease-free survival and distant metastasis-free survival) rather than overall survival. There was also some variation in both patient age and tumour size between the studies (Table 3). Variation in gene expression due to the heterogeneity of patient characteristics has begun to be addressed by studies that investigate the effects of age, race and differences in risk factors for breast cancer. Integrating large breast tumour gene expression datasets will potentially enable us to uncover more subtle population-level associations, providing that all clinical details and follow-up information is consistent, complete and made publicly available.

19Systematic multiplicative biases are introduced at many stages of microarray experiments, however they can easily be accounted for, which can enable raw data to be directly integrated from different gene expression datasets in order to generate results with improved statistical power and greater biological significance.

20Generation and processing of RNA comparing the two breast cell lines, MCF7 (cancer) and MCF10A (normal immortalised mammary epithelial) was described previously. Briefly, cells were grown in Dulbecco's modified Eagle's medium (DMEM) with 10% fetal calf serum (MCF7) or DMEM/F12 with 5% horse serum, 2 ng/ml, 0.5 μg/ml hydrocortisone, 0.5 μg/ml cholera toxin, and 5 μg/ml insulin (MCF10A). Minimally passaged cells (< 20) were obtained from the American Type Culture Collection ATCC. RNA was isolated using Trizol® (Ambion) according to manufacturer's instructions, purified using Qiagen RNeasy columns (Qiagen, Valencia, CA) and quantified using a Nanodrop spectrophotometer (Labtech). The quality and amount of starting RNA was confirmed with an Agilent Bioanalyzer 2100 (Agilent) prior to labelling and hybridisation to HG-U133A, HG-U133 plus 2.0, or Human Exon 1.0ST GeneChips (Affymetrix) using either the Affymetrix standard or small sample preparation protocols as previously described.

21Each experiment was repeated in triplicate, with three samples per cell line for each amount of starting total RNA, protocol or GeneChip used (1 dataset = 3 × MCF7 and 3 × MCF10A = 6 GeneChips). The HGU133A Genechips for the standard protocol and amplification experiments were scanned using a GeneArray 2500 scanner and the HG-U133 plus 2.0, and Exon 1.0ST Genechips were scanned using a GeneChip Scanner 3000. All MCF7 and MCF10A microarray data is MIAME compliant and accessible via MIAME VICE. All protocols are described in full here,. Raw spot readings were processed using R and Bionconductor. Probeset summarisation was done using MAS 5.0 and RMA as implemented in the Simpleaffy package or plier algorithm from Affymetrix ExACT software. Mappings between the Exon and U133A plus 2.0 GeneChips were performed as described previously. Alternative cell description files, relating probesets to unigene sequences were implemented as described previously. Lists of common significantly differentially expressed genes before and after mean batch-centering were identified using SAM analysis (siggenes BioConductor package) by adjust Δ value to find the top 1000 differentially expressed probesets using each protocol, as described previously.

22The mean expression level per probeset for a given dataset is subtracted from the individual GeneChip expression level on the Log2 scale. This can simply be performed in R using the 'rowMeans' function. Whilst preparing the manuscript we also noticed that this can be achieved using the 'pamr.batchadjust' function within the pamr Bioconductor package.

23Affymetrix data was downloaded from a total of ten datasets from published studies listed in Table 2 from the Gene Expression Omnibus or Array Express repositories. Raw.cel files were not available for the Wang et al. dataset, so all other datasets were normalized as in this study using the MAS 5.0 algorithm with a target intensity of 600 as implemented in the Simpleaffy package, using R within BioConductor. NetAffx was used to identify Affymetrix probesets representing the 'intrinsic gene set' previously used to classify human breast tumours. Centered average linkage clustering was performed using the Cluster and TreeView programs as described previously. Supervised principal components analysis using the Superpc for R package was used as previously described, in order to compare the predictive power of combining different published datasets. The follow up endpoints for the Loi et al., Pawitan et al. and Sotoriou et al. datasets were recurrence-free survival, for Desmedt et al. and Ivshina et al. datasets it was disease-free survival and for the Minn et al. and Wang et al. datasets it was distant metastasis-free survival.

24The authors declare that they have no competing interests.

25AHS conceived and designed the study. RBC, YH and SDP processed the lab experiments. AHS, GS and MJO performed the computational experiments and analysis. AHS, CJM, AH and RBC drafted the manuscript. All authors read and approved the final manuscript.

26The pre-publication history for this paper can be accessed here: