Sparse Partial Least Squares Regression for Simultaneous Dimension Reduction and Variable Selection

Abstract

Summary Partial least squares regression has been an alternative to ordinary least squares for handling multicollinearity in several areas of scientific research since the 1960s. It has recently gained much attention in the analysis of high dimensional genomic data. We show that known asymptotic consistency of the partial least squares estimator for a univariate response does not hold with the very large p and small n paradigm. We derive a similar result for a multivariate response regression with partial least squares. We then propose a sparse partial least squares formulation which aims simultaneously to achieve good predictive performance and variable selection by producing sparse linear combinations of the original predictors. We provide an efficient implementation of sparse partial least squares regression and compare it with well-known variable selection and dimension reduction approaches via simulation experiments. We illustrate the practical utility of sparse partial least squares regression in a joint analysis of gene expression and genomewide binding data.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 38

1. 1.

   Abramovich F, Benjamini Y, Donoho DL, Johnstone IM. Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 2006;34:584-653.

   Not in our library

2. 2.

   d'Aspremont A, Ghaoui LE, Jordan MI, Lanckriet GRG. A direct formulation for sparse pca using semidefinite programming. SIAM Rev. 2007;49:434-448.

   Not in our library

3. 3.

   Bair E, Hastie T, Paul D, Tibshirani R. Prediction by supervised principal components. J. Am. Statist. Ass. 2006;101:119-137.

   Not in our library

4. 4.

   Bendel RB, Afifi AA. A criterion for stepwise regression. Am. Statistn. 1976;30:85-87.

   Not in our library

5. 5.

   Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Statist. Soc. B. 1995;57:289-300.

   Not in our library

6. 6.

   Boulesteix A-L, Strimmer K. Predicting transcription factor activities from combined analysis of microarray and chip data: a partial least squares approach. Theor. Biol. Med. Modllng. 2005;2.

   Not in our library10.1186/1742-4682-2-23

7. 7.

   Boulesteix A-L, Strimmer K. Partial least squares: a versatile tool for the analysis of high-dimensional genomic data. Brief. Bioinform. 2006;7:32-44.

   Not in our library10.1093/bib/bbl016

8. 8.

   ter Braak CJF, de Jong S. The objective function of partial least squares regression. J. Chemometr. 1998;12:41-54.

   Not in our library

9. 9.

   Butler NA, Denham MC. The peculiar shrinkage properties of partial least squares regression. J. R. Statist. Soc B. 2000;62:585-593.

   Not in our library

10. 10.

    Chun H, Keleş S. Expression quantitative loci mapping with multivariate sparse partial least squares. Genetics. 2009;182:79-90.

    Not in our library10.1534/genetics.109.100362

11. 11.

    Efron B, Hastie T, Johnstone I, Tibshirani R. Least angle regression. Ann. Statist. 2004;32:407-499.

    Not in our library

12. 12.

    Frank IE, Friedman JH. A statistical view of some chemometrics regression tools. Technometrics. 1993;35:109-135.

    Not in our library

13. 13.

    Friedman JH, Popescu BE. Gradient directed regularization for linear regression and classification. Technical Report.

    Not in our library

14. 14.

    Geman S. A limit theorem for the norm of random matrices. Ann. Probab. 1980;8:252-261.

    Not in our library

15. 15.

    Golub GH, van Loan CF. Matrix Computations. 1987.

    Not in our library

16. 16.

    Goutis C. Partial least squares algorithm yields shrinkage estimators. Ann. Statist. 1996;24:816-824.

    Not in our library

17. 17.
    'Gene shaving' as a method for identifying distinct sets of genes with similar expression patterns · 2000

    Not checked
18. 18.

    Helland IS. Partial least squares regression and statistical models. Scand. J. Statist. 1990;17:97-114.

    Not in our library

19. 19.

    Helland IS. Model reduction for prediction in regression models. Scand. J. Statist. 2000;27:1-20.

    Not in our library

20. 20.

    Helland IS, Almoy T. Comparison of prediction methods when only a few components are relevant. J. Am. Statist. Ass. 1994;89:583-591.

    Not in our library

21. 21.

    Huang X, Pan W, Park S, Han X, Miller LW, Hall J. Modeling the relationship between lvad support time and gene expression changes in the human heart by penalized partial least squares. Bioinformatics. 2004;20:888-894.

    Not in our library10.1093/bioinformatics/btg499

22. 22.

    Johnstone IM, Lu AY. Sparse principal component analysis. Technical Report.

    Not in our library

23. 23.

    Jolliffe IT, Trendafilov NT, Uddin M. A modified principal component technique based on the lasso. J. Computnl Graph. Statist. 2003;12:531-547.

    Not in our library

24. 24.

    de Jong S. SIMPLS: an alternative approach to partial least squares regression. Chemometr. Intell. Lab. Syst. 1993;18:251-263.

    Not in our library

25. 25.

    Kosorok MR, Ma S. Marginal asymptotics for the ‘‘large p, small n’’ paradigm: with applications to microarray data. Ann. Statist. 2007;35:1456-1486.

    Not in our library

26. 26.

    Krämer N. An overview on the shrinkage properties of partial least squares regression. Computnl Statist. 2007;22:249-273.

    Not in our library

27. 27.

    Lee TI, Rinaldi NJ, Robert F, Odom DT, Bar-Joseph Z, Gerber GK, Hannett NM, Harbison CT, Thomson CM, Simon I, Zeitlinger J, Jennings EG, Murray HL, Gordon DB, Ren B, Wyrick JJ, Tagne J-B, Volkert TL, Fraenkel E, Gifford DK, Young RA. Transcriptional regulatory networks in saccharomyces cerevisiae. Science. 2002;298:799-804.

    Not in our library10.1126/science.1075090

28. 28.

    Nadler B, Coifman RR. The prediction error in cls and pls: the importance of feature selection prior to multivariate calibration. J. Chemometr. 2005;19:107-118.

    Not in our library

29. 29.

    Naik P, Tsai C-L. Partial least squares estimator for single-index models. J. R. Statist. Soc. B. 2000;62:763-771.

    Not in our library

30. 30.

    Pratt JW. On interchanging limits and integrals. Ann. Math. Statist. 1960;31:74-77.

    Not in our library

31. 31.

    Rosipal R, Krämer N, Saunders C, Grobelnik M, Gunn S, Shawe-Taylor J. Overview and recent advances in partial least squares. Subspace, Latent Structure and Feature Selection Techniques. 2006;34-51.

    Not in our library

32. 32.

    Spellman PT, Sherlock G, Zhang MQ, Iyer VR, Anders K, Eisen MB, Brown PO, Botstein D, Futcher B. Comprehensive identification of cell cycle-regulated genes of the yeast saccharomyces cerevisiae by microarray hybridization. Molec. Biol. Cell. 1998;9:3273-3279.

    Not in our library10.1091/mbc.9.12.3273

33. 33.

    Stoica P, Soderstorom T. Partial least squares: a first-order analysis. Scand. J. Statist. 1998;25:17-24.

    Not in our library

34. 34.

    Tibshirani R. Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B. 1996;58:267-288.

    Not in our library

35. 35.

    Wang L, Chen G, Li H. Group scad regression analysis for microarray time course gene expression data. Bioinformatics. 2007;23:1486-1494.

    Not in our library10.1093/bioinformatics/btm125

36. 36.

    Wold H. Estimation of Principal Components and Related Models by Iterative Least Squares. 1966.

    Not in our library

37. 37.

    Zou H, Hastie T. Regularization and variable selection via the elastic net. J. R. Statist. Soc. B. 2005;67:301-320.

    Not in our library

38. 38.

    Zou H, Hastie T, Tibshirani R. Sparse principal component analysis. J. Computnl Graph. Statist. 2006;15:265-286.

    Not in our library

Cited by papers we hold 9

• In-depth transcriptomic analysis of human retina reveals molecular mechanisms underlying diabetic retinopathy · 2021
  Not checked
• Metabolomics and Multi-Omics Integration: A Survey of Computational Methods and Resources · 2020
  No notice
• So you think you can PLS-DA? · 2020
  No notice
• Microbiome Multi-Omics Network Analysis: Statistical Considerations, Limitations, and Opportunities · 2019
  No notice
• Antibiotic perturbation of the murine gut microbiome enhances the adiposity, insulin resistance, and liver disease associated with high-fat diet · 2016
  No notice
• A Systematic Comparison of Linear Regression–Based Statistical Methods to Assess Exposome-Health Associations · 2016
  No notice
• Multiplexed Immunoassay Panel Identifies Novel CSF Biomarkers for Alzheimer's Disease Diagnosis and Prognosis · 2011
  No notice
• Sparse PLS discriminant analysis: biologically relevant feature selection and graphical displays for multiclass problems · 2011
  No notice
• Regularized estimation of large-scale gene association networks using graphical Gaussian models · 2009
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.