No retraction, correction or concern found when last checked on Nov 3, 2026.

Benchmarking Parametric and Machine Learning Models for Genomic Prediction of Complex Traits

Christina B Azodi, Emily Bolger, Andrew McCarren and 3 more (opens in a new tab)

G3 Genes|Genomes|Genetics · 2019

doi:10.1534/g3.119.400498 (opens in a new tab)

Abstract

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e., ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e., feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 70

1.
Deep learning for computational biology · 2016
No notice
2.
Beaulieu J., Doerksen T. K., MacKay J., Rainville A., and Bousquet J., 2014 Genomic selection accuracies within and between environments and small breeding groups in white spruce. BMC Genomics 15: 1048
Not in our library10.1186/1471-2164-15-1048 (opens in a new tab)
3.
Bellot P., de los Campos G., and Pérez-Enciso M., 2018 Can Deep Learning Improve Genomic Prediction of Complex Human Traits? Genetics genetics.2018.
Not in our library10.1534/genetics.118.301298 (opens in a new tab)
4.
Benjamini Y., and Hochberg Y., 1995 Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. J R Stat Soc Ser. B Stat Methodol 57: 289–300.
Not in our library
5.
Application of high-dimensional feature selection: evaluation for genomic prediction in man · 2015
No notice
6.
Blondel M., Onogi A., Iwata H., and Ueda N., 2015 A Ranking Approach to Genomic Selection. PLoS One 10: e0128570
Not in our library10.1371/journal.pone.0128570 (opens in a new tab)
7.
Breiman L., 2001 Random Forests. Mach. Learn. 45: 5–32.
Not in our library10.1023/a:1010933404324 (opens in a new tab)
8.
de los Campos G., Gianola D., Rosa G. J., Weigel K. A., and Crossa J., 2010 Semi-parametric genomic-enabled prediction of genetic values using reproducing kernel Hilbert spaces methods. Genet. Res. 92: 295–308.
Not in our library10.1017/s0016672310000285 (opens in a new tab)
9.
de los Campos G., Hickey J. M., Pong-Wong R., Daetwyler H. D., and Calus M. P. L., 2013 Whole-Genome Regression and Prediction Methods Applied to Plant and Animal Breeding. Genetics 193: 327–345.
Not in our library10.1534/genetics.112.143313 (opens in a new tab)
10.
de los Campos G., Naya H., Gianola D., Crossa J., Legarra A., 2009 Predicting quantitative traits with regression models for dense molecular markers and pedigree. Genetics 182: 375–385.
Not in our library10.1534/genetics.109.101501 (opens in a new tab)
11.
Chapelle O., Vapnik V., Bousquet O., and Mukherjee S., 2002 Choosing Multiple Parameters for Support Vector Machines. Mach. Learn. 46: 131–159.
Not in our library10.1023/a:1012450327387 (opens in a new tab)
12.
Desta Z. A., and Ortiz R., 2014 Genomic selection: genome-wide prediction in plant improvement. Trends Plant Sci. 19: 592–601.
Not in our library10.1016/j.tplants.2014.05.006 (opens in a new tab)
13.
Dietterich T. G., 2000 Ensemble methods in machine learning. Int. Workshop Mult. Classif. Syst.
Not in our library10.1007/3-540-45014-9_1 (opens in a new tab)
14.
Ehret A., Hochstuhl D., Gianola D., and Thaller G., 2015 Application of neural networks with back-propagation to genome-enabled prediction of complex traits in Holstein-Friesian and German Fleckvieh cattle. Genet. Sel. Evol. 47: 22
Not in our library10.1186/s12711-015-0097-5 (opens in a new tab)
15.
Endelman J. B., 2011 Ridge regression and other kernels for genomic selection with R package rrBLUP. Plant Genome 4: 250
Not in our library10.3835/plantgenome2011.08.0024 (opens in a new tab)
16.
Evans J., Crisovan E., Barry K., Daum C., Jenkins J., 2015 Diversity and population structure of northern switchgrass as revealed through exome capture sequencing. Plant J. 84: 800–815.
Not in our library10.1111/tpj.13041 (opens in a new tab)
17.
Evans J., Sanciangco M. D., Lau K. H., Crisovan E., Barry K., 2018 Extensive Genetic Diversity is Present within North American Switchgrass Germplasm. Plant Genome. 11
Not in our library10.3835/plantgenome2017.06.0055 (opens in a new tab)
18.
Fernandes S. B., Dias K. O. G., Ferreira D. F., and Brown P. J., 2017 Efficiency of multi-trait, indirect, and trait-assisted genomic selection for improvement of biomass sorghum. Theor. Appl. Genet. 131: 747–755.
Not in our library10.1007/s00122-017-3033-y (opens in a new tab)
19.
Friedman J. H., 2001 Greedy function approximation: A gradient boosting machine. Ann. Stat. 29: 1189–1232.
Not in our library10.1214/aos/1013203451 (opens in a new tab)
20.
Gianola D., Fernando R. L., and Stella A., 2006 Genomic-assisted prediction of genetic value with semiparametric procedures. Genetics 173: 1761–1776.
Not in our library10.1534/genetics.105.049510 (opens in a new tab)
21.
Girija, S. S., 2016 Tensorflow: Large-scale machine learning on heterogeneous distributed systems.
Not in our library
22.
Glorot X., and Bengio Y. Understanding the difficulty of training deep feedforward neural networks. 2010.
Not in our library
23.
Goddard M., 2009 Genomic selection: prediction of accuracy and maximisation of long term response. Genetica 136: 245–257.
Not in our library10.1007/s10709-008-9308-0 (opens in a new tab)
24.
Genome-enabled prediction of genetic values using radial basis function neural networks · 2012
No notice
25.
González-Camacho J. M., Crossa J., Pérez-Rodríguez P., Ornella L., and Gianola D., 2016 Genome-enabled prediction using probabilistic neural network classifiers. BMC Genomics 17: 208
Not in our library10.1186/s12864-016-2553-1 (opens in a new tab)
26.
Applications of Machine Learning Methods to Genomic Selection in Breeding Wheat for Rust Resistance · 2018
Not checked
27.
González-Recio O., and Forni S., 2011 Genome-wide prediction of discrete traits using Bayesian regressions and machine learning. Genet. Sel. Evol. 43: 7
Not in our library10.1186/1297-9686-43-7 (opens in a new tab)
28.
González-Recio O., Jiménez-Montero J. A., and Alenda R., 2013 The gradient boosting algorithm and random boosting for genome-assisted evaluation in large data sets. J. Dairy Sci. 96: 614–624.
Not in our library10.3168/jds.2012-5630 (opens in a new tab)
29.
Extension of the bayesian alphabet for genomic selection · 2011
No notice
30.
Hansey C. N., Johnson J. M., Sekhon R. S., Kaeppler S. M., and de Leon N., 2011 Genetic diversity of a maize association population with restricted phenology. Crop Sci. 51: 704–715.
Not in our library10.2135/cropsci2010.03.0178 (opens in a new tab)
31.
Heffner E. L., Sorrells M. E., and Jannink J.-L., 2009 Genomic Selection for Crop Improvement. Crop Sci. 49: 1–12.
Not in our library10.2135/cropsci2008.08.0512 (opens in a new tab)
32.
Heslot N., Yang H.-P., Sorrells M. E., and Jannink J.-L., 2012 Genomic Selection in Plant Breeding: A Comparison of Models. Crop Sci. 52: 146–160.
Not in our library10.2135/cropsci2011.06.0297 (opens in a new tab)
33.
Data and Theory Point to Mainly Additive Genetic Variance for Complex Traits · 2008
No notice
34.
Hirsch C. N., Foerster J. M., Johnson J. M., Sekhon R. S., Muttoni G., 2014 Insights into the maize pan-genome and pan-transcriptome. Plant Cell 26: 121–135.
Not in our library10.1105/tpc.113.119982 (opens in a new tab)
35.
Holland J. B., 2007 Genetic architecture of complex traits in plants. Curr. Opin. Plant Biol. 10: 156–161.
Not in our library10.1016/j.pbi.2007.01.003 (opens in a new tab)
36.
Jonas E., and de Koning D.-J., 2013 Does genomic selection have a future in plant breeding? Trends Biotechnol. 31: 497–504.
Not in our library10.1016/j.tibtech.2013.06.003 (opens in a new tab)
37.
Kasnavi S. A., Afshar M. A., Shariati M. M., Kashan N. E. J., and Honarvar M., 2017 Performance evaluation of support vector machine (SVM)-based predictors in genomic selection. Indian J. Anim. Sci. 87: 1226–1231.
Not in our library
38.
Kingma D. P., and Ba J., 2014 Adam: A Method for Stochastic Optimization. arXiv. https://arxiv.org/abs/1412.6980
Not in our library
39.
Klimasauskas, C. C., 1993 Applying Neural Networks In: Neural Networks in Finance and Investing: Using Artificial Intelligence to Improve Real World Performance, pp. 64–65. Trippi R. R. and Turban E., Probus, Chicago. ISBN: 1557384525.
Not in our library
40.
Kuhn M., and Johnson K., 2013 Over-Fitting and Model Tuning. Applied Predictive Modeling. Springer, New York, NY 61–92.
Not in our library10.1007/978-1-4614-6849-3_4 (opens in a new tab)
41.
LeBaron B., and Weigend A. S., 1998 A bootstrap evaluation of the effect of data splitting on financial time series. IEEE Trans. Neural Netw. 9: 213–220.
Not in our library10.1109/72.655043 (opens in a new tab)
42.
Accelerating the switchgrass (Panicum virgatum L.) breeding cycle using genomic selection approaches. · 2014
Not checked
43.
Long N., Gianola D., Rosa G. J. M., and Weigel K. A., 2011 Application of support vector regression to genome-assisted prediction of quantitative traits. Theor. Appl. Genet. 123: 1065–1074.
Not in our library10.1007/s00122-011-1648-y (opens in a new tab)
44.
Lorenz A. J., Chao S., Asoro F. G., Heffner E. L., Hayashi T., 2011 Genomic Selection in Plant Breeding: Knowledge and Prospects, chap 2. Elsevier, Amsterdam, Netherlands
Not in our library10.1016/b978-0-12-385531-2.00002-5 (opens in a new tab)
45.
Ma W., Qiu Z., Song J., Li J., Cheng Q., 2018 A deep convolutional neural network approach for predicting phenotypes from genotypes. Planta 248: 1307–1318.
Not in our library10.1007/s00425-018-2976-9 (opens in a new tab)
46.
Accuracy of breeding values of 'unrelated' individuals predicted by dense SNP genotyping · 2009
No notice
47.
Meuwissen T. H.., Hayes B. J., and Goddard M. E., 2001 Prediction of Total Genetic Value Using Genome-Wide Dense Marker Maps. Genetics 157: 1819–1829.
Not in our library10.1093/genetics/157.4.1819 (opens in a new tab)
48.
Monir M. M., and Zhu J., 2018 Dominance and Epistasis Interactions Revealed as Important Variants for Leaf Traits of Maize NAM Population. Front. Plant Sci. 9: 627
Not in our library10.3389/fpls.2018.00627 (opens in a new tab)
49.
A comparison of five methods to predict genomic breeding values of dairy bulls from genome-wide SNP markers · 2009
No notice
50.
Neves, H. H., R. Carvalheiro, and S. A. Queiroz, 2012 A comparison of statistical methods for genomic selection in a mice population. 13: 100
Not in our library10.1186/1471-2156-13-100 (opens in a new tab)
51.
Optimising Genomic Selection in Wheat: Effect of Marker Density, Population Size and Population Structure on Prediction Accuracy · 2018
No notice
52.
Okut H., Gianola D., Rosa G. J. M., and Weigel K. A., 2011 Prediction of body mass index in mice using dense molecular markers and a regularized neural network. Genet. Res. 93: 189–201.
Not in our library10.1017/s0016672310000662 (opens in a new tab)
53.
Parker, D. B., 1987 Optimal algorithms for adaptive networks: Second order backpropagation, second order direct backpropagation, and second order hebbing learning. Proceedings of the IEEE First International Conference on Neural Networks. 2: 593–600.
Not in our library
54.
Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., 2011 Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 12: 2825–2830.
Not in our library
55.
Pérez P., and de los Campos G., 2014 Genome-wide regression and prediction with the BGLR statistical package. Genetics 198: 483–495.
Not in our library10.1534/genetics.114.164442 (opens in a new tab)
56.
Pouladi F., Salehinejad H., and Gilani A. M., 2015 Deep Recurrent Neural Networks for Sequential Phenotype Prediction in Genomics. arXiv:1511.02554
Not in our library
57.
Prechelt L., 1998 Early Stopping - But When? pp. 55–69 in Neural Networks: Tricks of the Trade, edited by Orr G. B. and Müller K.-R. Springer Berlin Heidelberg, Berlin, Heidelberg.
Not in our library
58.
Ramstein G. P., Evans J., Kaeppler S. M., Mitchell R. B., Vogel K. P., 2016 Accuracy of Genomic Prediction in Switchgrass (Panicum virgatum L.) Improved by Accounting for Linkage Disequilibrium. G3 (Bethesda) 6: 1049–1062.
Not in our library10.1534/g3.115.024950 (opens in a new tab)
59.
Ribaut J.-M., and Ragot M., 2007 Marker-assisted selection to improve drought adaptation in maize: the backcross approach, perspectives, limitations, and alternatives. J. Exp. Bot. 58: 351–360.
Not in our library10.1093/jxb/erl214 (opens in a new tab)
60.
Riedelsheimer, C., F. Technow, and A. E. Melchinger, 2013 Comparison of whole-genome prediction models for traits with contrasting genetic architecture in a diversity panel of maize inbred lines. BMC Genomics 13: 452
Not in our library10.1186/1471-2164-13-452 (opens in a new tab)
61.
Genome-enabled prediction models for yield related traits in chickpea · 2016
Not checked
62.
Rumelhart, D. E., G. E. Hinton, and R. J. Williams, 1986 Learning internal representation by error propagation, Parallel Distributed Processing, DE Rumelhart and JL McClelland, eds. ISBN: 0-262-68053-X. 318–326.
Not in our library
63.
Shaikhina T., and Khovanova N. A., 2017 Handling limited datasets with neural networks in medical applications: A small-data approach. Artif. Intell. Med. 75: 51–63.
Not in our library10.1016/j.artmed.2016.12.003 (opens in a new tab)
64.
Genomic Selection and Association Mapping in Rice (Oryza sativa): Effect of Trait Genetic Architecture, Training Population Composition, Marker Number and Statistical Model on Accuracy of Rice Genomic Selection in Elite, Tropical Rice Breeding Lines · 2015
Corrected
65.
Usai M. G., Goddard M. E., and Hayes B. J., 2009 LASSO with cross-validation for genomic selection. Genet. Res. 91: 427–436.
Not in our library10.1017/s0016672309990334 (opens in a new tab)
66.
Vazquez A. I., Rosa G. J. M., Weigel K. A., de los Campos G., Gianola D., 2010 Predictive ability of subsets of single nucleotide polymorphisms with and without parent average in US Holsteins. J. Dairy Sci. 93: 5942–5949.
Not in our library10.3168/jds.2010-3335 (opens in a new tab)
67.
Webb S., 2018 Deep learning for biology. Nature 554: 555–557.
Not in our library10.1038/d41586-018-02174-z (opens in a new tab)
68.
Xavier A., Muir W. M., and Rainey K. M., 2016 Assessing Predictive Properties of Genome-Wide Selection in Soybeans. G3 (Bethesda) 6: 2611–2616.
Not in our library10.1534/g3.116.032268 (opens in a new tab)
69.
Xu Y., Wang X., Ding X., Zheng X., Yang Z., 2018 Genomic selection of agronomic traits in hybrid rice using an NCII population. Rice (N. Y.) 11: 32
Not in our library10.1186/s12284-018-0223-4 (opens in a new tab)
70.
Zou, H., and T. Hastie, 2005 Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. 67: 301–320. http://www.jstor.org/stable/3647580
Not in our library

Cited by papers we hold 5

Prediction of Maize Phenotypic Traits With Genomic and Environmental Predictors Using Gradient Boosting Frameworks · 2021
Not checked
Harnessing Genetic Diversity in the USDA Pea Germplasm Collection Through Genomic Prediction · 2021
Not checked
A review of deep learning applications for genomic selection · 2021
No notice
LightGBM: accelerated genomically designed crop breeding through ensemble learning · 2021
No notice
Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species · 2020
No notice

Questions about this paper

We email you if it is retracted, withdrawn, corrected or flagged.

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.