Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics

Abstract

We introduce a new representation and feature extraction method for biological sequences. Named bio-vectors (BioVec) to refer to biological sequences in general with protein-vectors (ProtVec) for proteins (amino-acid sequences) and gene-vectors (GeneVec) for gene sequences, this representation can be widely used in applications of deep learning in proteomics and genomics. In the present paper, we focus on protein-vectors that can be utilized in a wide array of bioinformatics investigations such as family classification, protein visualization, structure prediction, disordered protein identification, and protein-protein interaction prediction. In this method, we adopt artificial neural network approaches and represent a protein sequence with a single dense n-dimensional vector. To evaluate this method, we apply it in classification of 324,018 protein sequences obtained from Swiss-Prot belonging to 7,027 protein families, where an average family classification accuracy of 93%±0.06% is obtained, outperforming existing family classification methods. In addition, we use ProtVec representation to predict disordered proteins from structured proteins. Two databases of disordered sequences are used: the DisProt database as well as a database featuring the disordered regions of nucleoporins rich with phenylalanine-glycine repeats (FG-Nups). Using support vector machine classifiers, FG-Nup sequences are distinguished from structured protein sequences found in Protein Data Bank (PDB) with a 99.8% accuracy, and unstructured DisProt sequences are differentiated from structured DisProt sequences with 100.0% accuracy. These results indicate that by only providing sequence data for various proteins into this model, accurate information about protein structure can be determined. Importantly, this model needs to be trained only once and can then be applied to extract a comprehensive set of information regarding proteins of interest. Moreover, this representation can be considered as pre-training for various applications of deep learning in bioinformatics. The related data is available at Life Language Processing Website: http://llp.berkeley.edu and Harvard Dataverse: http://dx.doi.org/10.7910/DVN/JMFHTN.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 39

1. 1.

   Yandell MD, Majoros WH. Genomics and natural language processing. Nature Reviews Genetics. 2002;3(8):601–610.

   Not in our library10.1038/nrg861

2. 2.

   Searls DB. The language of genes. Nature. 2002;420(6912):211–217.

   Not in our library10.1038/nature01255

3. 3.

   Motomura K, Fujita T, Tsutsumi M, Kikuzato S, Nakamura M, Otaki JM. Word decoding of protein amino acid sequences with availability analysis: a linguistic approach. PloS one. 2012;7(11):e50039

   Not in our library10.1371/journal.pone.0050039

4. 4.

   Cai Y, Lux MW, Adam L, Peccoud J. Modeling structure-function relationships in synthetic DNA sequences using attribute grammars. PLoS Comput Biol. 2009;5(10):e1000529

   Not in our library10.1371/journal.pcbi.1000529

5. 5.

   Suykens JA, Vandewalle J. Least squares support vector machine classifiers. Neural processing letters. 1999;9(3):293–300.

   Not in our library10.1023/a:1018628609742

6. 6.

   Hinton GE. Distributed representations. School of Computer Science at Carnegie Mellon University. 1984;.

   Not in our library

7. 7.
   Computational Phenotype Discovery Using Unsupervised Feature Learning over Noisy, Sparse, and Irregular Clinical Data · 2013

   Corrected
8. 8.

   Xiong HY, Alipanahi B, Lee LJ, Bretschneider H, Merico D, Yuen RK, et al The human splicing code reveals new insights into the genetic determinants of disease. Science. 2015;347(6218):1254806

   Not in our library10.1126/science.1254806

9. 9.

   Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Natural language processing (almost) from scratch. The Journal of Machine Learning Research. 2011;12:2493–2537.

   Not in our library

10. 10.

    Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems; 2013 p. 3111–3119.

    Not in our library

11. 11.

    Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv preprint arXiv:13013781. 2013;.

    Not in our library

12. 12.

    Enright AJ, Van Dongen S, Ouzounis CA. An efficient algorithm for large-scale detection of protein families. Nucleic acids research. 2002;30(7):1575–1584.

    Not in our library10.1093/nar/30.7.1575

13. 13.

    Bork P, Dandekar T, Diaz-Lazcoz Y, Eisenhaber F, Huynen M, Yuan Y. Predicting function: from genes to genomes and back. Journal of molecular biology. 1998;283(4):707–725.

    Not in our library10.1006/jmbi.1998.2144

14. 14.

    Remmert M, Biegert A, Hauser A, Söding J. HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment. Nature methods. 2012;9(2):173–175.

    Not in our library10.1038/nmeth.1818

15. 15.
    Pfam: the protein families database · 2013

    No notice
16. 16.

    Cai C, Han L, Ji ZL, Chen X, Chen YZ. SVM-Prot: web-based support vector machine software for functional classification of a protein from its primary sequence. Nucleic acids research. 2003;31(13):3692–3697.

    Not in our library10.1093/nar/gkg600

17. 17.

    Leslie CS, Eskin E, Noble WS. The spectrum kernel: A string kernel for SVM protein classification. In: Pacific symposium on biocomputing. vol. 7. World Scientific; 2002. p. 566–575.

    Not in our library

18. 18.

    Huynen M, Snel B, Lathe W, Bork P. Predicting protein function by genomic context: quantitative evaluation and qualitative inferences. Genome research. 2000;10(8):1204–1210.

    Not in our library10.1101/gr.10.8.1204

19. 19.

    Murzin AG, Brenner SE, Hubbard T, Chothia C. SCOP: a structural classification of proteins database for the investigation of sequences and structures. Journal of molecular biology. 1995;247(4):536–540.

    Not in our library10.1016/s0022-2836(05)80134-2

20. 20.

    Aragues R, Sali A, Bonet J, Marti-Renom MA, Oliva B. Characterization of protein hubs by inferring interacting motifs from protein interactions. PloS Computational Biology. 2007;3.9:e178

    Not in our library10.1371/journal.pcbi.0030178

21. 21.

    Dunker AK, Silman I, Uversky VN, Sussman JL. Function and structure of inherently disordered proteins. Current opinion in structural biology. 2008;18(6):756–764.

    Not in our library10.1016/j.sbi.2008.10.002

22. 22.

    Dyson HJ, Wright PE. Intrinsically unstructured proteins and their functions. Nature reviews Molecular cell biology. 2005;6(3):197–208.

    Not in our library10.1038/nrm1589

23. 23.

    Sugase K, Dyson HJ, Wright PE. Mechanism of coupled folding and binding of an intrinsically disordered protein. Nature. 2007;447(7147):1021–1025.

    Not in our library10.1038/nature05858

24. 24.

    He B, Wang K, Liu Y, Xue B, Uversky VN, Dunker AK. Predicting intrinsic disorder in proteins: an overview. Cell research. 2009;19(8):929–949.

    Not in our library10.1038/cr.2009.87

25. 25.

    Jamali T, Jamali Y, Mehrbod M, Mofrad M. Nuclear pore complex: biochemistry and biophysics of nucleocytoplasmic transport in health and disease. Int Rev Cell Mol Biol. 2011;287:233–286.

    Not in our library10.1016/b978-0-12-386043-9.00006-2

26. 26.

    Sickmeier M, Hamilton JA, LeGall T, Vacic V, Cortese MS, Tantos A, et al DisProt: the database of disordered proteins. Nucleic acids research. 2007;35(suppl 1):D786–D793.

    Not in our library10.1093/nar/gkl893

27. 27.

    Ando D, Colvin M, Rexach M, Gopinathan A. Physical motif clustering within intrinsically disordered nucleoporin sequences reveals universal functional features. PloS one. 2013;8(9):e73831

    Not in our library10.1371/journal.pone.0073831

28. 28.

    Azimi M, Mofrad MR. Higher Nucleoporin-Importinβ Affinity at the Nuclear Basket Increases Nucleocytoplasmic Import. PloS one. 2013;8(11):e81741

    Not in our library10.1371/journal.pone.0081741

29. 29.

    Peyro M, Soheilypour M, Lee BL, Mofrad M. Evolutionary conserved sequence features optimizes nucleoporins behavior for cargo transportation through nuclear pore complex. Scientific Reports. In press 2015;.

    Not in our library10.1038/srep15795

30. 30.

    Procter JB, Thompson J, Letunic I, Creevey C, Jossinet F, Barton GJ. Visualization of multiple alignments, phylogenies and gene family evolution. Nature methods. 2010;7:S16–S25.

    Not in our library10.1038/nmeth.1434

31. 31.

    Rutherford K, Parkhill J, Crook J, Horsnell T, Rice P, Rajandream MA, et al Artemis: sequence visualization and annotation. Bioinformatics. 2000;16(10):944–945.

    Not in our library10.1093/bioinformatics/16.10.944

32. 32.

    Ganapathiraju M, Weisser D, Rosenfeld R, Carbonell J, Reddy R, Klein-Seetharaman J. Comparative n-gram analysis of whole-genome protein sequences. In: Proceedings of the second international conference on Human Language Technology Research. Morgan Kaufmann Publishers Inc.; 2002. p. 76–81.

    Not in our library

33. 33.

    Srinivasan SM, Vural S, King BR, Guda C. Mining for class-specific motifs in protein sequence classification. BMC bioinformatics. 2013;14(1):96

    Not in our library10.1186/1471-2105-14-96

34. 34.

    Vries JK, Liu X. Subfamily specific conservation profiles for proteins based on n-gram patterns. BMC bioinformatics. 2008;9(1):72

    Not in our library10.1186/1471-2105-9-72

35. 35.

    Goldberg Y, Levy O. word2vec Explained: deriving Mikolov et al.’s negative-sampling word-embedding method. arXiv preprint arXiv:14023722. 2014;.

    Not in our library

36. 36.

    Van der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research. 2008;9(2579–2605):85.

    Not in our library

37. 37.

    McGregor E. Proteins and proteomics: A laboratory manual. Journal of Proteome Research. 2004;3(4):694–694.

    Not in our library10.1021/pr040022a

38. 38.

    Rose PW, Bi C, Bluhm WF, Christie CH, Dimitropoulos D, Dutta S, et al The RCSB Protein Data Bank: new resources for research and education. Nucleic acids research. 2013;41(D1):D475–D482.

    Not in our library10.1093/nar/gks1200

39. 39.

    Platzer A. Visualization of SNPs with t-SNE. PloS one. 2013;8(2):e56883

    Not in our library10.1371/journal.pone.0056883

Cited by papers we hold 14

• Improving the generalizability of protein-ligand binding predictions with AI-Bind · 2023
  No notice
• Affinity2Vec: drug-target binding affinity prediction through representation learning, graph mining, and machine learning · 2022
  Not checked
• Neural networks to learn protein sequence–function relationships from deep mutational scanning data · 2021
  No notice
• Deep Learning in Proteomics · 2020
  No notice
• Bioinformatics Methods for Mass Spectrometry-Based Proteomics Data Analysis · 2020
  No notice
• UDSMProt: universal deep sequence models for protein classification · 2020
  No notice
• Antimicrobial peptide identification using multi-scale convolutional network · 2019
  Not checked
• Identifying SNAREs by Incorporating Deep Learning Architecture and Amino Acid Embedding Representation · 2019
  Not checked
• Gene2vec: distributed representation of genes based on co-expression · 2019
  No notice
• Modeling aspects of the language of life through transfer-learning protein sequences · 2019
  No notice
• Distributed Representation of Chemical Fragments · 2018
  Not checked
• Recurrent Neural Network for Predicting Transcription Factor Binding Sites · 2018
  No notice
• Deep Learning and Its Applications in Biomedicine · 2018
  No notice
• Opportunities and obstacles for deep learning in biology and medicine · 2018
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.