Quantum-chemical insights from deep tensor neural networks

Abstract

Learning from data has led to paradigm shifts in a multitude of disciplines, including web, text and image search, speech recognition, as well as bioinformatics. Can machine learning enable similar breakthroughs in understanding quantum many-body systems? Here we develop an efficient deep learning approach that enables spatially and chemically resolved insights into quantum-mechanical observables of molecular systems. We unify concepts from many-body Hamiltonians with purpose-designed deep tensor neural networks, which leads to size-extensive and uniformly accurate (1 kcal mol −1 ) predictions in compositional and configurational chemical space for molecules of intermediate size. As an example of chemical relevance, the model reveals a classification of aromatic rings with respect to their stability. Further applications of our model for predicting atomic energies and local chemical potentials in molecules, reliable isomer energies, and molecules with peculiar electronic structure demonstrate the potential of machine learning for revealing insights into complex quantum-chemical systems.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 53

1. 1.

   Kang B. & Ceder G. Battery materials for ultrafast charging and discharging. Nature 458, 190–193 (2009 ).

   Not in our library10.1038/nature07853

2. 2.

   Nørskov J. K., Bligaard T., Rossmeisl J. & Christensen C. H. Towards the computational design of solid catalysts. Nat. Chem. 1, 37–46 (2009 ).

   Not in our library10.1038/nchem.121

3. 3.

   Hachmann J.. The Harvard clean energy project: large-scale computational screening and design of organic photo-voltaics on the world community grid. J. Phys. Chem. Lett. 2, 2241–2251 (2011 ).

   Not in our library

4. 4.

   Pyzer-Knapp E. O., Suh C., Gomez-Bombarelli R., Aguilera-Iparraguirre J. & Aspuru-Guzik A. What is high-throughput virtual screening? A perspective from organic materials discovery. Annu. Rev. Mater. Res. 45, 195–216 (2015 ).

   Not in our library

5. 5.

   Curtarolo S.. The high-throughput highway to computational materials design. Nat. Mater. 12, 191–201 (2013 ).

   Not in our library10.1038/nmat3568

6. 6.

   Snyder J. C., Rupp M., Hansen K., Müller K.-R. & Burke K. Finding density functionals with machine learning. Phys. Rev. Lett. 108, 253002 (2012 ).

   Not in our library10.1103/physrevlett.108.253002

7. 7.

   Rupp M., Tkatchenko A., Muller K.-R. & Von Lilienfeld O. A. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 108, 058301 (2012 ).

   Not in our library10.1103/physrevlett.108.058301

8. 8.

   Ramakrishnan R., Dral P. O., Rupp M. & von Lilienfeld O. A. Big data meets quantum chemistry approximations: the Δ-machine learning approach. J. Chem. Theory Comput. 11, 2087–2096 (2015 ).

   Not in our library10.1021/acs.jctc.5b00099

9. 9.

   Bishop C. M. Pattern Recognition and Machine Learning Springer (2006 ).

   Not in our library

10. 10.

    Ghiringhelli L. M., Vybiral J., Levchenko S. V., Draxl C. & Scheffler M. Big data of materials science: critical role of the descriptor. Phys. Rev. Lett. 114, 105503 (2015 ).

    Not in our library10.1103/physrevlett.114.105503

11. 11.

    Schütt K.. How to represent crystal structures for machine learning: towards fast prediction of electronic properties. Phys. Rev. B 89, 205118 (2014 ).

    Not in our library

12. 12.

    Montavon G.. Machine learning of molecular electronic properties in chemical compound space. New J. Phys. 15, 095003 (2013 ).

    Not in our library

13. 13.

    Hansen K.. Assessment and validation of machine learning methods for predicting molecular atomization energies. J. Chem. Theory Comput. 9, 3404–3419 (2013 ).

    Not in our library10.1021/ct400195d

14. 14.

    Hirn M., Poilvert N. & Mallat S. Quantum energy regression using scattering transforms. Preprint at https://arxiv.org/abs/1502.02077 (2015 ).

    Not in our library

15. 15.

    Hansen K.. Machine learning predictions of molecular properties: accurate many-body potentials and nonlocality in chemical space. J. Phys. Chem. Lett. 6, 2326 (2015 ).

    Not in our library10.1021/acs.jpclett.5b00831

16. 16.

    Bartók A. P., Kondor R. & Csanyi G. On representing chemical environments. Phys. Rev. B 87, 184115 (2013 ).

    Not in our library

17. 17.

    Bartók A. P., Payne M. C., Kondor R. & Csanyi G. Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett. 104, 136403 (2010 ).

    Not in our library10.1103/physrevlett.104.136403

18. 18.

    Behler J. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. J. Chem. Phys. 134, 074106 (2011 ).

    Not in our library10.1063/1.3553717

19. 19.

    Behler J. Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations. Phys. Chem. Chem. Phys. 13, 17930–17955 (2011 ).

    Not in our library10.1039/c1cp21668f

20. 20.
    Deep learning · 2015

    No notice
21. 21.

    Montavon G., Braun M. L. & Müller K.-R. Kernel analysis of deep networks. J. Mach. Learn. Res. 12, 2563–2581 (2011 ).

    Not in our library

22. 22.

    Ciresan D., Meier U. & Schmidhuber J. Multi-column deep neural networks for image classification. In Proc. Conference on Computer Vision and Pattern Recognition. 3642–3649 (2012 ).

    Not in our library

23. 23.

    Krizhevsky A., Sutskever I. & Hinton G. E. ImageNet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems. 25, 1097–1105 (2012 ).

    Not in our library

24. 24.

    LeCun Y. & Bengio Y. in The Handbook of Brain Theory and Neural Networks (ed. Arbib M.A.) 255–257 (The MIT Press, Cambridge, MA, USA, 1995).

    Not in our library

25. 25.

    Hinton G.. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29, 82–97 (2012 ).

    Not in our library

26. 26.

    Sainath T. N.. Deep convolutional neural networks for large-scale speech tasks. Neural Netw. 64, 39–48 (2015 ).

    Not in our library10.1016/j.neunet.2014.08.005

27. 27.

    Collobert R. & Weston J. A unified architecture for natural language processing: deep neural networks with multitask learning. In Proc. 25th International Conference on Machine Learning. 160–167 (2008 ).

    Not in our library

28. 28.

    Scarselli F., Gori M., Tsoi A. C., Hagenbuchner M. & Mon-fardini G. The graph neural network model. IEEE Trans. Neural Netw. 20, 61–80 (2009 ).

    Not in our library10.1109/tnn.2008.2005605

29. 29.

    Duvenaud D. K.. Convolutional networks on graphs for learning molecular fingerprints. In Proc. Advances in Neural Information Processing Systems. 28, 2224–2232 (2015 ).

    Not in our library

30. 30.

    Socher R.. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. of the conference on empirical methods in natural language processing (EMNLP) 1631–1642 (2013 ).

    Not in our library

31. 31.

    Sutskever I., Martens J. & Hinton G. E. Generating text with recurrent neural networks. Proc. 28th Annu. Int. Conf. Mach. Learn. 1017–1024 (2011 ).

    Not in our library

32. 32.

    Socher R., Chen D., Manning C. D. & Ng A. Reasoning with neural tensor networks for knowledge base completion. In Proc. Advances in Neural Information Processing Systems. 26, 926–934 (2013 ).

    Not in our library

33. 33.

    Taylor G. W. & Hinton G. E. Factored conditional restricted Boltzmann machines for modeling motion style. In Proc. 26th Annual International Conference on Machine Learning. 1025–1032 (2009 ).

    Not in our library

34. 34.

    Blum L. C. & Reymond J.-L. 970 Million Druglike Small Molecules for Virtual Screening in the Chemical Universe Database GDB-13. J. Am. Chem. Soc. 131, 8732 (2009 ).

    Not in our library10.1021/ja902302h

35. 35.

    Reymond J.-L. The chemical space project. Acc. Chem. Res. 48, 722–730 (2015 ).

    Not in our library10.1021/ar500432k

36. 36.

    Ramakrishnan R., Dral P. O., Rupp M. & von Lilienfeld O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 140022 (2014 ).

    Not in our library10.1038/sdata.2014.22

37. 37.

    von Lilienfeld O. A. First principles view on chemical compound space: gaining rigorous atomistic control of molecular properties. Int. J. Quantum Chem. 113, 1676–1689 (2013 ).

    Not in our library

38. 38.

    De S., Bartok A. P., Csanyi G. & Ceriotti M. Comparing molecules and solids across structural and alchemical space. Phys. Chem. Chem. Phys. 18, 13754–13769 (2016 ).

    Not in our library10.1039/c6cp00415f

39. 39.

    Malshe M.. Development of generalized potential-energy surfaces using many-body expansions, neural networks, and moiety energy approximations. J. Chem. Phys. 130, 184102 (2009 ).

    Not in our library10.1063/1.3124802

40. 40.

    Manzhos S. & Carrington T. Jr A random-sampling high dimensional model representation neural network for building potential energy surfaces. J. Chem. Phys. 125, 084109 (2006 ).

    Not in our library10.1063/1.2336223

41. 41.

    Manzhos S. & Carrington T. Jr Using neural networks, optimized coordinates, and high-dimensional model representations to obtain a vinyl bromide potential surface. J. Chem. Phys. 129, 224104 (2008 ).

    Not in our library10.1063/1.3021471

42. 42.

    Behler J. & Parrinello M. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett. 98, 146401 (2007 ).

    Not in our library10.1103/physrevlett.98.146401

43. 43.

    Hohenberg P. & Kohn W. Inhomogeneous electron gas. Phys. Rev. 136, B864–B871 (1964 ).

    Not in our library

44. 44.

    Perdew J. P., Ernzerhof M. & Burke K. Rationale for mixing exact exchange with density functional approximations. J. Chem. Phys. 105, 9982–9985 (1996 ).

    Not in our library

45. 45.

    Becke A. D. Density-functional exchange-energy approximation with correct asymptotic behavior. Phys. Rev. A 38, 3098–3100 (1988 ).

    Not in our library10.1103/physreva.38.3098

46. 46.

    Lee C., Yang W. & Parr R. G. Development of the Colle- Salvetti correlation-energy formula into a functional ofthe electron density. Phys. Rev. B 37, 785–789 (1988 ).

    Not in our library10.1103/physrevb.37.785

47. 47.

    Vosko S. H., Wilk L. & Nusair M. Accurate spin-dependent electron liquid correlation energies for local spin density calculations: a critical analysis. Can. J. Phys. 58, 1200–1211 (1980 ).

    Not in our library

48. 48.

    Stephens P., Devlin F., Chabalowski C. & Frisch M. J. Ab initio calculation of vibrational absorption and circular dichro-ism spectra using density functional force fields. J. Phys. Chem. 98, 11623–11627 (1994 ).

    Not in our library

49. 49.

    Becke A. d. Beckes 3 parameter functional combined with the non-local correlation LYP. J. Chem. Phys. 98, 5648 (1993 ).

    Not in our library

50. 50.

    Perdew J. P., Burke K. & Ernzerhof M. Generalized gradient approximation made simple. Phys. Rev. Lett. 77, 3865–3868 (1996 ).

    Not in our library10.1103/physrevlett.77.3865

51. 51.

    Glorot X. & Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. 13th International Conference on Artificial Intelligence and Statistics. 249–256 (2010 ).

    Not in our library

52. 52.

    LeCun Y. A., Bottou L., Orr G. B. & Müller K.-R. in Neural Networks: Tricks of the Trade 9–48Springer (2012 ).

    Not in our library

53. 53.

    Ramachandran P. & Varoquaux G. Mayavi: 3D visualization of scientific data. Comput. Sci. Eng. 13, 40–51 (2011 ).

    Not in our library

Cited by papers we hold 23

• Machine Learning Interatomic Potentials and Long-Range Physics · 2023
  No notice
• DeePMD-kit v2: A software package for deep potential models · 2023
  No notice
• Learning local equivariant representations for large-scale atomistic dynamics · 2023
  No notice
• E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials · 2022
  No notice
• Impact of training and validation data on the performance of neural network potentials: A case study on carbon using the CA-9 dataset · 2021
  Not checked
• Unsupervised Learning Methods for Molecular Simulation Data · 2021
  No notice
• SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects · 2021
  No notice
• Complex reaction processes in combustion unraveled by neural network-based molecular dynamics simulation · 2020
  Not checked
• Machine Learning: Quantum vs Classical · 2020
  Not checked
• The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules · 2020
  No notice
• Quantum chemical accuracy from density functional approximations via machine learning · 2020
  No notice
• Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning · 2019
  No notice
• Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions · 2019
  No notice
• Explaining the unique nature of individual gait patterns with deep learning · 2019
  No notice
• Data‐Driven Materials Science: Status, Challenges, and Perspectives · 2019
  Corrected
• Deep Learning Spectroscopy: Neural Networks for Molecular Excitation Spectra · 2019
  No notice
• Unmasking Clever Hans predictors and assessing what machines really learn · 2019
  No notice
• ElemNet: Deep Learning the Chemistry of Materials From Only Elemental Composition · 2018
  No notice
• Towards exact molecular dynamics simulations with machine-learned force fields · 2018
  No notice
• Insightful classification of crystal structures using deep learning · 2018
  No notice

and 3 more

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.