SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects

Abstract

Machine-learned force fields combine the accuracy of ab initio methods with the efficiency of conventional force fields. However, current machine-learned force fields typically ignore electronic degrees of freedom, such as the total charge or spin state, and assume chemical locality, which is problematic when molecules have inconsistent electronic states, or when nonlocal effects play a significant role. This work introduces SpookyNet, a deep neural network for constructing machine-learned force fields with explicit treatment of electronic degrees of freedom and nonlocality, modeled via self-attention in a transformer architecture. Chemically meaningful inductive biases and analytical corrections built into the network architecture allow it to properly model physical limits. SpookyNet improves upon the current state-of-the-art (or achieves similar performance) on popular quantum chemistry data sets. Notably, it is able to generalize across chemical and conformational space and can leverage the learned chemical insights, e.g. by predicting unknown spin states, thus helping to close a further important remaining gap for today’s machine learning models in quantum chemistry.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 112

1. 1.

   Warshel A. Molecular dynamics simulations of biological reactions. Acc. Chem. Res. 2002;35:385.

   Not in our library10.1021/ar010033z

2. 2.

   Karplus M, McCammon JA. Molecular dynamics simulations of biomolecules. Nat. Struct. Biol. 2002;9:646.

   Not in our library10.1038/nsb0902-646

3. 3.

   Dirac PAM. Quantum mechanics of many-electron systems. Proc. R. Soc. Lond. A. 1929;123:714.

   Not in our library

4. 4.

   Dykstra, C., Frenking, G., Kim, K., & Scuseria, G. (Eds.). Theory and applications of computational chemistry: the first forty years. (Elsevier 2005).

   Not in our library

5. 5.

   Unke OT, Koner D, Patra S, Käser S, Meuwly M. High-dimensional potential energy surfaces for molecular simulations: from empiricism to machine learning. Mach. Learn. Sci. Technol. 2020;1:013001.

   Not in our library

6. 6.

   Bartók AP, Payne MC, Kondor R, Csányi G. Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett. 2010;104:136403.

   Not in our library10.1103/physrevlett.104.136403

7. 7.

   Rupp M, Tkatchenko A, Müller K-R, Von Lilienfeld OA. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 2012;108:058301.

   Not in our library10.1103/physrevlett.108.058301

8. 8.

   Bartók AP, et al. Machine learning unifies the modeling of materials and molecules. Sci. Adv. 2017;3:e1701816.

   Not in our library10.1126/sciadv.1701816

9. 9.

   Gastegger M, Schwiedrzik L, Bittermann M, Berzsenyi F, Marquetand P. wACSF—weighted atom-centered symmetry functions as descriptors in machine learning potentials. J. Chem. Phys. 2018;148:241709.

   Not in our library10.1063/1.5019667

10. 10.
    Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions · 2019

    No notice
11. 11.

    Unke OT, et al. Machine learning force fields. Chem. Rev. 2021;121:10142-10186.

    Not in our library10.1021/acs.chemrev.0c01111

12. 12.

    von Lilienfeld OA, Müller K-R, Tkatchenko A. Exploring chemical compound space with quantum-based machine learning. Nat. Rev. Chem. 2020;4:347.

    Not in our library10.1038/s41570-020-0189-9

13. 13.

    Noé F, Tkatchenko A, Müller K-R, Clementi C. Machine learning for molecular simulation. Annu. Rev. Phys. Chem. 2020;71:361.

    Not in our library10.1146/annurev-physchem-042018-052331

14. 14.
    Unsupervised Learning Methods for Molecular Simulation Data · 2021

    No notice
15. 15.

    Keith JA, et al. Combining machine learning and computational chemistry for predictive insights into chemical systems. Chem. Rev. 2021;121:9816-9872.

    Not in our library10.1021/acs.chemrev.1c00107

16. 16.

    Sauceda HE, Gastegger M, Chmiela S, Müller K-R, Tkatchenko A. Molecular force fields with gradient-domain machine learning (GDML): Comparison and synergies with classical force fields. J. Chem. Phys. 2020;153:124109.

    Not in our library10.1063/5.0023005

17. 17.

    Behler J, Parrinello M. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett. 2007;98:146401.

    Not in our library10.1103/physrevlett.98.146401

18. 18.

    Behler J. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. J. Chem. Phys. 2011;134:074106.

    Not in our library10.1063/1.3553717

19. 19.

    McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943;5:115.

    Not in our library

20. 20.

    Unke OT, Meuwly M. Toolkit for the construction of reproducing kernel-based representations of data: application to multidimensional potential energy surfaces. J. Chem. Inf. Model. 2017;57:1923.

    Not in our library10.1021/acs.jcim.7b00090

21. 21.

    Chmiela S, et al. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. 2017;3:e1603015.

    Not in our library10.1126/sciadv.1603015

22. 22.

    Chmiela S, Sauceda HE, Poltavsky I, Müller K-R, Tkatchenko A. sGDML: Constructing accurate and data efficient molecular force fields using machine learning. Computer Phys. Commun. 2019;240:38.

    Not in our library

23. 23.

    Smith JS, Isayev O, Roitberg AE. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chem. Sci. 2017;8:3192.

    Not in our library10.1039/c6sc05720a

24. 24.

    Zhang L, Han J, Wang H, Car R, Weinan E. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett. 2018;120:143001.

    Not in our library10.1103/physrevlett.120.143001

25. 25.

    Unke OT, Meuwly M. A reactive, scalable, and transferable model for molecular energies from a neural network approach based on local information. J. Chem. Phys. 2018;148:241708.

    Not in our library10.1063/1.5017898

26. 26.

    Christensen AS, Bratholm LA, Faber FA, Anatole von Lilienfeld O. Anatole von Lilienfeld, O. FCHL revisited: faster and more accurate quantum machine learning. J. Chem. Phys. 2020;152:044107.

    Not in our library10.1063/1.5126701

27. 27.

    Vapnik, V. N. The Nature of Statistical Learning Theory. (Springer, 1995).

    Not in our library

28. 28.

    Cortes C, Vapnik V. Support-vector networks. Mach. Learn. 1995;20:273.

    Not in our library

29. 29.

    Müller K-R, Mika S, Ratsch G, Tsuda K, Schölkopf B. An introduction to kernel-based learning algorithms. IEEE Trans. Neural Netw. 2001;12:181.

    Not in our library10.1109/72.914517

30. 30.

    Schölkopf, B. & Smola, A. J. Learning with kernels: support vector machines, regularization, optimization, and beyond. (MIT press, 2002).

    Not in our library

31. 31.

    Braun ML, Buhmann JM, Müller K-R. On relevant dimensions in kernel feature spaces. J. Mach. Learn. Res. 2008;9:1875.

    Not in our library

32. 32.

    Hansen K, et al. Assessment and validation of machine learning methods for predicting molecular atomization energies. J. Chem. Theory Comput. 2013;9:3404.

    Not in our library10.1021/ct400195d

33. 33.
    Quantum-chemical insights from deep tensor neural networks · 2017

    No notice
34. 34.

    Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry, in International Conference on Machine Learning, 1263–1272 https://proceedings.mlr.press/v70/gilmer17a.html (2017).

    Not in our library

35. 35.

    Schütt KT, Sauceda HE, Kindermans P-J, Tkatchenko A, Müller K-R. SchNet—a deep learning architecture for molecules and materials. J. Chem. Phys. 2018;148:241722.

    Not in our library10.1063/1.5019779

36. 36.

    Lubbers N, Smith JS, Barros K. Hierarchical modeling of molecular energies using a deep neural network. J. Chem. Phys. 2018;148:241715.

    Not in our library10.1063/1.5011181

37. 37.

    Unke OT, Meuwly M. PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput. 2019;15:3678.

    Not in our library10.1021/acs.jctc.9b00181

38. 38.

    Friesner RA. Ab initio quantum chemistry: methodology and applications. Proc. Natl Acad. Sci. USA. 2005;102:6648.

    Not in our library10.1073/pnas.0408036102

39. 39.

    Born, M. & Einstein, A. The Born-Einstein Letters 1916–1955 (Macmillan, 2005).

    Not in our library

40. 40.

    Noodleman L, Peng C, Case D, Mouesca J-M. Orbital interactions, electron delocalization and spin coupling in iron-sulfur clusters. Coord. Chem. Rev. 1995;144:199.

    Not in our library

Cited by papers we hold 6

• AIMNet2: a neural network potential to meet your neutral, charged, organic, and elemental-organic needs · 2025
  No notice
• MACE-OFF: Short-Range Transferable Machine Learning Force Fields for Organic Molecules · 2025
  No notice
• SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentials · 2023
  No notice
• Machine Learning Interatomic Potentials and Long-Range Physics · 2023
  No notice
• E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials · 2022
  No notice
• Graph neural networks for materials science and chemistry · 2022
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.