E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials

Abstract

This work presents Neural Equivariant Interatomic Potentials (NequIP), an E(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations for molecular dynamics simulations. While most contemporary symmetry-aware models use invariant convolutions and only act on scalars, NequIP employs E(3)-equivariant convolutions for interactions of geometric tensors, resulting in a more information-rich and faithful representation of atomic environments. The method achieves state-of-the-art accuracy on a challenging and diverse set of molecules and materials while exhibiting remarkable data efficiency. NequIP outperforms existing models with up to three orders of magnitude fewer training data, challenging the widely held belief that deep neural networks require massive training sets. The high data efficiency of the method allows for the construction of accurate potentials using high-order quantum chemical level of theory as reference and enables high-fidelity molecular dynamics simulations over long time scales.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 62

1. 1.

   Richards WD, et al. Design and synthesis of the superionic conductor na 10 snp 2 s 12. Nat. Commun. 2016;7:1-8.

   Not in our library10.1038/ncomms11009

2. 2.

   Boero M, Parrinello M, Terakura K. First principles molecular dynamics study of ziegler- natta heterogeneous catalysis. J. Am. Chem. Soc. 1998;120:2746-2752.

   Not in our library10.1021/ja972367i

3. 3.

   Lindorff-Larsen K, Piana S, Dror RO, Shaw DE. How fast-folding proteins fold. Science. 2011;334:517-520.

   Not in our library10.1126/science.1208351

4. 4.

   Behler J, Parrinello M. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett. 2007;98:146401.

   Not in our library10.1103/physrevlett.98.146401

5. 5.

   Bartók AP, Payne MC, Kondor R, Csányi G. Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett. 2010;104:136403.

   Not in our library10.1103/physrevlett.104.136403

6. 6.

   Shapeev AV. Moment tensor potentials: a class of systematically improvable interatomic potentials. Multiscale Model. Simul. 2016;14:1153-1173.

   Not in our library10.1137/15m1054183

7. 7.

   Thompson AP, Swiler LP, Trott CR, Foiles SM, Tucker GJ. Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials. J. Comput. Phys. 2015;285:316-330.

   Not in our library10.1016/j.jcp.2014.12.018

8. 8.

   Vandermause J, et al. On-the-fly active learning of interpretable bayesian force fields for atomistic rare events. Npj Comput. Mater. 2020;6:1-11.

   Not in our library10.1038/s41524-020-0283-z

9. 9.

   Schütt, K. et al. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. Adv. Neural Inf. Process. Syst.30, 991–1001 (2017).

   Not in our library

10. 10.

    Unke OT, Meuwly M. Physnet: A neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput. 2019;15:3678-3693.

    Not in our library10.1021/acs.jctc.9b00181

11. 11.

    Klicpera, J., Groß, J. & Günnemann, S. Directional message passing for molecular graphs. arXiv preprint arXiv:2003.03123 (2020).

    Not in our library

12. 12.

    Mailoa JP, et al. A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems. Nat. Mach. Intell. 2019;1:471-479.

    Not in our library10.1038/s42256-019-0098-0

13. 13.

    Park, C. W. et al. Accurate and scalable multi-element graph neural network force field and molecular dynamics with direct force architecture. arXiv preprint arXiv:2007.14444 (2020).

    Not in our library

14. 14.

    Artrith N, Kolpak AM. Understanding the composition and activity of electrocatalytic nanoalloys in aqueous solvents: A combination of dft and accurate neural network potentials. Nano Lett. 2014;14:2670-2676.

    Not in our library10.1021/nl5005674

15. 15.

    Zhang L, Han J, Wang H, Car R, Weinan E. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett. 2018;120:143001.

    Not in our library10.1103/physrevlett.120.143001

16. 16.

    Smith JS, Isayev O, Roitberg AE. Ani-1: an extensible neural network potential with dft accuracy at force field computational cost. Chem. Sci. 2017;8:3192-3203.

    Not in our library10.1039/c6sc05720a

17. 17.

    Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212 (2017).

    Not in our library

18. 18.

    Anderson, B., Hy, T. S. & Kondor, R. Cormorant: Covariant molecular neural networks. Adv. Neural Inf. Process. Syst. 14537–14546 (2019).

    Not in our library

19. 19.

    Townshend, R. J., Townshend, B., Eismann, S. & Dror, R. O. Geometric prediction: Moving beyond scalars. arXiv preprint arXiv:2006.14163 (2020).

    Not in our library

20. 20.

    Thomas, N. et al. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219 (2018).

    Not in our library

21. 21.
    E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials · 2022

    No notice
22. 22.

    Schütt, K. T., Unke, O. T. & Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. arXiv preprint arXiv:2102.03150 (2021).

    Not in our library

23. 23.

    Haghighatlari, M. et al. Newtonnet: A newtonian message passing network for deep learning of interatomic potentials and forces. arXiv preprint arXiv:2108.02913 (2021).

    Not in our library10.1039/d2dd00008c

24. 24.

    Klicpera, J., Becker, F. & Günnemann, S. Gemnet: Universal directional graph neural networks for molecules. arXiv preprint arXiv:2106.08903 (2021).

    Not in our library

25. 25.
    SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects · 2021

    No notice
26. 26.

    Qiao, Z. et al. Unite: Unitary n-body tensor equivariant network with applications to quantum chemistry. arXiv preprint arXiv:2105.14655 (2021).

    Not in our library

27. 27.

    Grisafi, A., Wilkins, D. M., Willatt, M. J. & Ceriotti, M. Atomic-scale representation and statistical learning of tensorial properties. In Machine Learning in Chemistry: Data-Driven Algorithms, Learning Systems, and Predictions, 1–21 (ACS Publications, 2019).

    Not in our library

28. 28.

    Weiler, M., Geiger, M., Welling, M., Boomsma, W. & Cohen, T. S. 3d steerable cnns: Learning rotationally equivariant features in volumetric data. Adv. Neural Inf. Process. Syst.31, 10381–10392 (2018).

    Not in our library

29. 29.

    Kondor, R. N-body networks: a covariant hierarchical neural network architecture for learning atomic potentials. arXiv preprint arXiv:1803.01588 (2018).

    Not in our library

30. 30.

    Kondor, R., Lin, Z. & Trivedi, S. Clebsch–gordan nets: a fully fourier space spherical convolutional neural network. Adv. Neural Inf. Process. Syst.31, 10117–10126 (2018).

    Not in our library

31. 31.

    Geiger, M. et al. e3nn/e3nn: 2021-05-04 10.5281/zenodo.4735637 (2021).

    Not in our library

32. 32.

    Hendrycks, D. & Gimpel, K. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016).

    Not in our library

33. 33.

    Larsen AH, et al. The atomic simulation environment—a python library for working with atoms. J. Phys. Condens. Matter. 2017;29:273002.

    Not in our library10.1088/1361-648x/aa680e

34. 34.

    He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016).

    Not in our library

35. 35.

    Chmiela S, et al. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. 2017;3:e1603015.

    Not in our library10.1126/sciadv.1603015

36. 36.
    Quantum-chemical insights from deep tensor neural networks · 2017

    No notice
37. 37.
    Towards exact molecular dynamics simulations with machine-learned force fields · 2018

    No notice
38. 38.

    Ko H-Y, et al. Isotope effects in liquid water via deep potential molecular dynamics. Mol. Phys. 2019;117:3269-3281.

    Not in our library10.1080/00268976.2019.1652366

39. 39.

    Christensen AS, von Lilienfeld OA. On the role of gradients for machine learning of molecular energies and forces. Mach. learn.: sci. technol. 2020;1:045018.

    Not in our library

40. 40.

    Devereux C, et al. Extending the applicability of the ani deep learning molecular potential to sulfur and halogens. J. Chem. Theory Comput. 2020;16:4192-4202.

    Not in our library10.1021/acs.jctc.0c00121

41. 41.

    Christensen AS, Bratholm LA, Faber FA, Anatole von Lilienfeld O. Fchl revisited: Faster and more accurate quantum machine learning. J. Chem. Phys. 2020;152:044107.

    Not in our library10.1063/1.5126701

42. 42.

    Drautz R. Atomic cluster expansion for accurate and transferable interatomic potentials. Phys. Rev. B. 2019;99:014104.

    Not in our library10.1103/physrevb.99.014104

43. 43.

    Kovács, D. P. et al. Linear atomic cluster expansion force fields for organic molecules: beyond rmse. J. Chem. Theory Comput.17, 7696–711 (2021).

    Not in our library10.1021/acs.jctc.1c00647

44. 44.

    Zhang, L. et al. End-to-end symmetry preserving inter-atomic potential energy model for finite and extended systems. Adv. Neural Inf. Process. Syst.31, (2018).

    Not in our library

45. 45.

    Sim WS, Gardner P, King DA. Multiple bonding configurations of adsorbed formate on ag111. J. Phys. Chem. 1996;100:12509-12516.

    Not in our library10.1021/jp9600726

46. 46.

    Wang G, Morikawa Y, Matsumoto T, Nakamura J. Why is formate synthesis insensitive to copper surface structures? J. Phys. Chem. B. 2006;110:9-11.

    Not in our library10.1021/jp055689e

47. 47.

    Yu X, Bates JB, Jellison GE, Hart FX. A stable thin-film lithium electrolyte: lithium phosphorus oxynitride. J. Electrochem. Soc. 1997;144:524-532.

    Not in our library10.1149/1.1837443

48. 48.

    Westover AS, et al. Plasma synthesis of spherical crystalline and amorphous electrolyte nanopowders for solid-state batteries. ACS Appl. Mater. Interfaces. 2020;12:11570-11578.

    Not in our library10.1021/acsami.9b20812

49. 49.

    Li W, Ando Y, Minamitani E, Watanabe S. Study of li atom diffusion in amorphous li3po4 with neural network potential. J. Chem. Phys. 2017;147:214106.

    Not in our library10.1063/1.4997242

50. 50.

    Cheng B, Engel EA, Behler J, Dellago C, Ceriotti M. Ab initio thermodynamics of liquid and solid water. Proceedings of the National Academy of Sciences. 2019;116:1110-1115.

    Not in our library10.1073/pnas.1815117116

51. 51.

    Hestness, J. et al. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409 (2017).

    Not in our library

52. 52.

    Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst.32, 8026–8037 (2019).

    Not in our library

53. 53.

    Fey, M. & Lenssen, J. E. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 (2019).

    Not in our library

54. 54.

    Hutter J, Iannuzzi M, Schiffmann F, VandeVondele J. cp2k: atomistic simulations of condensed matter systems. WIREs Computational Molecular Science. 2014;4:15-25.

    Not in our library10.1002/wcms.1159

55. 55.

    Kresse G, Hafner J. Ab initiomolecular dynamics for liquid metals. Phys. Rev. B. 1993;47:558-561.

    Not in our library10.1103/physrevb.47.558

56. 56.

    Kresse G, Furthmüller J. Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Comput. Mater. Sci. 1996;6:15-50.

    Not in our library10.1016/0927-0256(96)00008-0

57. 57.

    Kresse G, Furthmüller J. Efficient iterative schemes forab initiototal-energy calculations using a plane-wave basis set. Phys. Rev. B. 1996;54:11169-11186.

    Not in our library10.1103/physrevb.54.11169

58. 58.

    Perdew JP, Burke K, Ernzerhof M. Generalized gradient approximation made simple. Phys. Rev. Lett. 1996;77:3865-3868.

    Not in our library10.1103/physrevlett.77.3865

59. 59.

    Kresse G, Joubert D. From ultrasoft pseudopotentials to the projector augmented-wave method. Phys. Rev. B. 1999;59:1758-1775.

    Not in our library10.1103/physrevb.59.1758

60. 60.

    Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

    Not in our library

61. 61.

    Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).

    Not in our library

62. 62.

    Reddi, S. J., Kale, S. & Kumar, S. On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237 (2019).

    Not in our library

Cited by papers we hold 13

• High-accuracy physical property prediction for pure organics via molecular representation learning: bridging data to discovery · 2025
  Not checked
• AIMNet2: a neural network potential to meet your neutral, charged, organic, and elemental-organic needs · 2025
  No notice
• The design space of E(3)-equivariant atom-centred interatomic potentials · 2025
  No notice
• MACE-OFF: Short-Range Transferable Machine Learning Force Fields for Organic Molecules · 2025
  No notice
• DynamicBind: predicting ligand-specific protein-ligand complex structure with a deep equivariant generative model · 2024
  No notice
• SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentials · 2023
  No notice
• Scaling deep learning for materials discovery · 2023
  No notice
• PeSTo: parameter-free geometric deep learning for accurate prediction of protein binding interfaces · 2023
  No notice
• Machine Learning-Guided Protein Engineering · 2023
  No notice
• DeePMD-kit v2: A software package for deep potential models · 2023
  No notice
• Learning local equivariant representations for large-scale atomistic dynamics · 2023
  No notice
• Graph neural networks for materials science and chemistry · 2022
  No notice
• SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects · 2021
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.