Making brain–machine interfaces robust to future neural variability

Abstract

A major hurdle to clinical translation of brain–machine interfaces (BMIs) is that current decoders, which are trained from a small quantity of recent data, become ineffective when neural recording conditions subsequently change. We tested whether a decoder could be made more robust to future neural variability by training it to handle a variety of recording conditions sampled from months of previously collected data as well as synthetic training data perturbations. We developed a new multiplicative recurrent neural network BMI decoder that successfully learned a large variety of neural-to-kinematic mappings and became more robust with larger training data sets. Here we demonstrate that when tested with a non-human primate preclinical BMI model, this decoder is robust under conditions that disabled a state-of-the-art Kalman filter-based decoder. These results validate a new BMI strategy in which accumulated data history are effectively harnessed, and may facilitate reliable BMI use by reducing decoder retraining downtime.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 67

1. 1.

   Hochberg L. R.. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012 ).

   Not in our library10.1038/nature11076

2. 2.

   Collinger. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381, 557–564 (2013 ).

   Not in our library10.1016/s0140-6736(12)61816-9

3. 3.

   Gilja V.. Clinical translation of a high-performance neural prosthesis. Nat. Med. 21, 1142–1145 (2015 ).

   Not in our library10.1038/nm.3953

4. 4.

   Jarosiewicz B.. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Sci. Transl. Med. 7, 313ra179–313ra179 (2015 ).

   Not in our library10.1126/scitranslmed.aac7328

5. 5.

   Bacher D.. Neural point-and-click communication by a person with incomplete locked-in syndrome. Neurorehabil. Neural Repair 29, 462–471 (2015 ).

   Not in our library10.1177/1545968314554624

6. 6.

   Gilja V.. A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci. 15, 1752–1757 (2012 ).

   Not in our library10.1038/nn.3265

7. 7.

   Nuyujukian P., Fan J. M., Kao J. C., Ryu S. I. & Shenoy K. V. A high-performance keyboard neural prosthesis enabled by task optimization. IEEE Trans. Biomed. Eng. 62, 21–29 (2015 ).

   Not in our library10.1109/tbme.2014.2354697

8. 8.
   Emergence of a Stable Cortical Map for Neuroprosthetic Control · 2009

   No notice
9. 9.

   Flint R. D., Ethier C., Oby E. R., Miller L. E. & Slutzky M. W. Local field potentials allow accurate decoding of muscle activity. J. Neurophysiol. 108, 18–24 (2012 ).

   Not in our library10.1152/jn.00832.2011

10. 10.

    Nuyujukian P.. Performance sustaining intracortical neural prostheses. J. Neural Eng. 11, 66003 (2014 ).

    Not in our library10.1088/1741-2560/11/6/066003

11. 11.

    Chestek C. A.. Single-neuron stability during repeated reaching in macaque premotor cortex. J. Neurosci. 27, 10742–10750 (2007 ).

    Not in our library10.1523/jneurosci.0959-07.2007

12. 12.

    Simeral J. D., Kim S.-P., Black M. J., Donoghue J. P. & Hochberg L. R. Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array. J. Neural Eng. 8, 25027 (2011 ).

    Not in our library10.1088/1741-2560/8/2/025027

13. 13.

    Perge J. A.. Intra-day signal instabilities affect decoding performance in an intracortical neural interface system. J. Neural Eng. 10, 36004 (2013 ).

    Not in our library10.1088/1741-2560/10/3/036004

14. 14.

    Perge J. A.. Reliability of directional information in unsorted spikes and local field potentials recorded in human motor cortex. J. Neural Eng. 11, 46007 (2014 ).

    Not in our library10.1088/1741-2560/11/4/046007

15. 15.

    Wodlinger B.. Ten-dimensional anthropomorphic arm control in a human brain−machine interface: difficulties, solutions, and limitations. J. Neural Eng. 12, 16011 (2015 ).

    Not in our library10.1088/1741-2560/12/1/016011

16. 16.

    Aflalo T.. Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science 348, 906–910 (2015 ).

    Not in our library10.1126/science.aaa5417

17. 17.

    Jarosiewicz B.. Advantages of closed-loop calibration in intracortical brain-computer interfaces for people with tetraplegia. J. Neural Eng. 10, 46012 (2013 ).

    Not in our library10.1088/1741-2560/10/4/046012

18. 18.

    Huggins J. E., Wren P. A. & Gruis K. L. What would brain-computer interface users want? Opinions and priorities of potential users with amyotrophic lateral sclerosis. Amyotroph. Lateral Scler. 12, 318–324 (2011 ).

    Not in our library10.3109/17482968.2011.572978

19. 19.

    Collinger. Functional priorities, assistive technology, and brain-computer interfaces after spinal cord injury. J. Rehabil. Res. Dev. 50, 145 (2013 ).

    Not in our library10.1682/jrrd.2011.11.0213

20. 20.

    Srinivasan L., Eden U. T., Mitter S. K. & Brown E. N. General-purpose filter design for neural prosthetic devices. J. Neurophysiol. 98, 2456–2475 (2007 ).

    Not in our library10.1152/jn.01118.2006

21. 21.

    Wu W. & Hatsopoulos N. G. Real-time decoding of nonstationary neural activity in motor cortex. IEEE Trans. Neural Syst. Rehabil. Eng 16, 213–222 (2008 ).

    Not in our library10.1109/tnsre.2008.922679

22. 22.

    Shpigelman L., Lalazar H. & Vaadia E. Kernel-ARMA for hand tracking and brain-machine interfacing during 3D motor control. Neural Inf. Process. Syst. 1489–1496 (2008 ).

    Not in our library

23. 23.

    DiGiovanna J., Mahmoudi B., Fortes J., Principe J. C. & Sanchez J. C. Coadaptive brain - machine interface via reinforcement learning. IEEE Trans. Biomed. Eng. 56, 54–64 (2009 ).

    Not in our library10.1109/tbme.2008.926699

24. 24.

    Li Z., O'Doherty J. E., Lebedev M. A. & Nicolelis M. A. L. Adaptive decoding for brain-machine interfaces through bayesian parameter updates. Neural Comput. 23, 3162–3204 (2011 ).

    Not in our library10.1162/neco_a_00207

25. 25.

    Dangi S., Gowda S., Heliot R. & Carmena J. M. in 2011 5th International IEEE/EMBS Conference on Neural Engineering (NER) 609–612 (Cancun, Mexico, 2011).

    Not in our library

26. 26.

    Mahmoudi B. & Sanchez J. C. A symbiotic brain-machine interface through value-based decision making. PLoS ONE 6, e14760 (2011 ).

    Not in our library10.1371/journal.pone.0014760

27. 27.

    Orsborn A. L.. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014 ).

    Not in our library10.1016/j.neuron.2014.04.048

28. 28.

    Bishop W.. Self-recalibrating classifiers for intracortical brain-computer interfaces. J. Neural Eng. 11, 26001 (2014 ).

    Not in our library10.1088/1741-2560/11/2/026001

29. 29.

    Pohlmeyer E. A., Mahmoudi B., Geng S., Prins N. W. & Sanchez J. C. Using reinforcement learning to provide stable brain-machine interface control despite neural input reorganization. PLoS ONE 9, e87253 (2014 ).

    Not in our library10.1371/journal.pone.0087253

30. 30.

    Chestek C. A.. Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex. J. Neural Eng. 8, 45005 (2011 ).

    Not in our library10.1088/1741-2560/8/4/045005

31. 31.

    Flint R. D., Wright Z. A., Scheid M. R. & Slutzky M. W. Long term, stable brain machine interface performance using local field potentials and multiunit spikes. J. Neural Eng. 10, 56005 (2013 ).

    Not in our library10.1088/1741-2560/10/5/056005

32. 32.

    Stavisky S. D., Kao J. C., Nuyujukian P., Ryu S. I. & Shenoy K. V. A high performing brain–machine interface driven by low-frequency local field potentials alone and together with spikes. J. Neural Eng. 12, 36009 (2015 ).

    Not in our library10.1088/1741-2560/12/3/036009

33. 33.

    Homer M. L., Nurmikko A. V, Donoghue J. P. & Hochberg L. R. Sensors and decoding for intracortical brain computer interfaces. Annu. Rev. Biomed. Eng. 15, 383–405 (2013 ).

    Not in our library10.1146/annurev-bioeng-071910-124640

34. 34.

    Kao J. C., Stavisky S. D., Sussillo D., Nuyujukian P. & Shenoy K. V. Information systems opportunities in brain-machine interface decoders. Proc. IEEE 102, 666–682 (2014 ).

    Not in our library

35. 35.

    Sutskever I., Martens J. & Hinton G. in Proceedings of the 28th International Conference on Machine Learning (ICML) (Bellevue, WA, USA, 2011).

    Not in our library

36. 36.

    Martens J. & Sutskever I. in Proceedings of the 28th International Conference on Machine Learning, (ICML) 1033–1040 (Bellevue, WA, USA, 2011).

    Not in our library

37. 37.

    Churchland M. M. & Shenoy K. V. Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex. J. Neurophysiol. 97, 4235–4257 (2007 ).

    Not in our library10.1152/jn.00095.2007

38. 38.

    Sussillo D.. A recurrent neural network for closed-loop intracortical brain–machine interface decoders. J. Neural Eng. 9, 26027 (2012 ).

    Not in our library10.1088/1741-2560/9/2/026027

39. 39.

    Hochreiter S. & Schmidhuber J. Long Short-Term Memory. Neural Comput. 9, 1735–1780 (1997 ).

    Not in our library10.1162/neco.1997.9.8.1735

40. 40.

    Fan J. M.. Intention estimation in brain–machine interfaces. J. Neural Eng. 11, 16004 (2014 ).

    Not in our library10.1088/1741-2560/11/1/016004

41. 41.

    Chase S. M., Schwartz A. B. & Kass R. E. Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms. Neural Netw. 22, 1203–1213 (2009 ).

    Not in our library10.1016/j.neunet.2009.05.005

42. 42.

    Koyama S.. Comparison of brain-computer interface decoding algorithms in open-loop and closed-loop control. J. Comput. Neurosci. 29, 73–87 (2010 ).

    Not in our library10.1007/s10827-009-0196-9

43. 43.

    Cunningham J. P.. A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces. J. Neurophysiol. 105, 1932–1949 (2011 ).

    Not in our library10.1152/jn.00503.2010

44. 44.

    Seo D.. Wireless recording in the peripheral nervous system with ultrasonic neural dust neuron neuroresource wireless recording in the peripheral nervous system with ultrasonic neural dust. Neuron 91, 1–11 (2016 ).

    Not in our library10.1016/j.neuron.2016.06.034

45. 45.

    Fraser G. W., Chase S. M., Whitford A. & Schwartz A. B. Control of a brain-computer interface without spike sorting. J. Neural Eng. 6, 55004 (2009 ).

    Not in our library10.1088/1741-2560/6/5/055004

46. 46.

    So K., Dangi S., Orsborn A. L., Gastpar M. C. & Carmena J. M. Subject-specific modulation of local field potential spectral power during brain–machine interface control in primates. J. Neural Eng. 11, 26002 (2014 ).

    Not in our library10.1088/1741-2560/11/2/026002

47. 47.

    Shenoy K. V. & Carmena J. M. Combining decoder design and neural adaptation in brain-machine interfaces. Neuron 84, 665–680 (2014 ).

    Not in our library10.1016/j.neuron.2014.08.038

48. 48.

    Barrese J. C.. Failure mode analysis of silicon-based intracortical microelectrode arrays in non-human primates. J. Neural Eng. 10, 66014 (2013 ).

    Not in our library10.1088/1741-2560/10/6/066014

49. 49.

    Gold C., Henze D. A., Koch C. & Buzsáki G. On the origin of the extracellular action potential waveform: A modeling study. J. Neurophysiol. 95, 3113–3128 (2006 ).

    Not in our library10.1152/jn.00979.2005

50. 50.

    Kao J. C.. in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2013, 293–298 (IEEE, 2013).

    Not in our library10.1109/embc.2013.6609495

51. 51.

    Cunningham J. P.. A closed-loop human simulator for investigating the role of feedback-control in brain-machine interfaces. J. Neurophysiol. 105, 1932–1949 (2010 ).

    Not in our library10.1152/jn.00503.2010

52. 52.

    Schwartz A., Ketter R. & Georgopoulos A. Primate motor cortex and free arm movements to visual targets in three-dimensional space. I. Relations between single cell discharge and direction of movement. J. Neurosci. 8, 2913–2927 (1988 ).

    Not in our library10.1523/jneurosci.08-08-02913.1988

53. 53.

    Riehle A. & Requin J. Monkey primary motor and premotor cortex: single-cell activity related to prior information about direction and extent of an intended movement. J. Neurophysiol. 61, 534–549 (1989 ).

    Not in our library10.1152/jn.1989.61.3.534

54. 54.

    Wise S. P., Moody S. L., Blomstrom K. J. & Mitz A. R. Changes in motor cortical activity during visuomotor adaptation. Exp. Brain Res. 121, 285–299 (1998 ).

    Not in our library10.1007/s002210050462

55. 55.

    Crammond D. & Kalaska J. Differential relation of discharge in primary motor cortex and premotor cortex to movements versus actively maintained postures during a reaching task. Exp. Brain Res. 108, 45–61 (1996 ).

    Not in our library10.1007/bf00242903

56. 56.

    Messier J. & Kalaska J. F. Covariation of primate dorsal premotor cell activity with direction and amplitude during a memorized-delay reaching task. J. Neurophysiol. 84, 152–165 (2000 ).

    Not in our library10.1152/jn.2000.84.1.152

57. 57.

    Stark E. & Abeles M. Predicting movement from multiunit activity. J. Neurosci. 27, 8387–8394 (2007 ).

    Not in our library10.1523/jneurosci.1321-07.2007

58. 58.

    Perel S.. Single-unit activity, threshold crossings, and local field potentials in motor cortex differentially encode reach kinematics. J. Neurophysiol. 114, 1500–1512 (2015 ).

    Not in our library10.1152/jn.00293.2014

59. 59.

    Christie B. P.. Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain–machine interface performance. J. Neural Eng. 12, 16009 (2015 ).

    Not in our library10.1088/1741-2560/12/1/016009

60. 60.

    Nuyujukian P.. in 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 1301–1305 (Boston, USA, 2011).

    Not in our library10.1109/iembs.2011.6090306

61. 61.

    Kaufman M. T., Churchland M. M., Ryu S. I. & Shenoy K. V. Cortical activity in the null space: permitting preparation without movement. Nat. Neurosci. 17, 440–448 (2014 ).

    Not in our library10.1038/nn.3643

62. 62.

    Suminski A. J., Tkach D. C., Fagg A. H. & Hatsopoulos N. G. Incorporating feedback from multiple sensory modalities enhances brain-machine interface control. J. Neurosci. 30, 16777–16787 (2010 ).

    Not in our library10.1523/jneurosci.3967-10.2010

63. 63.

    Martens J. in 27th International Conference on Machine Learning 951, 735–742 (Haifa, Israel, 2010).

    Not in our library

64. 64.

    Hinton G. E., Srivastava N., Krizhevsky A., Sutskever I. & Salakhutdinov R. R. Improving neural networks by preventing co-adaptation of feature detectors. Preprint at arXiv:1207.0580. (2012).

    Not in our library

65. 65.

    Kim S.-P., Simeral J. D., Hochberg L. R., Donoghue J. P. & Black M. J. Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5, 455–476 (2008 ).

    Not in our library10.1088/1741-2560/5/4/010

66. 66.

    Wu W.. Neural decoding of cursor motion using a Kalman filter. Adv. Neural Inf. Process. Syst. 15, 133–140 (2003).

    Not in our library

67. 67.

    Knyazev A. V & Argentati M. E. Principal angles between subspaces in an a-based scalar product: algorithms and perturbation estimates. SIAM J. Sci. Comput. 23, 2008–2040 (2002 ).

    Not in our library

Cited by papers we hold 2

• Machine Learning for Neural Decoding · 2020
  No notice
• Structure in Neural Activity during Observed and Executed Movements Is Shared at the Neural Population Level, Not in Single Neurons · 2020
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.