Skip to content
naulon

No retraction, correction or concern found when last checked on Oct 29, 2026.

Measuring behavior across scales

Gordon J. Berman

BMC Biology · 2018 · Review · CC BY

doi:10.1186/s12915-018-0494-7

Abstract

The need for high-throughput, precise, and meaningful methods for measuring behavior has been amplified by our recent successes in measuring and manipulating neural circuitry. The largest challenges associated with moving in this direction, however, are not technical but are instead conceptual: what numbers should one put on the movements an animal is performing (or not performing)? In this review, I will describe how theoretical and data analytical ideas are interfacing with recently-developed computational and experimental methodologies to answer these questions across a variety of contexts, length scales, and time scales. I will attempt to highlight commonalities between approaches and areas where further advances are necessary to place behavior on the same quantitative footing as other scientific fields.

References 111

  1. 1.

    Anderson DJ, Perona P. Toward a Science of Computational Ethology. Neuron. 2014;84(1):18-31.

    Not in our library10.1016/j.neuron.2014.09.005

  2. 2.

    Krakauer JW, Ghazanfar AA, Gomez-Marin A, Maciver MA, Poeppel D. Neuroscience Needs Behavior: Correcting a Reductionist Bias. Neuron. 2017;93(3):480-90.

    Not in our library10.1016/j.neuron.2016.12.041

  3. 3.

    Wang XJ, Krystal JH. Computational psychiatry. Neuron. 2014;84(3):638-54.

    Not in our library10.1016/j.neuron.2014.10.018

  4. 4.

    Lebedev MA, Nicolelis MAL. Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. Physiol Rev. 2017;97(2):767-837.

    Not in our library10.1152/physrev.00027.2016

  5. 5.

    Anderson PW. More Is Different. Science. 1972;177(4047):393-6.

    Not in our library10.1126/science.177.4047.393

  6. 6.

    Goldenfeld N, Kadanoff LP. Simple Lessons from Complexity. Science. 1999;284(5411):87-9.

    Not in our library10.1126/science.284.5411.87

  7. 7.

    Dell AI, Bender JA, Branson K, Couzin ID, De Polavieja GG, Noldus LPJJ, et al. Automated image-based tracking and its application in ecology. Trends Ecol Evol. 2014;29(7):417-28.

    Not in our library10.1016/j.tree.2014.05.004

  8. 8.

    Egnor SER, Branson K. Computational Analysis of Behavior. Ann Rev Neuro. 2016;39:217-36.

    Not in our library10.1146/annurev-neuro-070815-013845

  9. 9.

    Robie AA, Seagraves KM, Egnor SER, Branson K. Machine vision methods for analyzing social interactions. J Exp Bio. 2017;220(1):25-34.

    Not in our library10.1242/jeb.142281

  10. 10.

    Calhoun AJ, Murthy M. Quantifying behavior to solve sensorimotor transformations: advances from worms and flies. Curr Opin Neurobiol. 2017;46:90-8.

    Not in our library10.1016/j.conb.2017.08.006

  11. 11.

    Stephens GJ, Osborne LC, Bialek W. Searching for simplicity in the analysis of neurons and behavior. Proc Nat Acad Sci. 2011;108(Supp 3):15565-71.

    Not in our library10.1073/pnas.1010868108

  12. 12.

    Gomez-Marin A, Paton JJ, Kampff AR, Costa RM, Mainen ZF. Big behavioral data: psychology, ethology and the foundations of neuroscience. Nat Neurosci. 2014;17(11):1455-62.

    Not in our library10.1038/nn.3812

  13. 13.

    Montague PR, Dolan RJ, Friston KJ, Dayan P. Computational psychiatry. Trends Cogn Sci. 2012;16(1):72-80.

    Not in our library10.1016/j.tics.2011.11.018

  14. 14.

    Anderson DJ, Adolphs R. A Framework for Studying Emotions across Species. Cell. 2014;157(1):187-200.

    Not in our library10.1016/j.cell.2014.03.003

  15. 15.

    Anderson DJ. Circuit modules linking internal states and social behaviour in flies and mice. Nature Rev Neuro. 2016;17(11):692-704.

    Not in our library10.1038/nrn.2016.125

  16. 16.

    Huys QJM, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci. 2016;19(3):404-13.

    Not in our library10.1038/nn.4238

  17. 17.

    Stephan KE, Mathys C. Computational approaches to psychiatry. Curr Opin Neurobiol. 2014;25:85-92.

    Not in our library10.1016/j.conb.2013.12.007

  18. 18.

    Leshner A, Pfaff DW. Quantification of behavior. Proc Nat Acad Sci. 2011;108(Supp 3):15537-41.

    Not in our library10.1073/pnas.1010653108

  19. 19.

    Fonio E, Golani I, Benjamini Y. Measuring behavior of animal models: faults and remedies. Nat Methods. 2012;9(12):1167-70.

    Not in our library10.1038/nmeth.2252

  20. 20.

    Altmann J. Observational study of behavior: sampling methods. Behaviour. 1974;49(3):227-67.

    Not in our library10.1163/156853974x00534

  21. 21.

    Martin P, Bateson P. Measuring behaviour: an introductory guide. Cambridge. 2007.

    Not in our library

  22. 22.

    Gao P, Ganguli S. On simplicity and complexity in the brave new world of large-scale neuroscience. Curr Opin Neurobiol. 2015;32:148-55.

    Not in our library10.1016/j.conb.2015.04.003

  23. 23.

    Panda S, Hogenesch JB, Kay SA. Circadian rhythms from flies to human. Nature. 2002;417(6886):329-35.

    Not in our library10.1038/417329a

  24. 24.

    Yemini E, Jucikas T, Grundy LJ, Brown AEX, Schafer WR. A database of Caenorhabditis elegans behavioral phenotypes. Nat Methods. 2013;10(9):877-9.

    Not in our library10.1038/nmeth.2560

  25. 25.
  26. 26.

    Ayroles JF, Buchanan SM, O’Leary C, Skutt-Kakaria K, Grenier JK, Clark AG, et al. Behavioral idiosyncrasy reveals genetic control of phenotypic variability. Proc Nat Acad Sci. 2015;112(21):6706-11.

    Not in our library10.1073/pnas.1503830112

  27. 27.

    Branson K, Robie AA, Bender JA, Perona P, Dickinson MH. High-throughput ethomics in large groups of Drosophila. Nat Methods. 2009;6(6):451-7.

    Not in our library10.1038/nmeth.1328

  28. 28.

    Pérez-Escudero A, Vicente-Page J, Hinz RC, Arganda S, De Polavieja GG. idTracker: tracking individuals in a group by automatic identification of unmarked animals. Nat Methods. 2014;11(7):743-8.

    Not in our library10.1038/nmeth.2994

  29. 29.

    Naumann EA, Fitzgerald JE, Dunn TW, Rihel J, Sompolinsky H, Engert F. From Whole-Brain Data to Functional Circuit Models: The Zebrafish Optomotor Response. Cell. 2016;167(4):947-960.e20.

    Not in our library10.1016/j.cell.2016.10.019

  30. 30.

    Orger MB, De Polavieja GG. Zebrafish Behavior: Opportunities and Challenges. Ann Rev Neuro. 2017;40:125-147.

    Not in our library10.1146/annurev-neuro-071714-033857

  31. 31.

    Dankert H, Wang L, Hoopfer ED, Anderson DJ, Perona P. Automated monitoring and analysis of social behavior in Drosophila. Nat Methods. 2009;6(4):297-303.

    Not in our library10.1038/nmeth.1310

  32. 32.

    Kabra M, Robie AA, Rivera-Alba M, Branson S, Branson K. JAABA: interactive machine learning for automatic annotation of animal behavior. Nat Methods. 2013;10(1):64-7.

    Not in our library10.1038/nmeth.2281

  33. 33.

    de Chaumont F, Coura RDS, Serreau P, Cressant A, Chabout J, Granon S, et al. Computerized video analysis of social interactions in mice. Nat Methods. 2012;9(4):410-7.

    Not in our library10.1038/nmeth.1924

  34. 34.

    Kain J, Stokes C, Gaudry Q, Song X, Foley J, Wilson R, et al. Leg-tracking and automated behavioural classification in Drosophila. Nat Communications. 2013;4:1910.

    Not in our library10.1038/ncomms2908

  35. 35.

    Hong W, Kennedy A, Burgos-Artizzu XP, Zelikowsky M, Navonne SG, Perona P, et al. Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning. Proc Nat Acad Sci. 2015;112(38):E5351-E5360.

    Not in our library10.1073/pnas.1515982112

  36. 36.

    Dickinson MH, Farley CT, Full RJ, Koehl MAR, Kram R, Lehman S. How Animals Move: An Integrative View. Science. 2000;288(5):100-6.

    Not in our library10.1126/science.288.5463.100

  37. 37.

    Childress S, Hosoi A, Schultz WW, Wang ZJ. Natural locomotion in fluids and on surfaces: swimming, flying, and sliding. 2012.

    Not in our library

  38. 38.

    Holmes P, Full RJ, Koditschek D, Guckenheimer J. The Dynamics of Legged Locomotion: Models, Analyses, and Challenges. SIAM Rev. 2006;48:207-304.

    Not in our library

  39. 39.

    Miller LA, Goldman DI, Hedrick TL, Tytell ED, Wang ZJ, Yen J, et al. Using computational and mechanical models to study animal locomotion. Int Comp Bio. 2012;52(5):553-75.

    Not in our library10.1093/icb/ics115

  40. 40.

    McInroe B, Astley HC, Gong C, Kawano SM, Schiebel PE, Rieser JM, et al. Tail use improves soft substrate performance in models of early vertebrate land locomotors. Science. 2016;353(6295):154-8.

    Not in our library10.1126/science.aaf0984

  41. 41.

    Aguilar J, Zhang T, Qian F, Kingsbury M, McInroe B, Mazouchova N, et al. A review on locomotion robophysics: the study of movement at the intersection of robotics, soft matter and dynamical systems. Rep Prog Phys. 2016;79(11):110001.

    Not in our library10.1088/0034-4885/79/11/110001

  42. 42.

    Dickinson MH, Lehmann FO, Sane SP. Wing rotation and the aerodynamic basis of insect flight. Science. 1999;284(5422):1954-60.

    Not in our library10.1126/science.284.5422.1954

  43. 43.

    Cowan NJ, Ankarali MM, Dyhr JP, Madhav MS, Roth E, Sefati S, et al. Feedback control as a framework for understanding tradeoffs in biology. Integr Comp Biol. 2014;54(2):223-37.

    Not in our library10.1093/icb/icu050

  44. 44.

    Roth E, Sponberg S, Cowan NJ. A comparative approach to closed-loop computation. Curr Opin Neurobiol. 2014;25:54-62.

    Not in our library10.1016/j.conb.2013.11.005

  45. 45.

    Alexander RM. Optima for Animals. 1996.

    Not in our library

  46. 46.

    Full RJ, Koditschek DE. Templates and anchors: Neuromechanical hypotheses of legged locomotion on land. J Exp Bio. 1999;202(23):3325-32.

    Not in our library10.1242/jeb.202.23.3325

  47. 47.

    Monty Python’s FlyingCircus. “The Ministry of Silly Walks”. BBC. 1970;2(1).

    Not in our library

  48. 48.

    Ristroph L, Bergou AJ, Ristroph G, Coumes K, Berman GJ, Guckenheimer J, et al. Discovering the flight autostabilizer of fruit flies by inducing aerial stumbles. Proc Nat Acad Sci. 2010;107(11):4820-4.

    Not in our library10.1073/pnas.1000615107

  49. 49.

    Jindrich DL, Full RJ. Dynamic stabilization of rapid hexapedal locomotion. J Exp Bio. 2002;205(18):2803-23.

    Not in our library10.1242/jeb.205.18.2803

  50. 50.

    Revzen S, Guckenheimer JM. Finding the dimension of slow dynamics in a rhythmic system. J Royal Soc Interface. 2012;9(70):957-71.

    Not in our library10.1098/rsif.2011.0431

  51. 51.

    Osborne LC, Lisberger SG, Bialek W. A sensory source for motor variation. Nature. 2005;437(7057):412-6.

    Not in our library10.1038/nature03961

  52. 52.

    Stephens GJ, de Mesquita MB, Ryu WS, Bialek W. Emergence of long timescales and stereotyped behaviors in Caenorhabditis elegans. Proc Nat Acad Sci. 2011;108(18):7286-9.

    Not in our library10.1073/pnas.1007868108

  53. 53.
  54. 54.
  55. 55.

    Brown AEX, Yemini EI, Grundy LJ, Jucikas T, Schafer WR. A dictionary of behavioral motifs reveals clusters of genes affecting Caenorhabditis elegans locomotion. Proc Nat Acad Sci. 2013;110(2):791-6.

    Not in our library10.1073/pnas.1211447110

  56. 56.

    Wiltschko AB, Johnson MJ, Iurilli G, Peterson RE, Katon JM, Pashkovski SL, et al. Mapping Sub-Second Structure in Mouse Behavior. Neuron. 2015;88(6):1121-35.

    Not in our library10.1016/j.neuron.2015.11.031

  57. 57.

    Kato S, Kaplan HS, Schrödel T, Skora S, Lindsay TH, Yemini E, et al. Global Brain Dynamics Embed the Motor Command Sequence of Caenorhabditis elegans. Cell. 2015;163(3):656-669.

    Not in our library10.1016/j.cell.2015.09.034

  58. 58.

    Todd JG, Kain JS, de Bivort BL. Systematic exploration of unsupervised methods for mapping behavior. Phys Biol. 2017;14(1):015002.

    Not in our library10.1088/1478-3975/14/1/015002

  59. 59.

    Klibaite U, Berman GJ, Cande J, Stern DL, Shaevitz JW. An unsupervised method for quantifying the behavior of paired animals. Phys Biol. 2017;14(1):015006.

    Not in our library10.1088/1478-3975/aa5c50

  60. 60.
  61. 61.
  62. 62.

    Nguyen JP, Shipley FB, Linder AN, Plummer GS, Liu M, Setru SU, et al. Whole-brain calcium imaging with cellular resolution in freely behaving Caenorhabditis elegans. Proc Nat Acad Sci. 2016;113(8):E1074-E1081.

    Not in our library10.1073/pnas.1507110112

  63. 63.
  64. 64.
  65. 65.

    Ristroph L, Berman GJ, Bergou AJ, Wang ZJ, Cohen I. Automated hull reconstruction motion tracking (HRMT) applied to sideways maneuvers of free-flying insects. J Exp Bio. 2009;212(9):1324-35.

    Not in our library10.1242/jeb.025502

  66. 66.

    Fontaine EI, Zabala F, Dickinson MH, Burdick JW. Wing and body motion during flight initiation in Drosophila revealed by automated visual tracking. J Exp Bio. 2009;212(9):1307-23.

    Not in our library10.1242/jeb.025379

  67. 67.

    Uhlmann V, Ramdya P, Delgado-Gonzalo R, Benton R, Unser M. FlyLimbTracker: An active contour based approach for leg segment tracking in unmarked, freely behaving Drosophila. PLoS ONE. 2017;12(4):e0173433.

    Not in our library10.1371/journal.pone.0173433

  68. 68.

    Vogelstein JT, Park Y, Ohyama T, Kerr RA, Truman JW, Priebe CE, et al. Discovery of brainwide neural-behavioral maps via multiscale unsupervised structure learning. Science. 2014;344(6182):386-92.

    Not in our library10.1126/science.1250298

  69. 69.

    Geng W, Cosman P, Baek JH, Berry CC, Schafer WR. Quantitative classification and natural clustering of Caenorhabditis elegans behavioral phenotypes. Genetics. 2003;165(3):1117-26.

    Not in our library10.1093/genetics/165.3.1117

  70. 70.

    Ghosh R, Mohammadi A, Kruglyak L, Ryu WS. Multiparameter behavioral profiling reveals distinct thermal response regimes in Caenorhabditis elegans. BMC Biol. 2012;10:85.

    Not in our library10.1186/1741-7007-10-85

  71. 71.

    Jhuang H, Garrote E, Mutch J, Yu X, Khilnani V, Poggio T, et al. Automated home-cage behavioural phenotyping of mice. Nat Commun. 2010;1:68.

    Not in our library10.1038/ncomms1064

  72. 72.

    Golani I, Kafkafi N, Drai D. Phenotyping stereotypic behaviour: collective variables, range of variation and predictability. Appl Anim Behav Sci. 1999;65(3):191-220.

    Not in our library

  73. 73.

    Bongard J, Lipson H. Automated reverse engineering of nonlinear dynamical systems. Proc Nat Acad Sci. 2007;104(24):9943-8.

    Not in our library10.1073/pnas.0609476104

  74. 74.

    Natale JL, Hofmann D, Hernández DG, Nemenman I. Reverse-engineering biological networks from large data sets. arXiv. 2017.

    Not in our library

  75. 75.

    Daniels BC, Nemenman I. Automated adaptive inference of phenomenological dynamical models. Nat Commun. 2015;6(1):8133.

    Not in our library10.1038/ncomms9133

  76. 76.

    Berman GJ, Bialek W, Shaevitz JW. Predictability and hierarchy in Drosophila behavior. Proc Nat Acad Sci. 2016;113(42):11943-8.

    Not in our library10.1073/pnas.1607601113

  77. 77.

    Gomez-Marin A, Stephens GJ, Brown AEX. Hierarchical compression of Caenorhabditis elegans locomotion reveals phenotypic differences in the organization of behaviour. J Royal Soc Interface. 2016;13(121):20160466.

    Not in our library10.1098/rsif.2016.0466

  78. 78.

    Schwarz RF, Branicky R, Grundy LJ, Schafer WR, Brown AEX. Changes in Postural Syntax Characterize Sensory Modulation and Natural Variation of C. elegans Locomotion. PLoS Comp Bio. 2015;11(8):e1004322.

    Not in our library10.1371/journal.pcbi.1004322

  79. 79.
  80. 80.

    Debnath L, Shah FA. Wavelet transforms and their applications. 2015.

    Not in our library

  81. 81.

    Benjamini Y, Fonio E, Galili T, Havkin GZ, Golani I. Quantifying the buildup in extent and complexity of free exploration in mice. Proc Nat Acad Sci. 2011;108(Supp 3):15580-7.

    Not in our library10.1073/pnas.1014837108

  82. 82.

    Katz Y, Tunstrom K, Ioannou CC, Huepe C, Couzin ID. Inferring the structure and dynamics of interactions in schooling fish. Proc Natl Acad Sci. 2011;108(46):18720-5.

    Not in our library10.1073/pnas.1107583108

  83. 83.

    Vicsek T, Czirók A, Ben-Jacob E, Cohen I, Shochet O. Novel type of phase transition in a system of self-driven particles. Phys Rev Lett. 1995;75(6):1226-9.

    Not in our library10.1103/physrevlett.75.1226

  84. 84.

    Toner J, Tu Y. Flocks, herds, and schools: A quantitative theory of flocking. Phys Rev E. 1998;58(4):4828-58.

    Not in our library

  85. 85.

    Buhl J, Sumpter D, Couzin ID, Hale J, Despland E, Miller E, et al. From disorder to order in marching locusts. Science. 2006;312(5778):1402-6.

    Not in our library10.1126/science.1125142

  86. 86.

    Haller G. Lagrangian Coherent Structures. Ann Rev Fluid Mech. 2015;47(1):137-62.

    Not in our library

  87. 87.

    Brunton BW, Johnson LA, Ojemann JG, Kutz JN. Extracting spatial-temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition. J Neurosci Methods. 2016;258:1-15.

    Not in our library10.1016/j.jneumeth.2015.10.010

  88. 88.

    Coen P, Clemens J, Weinstein AJ, Pacheco DA, Deng Y, Murthy M. Dynamic sensory cues shape song structure in Drosophila. Nature. 2014;507(7491):233-7.

    Not in our library10.1038/nature13131

  89. 89.
  90. 90.
  91. 91.

    Arthur BJ, Sunayama-Morita T, Coen P, Murthy M, Stern DL. Multi-channel acoustic recording and automated analysis of Drosophila courtship songs. BMC Biol. 2013;11:11.

    Not in our library10.1186/1741-7007-11-11

  92. 92.
  93. 93.

    Mets DG, Brainard MS. An Automated Approach to the Quantitation of Vocalizations and Vocal Learning in the Songbird. bioRxiv. 2017.

    Not in our library10.1371/journal.pcbi.1006437

  94. 94.

    Van Segbroeck M, Knoll AT, Levitt P, Narayanan S. MUPET–Mouse Ultrasonic Profile ExTraction: A Signal Processing Tool for Rapid and Unsupervised Analysis of Ultrasonic Vocalizations. Neuron. 2017; 94(3):465–85.

    Not in our library10.1016/j.neuron.2017.04.005

  95. 95.

    Robie AA, Hirokawa J, Edwards AW, Umayam LA, Lee A, Phillips ML, et al. Mapping the Neural Substrates of Behavior. Cell. 2017;170(2):393-406.e28.

    Not in our library10.1016/j.cell.2017.06.032

  96. 96.

    Clemens J, Girardin CC, Coen P, Guan XJ, Dickson BJ, Murthy M. Connecting Neural Codes with Behavior in the Auditory System of Drosophila. Neuron. 2015;87(6):1332-43.

    Not in our library10.1016/j.neuron.2015.08.014

  97. 97.

    Wang NXR, Olson JD, Ojemann JG, Rao RPN, Brunton BW. Unsupervised Decoding of Long-Term, Naturalistic Human Neural Recordings with Automated Video and Audio Annotations. Front Hum Neurosci. 2016;10(251):78-13.

    Not in our library10.3389/fnhum.2016.00165

  98. 98.

    Billings J, Medda A, Shakil S, Shen X, Kashyap A, Chen S, et al. Instantaneous brain dynamics mapped to a continuous state space. Neuroimage. 2017;162:344-52.

    Not in our library10.1016/j.neuroimage.2017.08.042

  99. 99.
  100. 100.
  101. 101.
  102. 102.
  103. 103.

    Dawkins R, Bateson PPG, Hinde RA. Hierarchical organisation: A candidate principle for ethology. Growing Points in Ethology. 1976.

    Not in our library

  104. 104.

    Tinbergen N. The Study of Instinct. Oxford. 1951.

    Not in our library

  105. 105.

    Simon HA. The Architecture of Complexity. Proc Am Philos Soc. 1962;106(6):467-82.

    Not in our library

  106. 106.

    Simon HA, Pattee HH. The organization of complex systems. Hierarchy Theory. 1973.

    Not in our library

  107. 107.

    Dawkins R, Dawkins M. Hierarchical Organization and Postural Facilitation - Rules for Grooming in Flies. Anim Behav. 1976;24(4):739-55.

    Not in our library

  108. 108.

    Lefebvre L. Grooming in crickets: timing and hierarchical organization. Anim Behav. 1981;29(4):973-84.

    Not in our library

  109. 109.

    Solway A, Diuk C, Córdova N, Yee D, Barto AG, Niv Y, et al. Optimal behavioral hierarchy. PLoS Comput Biol. 2014;10(8):e1003779.

    Not in our library10.1371/journal.pcbi.1003779

  110. 110.

    Sponberg S. The Emergent Physics of Animal Locomotion. Phys Today. 2017;70(9):34.

    Not in our library

  111. 111.

    Niv Y. Reinforcement learning in the brain. J Math Psychol. 2009;53(3):139-54.

    Not in our library

Cited by papers we hold 7

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.