Computational bioacoustics with deep learning: a review and roadmap

Abstract

Animal vocalisations and natural soundscapes are fascinating objects of study, and contain valuable evidence about animal behaviours, populations and ecosystems. They are studied in bioacoustics and ecoacoustics, with signal processing and analysis an important component. Computational bioacoustics has accelerated in recent decades due to the growth of affordable digital sound recording devices, and to huge progress in informatics such as big data, signal processing and machine learning. Methods are inherited from the wider field of deep learning, including speech and image processing. However, the tasks, demands and data characteristics are often different from those addressed in speech or music analysis. There remain unsolved problems, and tasks for which evidence is surely present in many acoustic signals, but not yet realised. In this paper I perform a review of the state of the art in deep learning for computational bioacoustics, aiming to clarify key concepts and identify and analyse knowledge gaps. Based on this, I offer a subjective but principled roadmap for computational bioacoustics with deep learning: topics that the community should aim to address, in order to make the most of future developments in AI and informatics, and to use audio data in answering zoological and ecological questions.

References 241

1. 1.
   A Review of Deep Learning Based Methods for Acoustic Scene Classification · 2020

   Not checked
2. 2.

   Acconcjaioco M, Ntalampiras S. One-shot learning for acoustic identification of bird species in non-stationary environments. 2021;755-762.

   Not in our library

3. 3.

   Adavanne S, Drossos K, Çakr E, Virtanen T. Stacked convolutional and recurrent neural networks for bird audio detection. 2017;1729-1733.

   Not in our library

4. 4.

   Adavanne S, Politis A, Virtanen T, Adavanne S, Politis A, Virtanen T. Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network. 2018;1462-1466.

   Not in our library

5. 5.

   Adi K, Johnson MT, Osiejuk TS. Acoustic censusing using automatic vocalization classification and identity recognition. Journal of the Acoustical Society of America. 2010;127(2):874-883.

   Not in our library10.1121/1.3273887

6. 6.

   Allen AN, Harvey M, Harrell L, Jansen A, Merkens KP, Wall CC, Cattiau J, Oleson EM. A convolutional neural network for automated detection of humpback whale song in a diverse, long-term passive acoustic dataset. Frontiers in Marine Science. 2021;8:165.

   Not in our library10.3389/fmars.2021.607321

7. 7.

   Arteta C, Lempitsky V, Zisserman A. Counting in the wild. 2016;483-498.

   Not in our library

8. 8.

   Baevski A, Zhou Y, Mohamed A, Auli M. wav2vec 2.0: a framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems (NeurIPS). 2020;33:12449-12460.

   Not in our library

9. 9.

   Bai S, Kolter JZ, Koltun V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. ArXiv preprint. 2018.

   Not in our library

10. 10.

    Bain M, Nagrani A, Schofield D, Berdugo S, Bessa J, Owen J, Hockings KJ, Matsuzawa T, Hayashi M, Biro D, Carvalho S, Zisserman A. Automated audiovisual behavior recognition in wild primates. Science Advances. 2021;7(46):eabi4883.

    Not in our library10.1126/sciadv.abi4883

11. 11.

    Baker E, Vincent S. A deafening silence: a lack of data and reproducibility in published bioacoustics research? Biodiversity Data Journal. 2019;7:e36783.

    Not in our library10.3897/bdj.7.e36783

12. 12.

    Balestriero R, Cosentino R, Glotin H, Baraniuk R. Spline filters for end-to-end deep learning. 2018.

    Not in our library

13. 13.

    Baumann J, Lohrenz T, Roy A, Fingscheidt T. Beyond the dcase 2017 challenge on rare sound event detection: a proposal for a more realistic training and test framework. 2020.

    Not in our library

14. 14.

    Beecher MD. Signalling systems for individual recognition: an information theory approach. Animal Behaviour. 1989;38(2):248-261.

    Not in our library10.1016/s0003-3472(89)80087-9

15. 15.

    Beery S, Liu Y, Morris D, Piavis J, Kapoor A, Meister M, Perona P. Synthetic examples improve generalization for rare classes. 2020;abs/1904.05916:863-873.

    Not in our library

16. 16.

    Bergler C, Schmitt M, Cheng RX, Schröter H, Maier A, Barth V, Weber M, Nöth E, Ekštein K. Deep representation learning for orca call type classification. Text, Speech, and Dialogue. TSD 2019. Lecture Notes in Computer Science. 2019a;11697:274-286.

    Not in our library10.1007/978-3-030-27947-9_23

17. 17.

    Bergler C, Schröter H, Cheng RX, Barth V, Weber M, Nöth E, Hofer H, Maier A. Orca-spot: an automatic killer whale sound detection toolkit using deep learning. Scientific Reports. 2019b;9(1):1-17.

    Not in our library10.1038/s41598-019-47335-w

18. 18.

    Best P, Ferrari M, Poupard M, Paris S, Marxer R, Symonds H, Spong P, Glotin H. Deep learning and domain transfer for orca vocalization detection. 2020.

    Not in our library

19. 19.

    Bhatia R. Bird song synthesis using neural vocoders. 2021.

    Not in our library

20. 20.

    Bjorck J, Rappazzo BH, Chen D, Bernstein R, Wrege PH, Gomes CP. Automatic detection and compression for passive acoustic monitoring of the African forest elephant. Proceedings of the AAAI Conference on Artificial Intelligence. 2019;33:476-484.

    Not in our library10.1609/aaai.v33i01.3301476

21. 21.

    Bravo Sanchez FJ, Hossain MR, English NB, Moore ST. Bioacoustic classification of avian calls from raw sound waveforms with an open-source deep learning architecture. Scientific Reports. 2021;11(1):1-12.

    Not in our library10.1038/s41598-021-95076-6

22. 22.

    Brown A, Montgomery J, Garg S. Automatic construction of accurate bioacoustics workflows under time constraints using a surrogate model. Applied Soft Computing. 2021;107944(12):107944.

    Not in our library10.1016/j.asoc.2021.107944

23. 23.

    Brown CH, Riede T. Comparative bioacoustics: an overview. 2017.

    Not in our library

24. 24.

    Cakir E, Adavanne S, Parascandolo G, Drossos K, Virtanen T. Convolutional recurrent neural networks for bird audio detection. 2017;1744-1748.

    Not in our library

25. 25.

    Çakır E, Parascandolo G, Heittola T, Huttunen H, Virtanen T. Convolutional recurrent neural networks for polyphonic sound event detection. ArXiv preprint. 2017.

    Not in our library

26. 26.

    Canziani A, Paszke A, Culurciello E. An analysis of deep neural network models for practical applications. ArXiv preprint. 2016.

    Not in our library

27. 27.

    Chait M. How the brain discovers structure in sound sequences. Acoustical Science and Technology. 2020;41(1):48-53.

    Not in our library10.1250/ast.41.48

28. 28.

    Chen H, Xie W, Vedaldi A, Zisserman A. VGGSound: a large-scale audio-visual dataset. 2020a;721-725.

    Not in our library

29. 29.

    Chen RTQ, Amos B, Nickel M. Neural spatio-temporal point processes. ArXiv preprint. 2020b.

    Not in our library

30. 30.

    Chen X, Zhao J, Chen YH, Zhou W, Hughes AC. Automatic standardized processing and identification of tropical bat calls using deep learning approaches. Biological Conservation. 2020c;241(2):108269.

    Not in our library10.1016/j.biocon.2019.108269

31. 31.

    Chollet F. Xception: deep learning with depthwise separable convolutions. 2017;1251-1258.

    Not in our library

32. 32.

    Chorowski JK, Bahdanau D, Serdyuk D, Cho K, Bengio Y. Attention-based models for speech recognition. 2015;577-585.

    Not in our library

33. 33.

    Clementino T, Colonna J. Using triplet loss for bird species recognition on BirdCLEF 2020. 2020.

    Not in our library

34. 34.

    Çoban EB, Pir D, So R, Mandel MI. Transfer learning from Youtube soundtracks to tag arctic ecoacoustic recordings. 2020;726-730.

    Not in our library

35. 35.

    Coffey KR, Marx RG, Neumaier JF. DeepSqueak: a deep learning-based system for detection and analysis of ultrasonic vocalizations. Neuropsychopharmacology. 2019;44(5):859-868.

    Not in our library10.1038/s41386-018-0303-6

36. 36.

    Cohen-Hadria A, Cartwright M, McFee B, Bello JP. Voice anonymization in urban sound recordings. 2019;1-6.

    Not in our library

37. 37.

    Colonna J, Peet T, Ferreira CA, Jorge AM, Gomes EF, Gama J. Automatic classification of anuran sounds using convolutional neural networks. 2016;73-78.

    Not in our library

38. 38.

    Cramer J, Lostanlen V, Farnsworth A, Salamon J, Bello JP. Chirping up the right tree: incorporating biological taxonomies into deep bioacoustic classifiers. 2020;901-905.

    Not in our library

39. 39.

    Dekkers G, Rosas F, van Waterschoot T, Vanrumste B, Karsmakers P. Dynamic sensor activation and decision-level fusion in wireless acoustic sensor networks for classification of domestic activities. Information Fusion. 2022;77(5):196-210.

    Not in our library10.1016/j.inffus.2021.07.022

40. 40.

    Dias FF, Ponti MA, Minghim R. A classification and quantification approach to generate features in soundscape ecology using neural networks. Neural Computing and Applications. 2021;34:1923-1937.

    Not in our library10.1007/s00521-021-06501-w

Cited by papers we hold 1

• Analysing vocal complexity in relation to sociality in orcas of British Columbia: An application of long-term computational passive acoustics · 2025
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.