The Hippocampal Film Editor: Sensitivity and Specificity to Event Boundaries in Continuous Experience

Abstract

The function of the human hippocampus is normally investigated by experimental manipulation of discrete events. Less is known about what triggers hippocampal activity during more naturalistic, continuous experience. We hypothesized that the hippocampus would be sensitive to the occurrence of event boundaries, that is, moments in time identified by observers as a transition between events. To address this, we analyzed functional MRI data from two groups: one ( n = 253, 131 female) who viewed an 8.5 min film and another ( n = 15, 6 female) who viewed a 120 min film. We observed a strong hippocampal response at boundaries defined by independent observers, which was modulated by boundary salience (the number of observers that identified each boundary). In the longer film, there were sufficient boundaries to show that this modulation remained after covarying out a large number of perceptual factors. This hypothesis-driven approach was complemented by a data-driven approach, in which we identified hippocampal events as moments in time with the strongest hippocampal activity. The correspondence between these hippocampal events and event boundaries was highly significant, revealing that the hippocampal response is not only sensitive, but also specific to event boundaries. We conclude that event boundaries play a key role in shaping hippocampal activity during encoding of naturalistic events. SIGNIFICANCE STATEMENT Recent years have seen the field of human neuroscience research transitioning from experiments with simple stimuli to the study of more complex and naturalistic experience. Nonetheless, our understanding of the function of many brain regions, such as the hippocampus, is based primarily on the study of brief, discrete events. As a result, we know little of what triggers hippocampal activity in real-life settings when we are exposed to a continuous stream of information. When does the hippocampus “decide” to respond during the encoding of naturalistic experience? We reveal here that hippocampal activity measured by fMRI during film watching is both sensitive and specific to event boundaries, identifying a potential mechanism whereby event boundaries shape experience by modulation of hippocampal activity.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 81

1. 1.

   Allen TA, Salz DM, McKenzie S, Fortin NJ (2016 ) Nonspatial sequence coding in CA1 neurons. J Neurosci 36:1547–1563.

   Not in our library10.1523/jneurosci.2874-15.2016

2. 2.

   Axmacher N, Cohen MX, Fell J, Haupt S, Dümpelmann M, Elger CE, Schlaepfer TE, Lenartz D, Sturm V, Ranganath C (2010 ) Intracranial EEG correlates of expectancy and memory formation in the human hippocampus and nucleus accumbens. Neuron 65:541–549.

   Not in our library10.1016/j.neuron.2010.02.006

3. 3.

   Baayen RH, Davidson DJ, Bates DM (2008 ) Mixed-effects modeling with crossed random effects for subjects and items. J Mem Lang 59:390–412.

   Not in our library10.1016/j.jml.2007.12.005

4. 4.

   Baker LJ, Levin DT (2015 ) The role of relational triggers in event perception. Cognition 136:14–29.

   Not in our library10.1016/j.cognition.2014.11.030

5. 5.

   Baldassano C, Chen J, Zadbood A, Pillow JW, Hasson U, Norman KA (2017 ) Discovering event structure in continuous narrative perception and memory. Neuron 95:709–721.e5.

   Not in our library10.1016/j.neuron.2017.06.041

6. 6.

   Ben-Yakov A, Dudai Y (2011 ) Constructing realistic engrams: poststimulus activity of hippocampus and dorsal striatum predicts subsequent episodic memory. J Neurosci 31:9032–9042.

   Not in our library10.1523/jneurosci.0702-11.2011

7. 7.

   Ben-Yakov A, Eshel N, Dudai Y (2013 ) Hippocampal immediate poststimulus activity in the encoding of consecutive naturalistic episodes. J Exp Psychol Gen 142:1255–1263.

   Not in our library10.1037/a0033558

8. 8.

   Ben-Yakov A, Rubinson M, Dudai Y (2014 ) Shifting gears in hippocampus: temporal dissociation between familiarity and novelty signatures in a single event. J Neurosci 34:12973–12981.

   Not in our library10.1523/jneurosci.1892-14.2014

9. 9.

   Bird CM, Capponi C, King JA, Doeller CF, Burgess N (2010 ) Establishing the boundaries: the hippocampal contribution to imagining scenes. J Neurosci 30:11688–11695.

   Not in our library10.1523/jneurosci.0723-10.2010

10. 10.

    Chen J, Olsen RK, Preston AR, Glover GH, Wagner AD (2011 ) Associative retrieval processes in the human medial temporal lobe: hippocampal retrieval success and CA1 mismatch detection. Learn Mem 18:523–528.

    Not in our library10.1101/lm.2135211

11. 11.

    Clark HH (1973 ) The language-as-fixed-effect fallacy: a critique of language statistics in psychological research. J Verbal Learning Verbal Behav 12:335–359.

    Not in our library10.1016/s0022-5371(73)80014-3

12. 12.

    Cusack R, Vicente-Grabovetsky A, Mitchell DJ, Wild CJ, Auer T, Linke AC, Peelle JE (2014 ) Automatic analysis (aa): efficient neuroimaging workflows and parallel processing using matlab and XML. Front Neuroinform 8:90.

    Not in our library10.3389/fninf.2014.00090

13. 13.

    Desikan RS, Ségonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, Buckner RL, Dale AM, Maguire RP, Hyman BT, Albert MS, Killiany RJ (2006 ) An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31:968–980.

    Not in our library10.1016/j.neuroimage.2006.01.021

14. 14.

    Doeller CF, King JA, Burgess N (2008 ) Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory. Proc Natl Acad Sci U S A 105:5915–5920.

    Not in our library10.1073/pnas.0801489105

15. 15.

    DuBrow S, Davachi L (2013 ) The influence of context boundaries on memory for the sequential order of events. J Exp Psychol Gen 142:1277–1286.

    Not in our library10.1037/a0034024

16. 16.

    DuBrow S, Davachi L (2014 ) Temporal memory is shaped by encoding stability and intervening item reactivation. J Neurosci 34:13998–14005.

    Not in our library10.1523/jneurosci.2535-14.2014

17. 17.

    DuBrow S, Davachi L (2016 ) Temporal binding within and across events. Neurobiol Learn Mem 134:107–114.

    Not in our library10.1016/j.nlm.2016.07.011

18. 18.

    Dubrow S, Rouhani N, Niv Y, Norman KA (2017 ) Does mental context drift or shift? Curr Opin Behav Sci 17:141–146.

    Not in our library10.1016/j.cobeha.2017.08.003

19. 19.

    Eickhoff SB, Stephan KE, Mohlberg H, Grefkes C, Fink GR, Amunts K, Zilles K (2005 ) A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 25:1325–1335.

    Not in our library10.1016/j.neuroimage.2004.12.034

20. 20.

    Ezzyat Y, Davachi L (2011 ) What constitutes an episode in episodic memory? Psychol Sci 22:243–252.

    Not in our library10.1177/0956797610393742

21. 21.

    Güçlü U, van Gerven MA (2015 ) Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J Neurosci 35:10005–10014.

    Not in our library10.1523/jneurosci.5023-14.2015

22. 22.

    Gupta AS, van der Meer MA, Touretzky DS, Redish AD (2012 ) Segmentation of spatial experience by hippocampal theta sequences. Nat Neurosci 15:1032–1039.

    Not in our library10.1038/nn.3138

23. 23.
    A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie · 2014

    No notice
24. 24.

    Hanke M, Adelhöfer N, Kottke D, Iacovella V, Sengupta A, Kaule FR, Nigbur R, Waite AQ, Baumgartner F, Stadler J (2016 ) A studyforrest extension, simultaneous fMRI and eye gaze recordings during prolonged natural stimulation. Sci Data 3:160092.

    Not in our library10.1038/sdata.2016.92

25. 25.

    Hasson U, Landesman O, Knappmeyer B, Vallines I, Rubin N, Heeger DJ (2008 ) Neurocinematics: The neuroscience of film. Projections 2:1–26.

    Not in our library

26. 26.

    Hasson U, Malach R, Heeger DJ (2010 ) Reliability of cortical activity during natural stimulation. Trends Cogn Sci 14:40–48.

    Not in our library10.1016/j.tics.2009.10.011

27. 27.
    An annotation of cuts, depicted locations, and temporal progression in the motion picture "Forrest Gump" [version 1; referees: 2 approved] · 2016

    Not checked
28. 28.

    Heusser AC, Ezzyat Y, Shiff I, Davachi L (2018 ) Perceptual boundaries cause mnemonic trade-offs between local boundary processing and across-trial associative binding. J Exp Psychol Learn Mem Cogn 44:1075–1090.

    Not in our library10.1037/xlm0000503

29. 29.

    Hsieh LT, Gruber MJ, Jenkins LJ, Ranganath C (2014 ) Hippocampal activity patterns carry information about objects in temporal context. Neuron 81:1165–1178.

    Not in our library10.1016/j.neuron.2014.01.015

30. 30.

    Huff M, Meitz TG, Papenmeier F (2014 ) Changes in situation models modulate processes of event perception in audiovisual narratives. J Exp Psychol Learn Mem Cogn 40:1377–1388.

    Not in our library10.1037/a0036780

31. 31.

    Kim H (2011 ) Neural activity that predicts subsequent memory and forgetting: a meta-analysis of 74 fMRI studies. Neuroimage 54:2446–2461.

    Not in our library10.1016/j.neuroimage.2010.09.045

32. 32.

    Köhler S, Danckert S, Gati JS, Menon RS (2005 ) Novelty responses to relational and non-relational information in the hippocampus and the parahippocampal region: a comparison based on event-related fMRI. Hippocampus 15:763–774.

    Not in our library10.1002/hipo.20098

33. 33.

    Krizhevsky A, Sutskever I, Hinton GE (2012 ) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105.

    Not in our library

34. 34.

    Kumaran D, Maguire EA (2006 ) An unexpected sequence of events: mismatch detection in the human hippocampus. PLoS Biol 4:e424.

    Not in our library10.1371/journal.pbio.0040424

35. 35.

    Kumaran D, Maguire EA (2009 ) Novelty signals: a window into hippocampal information processing. Trends Cogn Sci 13:47–54.

    Not in our library10.1016/j.tics.2008.11.004

36. 36.

    Kurby CA, Zacks JM (2008 ) Segmentation in the perception and memory of events. Trends Cogn Sci 12:72–79.

    Not in our library10.1016/j.tics.2007.11.004

37. 37.

    Kuznetsova A, Brockhoff PB, Christensen RHB (2017 ) lmerTest package: tests in linear mixed effects models. J Stat Softw 82:1–26.

    Not in our library

38. 38.

    Lee SA, Miller JF, Watrous AJ, Sperling MR, Sharan A, Worrell GA, Berry BM, Aronson JP, Davis KA, Gross RE, Lega B, Sheth S, Das SR, Stein JM, Gorniak R, Rizzuto DS, Jacobs J (2018 ) Electrophysiological signatures of spatial boundaries in the human subiculum. J Neurosci 38:3265–3272.

    Not in our library10.1523/jneurosci.3216-17.2018

39. 39.

    Luke SG (2017 ) Evaluating significance in linear mixed-effects models in R. Behav Res Methods 49:1494–1502.

    Not in our library10.3758/s13428-016-0809-y

40. 40.

    Magliano JP, Zacks JM (2011 ) The impact of continuity editing in narrative film on event segmentation. Cogn Sci 35:1489–1517.

    Not in our library10.1111/j.1551-6709.2011.01202.x

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.