The brain and its time: intrinsic neural timescales are key for input processing

1Our environment bombards the brain with a variety of regular and irregular inputs on different timescales. Consider one of the temporally most complex inputs, music. We can simultaneously perceive the music’s different timescales and, even better, are able to integrate them into one meaningful whole like a melody. Moreover, the melody can be distinguished from the ongoing accompaniment in the background.

2How can our brain process and integrate such multiscale inputs? Recent evidence suggests that the brain itself exhibits intrinsic neural timescales (INT)–. As measured by the autocorrelation window (ACW) in resting state, lower-order unimodal sensory regions—the primary visual cortex, for example—shows short timescales compared to higher-order transmodal regions like the default-mode network (DMN)–,–. However, the specific function or role of the INT in the brain and its neural processing still remains unclear.

3Reviewing various findings on INT in both human and non-human species, we propose that they play a key role in processing and structuring inputs in different timescales (Fig. 1 for a general framework). Rather than focusing on specific kinds of inputs (like visual, somatosensory, or auditory; see Box 1), we here aim to describe basic dynamic principles of the temporal nature of input processing that are shared across different inputs. Specifically, the brain utilizes its own INT to process and actively shape the extrinsic timescales of the multiscale inputs it receives from both environment and body. This allows the brain to encode the stochastic structure of its environmental inputs according to its own stochastic structure. Its own stochastic structure is determined by its INT and its unimodal–transmodal hierarchy. That, as we will detail, is mediated by specific computational mechanisms like temporal integration/segregation as well as input sampling with consecutive shifting towards slower frequency modes within the processing hierarchy.

4We first review empirical evidence on INT during both resting and task states. That is followed by a second part where we link INT to distinct facets of input processing, like encoding of stochastics and their sharing by different species. In the third part, borrowing from Physics and Mathematics, we explore the computational mechanisms driving input processing by the INT; this includes temporal integration and segregation of the input, as well as a sampling mechanism that shifts subsequent stages of input processing towards slower frequency modes. We conclude that the role or function of INT consists in processing and structuring multiscale inputs from the environment which, to some degree, is evolutionarily shared across human and non-human species. Given its key role in the brain’s input processing, INT carries major implications for psychiatric disorders (Box 2) and artificial intelligence (Box 3).

5The notion of input is a concept that can include different meanings. One first and foremost associates sensory functions with the notion of input. For instance, one can distinguish somatosensory, visual, and auditory inputs. Recent studies, demonstrate that these sensory input systems are related, with different input streams in the brain in terms of both their functional connectivity and INT. Most interestingly, these input streams share continuous progression from unimodal (like primary sensory cortex) to more transmodal regions like the dorsolateral prefrontal cortex. The transmodal regions are, in part, shared by the different sensory input systems.

6Hence, despite their distinctions in terms of their modality-specific sensory input, these input systems may nevertheless converge in the higher-order transmodal regions. Such convergence of the sensory input streams with the unimodal–transmodal hierarchy of INT suggests that some dynamic, i.e., temporal features and mechanisms of input processing, as the ones discussed above, may be shared among the different sensory modalities.

7In addition to sensory inputs from the external environment, the brain also receives inputs from the body, that is, its interoceptive inputs (and also its proprioceptive inputs). Taylor et al. demonstrate that the brain’s interoceptive input streams (with insula as the key region) again follow the hierarchical progression from unimodal to transmodal regions. Remarkably, even the interoceptive input system converged with the exteroceptive sensory input streams in the higher transmodal regions. That suggests some commonality among intero- exteroceptive input processing.

8Complementing the continuous “bombardment” by the environment’s exteroceptive inputs and the body’s interoceptive inputs, the brain itself shows activity changes in its spontaneous activity. These changes may be weak or, alternatively, stronger, inducing the degree of activity changes usually elicited by exteroceptive stimuli. This may, for instance, apply to spontaneous activity changes in the auditory cortex which then may be perceived as external voices while being hallucinatory in their nature,,. These spontaneous activity changes may even exist in the “healthy” brain (as healthy persons can show auditory hallucinations) and may therefore be designated as “neuronal input”. These spontaneous changes may be predominantly associated with internally oriented cognition like mind-wandering,, self-referential processing,, and mental time travel/episodic simulation,. Importantly, these spontaneous activity changes, the neuronal inputs, occur in all regions of the brain and, interestingly, are significantly stronger in the higher-order transmodal regions when compared to the unimodal regions,–. Hence, we can observe again that even the brain’s own neuronal input seems to follow the progression from the unimodal to the transmodal hierarchy.

9Together, these observations suggest that beyond the distinction between specific sensory inputs, as well as the different sources of inputs (environment, body, brain), there is some deeper level to the notion of input. Unlike the more superficial level with the distinction of different inputs and sources, there seems to be a deeper level that is shared among the different inputs. This deeper level of the input seems to be strongly shaped by the brain’s INT in strongly temporal and thus dynamic way. Our description of the different features and mechanisms of input processing in this paper targets this deeper temporal, highly dynamic level of input processing.

10Are INT relevant for behavior and cognition? Evidence for that comes from their changes in psychiatric disorders. A recent resting state fMRI study applied the ACW in subjects with autism. They observed significantly shorter ACW in primary sensory regions (visual, sensorimotor, auditory) in adult autism spectrum disorder (ASD) compared to healthy subjects; these changes correlated negatively with the severity of autism. In contrast, ACW in right caudate was significantly longer in ASD which also correlated with the degree of repetitive restrictive behavior in ASD subjects.

11They then investigated fMRI resting state ACW in an adolescent children ASD dataset where they obtained similar ACW changes in the same regions including analogous correlation results. This means that there is a developmental component to the intrinsic timescales. That was complemented by investigating the neuro-anatomical basis through calculating the local gray matter volume. Significant positive correlation of local gray matter volume with the duration of ACW in the same region was observed which also hold for the regions altered in ASD. Finally, they calculated mediation analysis showing that the gray matter volume in the above-mentioned regions was mediated in their impact on autistic symptoms by the duration of the ACW.

12The relevance of intrinsic timescales in autism is further supported by another fMRI resting state study in ASD. Operating in the frequency domain (rather than time domain), they calculated the power-law exponent (PLE) (and spectral entropy). They observed that ASD showed increased PLE with stronger power in slow frequencies in specifically regions of the salience network (insula, supragenual anterior cingulate cortex, and thalamus) (see also ref. showing that the salience network exhibits the highest variability and flexibility among the different networks). Moreover, they demonstrated that such increased PLE in salience network was not observed in schizophrenia. Finally, these findings were specific for PLE as distinguished from others like regional homogeneity and neural variability that did not show any changes in these regions in ASD,,.

13Using EEG, one recent study in schizophrenia including mostly post-acute first-episode subjects showed abnormally long ACW (and high PLE) in several electrodes during a task state involving self-specificity (i.e., enfacement task). They also demonstrated that the degree of change in ACW from rest to task was significantly lower in schizophrenia subjects, that is, unlike in healthy subjects, they barely shortened their ACW during the task. Most remarkably, analogous ACW prolongation during task and its reduced rest–task difference was not observed during a non-self-task, i.e., auditory oddball—this suggests a close relationship of ACW with self-specificity.

14Moreover, applying a moderation model, they showed that the degree of ACW mediated the relationship between self-disturbance and negative symptoms in schizophrenia participants. As in the autism study by Watanabe and colleagues, these data support the assumption that changes in INT may be related to psychopathological symptoms and, more generally, be relevant for behavior and cognition. This is further supported in a recent study on psychosis in schizophrenia.

15Using fMRI, they show globally reduced (i.e., shortened) INT in psychosis/schizophrenia compared to healthy controls. Moreover, they observed specific changes within the neural hierarchy of auditory and somatosensory input streams: the increase in INT at lower levels of the neural hierarchies may reflect hallucinations (comparing psychotic subjects with severe vs. mild hallucinations). While the increased timescales at higher levels of neural hierarchies may reflect delusions (comparing psychotic subjects with severe vs. mild delusions). Together, the findings by, support the importance of INT in mediating specific psychopathological symptoms of psychosis.

16We saw that the environmental hierarchies of different events may be recapitulated and thus modeled by the brain itself within its own intrinsic hierarchical organization, i.e., its temporo-spatial hierarchy. There is no need for the living to “represent” a model of the environment in their head: “An agent does not have a model of its world—it is a model. In other words, the form, structure, and states of our embodied brains do not contain a model of the sensorium—they are that model”.. Based on the temporal hierarchy of its INT, the human agent (and related non-human species) is a temporo-spatial model of its environment displaying a more or less analogous temporal structure albeit in a miniature scale-free way.

17The brain can be conceived as a free energy-driven temporo-spatial model of the temporo-spatial hierarchies of its environmental context. That results in temporal and spatial nestedness of the brain within its respective environmental context. Despite different temporal (and spatial) scales across body, brain, and environments, they are nevertheless connected through scale-free self-similarity in their shape or form. Just like the smaller Russian doll is contained within the next larger one (same shape, different size), the brain and its temporo-spatial organization nest in a scale-free self-similar way within the much larger environment. Given such temporo-spatial self-similarity between brain and environment, we may better focus on “what our head’s inside of” rather than searching for “what inside our heads”.

18This carries major implications for modeling artificial agents as it entails the need to include an intrinsic temporal and spatial hierarchy within the inner design and architecture of the agent itself. Future AI models may want to implement such intrinsic spatial and temporal organization in their artificial agents, including the different timescales and the core–periphery organization (see ref. for first steps in this direction in artificial agents using what they describe as “multiple time scale recurrent neural network”, refs., who emphasize the need for temporal hierarchies in artificial agents for their adaptation to the environment).

19Spatiotemporal hierarchies would extend the current—often module-based—models of artificial agents by combining top-down (providing the agent’s inner input) and bottom-up (providing the agent’s outer input) layers—that Tani uses in his compelling model of an artificial agent,,. Most importantly, by conjoining the temporo-spatial architecture with the free energy principle, the artificial agent’s intrinsic temporo-spatial hierarchy may be a small-scale but self-similar model of its own environmental context. To achieve that, the agent’s temporo-spatial hierarchy needs to be highly dynamic and continuously changing, so as to adapt to the changing environmental dynamics. More specifically, this means that the causal (or temporo-spatial) architecture of the environment must be recapitulated or installed in the agent’s temporo-spatial dynamics in such a way as to allow the agent to minimize its variational free energy with its respective environmental context.

20INT is commonly investigated in cellular, and systemic,–,,, granularity levels. On a systemic level, Hasson and colleagues,,, operationalize INT using functional connectivity during task state. They define temporal receptive windows as “the length of time before a response during which sensory information may affect that response” and it roughly correspond to what are described as temporal receptive fields on the cellular level.

21Recent studies operationalize the INT using the autocorrelation function of the signal during both resting and task state. Autocorrelation function is the correlation of a signal with shifted (time-lagged) versions of itself. Since autocorrelation function yields a series of numbers, different studies report slightly different properties of it. Murray and colleagues obtain INT by fitting an exponential curve to the autocorrelation; however, ACW is most commonly reported in the INT’s literature.

22In their fMRI studies, Watanabe and colleagues define ACW as the area under the curve of the autocorrelation function from zero to the time lag that the correlation reaches zero (see also refs.,). However, EEG/MEG studies define ACW as the time lag the correlation reaches half of its maximum value (Fig. 2a) or when it reaches 1⁄e (Golesorkhi et al. defines a new variant called ACW-0).

23Murray and colleagues investigated single-cell recording data in non-human primates and calculated their autocorrelation function in pre-stimulus intervals. From that, they measured the duration of the temporal window at a correlation decay of 50%, i.e., ACW. They observed a shorter ACW in lower-order unimodal sensory regions while higher-order transmodal regions, such as the prefrontal cortex, exhibited a longer ACW. Subsequent computational modeling studies employed large-scale non-human primate-based, human-based structural connectivity networks,, or a standard model of synchronization, i.e., Kuramoto model (0.01–0.1 Hz). They also demonstrated longer INT (as measured by the ACW) in prefrontal regions which remained shorter in sensory and motor cortex (see also ref., see also other models of neural dynamics in refs.,).

24The computational findings are supported by observations of a corresponding hierarchy of timescales in real human data using fMRI,. Operating in the infraslow frequency range (0.01–0.1 Hz), resting state fMRI studies applied the autocorrelation function to the BOLD signal and, following Murray and colleagues, determined the ACW at 50% of correlation decay,,. Employing small, or large-scale,, fMRI datasets, all studies observed shorter ACW in unimodal regions, including sensory and motor regions/networks on the cortical level. In contrast, transmodal regions, including higher-order networks such as the central-executive networks (CEN), dorsal attention networks (DAN), and default-mode network (DMN), generally show a longer ACW.

25In addition to a temporal hierarchy on the cortical level, Raut and colleagues also measured the ACW in subcortical regions like the thalamus, cerebellum, striatum, and hippocampus. Interestingly, they again observed that gradients of the ACW within each of these subcortical regions, especially in the thalamus and striatum, appear to mirror the temporal hierarchy on the cortical level. Together, these data strongly suggest that both cortical and subcortical regions display an intrinsic hierarchical organization with unimodal sensory and motor regions showing shorter timescales while transmodal higher-order association regions exhibit longer timescales.

26These findings were all obtained in human fMRI that measures BOLD activity in the infraslow frequency range (0.01–0.1 Hz). That raises the question as to whether the distinction of shorter unimodal and longer transmodal INTs are also present in faster frequency ranges between 1 and 70 Hz as can be typically measured with EEG/MEG. Indeed, two recent human resting state MEG studies, demonstrate a longer ACW in higher-order transmodal regions/networks like the CEN and DMN, whereas it was significantly shorter in unimodal sensory regions. Hence, these findings suggest that INT follows a similar topographical distribution in faster frequencies (1–70 Hz) as those in slower frequency ranges (0.01–0.1 Hz). Such ubiquitous occurrence suggests a most basic or fundamental, though unclear, role or function of INT in the brain.

27How are the intra-regional INTs related to inter-regional connections? The INT is constituted by both intra-regional cellular features,,, and inter-regional connectivity,,,. Intra-regional cellular features concern the excitation–inhibition balance with its local recurrent wiring as in supragranular feedforward and infra-granular feedback connections,,, (see also ref. for demonstrating the relevance of population codes). Cavanagh et al. demonstrate that even within regions there is considerable variability of the INT at the single neuron level. Specifically, the temporal receptive field of a single neuron can change over time and adapt to, for instance, task demands as during working memory (see also ref. ) and/or decision making–. Moreover, Spitmaan et al. observe less dependence upon the task context—this further underlines their adaptative nature. Moreover, as the authors put it, the timescales of different neurons during task-related activity suggest a certain independence, i.e., flexibility.

28In addition to the intra-regional cellular features, INT is also strongly shaped by inter-regional connectivity. Chaudhuri and colleagues demonstrated that purely local connectivity itself is insufficient to yield the diversity of timescales across the cortex. Moreover, in their non-human primate-based computational model, they remove all long-range projections which significantly restricts the range of different timescales and abolishes the intrinsic temporal hierarchy. The relationship of intra-regional INT and inter-regional functional connectivity holds again across different species as it can be observed in both non-human primates and humans,,,–.

29How exactly is inter-regional functional connectivity related to the INT? Two recent studies in human fMRI show that the duration of INT in different regions, as measured by the resting state ACW, is positively correlated with the degree of said region’s change in functional connectivity during task: the longer the region’s resting state ACW, the stronger its task-related change in its functional connectivity to other regions,. That is further supported by Raut and colleagues who demonstrated that the individual variability in ACW across different regions is directly related to the individual variation of the functional connectivity pattern of the same regions (see also,–).

30Together, these findings suggest a close relationship of INT to the brain’s inter-regional connectivity pattern—intra-regional temporal features are, in part, constituted by long-range inter-regional connections. Such an intimate link between intra-regional timescales and inter-regional connectivity means that the different timescales can interact and integrate with each other. This may enlarge the number of available timescales, i.e., the repertoire of timescales, as we will illustrate later.

31Is the resting state’s INT related to task states? A positive answer to this question would support their involvement in input processing. The relevance of INT for input processing is strongly suggested by the excellent studies of Hasson and colleagues,,,– (ref. for review). They demonstrate that shorter temporal segments of external stimuli (like single words of stories, or short episodes in movies) are processed preferentially in lower-order unimodal sensory regions. Longer intervals (like whole paragraphs in stories, or longer episodes in movies) are related to activity changes in higher-order transmodal regions. Given that external inputs are processed and structured in temporal terms, i.e., according to different durations, Hasson and colleagues speak of temporal receptive windows which roughly correspond to what are described as temporal receptive fields on the cellular level.

32Does the spatial, or topographical, pattern of the INT overlap between rest and task states, i.e., rest–task overlap? While such rest–task overlap has been well demonstrated for functional connectivity,–, it remains an open issue in the case of INT. The various task studies on the brain’s temporal receptive windows show a spatial pattern that is well in accordance with the hierarchical organization of INT in the resting state. In the same way, the ACW is longest in the DMN during rest. Task states also show that the DMN processes the longest sequence of inputs and information,, while the shorter resting state ACW in unimodal sensory regions seems to find its equivalent in the short sequences of inputs processed in these regions. Hence, comparison of rest ACW and task temporal receptive windows shows analogous hierarchical topographical organization. This suggests a close relationship between rest and task, i.e., rest–task modulation or interaction– (see below for the discussion of task-specific changes in INT).

33If there is such a rest–task overlap, one would assume that the hierarchical organization of resting state ACW is carried over to, and thus present in, the temporal receptive windows during task states. Evidence for such rest–task overlap comes from both computational modeling and brain imaging. Gollo and colleagues conducted a modeling study based on the synchronization model of Kuramoto with simulations of transcranial magnetic stimulation: they show that regions with longer ACW, as located in the transmodal core, display lower and more sluggish activity changes in response to external stimuli than sensory regions; the latter exhibit a shorter ACW at the more unimodal periphery, accompanied by higher amplitude and a faster response to external stimuli (see also refs.,). Analogous results were observed in the modeling study by Chaudhuri and colleagues who applied electrical stimulation to V1 in the visual cortex to his non-human primate-based network model (see also ref. ). One interesting finding here is that regions weakly connected to the input regions exhibit longer INT during stimulation. This again demonstrates that tasks exert effects beyond those at the stimulated regions themselves. These computational data on rest–task overlap of INT are supported by human brain imaging data. A recent human fMRI study by Ito and colleagues investigated the ACW in resting state and the amplitude during different task states. They demonstrated a negative correlation of resting state ACW duration (in different regions) with the magnitude of task-related activity, i.e., amplitude, in the same regions. Therefore, the longer the region’s resting state ACW, i.e., transmodal regions, the lower its task-related amplitude. While regions with shorter ACW, i.e., unimodal regions, exhibit higher amplitude during different tasks. These results support the idea that the resting state’s INT strongly shapes task-related activity and associated input processing,,. The mechanisms of this, however, remain unclear.

34The rest–task overlap strongly suggests that the resting state’s INT may also shape or modulate the temporal features of task states including associated cognition—this amounts to what we describe as rest–task interaction or modulation (see also refs.,,). This has recently been addressed by Golesorkhi and colleagues (see also ref. for initial steps). Applying MEG, they investigated the ACW-50 and ACW-0 (see above) not only during rest but also during three different task states (motor, story-math, working memory). They showed that the resting state’s ACW and its hierarchical core–periphery organization strongly predict their task states: the resting state’s core–periphery organization of ACW was essentially preserved during all three task states as topographical rest–task correlation yielded high values (0.8–0.9). These results suggest that the resting state’s hierarchical organization of its INT is essentially carried over, and preserved, during task states, irrespective of the task.

35Golesorkhi and colleagues also observed some task-specific changes (Fig. 2) when calculating the rest–task difference (which subtracts and cancels out the shared, i.e., correlating temporal hierarchical organization). Specifically, higher-order network regions showed a strong ACW, shortening during the story-math task (which was presented in 30 s intervals). Only minimal changes were seen in motor and working memory tasks. A reverse pattern was observed in lower-order network regions; the ACW was shortened considerably during working memory but minimally in story-math and motor tasks. These data suggest that, once one subtracts the hierarchical temporal organization present in both rest and task, task-specific changes can be observed. Furthermore, the ACW itself and, more generally, the INT, can be modulated during task states—they are dynamic and adaptive rather than static and non-adaptive. Though more studies are needed, task-related modulation seems to mainly concern the shortening of the ACW relative to rest. The adaptative nature of INT is also documented by them during the delay period of a working memory task (relative to pre-stimulus baseline) in human ECoG.

36In addition to task states, INT also shape behavior and cognition. Studies in non-human primates demonstrated that a longer duration of the resting state’s INT (as obtained during baseline intervals sandwiched between tasks) is associated with better behavioral performance in a variety of different tasks. These include a longer duration of delays in a delay discounting task, stronger spatial response coding in the delay period during a non-match-to-goal task, and modulating working memory performance during later periods, i.e., delay. On the human side, recent fMRI and/or EEG studies demonstrate that the resting state’s ACW is directly related to higher-order cognition like the level of consciousness,, the sleep stage, the sense of self–, and psychiatric disorders (see Box 2 for details). Tentatively, these data show that INT strongly shapes behavior, including perception and higher-order cognition like consciousness and self. Since task states, as well as perception and cognition, are dependent upon various kinds of inputs, together these data are compatible with the assumption that INT is key for input processing and structuring.

37Key findings of the INT are: (I) their topographical organization during both resting and task states along uni- and transmodal regions/networks; (II) their topographical carry-over and partial change during the transition from rest to task; and (III) their relation to the temporal structure of external inputs during task states,,,,,. Together, these findings suggest their involvement specifically in the brain’s input processing.

38We presuppose here a wide notion of input including stimuli from both one’s own body, i.e., interoceptive, and external environment, i.e., exteroceptive (Box 1). Our focus is primarily on the dynamic principles and mechanisms underlying input processing in general rather than describing the specifics of various inputs like auditory, visual, or somatosensory (see Box 1). Considering INT, in the following we first address the importance of input processing as distinguished from output processing and then discuss two important facets: (I) cross-species input sharing (II) and stochastic matching of the extrinsic environmental inputs with the brain’s intrinsic stochastic structure.

39Is INT engaged in either input or output processing? This was recently addressed by Zilio and colleagues, who investigated the ACW in resting state EEG in subjects with physiologic, pharmacologic, and pathological alterations of consciousness. Under such conditions, input processing is known to be altered in distinct ways, i.e., progressive decrease (NREM sleep stages N1, N2, N3), isolation from external inputs but preserved capacity for processing of internal inputs (from the own body and brain) (REM sleep and ketamine), and extreme deficiency or complete absence of both external and internal inputs (unresponsive wakefulness syndrome, sevoflurane). Additionally, they included subjects with complete loss of motor function, e.g., output processing, whereas input processing and consciousness are preserved (locked-in syndrome and amyotrophic lateral sclerosis).

40The results (Fig. 3) show abnormally long ACWs in the unresponsive wakefulness syndrome, through abnormal strengthening of slow frequency power (and concurrent weakening of fast frequency power). Also, both the physiologic and pharmacologic alterations of consciousness showed abnormal prolongation of the ACW in line with the progressive decrement of the capacity of input processing in the different behavioral states. The motor conditions, in contrast, exhibited a “normal” ACW with a preserved balance of slow and fast frequencies in the power spectrum. Together, these findings support the involvement of INT specifically in input processing. In contrast, INT does not appear to be significantly associated with output processing in subjects with motor deficits but preserved input processing. If the ACW was significantly involved in both input and output processing, it should have globally changed in both types of conditions, altered states of consciousness and altered motor conditions (although the ACW is significantly shorter in the parieto-occipital regions of amyotrophic lateral sclerosis patients than in healthy subjects). We need to be cautious, however. One can neither fully exclude output disturbances in the altered states of consciousness nor changes in input processing in the motor conditions (locked-in syndrome, amyotrophic lateral sclerosis). Hence, more direct support for the role of the ACW in input processing is required.

41Of note, however, is that Zilio and colleagues investigated only resting state activity. Therefore, the ACW can only be indirectly related to input processing; an investigation of task states with actual inputs are needed to provide a direct relation to input processing. Given that the disorders and the alterations of consciousness are known to exhibit deficient input processing–, their findings suggest that the resting state’s INT exerts the capacity for input processing, i.e., a neural predisposition–. Even when not exposed to actual multiscale inputs from the external environment, the resting state still exhibits its own INT, which index its capacity for processing the former. This is, for instance, the case in sleep where we can still be awoken at any time by sufficiently strong external inputs—the brain’s capacity or predisposition of input processing is preserved. In contrast, this remains impossible in total anesthesia and coma where even the strongest external inputs will not wake the individual—the brain’s capacity or predisposition of input processing is lost.

42We so far demonstrated the significance of INT in processing inputs from the external environment. Assuming that different species somewhat share one and the same external environment, then one would suppose that they should, to some degree, overlap or share, at least in part, their INT. There is indeed evidence for such “input sharing” across species as it is manifested in the cross-species evolutionary preservation of INT.

43The data suggests that the regional differentiation of the INT along the transmodal–unimodal gradient holds in both non-human primates,, and humans,,,. This can be extended to other species as it is supported by cross-species studies on both the cellular, and more regional-systemic levels. Shinomoto and colleagues, show, in a first step, regional differentiation in the cellular firing pattern of different regions in the non-human primate brain: spiking patterns are regular in motor areas, random in visual areas, and burst-like in the prefrontal cortex. In a second step, they demonstrate that such temporal fingerprinting in the regions’ temporal structure of their firing pattern holds across different species including non-human primates, cats, rats, and mice; the differences in firing patterns between different regions within one species are larger than the firing pattern differences within the same region across different species,. Together, these data demonstrate that temporal features of neural firing patterns on the cellular level of specific regions are preserved across different species.

44Analogous observations of cross-species evolutionary preservation have been made on the more regional-systemic level of oscillations. Buzsáki and colleagues demonstrate that various oscillatory rhythms such as alpha, spindles, and ripples are present in more or less the same frequency range in different species including humans, non-human primates, dogs, bat, gerbil, guinea pig, rabbit, mouse, and hamster (see also ref. ). Importantly, Buzsáki and colleagues observe that such preservation of the same frequencies across different species holds independent of brain size: even if the brain size changes and becomes larger throughout evolution, the frequency range of the rhythmic pattern remains the same in different mammals. They conclude that temporal organization of the brain is a key priority in evolution: “In summary, the preservation of temporal constants that govern brain operations across several orders of magnitude of timescales suggests that the brain’s architectural aspects—scaling of the ratios of neuron types, modular growth, system size, inter-system connectivity, synaptic path lengths, and axon caliber—are subordinated to a temporal organizational priority”. (ref., p.755).

45Is the human brain’s INT an evolutionarily preserved manifestation of our ancestors’ timescales, including their key role in processing external inputs from the environment? The findings by both Shinomoto and colleagues, and Buzsáki and colleagues suggest exactly that. One would consequently expect that human behavior, if based upon its evolutionarily preserved INT, should resemble the behavior of non-human species. For instance, Zhang and Ghazanfar propose that the timescales of human infant vocal production can be seen in line with the multiple INTs of vocal production in marmoset monkeys, songbirds, and other vertebrates. Together, these findings suggest that INT is, in part, preserved evolutionarily across different species which may be manifested in somewhat similar forms of behavior (Fig. 4a).

46How is it possible that different species share their INT? The reason for such cross-species similarity cannot be found in the brain itself as cross-species temporal similarities occur across different brain sizes. Picking up the suggestion by Shinomoto and colleagues,, cross-species similarity may rather be related to similarity in function and, as we specify, in the nature of the inputs. Different species share more or less the same environmental context, i.e., ecological niche and affordances–. They consequently are exposed to the same input that share similar timescales. Specifically, even if the inputs themselves are elaborated in somewhat distinct ways by the species-specific sensory organs, the input stochastics, i.e., the relation between inputs, may nevertheless be processed in similar ways across the different species.

47Is the brain’s INT related to the input stochastics in the environment? Stephens and colleagues demonstrate that regional differences in the brain’s INT, as indexed in their study by the power spectrum, are related to the temporal structure of the external information, with the former aligning to the latter: (I) regions with shorter INT and faster dynamics, i.e., early auditory cortex, are activated during shorter stimulus segments (e.g., single phonemes or words) (see also refs. – for more support in terms of entrainment); (II) regions with intermediate timescales and balanced slow–fast dynamics, i.e., superior temporal gyrus and inferior frontal gyrus, are recruited by intermediate durations in the temporal structure of stimuli (e.g., the structure of sentences); and (III) regions with longer intrinsic timescales and slower dynamics, i.e., precuneus and medial prefrontal cortex, are activated by slowly varying stimulus dynamics (e.g., stimulus narrative, see also refs.,,,,).

48These findings raise yet another question, though. How can the limited number of INT of the brain’s various regions process and sample a seemingly unlimited and constantly changing number of extrinsic neural timescales of the environment? Given that there is a positive relation of intra-regional INT and inter-regional functional connectivity,, (see above in Part I), we propose direct interaction between the different regions’ INT—such interaction would enlarge the number of possible timescales, i.e., the repertoire of timescales, as we say (Fig. 4b). We can take the structure of DNA as an analogy. This complex structure is created from only four bases of adenine, thymine, guanine and cytosine. If a species has a high number of regions with different INT, their degree of possible interaction through inter-regional functional connectivity is much higher than in a species with only a low number of regions exhibiting distinct INT and/or low inter-regional functional connectivity.

49A large repertoire of timescales may extend the organism’s ability to encode and sample the input stochastics of their respective environment in a more fine-grained and temporally differentiated way, that is, according to distinct timescales in the environment. That, in turn, may reduce the error in the brain’s encoding of the input stochastics relative to the latter’s stochastic occurrence in the natural world. Accordingly, we tentatively suppose that species with a higher number and thus large repertoire of INT are prone to lower degrees of error in their input processing—they can better align to their environmental context in a more fine-grained way than species with a low number of intrinsic timescales and/or a small repertoire (Fig. 4c).

50What are the mechanisms by which the INT processes input? The various task state studies conducted by Hasson and colleagues with the formulation of the temporal receptive windows suggest that the INT may structure the inputs into segments of different durations, e.g., short and long segments like single words, sentence, and paragraphs,,,,. Such temporal structuring may mean that certain inputs are processed together with high degrees of temporal integration, amounting to some form of “temporal smoothing”,. Other inputs may be processed in a more segregated and, therefore, temporally precise way entailing higher degrees of temporal segregation (see refs.,). Together, this amounts to a balance of temporal integration vs. segregation in input processing.

51How can INT modulate their balance of temporal integration vs. segregation during input processing? The ACW measures the degree of correlation of neural activity patterns across different time points. If only a low number of distinct time points correlate with each other, the correlation is low, indexing a short ACW. This means that inputs at more time points beyond those that correlate in ACW are sampled independent of each other—they will be processed with high degrees of temporal segregation but low temporal integration. Moreover, the processing of single inputs may then be more or less restricted to their actual durations as, due to low correlation with a low ACW, they are not expanded (in a virtual way) beyond their actual physical durations, i.e., temporal smoothing or expansion. Accordingly, short INT predisposes that inputs are processed with high temporal precision in both their specific time points and actual durations with low degrees of “temporal smoothing”. Such a pattern of input processing is strongly supported by the short duration of the intrinsic timescales in unimodal regions like the sensory cortex that display a short ACW in rest and a short temporal receptive window during task states–,,, (Fig. 5).

52The reverse pattern of input processing appears to hold in transmodal regions; their rest ACW and task temporal receptive windows are much longer than those in unimodal regions–,,,. A longer ACW indicates a higher correlation of neural activity across temporally more distant time points. Inputs at different time points are then not sampled independently of each other, but somewhat linked together across time resulting in “temporal summing and pooling” (see ref. who speak of temporal pooling and summing) and ultimately high degrees of temporal integration. Moreover, the duration of the single inputs’ neural processing may virtually, i.e., neuronally, expand beyond their actual physical duration: even if the stimulus is already physically absent (but still present in the neural activity), distant time points’ neural activities may still correlate highly with the preceding time points of the actual stimulus—this amounts to high temporal smoothing/expansion and low temporal precision. For instance, higher-order transmodal regions like the prefrontal cortex support temporal integration and expansion of sensory,–, motor, and cognitive information,,,. This is compatible with the idea that transmodal regions such as the prefrontal cortex is involved in higher-order cognition like memory, imagination, abstraction, self, and consciousness.

53So far, we have demonstrated how the INT processes the input in particular timescales by modulating them through temporal integration and segregation. However, the brain is confronted with a variety of different inputs in various or multiple scales, with the number of timescales in the environment far exceeding the available timescales of the brain,,. How can the brain bridge the gap between its own restricted timescales and the more expanded ones of its environmental context? Ideally, the brain encodes all inputs from the larger-scale environment within its own smaller-scale neural activity without losing any information, i.e., minimal error.

54The empirical data show hierarchical organization of the timescales within the brain according to a fast–slow gradient from uni- to transmodal regions. The inputs may be sampled in a more or less analogous way when transitioning from the faster unimodal to the slower transmodal regions; this implies what in mathematics is described as down-sampling from faster input stochastics to slower ones. We suggest that the INT acts as input samplers, that is, down-sampling across the hierarchy of unimodal and transmodal regions. Here, we first perform numerical simulations to provide some support for the differential response of unimodal and transmodal regions during input processing (this section). This will be complemented in a second step (next section) by illustrating the mathematical principles of the fast–slow gradient of down-sampling again showing some simulation data.

55Under our fast–slow gradient assumption, sensory networks would be the first to carry out this down-sampling process. Unimodal and sensory networks show shorter intrinsic timescales compared to transmodal ones,. This implies that the first sampling would be done at a higher frequency and, progressively, said sampling would be on more widely spaced timescales, i.e., down-sampling. In that case, one would expect a faster and more transient, i.e., fast-frequency response in unimodal regions as related to their shorter INT. Transmodal regions, in contrast, should show a slower, delayed and longer-lasting response.

56In a first step, we probed this in a computational network model (details are provided in the Supplementary material), applying inputs of short duration to lower-order regions, i.e., visual cortex V1, and tracking the response in both lower- and higher-order regions, i.e., anterior cingulate cortex 24c. Indeed, we observe immediate and short-lasting responses in V1, whereas 24c displays more delayed, i.e., sluggish and longer-lasting responses (Fig. 6a). This is compatible with a fast–slow gradient of consecutive input down-sampling throughout the lower-higher hierarchical processing stages.

57In a second step, we varied the duration of the input to V1. Inputs of shorter duration should yield a faster response saturation in lower-order regions like V1 while inputs of longer duration would be required to yield the same effect in the higher-order regions like 24c. This again was confirmed in our simulation data (Fig. 6b), providing indirect support for input down-sampling along the lines of a fast–slow gradient. In sum, there seems to be a close link of fast–slow input sampling during the encoding of the input stochastics, with the latter being sampled in a seemingly temporal way by the brain’s INT.

58Together, these simulation results are compatible with a recent study by Wengler and colleagues. They investigate fMRI-based ACW in subsequent regions of three sensory-based input streams, sensorimotor, visual, and auditory. In their data, the ACW shows a short to long gradient from primary over secondary sensory to higher-order sensory regions (like frontal eye field and dorsolateral prefrontal cortex). This holds for all three sensorimotor, visual, and auditory systems. Albeit indirectly, this supports our view of the fast–slow gradient mediating the continuous down-sampling of the input when transitioning from lower-order unimodal to higher-order transmodal regions.

59Mathematically speaking, the fast–slow gradient with continuous down-sampling entails a shift from faster to slower frequencies in the input stochastics, that is, a decrease in its maximum interpretable frequency. The more input processing advances towards the transmodal end of the fast–slow gradient, the more its fast frequency-based temporal precision decreases. This means that different inputs may no longer be distinguishable from each other, something which, in signal processing, is described as aliasing (Fig. 7a right). Accordingly, the stronger the down-sampling going along with lower sampling rate of the input, the slower the maximum frequency that can be reconstructed from a discrete signal, i.e., a signal after a sampling process,.

60On the neuronal side, down-sampling along the fast–slow gradient of the input stochastics results in a shift of the spectral content towards slower frequencies—this can be easily indexed by the median frequency. In fact, for signals where most of the spectral content is found in slow frequencies, as in brain’s fMRI and EEG / MEG signals, this is exactly what one can observe. They have a 1⁄f distribution with stronger power in slow frequencies and less power in faster frequencies.

61Analogously, the down-sampling process carried out throughout the unimodal–transmodal hierarchy of the INT necessarily leads to a removal of the faster frequencies to achieve good resolution of the inputs’ slow frequencies,. Throughout the course of the unimodal to transmodal hierarchical processing, one loses the inputs’ information in the faster frequencies but preserves its disproportionally strong slow frequency information (see Fig. 7a for a simulation of this model using pink noise). A reduction in the sampling frequency causes the information to be maintained in slower frequencies with their higher spectral power at the cost of losing the detailed information provided by the faster frequencies. In other words, the original inputs may be shifted towards the slower frequencies in the transmodal regions. Down-sampling along a fast–slow gradient may thus be optimal for preserving the maximal amount of input information along its whole temporal range including both fast and slow frequency input components.

62The assumption of such fast–slow down-sampling along the unimodal–transmodal hierarchy is compatible with the empirical data, that is, the relation of ACW with spectral content, i.e., the power spectrum. The longer ACW in transmodal regions are related to stronger power in infraslow frequencies as compared with faster ones. Shorter ACWs in unimodal regions are more dominated by their (relative) shift towards faster frequencies, meaning that their median frequency is lower,,,,, (Fig. 7b). This raises the question for the role or function of slower frequencies like delta (1–4 Hz), slow cortical potentials in the 0.1–1 Hz range, and the infraslow frequencies (0.001–0.1 Hz) in input processing. Unlike faster frequencies like gamma (>30 Hz), beta (14–30 Hz), alpha, and theta,, the role or function of these slower frequencies is currently unclear (see also refs. –).

63We know that INT is closely linked to slow frequencies and their long cycle durations (see also refs.,). Longer cycle durations means that more inputs can be temporally integrated within one cycle and subsequently be processed together. The longer INT, especially in the transmodal regions, thus samples, the inputs in favor of a slow mode—the originally faster inputs are filtered and processed in a slow frequency way, i.e., sub-sampling (see refs.,,,,–; see also ref. ). On a more cognitive level, this means that internally oriented cognition like mind-wandering or mental time travel, as mediated by transmodal regions, may be characterized by predominant slow modes (see also ref. ).

64One may now raise yet another question. We assumed that the INT is key in input encoding, that is, encoding the stochastics of the input. One would now assume that such input encoding should be closely related to the input sampling in terms of fast–slow down-sampling. Do input encoding and encoding sampling converge? This is indeed supported by a recent combined human EEG and modeling study. SanCristobal and colleagues show that the neural processing of the input stochastics of a looming sound (3 s) is directly related to the resting state’s ACW: the longer the resting state’s ACW, the better the task-related power changes in delta, alpha, and beta could track the physical dynamics of the looming sound. This supports the assumption that the resting state’s INT display the capacity for input sampling with the consequent bias towards the slow frequency mode.

65Next, SanCristobal et al. complement these data by a computational model probing for the slower mode (via Ornstein-Uhlenbeck process): a longer ACW exerts a lower sampling frequency with a shift towards slower frequencies in input sampling. This in turn makes it impossible to obtain information from faster frequencies. Together, these findings support the notion that INT mediates input sampling by tilting or biasing it towards the slow frequency mode.

66Even more important, these findings suggest that the seemingly stochastic nature of input encoding converges with the fast–slow gradient of down-sampling. Is the fast–slow gradient of down-sampling, with the emphasis on slow-frequency encoding, the best way of bridging the timescale differences of the brain and environment? The timescale of the environment exceeds far beyond that of the brain, especially in the slow frequency ranges (as, for instance the brain cannot process the ultra-slow frequency ranges of seismic earth waves). The fast–slow gradient of down-sampling may then best be suitable for overcoming the timescale differences between the environmental context (where the input is coming from) and the brain in order to encode best the former’s slow frequency modes.

67How can the brain process temporally complex inputs such as music and language and, even better, integrate them into one meaningful whole as, for instance, a melody or a sentence? We propose that the intrinsic neural timescales (INT) take on a key role or function for the brain’s input processing. This is supported by the major role of INT in both resting and task states, including carry-over from rest to task. Following these findings, we propose that a key function of the INT consists in the dynamic shaping and structuring of input processing, including its different facets like cross-species input sharing and encoding of input stochastics. This concerns input sharing across species as well as input encoding through matching the stochastics of both environment and brain. While that may be mediated by two key mechanisms, (I) input integration vs. segregation on temporal grounds as well as (II) fast–slow down-sampling along the unimodal–transmodal hierarchy of the INT. Taken together, their key role in input processing through distinct mechanisms renders INT highly relevant for current views of the brain’s function, including its role in mental features and psychiatric disorders (Box 2), as well as for designing artificial intelligence (Box 3).