The Nature of Shared Cortical Variability

1Repeated presentations of the same stimulus elicit highly variable responses in sensory cortex. This variability is correlated across neurons, so it cannot be easily removed by averaging across the population, and may thus place critical constraints on information transmission. Understanding its nature can thus shed light on the circuit mechanisms and computations performed by the cortex in health and disease.

2Cortical variability does not arise because neurons are intrinsically noisy. Indeed, cortical neurons can generate highly reliable spike trains. The variability of their responses is therefore more likely to arise from the variability of their synaptic inputs, reflecting cortical network dynamics. Traditionally, variability has been studied in single neurons or neuronal pairs, but a full description requires understanding factors operating at the population level.

3A clue toward understanding cortical variability comes from spontaneous activity patterns that the cortex produces in the absence of stimuli. These patterns share some features with responses evoked by sensory stimuli: for instance, pairwise correlations measured during spontaneous activity can resemble those seen during sensory stimulation. Voltage-sensitive dye imaging experiments in visual cortex suggest that the interaction between spontaneous and evoked activity is additive: activity would be the sum of a deterministic sensory response and a stochastic pattern originating from networks that generate spontaneous activity. Subsequent work showed that such an additive interaction could approximate the dependence of pairwise correlations on cortical state. Other work suggests a more complex picture. In auditory cortex, responses to prolonged tone stimuli show similar fluctuating activity to those seen in silence. Yet, sensory-evoked spikes do not occur independently of these fluctuations, as would be expected from addition, but occur together with them, suggesting that spontaneous fluctuations gate the representation of stimuli. In visual cortex, quantitative analyses suggest that the variability of single neurons and correlations of neuronal pairs are more consistent with a multiplicative gain change, whose gain factor fluctuates from trial to trial.

4Such multiplicative variability is consistent with what one might expect from top-down feedback from higher order cortices. By targeting layer 1, this feedback can change the gain with which neurons respond to activity in input layers. Top-down feedback might be involved in spatial attention, which can have a multiplicative effect on neuronal gain. Yet, other evidence suggests that the influence of spatial attention is additive. Other modulatory effects, such as those seen in visual cortex during locomotion, appear to be both additive and multiplicative.

5These observations raise multiple questions. If single-neuron variability is well-modeled by multiplicative gain changes, can a single, population-wide gain factor explain the coordinated fluctuations of the population? Are the additive and multiplicative models mutually exclusive or is there a common ground between them? And what are the effects of the multiplicative and additive fluctuations on the cortical code?

6Here, we answer these questions by analyzing the trial-by-trial activity of large, simultaneously recorded neural populations in primary visual cortex (V1) of anesthetized cats and quietly awake mice. We find that cortical variability is best understood at the population level, where it can be described by a simple mathematical model comprising two sources of shared variability: multiplicative and additive. This model explained the structure of trial-to-trial population variability and captured the complex dependence of neuronal correlations on stimulus and neuronal tuning. Our results suggest that neither additive nor multiplicative variability alone forms a complete model of cortical variability; instead, a combination of the additive and multiplicative components that invest the whole population can explain much of the shared response variability in visual cortex.

7We first analyzed the activity of large neural populations in V1 of anesthetized cats (seven recording sessions from three neuronal populations in three cats). Neuronal responses were recorded from a 10-by-10 electrode array that covered a 16 mm2-region with a diversity of orientation preferences (Figure 1A). All spikes detected on a given site of the array were pooled, as they originated from neurons having similar preferred orientations. Stimuli were contrast-reversing oriented gratings and plaids consisting of two superimposed component gratings. These data sets were previously analyzed after averaging across trials of the same stimulus; here, we examined them in individual trials.

8Repeated presentations of the same stimulus elicited highly variable responses, yet this variability was coordinated across the population. To illustrate the nature of this variability, we plotted “population tuning curves”, a graphical summary of the population response to a plaid stimulus (Figure 1B). As expected, a plaid with very different component contrasts evoked the largest mean activity at sites tuned for the orientation of the high-contrast component grating (Figure 1C), a form of winner-take-all competition. Yet, the responses of all tuned sites varied from trial to trial; for instance, firing rate tended to be higher on one trial than on another, and this difference affected most sites simultaneously (Figure 1D). As a result, the curves fitted to the population activity changed noticeably from trial to trial (Figure 1E). Similar results were obtained for different stimuli. For instance, as observed previously, a plaid with equal component contrasts elicited large mean responses at sites tuned for either of the component orientations (Figure 1F). Trial-by-trial responses were again highly variable, and the variability seemed to be coordinated at the population level (Figures 1G and 1H).

9Although trial-by-trial variability was clearly coordinated across the population, the nature of this shared variability was not immediately obvious. In the first example, population responses seemed to be scaled multiplicatively between trials (Figure 1D). Yet, in the second example, population responses seemed to be shifted by a common offset between trials (Figure 1G). There were also examples that spoke in favor of a mixture of additive and multiplicative effects (Figures 1E and 1H).

10To explore the structure of this variability and to gain an intuition into how it is shared between neurons, we characterized the population response on each trial as a single, compact population tuning curve determined by three free parameters. The population tuning curve (a vector whose values are the normalized firing rates of all orientation-tuned sites, ordered by their preferred orientations) on trial i was fit as a linear combination of prototypical responses to the two component gratings of a plaid, plus a constant shift (Figure 1B):Here, and are the component orientations of the plaid stimulus presented on trial i, and is the prototypical response (a circular Gaussian) to a grating of orientation. The population tuning curve Ri on each trial i is thus determined by three parameters: the tuned response to component grating 1,, the tuned response to component grating 2,, and a baseline untuned response,. These parameters were fit by least-squares to the population response; they effectively summarized the population activity of many tens of sites on each trial, accounting for 47 ± 18% (median ± median absolute deviation) of the variance. We used these fits to examine the population data and the predictions of various models of neuronal variability (but not to fit the models, which was done on the actual spike counts as a function of site and time, as described below).

11The parameters of the population tuning curves confirmed that the trial-to-trial variability included a shared multiplicative component (Figure 1I). For a plaid with equal component contrasts, the tuned-response components to gratings 1 and 2, and, were positively correlated across trials: the ellipse summarizing their distribution was clearly diagonal (Figure 1I, black; r = 0.45). By comparison, for a plaid with markedly different component contrasts, the tuned response to the low-contrast grating stayed close to zero regardless of the tuned response to the high-contrast grating (Figure 1I, blue; r = −0.05). These two elongated clouds radiating outward from the origin are what would be expected from shared multiplicative variability. They do not, however, rule out the presence of additive variability, which causes variations in the parameter βi (Figure S3).

12The shared nature of trial-to-trial variability described above suggests that trial-to-trial fluctuations in population activity might be accountable by a small number of factors, whose joint effect on each neuron can be multiplicative and/or additive. To formalize this idea, we turned to a more rigorous approach that seeks to predict the response of every site to every stimulus on every trial. We developed a set of models that operate at the level of spike trains of units (single neuron or multiunit) in a population (not at the level of the summary statistics α1, α2, and β previously described). In these models, the response of each unit on a trial depends on both the tuning of that unit to the stimulus shown on that trial and a trial-varying, global, shared factor that originates from the rest of the brain (Figure 2A). To account for the different factors of shared variability, we considered an additive model (Figure 2B), a multiplicative model (Figure 2C), and an affine model (Figure 2D), which encompasses them both.

13In the affine model, the expected spike count of unit c on trial i, during which stimulus s(i) is presented, iswhere is the deterministic sensory drive to unit c arising from stimulus s(i). This term reflects the unit’s sensory tuning as well as its contextual interactions; i.e., anything that contributes to the unit’s mean response to that stimulus; e.g., divisive normalization. The multiplicative gain gi scales the firing rate of all units in proportion to their sensory drive; i.e., it controls their response gain. The additive offset ai adds to their firing rates in proportion to their coupling factors hc. Constraining gi = 1 gives a purely additive model (Figure 2B); setting ai = 0 results in a purely multiplicative model (Figure 2C). Constraining both gi = 1 and ai = 0 gives a model that takes no shared variability into account (referred as the independent model; Figure S4A).

14To compare model predictions with recorded spike counts, we passed fc,i into a stochastic spike count generator, which yields an integer spike count nc,i from a negative binomial distribution with mean fc,i and cell- and stimulus-dependent Fano factor Fc,s(i), estimated by maximum likelihood. This stochastic spike count generator delivers the fraction of variability that is not shared, but private to each unit.

15To study how well these global, shared factors could explain cortical variability, we fit the affine model and compared the results with the purely additive and multiplicative models. We obtained the model parameters by fitting the model predictions fc,i to the spike counts recorded at each site, and we evaluated the fits with cross-validation (see Supplemental Information: cross-validation; Figure S1). Performance was measured by the quality index q: the improvement in cross-validated prediction compared to the independent model with no shared variability. The quality index is zero or negative for a model that offers no improvement over the independent model, and equals 1 for a perfect prediction.

16This analysis showed that the affine model is superior to both the additive and multiplicative models (Figures 2E–2G). The multiplicative model performed better than the additive model (Figure 2E; Table S1; p < 10−11 for all data together, p < 0.03 in five out of seven recording sessions evaluated individually; sign test). In all sessions, the affine model considerably outperformed both the additive model (Figure 2F; Table S1; p < 10−43 for all data together, p < 10−6 in five individual sessions, and p < 0.02 in the remaining two; sign test) and the multiplicative model (Figure 2G; Table S1; p < 10−43 for all data together, p < 10−8 in four individual sessions, and p < 0.001 for the rest; sign test). Because these results were cross-validated, the affine model could not gain a numerical advantage by over-fitting. Rather, the results indicate that neither the additive nor multiplicative factors alone suffice: combining the two forms a better model for shared cortical variability.

17Note that the additive model—unlike the multiplicative model—has a coupling term hc that allows each unit to be coupled to population activity differently, a phenomenon that has been previously described. Introducing a similar cell-coupling term to the multiplicative model did improve its performance, but this extended multiplicative model still performed worse than the affine model (Figures S2A–S2D; Table S2; Supplemental Information: the extended multiplicative model). The superior performance of the affine model thus reflects the presence of an additive component rather than simply its ability to model different coupling strengths of different neurons.

18To gain further intuition into the performance of the models, we returned to the reduced representation of the population activity (Figure 3). We first computed the population tuning curves for the recorded and simulated activities. We then fit ellipses to summarize their variability (as in Figure 1I) and examined the ellipses corresponding to all plaid stimuli (Figures 3, S3, and S4). In the data, the tuned responses to component gratings 1 and 2 tended to be correlated only for plaids with equal component contrasts, resulting in ellipses emanating from the origin (Figure 3A). The additive model produced tuned responses that were slanted (i.e., highly correlated) for all plaid stimuli, including those with unequal component contrasts (Figure 3B). The multiplicative and affine models, by contrast, captured the angle of the ellipses appropriately (Figures 3C and 3D). Indeed, the affine model outperformed both the additive and multiplicative models in predicting the ellipse angles across all seven sessions (Figures 3E–3G). The affine model also provided better predictions of ellipse lengths, which reflect the strength of correlated variability (Figures 3H–3J). Analyzing correlations between the tuned responses and the baseline, untuned activity gave similar conclusion (Figure S3). Finally, all three models outperformed the independent model (Figure S4).

19In short, the affine model was superior in capturing correlations between the two tuned-response components as well as correlations between the tuned- and untuned-response components. The affine model thus explains much of the response variability and surpasses both the additive and multiplicative models in predicting not only the raw individual spike counts (Figure 2), but also our summary analysis of ensemble activity, the population tuning curves (Figures 3, S3, and S4).

20Our results suggest that the correlated response variability of neuronal populations can be well modeled by only two global factors, additive and multiplicative, that are shared across the population. In a population of N neurons, there are order N2 pairwise correlations. To what extent can these pairwise correlations be explained by these two factors?

21To address this, we measured pairwise noise correlations for all pairs of orientation-tuned sites in response to single-grating stimuli. Correlations tended to be high because they were measured over long time windows and involved multiunit activity. We found that noise correlations predicted by the affine model were closer to the measured values, compared to either the additive model (Figures S5A–S5U; p = 10−48 for all data together, p < 10−4 in four individual sessions, and p < 0.05 in one session; sign test on the squared errors between measured and predicted noise correlations) or the multiplicative model (Figures S5A–S5U; p < 10−54 for all data together, p < 10−5 in five individual sessions, and p < 0.05 in the remaining two; sign test).

22We next asked whether the affine model could predict the well-known relationship between signal and noise correlations: noise correlations are larger between cells with similar sensory tuning (reviewed in ). This relationship is often attributed to increased connectivity between neurons with similar sensory tuning, but it can also arise from shared multiplicative variability. This is simply because multiplication affects neurons that are responding to a stimulus more than neurons that are not responding, thus introducing correlations among neurons with similar tuning.

23We measured the dependence of noise correlations on the difference in preferred orientations and compared it to the one calculated from spike counts simulated by the additive, multiplicative, and affine models (Figure 4). As expected, noise correlations showed a clear dependence on orientation-tuning similarity: sites preferring the same orientation were more strongly correlated (Figure 4A). This dependence could not be captured by the additive model, which predicted a negligible dependence on tuning similarity (Figure 4B). The multiplicative model predicted stronger noise correlations when preferred orientations were more similar, but correlations were generally underestimated (Figure 4C). The affine model almost completely accounted for the dependence of noise correlations on tuning similarity (Figure 4D); this superiority was statistically significant in all sessions (p < 0.05 for both additive versus affine and multiplicative versus affine; t test on each session’s sum of squared errors between running medians of data and model; Figure S6).

24Still, the affine model slightly underestimated the dependence of pairwise correlations on tuning difference (Figure 4D). To investigate this mild imperfection, we looked at the spontaneous correlations; i.e., the pairwise correlations measured in the absence of stimuli (Figure 4E). Spontaneous correlations have been reported to be strongest for similarly tuned cells. Our data showed this effect, albeit weakly (Figure 4E, red curve). Unsurprisingly, none of the three models could predict this dependence; for instance, the predictions of the affine model were essentially flat (Figure 4E, black curve). This small, but noticeable, error in accounting for spontaneous correlations hints at what might be missing from the affine model: a slightly higher coupling between neurons that are similarly tuned.

25To further understand the predictions of the three models, we considered an additional factor that determines noise correlations: the orientation of the stimulus relative to the preferred orientations and of the two sites. Noise correlations between a neuronal pair depend not only on the neurons’ sensory tuning, but also on the attributes (orientation in this case) of the stimulus. Assuming rotational symmetry, this dependence on stimulus orientation and cell tuning can be summarized by a function, where is the preferred orientation of site c relative to stimulus orientation (Figure 5A). This representation was particularly informative in three locations: (1) the “center” bin, where the preferred orientations of both sites match the stimulus orientation; (2) the “edge” bin, where one site’s tuning matches the stimulus and the other is orthogonal to it; and (3) the “corner” bin, where the preferred orientations of both sites are orthogonal to the stimulus orientation (Figure 5B).

26This representation revealed a rich structure of pairwise correlations (Figures 5C–5J). In a typical session, noise correlations peaked in the center bin (Figure 5C); this observation was well predicted by the multiplicative and affine models (Figures 5E and 5F), but not by the additive model (Figure 5D). As has been previously observed, correlations between similarly tuned sites were also high in the corner bins; this was predicted by the additive and affine models (Figures 5D and 5F), but not by the multiplicative model (Figure 5E). Finally, the data showed the lowest correlations in the edge bins; this was predicted by the affine and additive models (Figures 5D and 5F), but not by the multiplicative model (Figure 5E). While this particular structure of the correlation matrix was the most common (four out of seven sessions), it was not the only one we observed. Other recordings showed a different correlation structure that, again, was well captured by the affine model (e.g., Figures 5G–5J).

27To assess each model’s performance across sessions, we computed the median noise correlations that fell into the center, corner, and edge bins of the correlation matrix (Figures 5K–5M). As in previous examples, the additive model tended to underestimate correlations in the center bin (Figure 5K). The multiplicative model did better (Figure 5L), but not as well as the affine model (Figure 5M). Note that the correlation matrices predicted by the extended multiplicative model were similar to the multiplicative model (Figures S2H–S2O). Only the affine model could predict all possible structures of the correlation matrices. In fact, the rich structure of these correlation matrices can be analytically predicted by the affine model (see Supplemental Information: analytic calculation of pairwise correlations; Figure S7). We conclude that the affine model concisely accounts for the complex, session-dependent relationship of noise correlations to neuronal tuning and stimulus.

28We next asked whether the affine model—derived from anesthetized cat data—is also a good description of shared variability in the unanesthetized cortex. To this end, we used multisite silicon probes to record population activity in V1 of quietly awake, head-fixed mice (five sessions in five neuronal populations in four mice) in response to drifting gratings (12 directions at 100% or 60% contrast). The data were spike-sorted and only stable, well-isolated single units were used for further analysis. As expected, the mean firing rates were substantially lower for single neurons in mouse V1 than for multiunit activity in cat V1 (mean of 2.8 versus 30.3 spikes/s). Even with such low firing rates, a majority (58%) of neurons showed detectable shared variability (quality index q > 0.1 for at least one of the models; Table S3). We focused on these neurons, fit the models, and asked which model better described this shared variability.

29The results demonstrate that as in anesthetized cats, shared cortical variability in awake mice could be described by a combination of additive and multiplicative components (Figure 6). Differences between the multiplicative and additive models were small (Figure 6A; Table S3; p = 0.30 for all data together and p > 0.05 in all individual sessions; sign test). The affine model, on the other hand, performed significantly better than either the additive model (Figure 6B; Table S3; p < 10−12 for all data together and p < 0.008 in all individual sessions; sign test) or the multiplicative model (Figure 6C; Table S3; p < 10−16 for all data together and p < 0.002 in all individual sessions; sign test). Again, the extended multiplicative model did not fare as well as the affine model (Figures S2E–S2G; Table S4).

30The affine model also outperformed the additive and multiplicative models in predicting noise correlations in single-unit data from quietly awake mice. As expected on the basis of lower firing rates, measured correlations were considerably lower than in the multiunit data from anesthetized cats (noise-correlation means of 0.09 versus 0.38). Nonetheless, they could still be used to distinguish the performance of the models. As with the cat data, the affine model trumped both the additive model (Figures S5, p < 10−4 for all data together, p < 0.005 in two individual sessions, and p < 0.04 in two others; sign test on the squared errors between measured and predicted noise correlations) and the multiplicative model (Figures S5, p < 10−31 for all data together and p < 0.001 in all individual sessions; sign test).

31We have shown that cortical population variability can be well described by two sources of shared fluctuation: multiplicative and additive. How do these two sources of fluctuation impact information coding? To address this question, we used the responses of a neuronal population governed by the affine model to distinguish stimuli that differ subtly in orientation or contrast. To quantify this aspect of population decoding, we used a linear discriminability measure:Here and are the trial-averaged population response vectors to stimuli 1 and 2, and Σ is the population covariance matrix, derived from all presentations of the two stimuli. Since,, and Σ can be analytically calculated under the affine model (Supplemental Information: analytic calculation of pairwise correlations), we can estimate the dependence of d2 on model parameters. For comparison, we also evaluated this measure for a population of independent, uncorrelated neurons having the equivalent amount of variability. To obtain a discriminability measure based on those uncorrelated responses, we set all off-diagonal values of the covariance matrix Σ (corresponding to correlations between neurons) to zero. Depending on the structure of the correlation, its value can be superior or inferior to d2.

32We first considered the effect of changing the mean of either the additive offset or the multiplicative gain. Increasing the mean of the multiplicative gain enhanced coding, whereas increasing the mean additive offset slightly degraded it (data not shown). This accords with classic studies of tuning curves: the steeper the tuning curve and the lower the baseline, the larger the decoding accuracy.

33We next considered the effect of trial-to-trial fluctuations in the additive and multiplicative terms. We simulated a contrast discrimination task (Figure 7A) and an orientation discrimination task (Figure 8A). The contrast discrimination task involved distinguishing two gratings of the same orientation, but different contrasts (6% and 12%); the orientation discrimination task involved distinguishing two 12%-contrast gratings whose orientations differed by either 6° or 90°.

34In the contrast discrimination task, shared multiplicative fluctuations had a much larger effect on stimulus coding than additive ones. Increasing the variability of the additive offset hardly had any impact on discriminability, triggering a small decline that became noticeable only after trial-shuffling, when variability was uncorrelated across neurons (Figure 7B). On the other hand, increasing the variability of the multiplicative gain sharply decreased discriminability; e.g., discriminability dropped by a factor of 5 when the coefficient of variation of gi increased from 0 to 0.5. This decline was reduced in, indicating that it is specific to multiplicative variability that is shared across neurons (Figure 7C). These trends also held for tasks involving different pairs of contrasts (Figures S8B–S8D).

35To understand why contrast discrimination would particularly suffer from shared multiplicative fluctuations, we fit the simulated population response on a single trial as a linear combination of a unit circular Gaussian and a constant offset (Figure 7A). Population activity on each trial was thus summarized by two parameters: the tuned amplitude α and the baseline amplitude β. Discriminability could then be understood from the overlap of the distributions of the tuned responses α to the two stimuli: the more they overlap, the lower is discriminability. Increasing the variability of the additive offset hardly altered the shapes of these distributions, explaining the negligible effect of additive fluctuations on coding (Figure 7D). Rather, additive fluctuations affected the baseline amplitude β, which cannot be used to discriminate the two stimuli (data not shown). By contrast, increasing the variability of the multiplicative gain broadened the distributions of the tuned amplitudes α, making them much harder to discern (Figure 7E).

36We obtained further insight into the effect of correlations induced by shared additive and multiplicative fluctuations by comparing them to the effect of equivalent uncorrelated variability using a shuffling analysis (Figures 7F and 7G). Comparing the α distributions calculated from the additive-correlated responses to stimuli 1 and 2 (Figure 7D) to their shuffled counterparts (Figure 7F) showed that additive-correlated variability was less detrimental than its shuffled equivalent (see also Figure S8M). By contrast, the α distributions obtained with shared multiplicative fluctuations (Figure 7E) had a greater overlap than the ones from the corresponding shuffled trials (Figure 7G). This indicates that in the contrast discrimination task, additive-correlated variability had a minor effect, but multiplicative-correlated variability worsened discriminability.

37The effect of shared fluctuations on the orientation discrimination task depended on the type of fluctuations and the difficulty of the task (Figure 8). Shared additive fluctuations had no effect on discriminability, which deteriorated only after trial-shuffling, when variability was no longer shared among neurons (Figures 8B and 8C). Shared multiplicative fluctuations, on the other hand, had large consequences that depended on the difficulty of the task (Figures 8D and 8E). For most orientation differences, they had a strong detrimental effect, which was reduced by trial-shuffling (Figures 8D, S8G, S8H, S8K, and S8L). Yet, when the task involved discriminating stimuli of fine orientation difference, performance was actually worsened by trial-shuffling (Figures 8E, S8E, and S8I), indicating that shared multiplicative fluctuations allow better fine-orientation discrimination than the equivalent uncorrelated variability.

38To understand these effects, we fit population responses as a sum of responses to the two gratings being discriminated (Figure 8A) and plotted ellipses illustrating the means and covariances of the population responses (Figures 8F–8M; see also Supplemental Information: decoding analysis with reduced dimensionality). For two gratings of very different orientations, the mean responses were far apart; shared multiplicative fluctuations elongated the two ellipses in such a way that they started to overlap, which reduced their discriminability (Figure 8H). Introducing the same amount of variability to uncorrelated neurons by trial-shuffling did not increase the overlap significantly (Figure 8L). Conversely, for fine-orientation discrimination, the mean responses to the two stimuli were very close (Figure 8I). Shared multiplicative fluctuations again elongated the two ellipses, but because the direction of this elongation was almost orthogonal to the separation between the centers, its effects on overlap were relatively minor. When the equivalent variability was applied independently to the neurons via trial-shuffling, the elongated ellipses were replaced by nearly circular shapes that showed greater overlap (Figure 8M). We observed similar trends in orientation discrimination tasks that involved gratings of a different contrast (Figures S8I–S8L).

39In short, shared additive fluctuations are not detrimental to either contrast or orientation discrimination. Shared multiplicative fluctuations, on the other hand, are detrimental to the discrimination of contrast and of coarse orientation differences, but are less harmful to fine-orientation discrimination than an equivalent amount of uncorrelated variability.

40We analyzed the activity of large neuronal populations in V1. We found that much of the trial-by-trial variability is shared across the population and can be summarized using only two global factors: additive and multiplicative. These results may reconcile the long-held view that variability is additive with subsequent claims of it being predominantly multiplicative. The former view hypothesizes that the large response variability on individual trials comes from adding ongoing cortical activity onto a deterministic sensory response. The latter postulates that much of the response variability arises from fluctuations in excitability. We argue that it is not one or the other, but a combination of the two.

41Our data suggest that common multiplicative gain fluctuations play a dominant role in the structure of pairwise noise correlations. There are two long-standing hypotheses for the circuit mechanisms underlying the similarity of signal and noise correlations. The first is that correlations arise from synaptic connectivity patterns. In visual cortex, neurons with similar orientation preferences are preferentially connected, thus co-tuned neurons will also share a larger fraction of common input from cells of similar sensory preference, increasing their correlations. The second is that correlations reflect common modulation by multiplicative gain fluctuations. Indeed, even a pair of neurons that are unconnected and share no inputs would exhibit noise correlations if they experienced common multiplicative modulation, and the strength of these correlations would be stronger for neurons with similar sensory tuning. Our data provide evidence in favor of both hypotheses, but ascribe a larger role to the mechanism of shared multiplicative fluctuations. Indeed, the affine model generated a good fit to the dependence of noise correlation on orientation tuning (Figures 4 and S6) and only a small residual was left to be explained. Intriguingly, this residual resembled the small, but significant, dependence of spontaneous correlations on tuning difference. The affine model could not possibly capture this dependence, because it has no way to preferentially assign shared factors as a function of tuning similarity. The similarity of spontaneous and signal correlations, matching the residual error in the affine model’s prediction, thus suggests that specific synaptic connectivity does contribute to producing noise correlations, although this contribution is numerically smaller than the contribution of common gain fluctuations.

42Our results lend weight to the multiplicative model, but also reveal its possible limitations. For example, shared multiplicative fluctuations alone were insufficient to predict the complex relationship between correlations, the sensory tuning of both units, and the stimulus (Figure 5). The affine model—by adding an additional common additive offset on each trial—resolved this limitation.

43Nevertheless, there are certain features that may further improve the affine model. For instance, show that different neurons are coupled to global fluctuations differently. In the current affine model, only the additive component has a cell-coupling term that allows each neuron to be coupled to population activity to different degrees. We saw that it is possible to extend the multiplicative model in such a way; this extension improved the fit of the multiplicative model, but not to the extent that it matched the affine model (Figure S2). Future work could explore the possibility of combining the extended multiplicative model and the additive model; however, this would require substantially greater amounts of single-unit data than analyzed here.

44What candidate circuit mechanisms might underlie the multiplicative and additive fluctuations described here? A possibility lies in different interneuron classes: activation and inactivation of parvalbumin- or somatostatin-positive interneurons have been variously suggested to have multiplicative, additive, and combined (affine) effects on the firing rates of pyramidal cells. Another possibility lies in top-down connections from higher order cortices and thalamus. These inputs, which target distal apical dendrites in layer 1, can rarely elicit spikes in pyramidal cells, but may boost the gain of pyramidal cells’ responses to more proximal sensory inputs. Such top-down connections are believed to play a role in attentional modulation of V4 firing, which has been reported to have both a multiplicative and an additive effect on visual responses. In addition, multiplicative effects have been observed in other contexts such as normalization. The multiplicative and additive fluctuations might also share circuits with those that modulate responses based on locomotion, which has both an additive and a divisive effect.

45Additive and multiplicative fluctuations have very different consequences for the cortical coding of sensory information. An analytically tractable model based on our experimental results revealed that for both contrast and orientation discriminations, additive fluctuations had negligible effect on the discriminability of sensory stimuli, whereas multiplicative fluctuations had a much larger effect. These results have a simple intuitive explanation. Variability in population activity reduces the ability to discriminate stimuli, when it means that a single population-firing pattern can be induced by two different sensory stimuli. A change in multiplicative gain can affect population activity in a very similar way to a change in stimulus contrast; it is thus not surprising that contrast discrimination is impaired strongly by multiplicative fluctuations. Likewise, the small effect of additive fluctuations may be understood from the fact that the resulting changes in baseline could not have been generated by any of the stimuli presented.

46While correlated fluctuations were originally believed to only worsen stimulus discriminability, responses with correlated variability can often outperform responses with the same amount of uncorrelated variability. This does not mean that correlated variability helps discrimination (compared to zero variability); it means that its correlation structure interferes less with stimulus coding than might otherwise be expected. Indeed, we found that correlations frequently help discriminability. Shared additive fluctuations, for example, had a less detrimental effect on discriminability than their uncorrelated counterparts. This may be because the effect of shared additive fluctuations lies along a different direction in population vector space than differences between stimuli; shuffling thus adds variance to a dimension that can interfere with stimulus coding (Figure S8M). For shared multiplicative fluctuations, the effect of correlations on orientation discrimination depended on the difficulty of the discrimination task. For easy tasks (large orientation differences), correlations hurt. In this case, the pools of neurons responding to the two very different stimuli are largely distinct; the arguments of therefore apply, and correlations cause averages over populations to be taken less accurately. For fine discrimination, however, correlations improved performance, possibly because the multiplicative fluctuations move population responses in a different direction to differences in stimulus orientation (akin to differences in contrast). Our decoding results therefore suggest that the nature of shared variability in visual cortical populations is well-suited for stimulus discrimination: additive variability has little consequence, while multiplicative-derived correlations benefit fine-orientation discrimination over uncorrelated variability.

47But why should shared multiplicative and additive variability occur at all? Why not simply have private variability or no variability? Multiplicative variability might reflect the same circuit mechanisms that are responsible for top-down processes, such as attention. It has been suggested that visual attention modulates visual cortical responses in a similar manner to an increase in stimulus contrast, which could explain its primarily multiplicative effect similar to a contrast change. While fluctuations in this process would clearly impair fine-contrast discrimination, contrast discrimination may be a task only rarely required. Indeed, much of the visual system appears to be geared toward making contrast-invariant judgments. Because our results show that multiplicative fluctuations have little effect on fine-orientation discrimination, we conclude that multiplicative fluctuations might allow the visual system to modulate the salience of visual stimuli, while having relatively little impact on stimulus coding. Additive fluctuations may allow the population to include an additional dimension of salience or other non-sensory factors, with only minor impact on representation of sensory information.

48Anesthetized cat recordings were approved by the Animal Care and Use Committee of the Smith-Kettlewell Eye Research Institute. Experimental methods have been previously described by. Briefly, responses, primarily from layers 2/3, were recorded with a 10-by-10 electrode array. All threshold crossings on each channel were pooled and only orientation-tuned sites were considered in subsequent analyses. Sequences of 2 s contrast-reversing oriented gratings and plaids, interspersed with 2 s blanks, were shown in random order in blocks (see Supplemental Information: anesthetized cat recordings).

49Awake mouse recordings were conducted under personal and project licenses issued by the Home Office, in accordance with the UK Animals (Scientific Procedures) Act 1986. A head plate with a recording chamber was affixed to the skull. After 3 days of recovery and 3 head-restraint acclimatization sessions, a craniectomy (and durotomy, if necessary) was made over the left V1. The animal was allowed to recover for at least 1.5 hr before the recording. Multisite silicon probes were inserted to a depth of 500–800 μm (median 615 μm). Animals were judged to be quietly awake by video monitoring. Spikes were detected using NDManager and clustered using KlustaKwik, followed by manual adjustment using KlustaViewa. Detailed analysis was carried out only on well-isolated units that showed consistent firing throughout a recording session. Sequences of 1 s oriented drifting gratings, interspersed with either 1 or 6 s blanks were shown on three liquid-crystal display monitors, covering a field of view of ∼120° × 60° that extended in front and to the right of the animal (see Supplemental Information: awake mouse recordings).

50Population tuning curves were fit to population responses to plaid stimuli as a linear combination of prototypical response to the component gratings and a constant baseline shift. The percentage of variance of the population activity that could be explained by the population tuning curve analysis on each trial was estimated as, where rc,i is the normalized firing rate for each site c and trial i, and is the measured site-averaged response on trial i (see Supplemental Information: population tuning curve analysis).

51Full details of the models considered in this study can be found in Supplemental Information: the models. Briefly, we denote the experimentally measured spike counts of unit c (single neuron or multiunit) on trial i as Nc,i and the stimulus presented on trial i as s(i). Each model predicts an expected spike count fc,i of unit c on trial i that approximates Nc,i.

52In the independent model, the expected spike count on each trial is a deterministic quantity:where the matrix is estimated as the trial-averaged spike count of unit c to stimulus s(i).

53In the additive model, the expected spike count iswhere dc,s represents the sensory drive of unit c from stimulus s, ai the common additive offset on each trial i, and hc the degree to which each unit c is susceptible to this offset. The total number of parameters in this model is is estimated as in the independent model, and the parameters ai and hc were fit by least-squares.

54In the multiplicative model, the expected spike count is given bygi is the common multiplicative gain on trial i. Note that unlike the model of, where on each trial each cell has its own (private) gain (i.e., ), we propose a common multiplicative gain that is shared across the population. This model contains parameters, which were fit by least-squares.

55The affine model incorporates both the additive and multiplicative components:The total number of parameters in this model is. We fit the affine model by an alternation method that was repeated until the convergence criterion was met, specifically that the difference in squared error per unit per trial between two iterations was lower than 10−10. Typically this took several hundred iterations.

56The predictions of these models represent expected spike counts rather than actual integer observations. To generate spike counts nc,i from each model, we used a negative-binomial spike generator with mean fc,i and a Fano factor parameter that was estimated for each unit c and stimulus s by maximum likelihood (see Supplemental Information: spike count generator). Spike counts generated by this method were used for the analyses in Figures 3, 4, and 5.

57We assessed the models’ goodness of fit by cross-validating the simulated spike count nc,i (see Supplemental Information: cross-validation; Figure S1). Model performance was assessed for each unit c by quality index, where was the squared error of the independent model. This cross-validation method was used to generate the plots in Figures 2E–2G, 6, and S2B–S2G; only units that showed shared variability (qc > 0.1 for at least one of the models) were included in statistical analysis.

58To estimate how the linear discriminability measure d2depends on the fluctuations of additive offset and multiplicative gain, we constructed a homogeneous neural population with translation-invariant orientation tuning curves (see Supplemental Information: decoding). To visualize the population patterns produced by this population, we simulated population responses on 104 trials of the contrast and orientation discrimination tasks, assuming Gaussian distributions for both multiplicative gain and additive offset. The corresponding uncorrelated responses were obtained by trial-shuffling. These population responses were then projected onto a low-dimensional space for visualization (see Supplemental Information: decoding analysis with reduced dimensionality).

59Conceptualization: I.-C.L., M.O., M.C., and K.D.H.; Methodology: I.-C.L., M.O., M.C., and K.D.H.; Software: I.-C.L.; Formal Analysis: I.-C.L., M.O., M.C., and K.D.H.; Investigation: I.-C.L. and M.O.; Resources: M.C.; Writing – Original Draft: I.-C.L., M.O., M.C., and K.D.H.; Writing – Reviewing & Editing: I.-C.L., M.O., M.C., and K.D.H.; Visualization: I.-C.L.; Supervision: M.C. and K.D.H.; Project Administration: M.C. and K.D.H.; Funding Acquisition: M.C. and K.D.H.