Dopaminergic neurons write and update memories with cell-type-specific rules

1Animals use memories of past events to predict the future. In some cases, an animal is best served by making a prediction based solely on their most recent experience. In others, a series of experiences is integrated to make a probabilistic prediction, discounting an event experienced only once. How are such different strategies implemented in the brain? One idea is that individual components of a memory—often called engrams—are simultaneously stored in distinct sub-circuits whose outputs can then be combined upon recall to affect behavior. These sub-circuits would vary in their rules for writing, updating and retaining these engrams, having differences in synaptic plasticity and circuit properties. Experiments aimed at uncovering the mechanisms by which different forms of memory are established and maintained, and then coherently coordinated to drive behavior, are facilitated by using a model system in which the relevant cells and circuits can be identified and manipulated either individually or in specific combinations. In this report, we describe experiments performed in such a model system, the olfactory circuitry of Drosophila melanogaster.

2Neuronal circuits for learning associations often share a common architecture: a large array of anatomically similar neurons that represent the sensory environment converge onto a much smaller number of output neurons (Figure 1A). Punishment or reward activates modulatory neurons that in turn cause changes in the synaptic weight matrix between the neurons representing the sensory cues and the output neurons, resulting in memory formation. The mushroom body (MB) shares this architecture and is the major site of associative learning in Drosophila. Odor identity is represented by a pattern of sparse activity in the ~2000 Kenyon cells (KCs), whose parallel axons form the lobes of the MB. The dendrites of MB output neurons (MBONs) and terminals of dopaminergic input neurons (DANs) tile the KC axons; the extent of the arbors of individual MBONs and DANs overlap precisely, defining 15 distinct compartmental units that tile the MB lobes of adult Drosophila.

3While these compartmental units share a similar general structure, there are important anatomical and functional differences between them. Each compartment contains only one of the three major classes of KCs: γ, α′/β′ and α/β. Anatomical data suggest that each MBON samples from ~90 to ~2000 KCs in one or two compartments, while each KC forms en passant synapses on 5–6 types of MBONs along its axon in the lobes. During associative learning, each DAN is thought to modulate KC-MBON synapses only in its target compartment(s). Thus, the information encoded by KC activity might contribute to multiple distinct engrams by compartmental specific modulation of KC-MBON en passant synapses. A large body of previous work has established that the KCs, MBONs and DANs innervating individual compartments are differentially involved in forming memories with different valence—that is, appetitive and aversive memories—and with different stabilities. However little is known about the rules for writing and updating memory in each compartment. By using the anatomical map of the MB and cell type specific drivers we reported previously in conjunction with newly developed behavioral assays, we have been able to establish that different compartments can employ vastly different learning rules.

4Punitive or rewarding stimuli such as electric shock, heat, cold, bitter taste and sugar generally activate a complex pattern of DANs. Believing that a reduction in complexity will be essential to understand the roles played by DAN inputs to different MB compartments, we utilized intersectional split-GAL4 drivers to express CsChrimson, a red-shifted channelrhodopsin, in specific cell types (see Materials and methods). While activation of CsChrimson in these driver lines provides a stimulus that is unlikely to occur naturally, it allowed us to separately examine the memory components induced by individual DAN cell types.

5We developed an olfactory arena that allows fine temporal control of both odor delivery and DAN activation using optogenetics (Figure 1B, Video 1 and Figure 1—figure supplement 1; detailed construction documents are included as a Supplementary file 1). In this arena, freely moving flies can be repeatedly trained and tested without the manual handling or temperature changes required in previous assays, thereby minimizing variability that might obscure subtle behavioral effects. These methods allowed us to systematically examine the properties of memories induced in different MB compartments, including: (1) the temporal pairing requirements of odor presentation and DAN activation; (2) the amount of training required for memory formation; (3) retention time; (4) weakening of the conditioned response induced by either DAN activation in the absence of odor presentation or by odor presentation in the absence of DAN activation; (5) the ability to learn new associations; and (6) the capacity to store multiple memories.

6Dopamine signaling to KCs has been implicated in both learning and forgetting. However, it has not been determined if a single DAN cell type can drive both processes within the same compartment. Pairing one of two odors with activation of PPL1-γ1pedc, results in robust aversive memory to the paired odor (; Figure 1C–E and Video 2). That memory is fully retained after 10 min (blue line in Figure 1D) but has largely decayed by 24 hr (Figure 3B). Presentation of the odors alone a few minutes after training resulted in a modest reduction in the conditioned response (orange line in Figure 1D). A second activation of the same DAN a few minutes after training in the absence of odor can almost completely abolish the conditioned response (red line in Figure 1D). Recent imaging data of the γ4 MBON suggest that this reduction most likely results from restoration of the response of the MBON to the odor; that is, erasure of the memory. If, after the first training, contingencies are reversed such that the other odor is presented paired with DAN activation, the first memory is reduced and a memory of the new association is formed (purple line in Figure 1D). Taken together, these data indicate that the same DANs can write a new memory or reduce an existing conditioned response, enabling the flexibility to rapidly change the associations formed between a conditioned stimulus (CS), the odor, and an unconditioned stimulus (US) represented by dopamine release.

7In classical conditioning, both the rate of learning and the valence of the resultant memory depend on the relative timing between the CS and the US. Our ability to precisely control DAN stimulation and odor presentation enabled us to examine the CS-US timing relationship (Figure 1F). We found that when PPL1-γ1pedc stimulation fell within a 30-s time window following the onset of a 10-s odor presentation, an aversive memory was formed (negative PI, Figure 1F). Interestingly, we observed that DAN stimulation that precedes odor presentation by 20 to 60 s induced an appetitive memory (positive PI, Figure 1F). These observations are consistent with the notion that it is the predictive aspect of CS-US timing that matters. When the timing is such that the CS predicts a subsequent aversive US, animals learn to avoid the CS. However, animals can also learn that the CS predicts the end of an aversive US, and are consequently attracted to the CS. It has been suggested that this timing dependency could result from the dynamics of the biochemical signaling cascade acting downstream of dopamine receptors.

8Pairing activation of PPL1-γ1pedc with odor has been shown to depresses the subsequent spiking rate of the MBON from the γ1pedc compartments in response to the trained odor. In behavioral assays, optogenetic activation of this MBON was shown to attract flies. Taken together, these observations suggest that DAN activation paired with an odor produces an aversive behavioral response to that odor by decreasing the MBON’s attractive output. Thus our data can be most easily explained if this single DAN can bi-directionally alter the strength of KC-MBON synapses depending on the presence and relative timing of odor-driven KC activity; a full testing of this model awaits additional physiological measurements.

9In order to compare the parameters of learning in different MB compartments, we selected a set of additional split-GAL4 drivers that express at similar and high levels in different DAN cell types (diagrammed in Figure 2A; primary imaging data documenting expression patterns can be found at http://www.janelia.org/split-gal4). In addition, the three DAN cell types, which express CsChrimson using the MB320C and MB099C drivers, have been shown to have similar spiking responses to CsChrimson activation. In three of six cases, we chose drivers expressing in a combination of two cell types because we found that activation of only a single DAN cell type did not produce a sufficiently robust memory (Figure 2—figure supplements 1,2). We confirmed that the lines used in our optogenetic experiments (Figure 2A) showed comparable memory formation when trained with electric shock or sugar reward (Figure 2—figure supplement 3). Together, the selected drivers innervate 11 of the 15 MB compartments. Below we describe the results obtained in a number of different learning assays by activating these split-GAL4 drivers.

10Longer and repetitive training has been shown to induce stronger and more persistent memory across animal phyla. Consistent with those observations, we found that a 10-s training generally induced memory less effectively than a 60-s training in our immediate memory assay (Figure 2B). We also found that the optimal temporal relationship of DAN activation and odor presentation for memory formation was similar, but not identical, for DANs innervating different MB compartments (Figure 2—figure supplement 4).

11Long-term aversive memory in flies requires repetitive electric shock conditioning with resting intervals, so-called spaced training. We found that two sets of DANs, PPL1-α3 alone (MB630B) or the combination of PPL1-γ2α′1 and PPL1-α′2α2 (MB099C) can induce 1-day and 4-day aversive memory after spaced training (Figure 2C), suggesting that the effects of spaced training can be implemented in individual compartments. Making memory formation dependent on repetitive training might be beneficial by allowing an animal to ignore spurious one-time events. Recent work has shown that the γ2α′1 compartments play key roles in both sleep regulation and long-term memory. The observation that co-activation of PPL1-γ2α′1 and other DANs synergistically prolongs memory retention raises the possibility that PPL1-γ2α′1 might act broadly to facilitate memory consolidation by promoting sleep after learning.

12We found that a particular DAN’s ability to induce the formation of immediate, 1-day and 4-day memories is not correlated. For example, immediate memory after a single pairing with activation of PPL1-α3 (MB630B) was barely detectable, although multiple activations resulted in 4-day memory (Figure 2C). In contrast, PPL1-γ1pedc (MB320C) activation resulted in robust immediate memory acquisition after a single round of training, but its activation failed to induce 4-day memory even after extensive spaced training (Figure 2C). These results imply the stability of memory is an intrinsic property of the MB compartment, rather than a consequence of the training protocol. In this view, repetitive training with naturalistic stimuli that activate many DAN cell types would recruit additional compartments with slower acquisition rates and the behaviorally assayed retention of memory would reflect the combined memories formed in different compartments (Figure 2—figure supplement 5). It remains an open question whether short-term memories are converted into long-term memories as biochemical changes in the same synapses or whether these memories are formed separately and in parallel. For olfactory learning in Drosophila, our data are consistent with a model in which memory formation and consolidation can occur independently and in parallel in individual MB compartments; this view does not exclude the possibility that network activity facilitates memory consolidation.

13We found that the memories induced in different compartments have different stabilities, displaying different dynamics of spontaneous memory decay over a 1-day period (Figure 3A–B). Memories in each compartment also differed in the extent to which they were reduced by a second presentation of the trained odor without reinforcement (Figure 3C).

14Likewise, DAN activation without odor presentation significantly reduced immediate memory (Figure 3D) for four of the five sets of DANs tested. These two effects might be mechanistically linked as odor presentation alone can result in activation of a subset of dopaminergic neurons.

15In both the case of presentation of odor without dopamine and of dopamine without odor, the association the fly had previously learned is not confirmed. It would make sense for a memory to be diminished when the contingency upon which it is based is found to be unreliable. Consistent with this idea, repetitive spaced training with these same DANs can induce 1-day memory that is resistant to DAN activation (Figure 3—figure supplement 1). The differences we observed between compartments suggest that they weigh the importance of the reliability of the correlation between CS and US differently.

16The α1 compartment differed from the other compartments we tested in that it was resistant to memory reduction by DAN activation (Figure 3D). This compartment plays a key role in long-term appetitive memory of nutritious foods and has an unusual circuit structure: its MBON (MBON-α1) appears to form synapses on the dendrites of the DAN that innervates the α1 compartment (PAM-α1) forming a recurrent circuit necessary for long-term memory formation. The α1 compartment also showed the least ability to replace an older association with a new one (Figure 3E). This observation suggests that the initial memory may not be affected by the second training, resulting in co-existing appetitive memories for both odors. Indeed, flies were able to retain associations between each of two odors and PAM-α1 (MB043C) activation, while only the most recently learned association was remembered with PPL1-γ1pedc (MB320C) activation (3F). The higher memory capacity of the α1 compartment is not due to generalization, since training with one odor pair did not affect the innate odor preference observed with a different, untrained odor pair. Thus two distinct strategies for updating memories appear to be used in different MB compartments: (1) writing a new memory, while diminishing the old memory; or (2) writing a new memory, while retaining the old memory.

17Our results suggest that memory formation in each compartment is largely parallel and independent, with compartmental specific rules for updating memories. Such a model of independent memory storage should allow appetitive and aversive memories to be simultaneously formed for the same odor in different compartments. We tested this idea by simultaneously activating DANs to α1 and γ1pedc while exposing flies to an odor (Figure 4A). When flies were tested immediately after training, the odor was strongly aversive, but the same odor became appetitive after 1 day. These results are most easily explained by simultaneous formation of an aversive memory in γ1pedc and an appetitive memory in α1, with rapid decay of the memory in γ1pedc and slow decay in α1 resulting in a shift in valence of the conditioned response over time. However, the fact that we observed strongly aversive immediate memory, rather than an intermediate response, suggests that the MB network non-linearly integrates these conflicting signals. The known feedforward connection between γ1pedc and α1 provides a possible circuit mechanism (Figure 5B; ). Recent studies provide further examples most easily explained by parallel induction of conflicting memories of different decay rates. We also found that wild type flies are capable of efficiently switching odor preference when they had conflicting sequential experiences of sugar reward followed by shock punishment with the same odor (Figure 4B).

18Our results demonstrate that different MB compartments use distinct rules for writing and updating memories of odors (Figure 5A). By analyzing individual memory components–or engrams–induced by local dopamine release, we found that the interpretation of a common odor representation carried by sparse KC activity to multiple compartments could be modified differently in each of those compartments. We do not know the mechanisms that generate these distinct learning rules. They could arise from differences in the dopamine release properties of different DAN cell types or from local differences in the biochemical response to dopamine signaling in each MB compartment. For example, KCs express four distinct dopamine receptors, which might be deployed differently in each compartment. Or they could originate from circuit properties: we know from anatomical, behavioral and functional imaging studies that MB compartments can communicate through connections between their extrinsic neurons, the DANs and MBONs, as well as by a layered network within the MB (Figure 5B). In the mammalian brain, associative memories are also stored as distributed and parallel changes with partially overlapping functions; for example, different populations of dopaminergic neurons develop representations of a visual objects’ value with distinct learning rules. We expect many of the underlying strategies and mechanisms may be shared between flies and other species. Our work provides a foundation for experiments aimed at understanding the molecular and circuit mechanisms by which distributed memory components are written with distinct rules and later integrated to guide memory-based behaviors.

19Crosses of split-GAL4 lines for DANs and 20xUAS-CsChrimson-mVenus in attP18 were kept on standard cornmeal food supplemented with retinal (0.2 mM all-trans-retinal prior to eclosion and then 0.4 mM) at 22°C at 60% relative humidity in the dark. Female flies were sorted on cold plates at least 1 d prior to the experiments and 4–10 d old flies were used for experiments. The new split-GAL4 driver, MB630B was designed based on confocal image databases (http://flweb.janelia.org), BrainBase (http://brainbase.imp.ac.at), and constructed by inserting VT026773-p65ADZp in attP40 and R72B05-ZpGAL4DBD in attP2 as described previously. VT045661-LexA was constructed as described previously and injected into JK22C. The confocal images of expression patterns are available online (http://www.janelia.org/split-gal4). For driving CsChrimson by both MB109B and MB315C, 20xUAS-CsChrimson-mVenus in attP18 was first combined with MB315C, and then crossed with MB109B.

20The olfactory arena for optogenetics experiments was designed based on the four-field olfactometer and was briefly described in previous reports. The overview of the assay is described in Figure 1—figure supplement 1 and a detailed description of the apparatus is provided in Supplementary file 1; the Janelia TechTransfer office (techtransfer@janelia.hhmi.org) will provide complete construction documentation and CAD files upon request.

21Groups of approximately 20 females of 4–10 d post-eclosion were trained and tested at 25°C at 50% relative humidity in a dark chamber. The flow rate of input air from each of the four arms was maintained at 100 mL/min throughout the experiments by mass-flow controllers, and air was extracted from the central hole at 400 mL/min. Odors were delivered to the arena by switching the direction of airflow to the tubes containing diluted odors using solenoid valves. The odors were diluted in paraffin oil (Sigma–Aldrich): 3-octanol (OCT; 1:1000; Merck) and 4-methylcyclohexanol (MCH; 1:750; Sigma–Aldrich), Pentyl acetate (PA: 1:5000; Sigma–Aldrich) and ethyl lactate (EL: 1:5000; Sigma–Aldrich). Shock and sugar conditioning was performed as previously described by using tubes with sucrose absorbed Whatman 3 MM paper or copper grids (Figure 2—figure supplement 3). For the experiments in Figure 4B, a sheet of copper grid was placed at the bottom of arena. For appetitive memory assays, flies were starved for 24–48 hr on 1% agar. Videography was performed at 30 frames per second and analyzed using Fiji. Statistical comparisons were performed (Prism; Graphpad Inc, La Jolla, CA 92037) using the Kruskal Wallis test followed by Dunn's post-test for multiple comparison, except those in Figure 1F, Figure 2C and Figure 2—figure supplement 4 which used Wilcoxon signed-rank test with Bonferroni correction to compare from zero.