Reward signal in a recurrent circuit drives appetitive long-term memory formation

1Environmentally relevant information, such as poison or food, can trigger formation of long-term memory (LTM) in a single association event. Because LTM formation is energetically costly, it is advantageous for animals to refer to surrounding conditions when selecting what information to retain as stable LTM. Appetitive olfactory learning in Drosophila melanogaster provides an opportunity to uncover the mechanisms underlying such integration of information, as a single presentation of nutritious sugar and an odor drives robust short-term memory (STM) and LTM, while non-nutritious sugar induces only STM.

2In Drosophila, sugar reward is mediated by a group of dopamine neurons called the protocerebral anterior medial (PAM) cluster. Artificial activation of the PAM cluster neurons in the presence of odor drives robust appetitive STM and LTM formation. Recent studies subdivided the PAM cluster neurons into STM- and LTM-inducing cell types and identified several cell types responsible for LTM formation, including PAM-α1. Interestingly, the induced LTM is undetectable immediately after conditioning but gradually develops with time.

3Dopamine reward signals are conveyed by PAM cluster neurons to a brain structure called mushroom body (MB), where the convergence of the reward and the odor is assumed to take place. The MB represents odors using a sparse subset of ∼2000 intrinsic neurons, Kenyon cells (KCs). KCs send parallel fibers to form lobe structures, where multiple MB-output neuron (MBON) types elaborate spatially segregated dendritic arbors. The KC-MBON synapses are presumed to be modified in the course of memory formation by the reinforcing dopamine neurons.

4While there has been progress on the molecular and cellular mechanisms underlying LTM formation resulting from repetitive conditioning, how a single presentation of a reward can trigger LTM remains to be understood. Here, we identify and behaviorally characterize a recurrent reward circuit that is essential for appetitive LTM formation and consolidation. The recurrent circuit consists of PAM-α1 dopamine neurons and a single MBON type, MBON-α1. This recurrent dopamine reward circuit may provide insights into a general mechanism for LTM formation.

5PAM-α1 delivers reward signals essential for appetitive LTM formation. To understand the neuronal circuit that regulates rewards for LTM formation, we first sought to anatomically identify the input neurons to PAM-α1. The presynaptic terminals of PAM-α1 are localized to the basal compartment of the MB α lobe (α1), whereas the dendrites stretch horizontally across the superior medial, intermediate, and lateral protocerebra (SMP, SIP, and SLP, respectively) (Figure 1A,B,H). By segmenting the dendrites of PAM-α1 in confocal images, we queried a database of GAL4 expression patterns to computationally search for candidate upstream neurons. Strikingly, one of the input candidates we identified was the output neurons from the α1 compartment of the MB (MBON-α1) (Figure 1C,D). Transgenic expression of a presynaptic marker by MB310C-GAL4 revealed that the presynaptic terminals of MBON-α1 lie in close apposition to the dendrites of PAM-α1 (Figure 1E,I). The terminals of MBON-α1 were immunoreactive for the vesicular glutamate transporter (Figure 1F) but not for markers for GABAergic or cholinergic neurons (glutamic acid decarboxylase or choline acetyltransferase, respectively) (Figure 1—figure supplement 1), consistent with MBON-α1 being glutamatergic. Differential labeling of the individual MBON-α1 neurons in the same brain revealed that MBON-α1 comprises two cells with very similar morphology (Figure 1G).

6To precisely determine the relative arrangement of PAM-α1 and MBON-α1 neurites, we performed a double-labeling experiment using R72D01-LexA, which labels PAM-α1, and the split-GAL4 driver MB310C-GAL4, which labels MBON-α1. Confocal images revealed that the processes of MBON-α1 and PAM-α1 substantially intermingled both in the SIP and SLP, and in the α1 compartment of the MB—the input and output sites of PAM-α1, respectively (Figure 2A–E, Figure 2—figure supplement 1, Video 1). To observe these processes in SIP and SLP in even greater detail, we employed a microscope with a super resolution detection system and found bouton-like structures of MBON-α1 attached to the dendrites of PAM-α1 (Figure 2B–D). Furthermore, a GFP reconstitution across synaptic partners (GRASP) experiment using R72D01-LexA and MB310C-GAL4 revealed strong GRASP signals in SIP, SLP, and the α1 compartment (Figure 2F). This further supports the close juxtaposition between PAM-α1 and MBON-α1.

7If the MB feedback modulates PAM-α1, the output of MBON-α1 might be required for the acquisition of LTM of the sugar reward. We examined the requirement of PAM-α1 and MBON-α1 by expressing a temperature-sensitive dominant negative dynamin, Shibirets1 using the split-GAL4 drivers MB299B-GAL4 and MB310C-GAL4 (Figure 3B,F). MB299B-GAL4 has strong expression in PAM-α1 (Figure 3B). Consistent with our previous results, the blockade of PAM-α1 during conditioning did not significantly affect STM (Figure 3C), but led to a severe impairment of sucrose-rewarded LTM (Figure 3D). The unimpaired performance in appetitive STM demonstrates that flies of these genotypes have normal sugar and odor perception at the restrictive temperature. Intriguingly, we found a similar preferential impairment of LTM upon the blockade of MBON-α1, using MB310C-GAL4 driver (Figure 3H). The same blockade did not significantly impair STM (Figure 3G). We confirmed the preferential requirement of MBON-α1 for LTM using another driver, MB323B-GAL4 (Figure 3—figure supplement 1A), and obtained similar results (Figure 3G,H); neither MB310C/UAS-shi nor MB323B/UAS-shi flies showed a significant LTM impairment when trained and tested at the permissive temperature (Figure 3—figure supplement 1B).

8We next examined whether glutamatergic input to PAM-α1 is required for LTM formation by down-regulating the expression of glutamate receptors using MB299B-GAL4. For efficient and cell-type-specific manipulation, we used UAS-RNAi fly strains based on the shRNA technique. Intriguingly, knocking down either dNR-1 or dNR-2, which together form the functional NMDA receptor, impaired LTM (Figure 4C) but not STM (Figure 4B). This preferential requirement in LTM is similar to the blockade of MBON-α1 and PAM-α1, suggesting the direct glutamatergic feedback.

9Given that thermogenetic stimulation of PAM-α1 with a temperature-sensitive cation channel dTrpA1 can provide the reward signal for LTM formation, we asked if activation of MBON-α1 would also be capable of inducing appetitive LTM (Figure 5A). Indeed, we found appetitive LTM upon thermo-activation of MBON-α1 in the presence of an odor (Figure 5B). These results, together with the consistent requirements of MBON-α1 and NMDAR-signaling in PAM-α1, are most easily explained if the glutamatergic feedback regulation from MBON-α1 augments dopamine reward signaling by PAM-α1.

10Several lines of evidence suggest that D1-like dopamine receptor (DopR1) is required in KCs for appetitive memory. However, previous studies focused on only STM. To identify the receiving site of dopamine for LTM formation, we knocked down DopR1, either in α/β KCs or in MBON-α1, the potential targets of dopamine from PAM-α1, and measured appetitive LTM. To target the expression specifically to the α/β KCs, we employed two driver lines, MB008B-GAL4 (Figure 6—figure supplement 1A) and c739-GAL4. Interestingly, knocking down DopR1 in the α/β KCs, using c739-GAL4 or MB008B-GAL4, resulted in a severe impairment of appetitive LTM (Figure 6A) while leaving STM unaffected (Figure 6—figure supplement 1B). These results are consistent with the observation that the coincidence detector adenylate cyclase encoded by rutabaga, thought to act downstream of the dopamine receptor, functions for LTM in the α/β KCs. On the other hand, knocking down DopR1 in MBON-α1 did not significantly impair either appetitive LTM or STM (Figure 6A and Figure 6—figure supplement 1B).

11To further localize the role of DopR1 in KCs, we performed a rescue experiment, using a piggyBac insertion mutant, dumb2. dumb2 mutant flies showed severely impaired appetitive LTM. Targeting wild-type DopR1 gene expression in KCs fully rescued the defects (Figure 6B). Together, although we cannot exclude the possible role of other dopamine receptors in MBON-α1, we propose that DopR1-mediated memory trace is formed in α/β KCs not in MBON-α1.

12Previous studies have highlighted the role of the output from α/β KCs for memory retrieval. Given the requirement of MBON-α1 for the acquisition of appetitive LTM, the output of the α/β KCs during conditioning may be important for LTM formation. To test this possibility, we blocked the output of the α/β KCs with Shits1. Strikingly, the blockade of the α/β KCs during conditioning significantly impaired appetitive LTM, but not STM (Figure 7A,B), similar to the blockade of PAM-α1 and MBON-α1 (Figure 3). When the flies were trained and tested at the permissive temperature, no significant impairment of appetitive LTM was observed (Figure 7—figure supplement 1). The requirement for α/β KC synaptic signaling during LTM formation supports our model that MBON-α1 provides information about the memory trace, formed in the α/β KCs, to the reward signaling PAM-α1 neurons.

13If the nascent memory trace in α/β KCs is communicated by MBON-α1 during conditioning, MBON-α1 might also convey the read-out of LTM. Intriguingly, when MBON-α1 was blocked only during the test phase of LTM, we found a severe impairment of conditioned odor approach (Figure 8A). In contrast, the blockade of PAM-α1 during LTM retrieval did not have a significant effect (Figure 8B). These results imply that MBON-α1 plays a dual role in both formation and retrieval of LTM; different post-synaptic targets of MBON-α1 would be used in these two roles, with the recurrent PAM-α1-MBON-α1 circuit selectively involved in LTM formation.

14The glutamatergic input from MBON-α1 might provide an excitatory feedback on the PAM-α1 neurons via the NMDA receptor thereby prolonging the dopamine release after the cessation of sugar presentation. If this model were correct, the post-training blockade of the feedback circuit might be expected to impair appetitive LTM. Consistent with this idea, blocking any component of the feedback circuit for 1 hr immediately following conditioning resulted in a significant impairment of appetitive LTM (Figure 9A,C,E), whereas the same blockade 22 hr after training did not (Figure 9B,D,F). These results strongly suggest that the recurrent circuit has an ongoing activity, which plays a key role in early phase of appetitive LTM consolidation.

15Prolonged dopamine release in the absence of odor may also modulate KCs that do not respond to the rewarded odor. If so, the odor specificity of appetitive LTM might be compromised compared to STM. Thus, we measured the generalization profile of STM and LTM by testing with the graded mixtures of ‘contaminating’ odors (Figure 10). Odor mixtures were created by systematically varying the relative contents of the contaminant (trained odor: contaminant = 100:0, 80:20, 60:40, 40:60, 20:80, or 0:100): if the memory is specific to the trained odor, the performance quickly decreases with the increasing contaminant ratio. Strikingly, the performance decline is shallower in LTM compared to STM (Figure 10C,D). We confirmed these results with another set of odorants, suggesting that compromised stimulus specificity in LTM is general (Figure 10—figure supplement 1). Considering that the only difference between these groups is the retention time, there may be a trade-off between consolidation and specificity, possibly due to the prolonged dopamine release in the MB following memory acquisition.

16Prior studies on the role of MBONs in learning and memory have focused on memory retrieval. Here, we extend the known role of MBONs and show that MB output through MBON-α1 is required not only for the retrieval, but also for the acquisition and early consolidation of long-term appetitive olfactory memory. This unanticipated aspect of the MB output for LTM formation appears to function through a recurrent loop (Figure 11): MBON-α1 dendrites and the terminals of PAM-α1 occupy the same MB compartment; the DopR1 dopamine receptor is required in the KCs of that compartment (Figure 6); MBON-α1 in turn sends a portion of its outputs to the dendrites of PAM-α1 (Figures 1, 2) closing the loop; synaptic signaling by all three cell types—α/β KCs, MBON-α1, and PAM-α1—is required during LTM formation and consolidation (Figures 3, 7, 9). These findings strongly support our model that feedback signals from α/β KCs, mediated by MBON-α1, play a critical role in controlling the rewarding dopamine signals from PAM-α1 (Figure 11). Inhibitory recurrent circuits in the MBs have been implicated in appetitive learning of honeybees. Our present study reveals a new circuit motif, a feedback circuit that is required for dopamine neurons to provide reinforcement during appetitive LTM formation.

17Dopamine has been thought to simply relay rewarding or punitive stimuli. However, our results call into question this prevailing circuit model. More specifically, PAM-α1 might integrate simultaneous inputs from sugar perception and the nascent memory trace to augment and sustain the dopamine release to gate LTM formation (Figure 11). The requirement of the NMDA receptor in PAM-α1 (Figure 4) further supports this model. Such integration by PAM-α1 might enhance the reward gain selectively in relevant sensory environments. In other words, the reinforcing signal for LTM formation may depend on the presence of other sensory stimuli (e.g., odor). As LTM formation is energetically costly, this gain control of the reinforcing signal might serve to restrict LTM formation to ecologically relevant situations. Although the requirement of NMDA receptor in PAM-α1 implies that MBON-α1 provides excitatory input, the synaptic connectivity between MBON-α1 and PAM-α1 awaits physiological demonstration, given functional inhibitory glutamate receptor signaling.

18In addition to the role in memory acquisition, the PAM-α1 output is required during consolidation (Figure 9A,B). The dopamine release is likely to be sustained by the ongoing activity of the recurrent circuit (Figure 9C–F). A recent study reported a consistent finding that the output from α/β KCs during consolidation has an essential role for appetitive and aversive LTM in addition to their role in retrieval. Recent studies revealed a gradual development of appetitive LTM after conditioning. Ongoing activity in the recurrent loop we propose here may provide a circuit mechanism for the gradual LTM development. In addition, previous studies reported that LTM consolidation in Drosophila or in rats requires post-training dopamine inputs.

19Our generalization experiments show that appetitive LTM is less specific to the trained odor, compared to STM (Figure 10). This result fits well into previous studies in mice and bees, demonstrating that long-lasting memories are more generalized. More generalizable LTM makes sense from an ecological viewpoint: it is worth taking a chance for survival even if the smell is not exactly the same as learned.

20In the fly brain, distinct types of dopamine neurons signal reward for STM and LTM, and these memories are independently formed in different MB lobes. Octopamine has been widely accepted as a positive reinforcement signal in insects. In Drosophila, it is considered to provide inputs to dopamine neurons that induce STM formation. Recent studies in mammals reached a similar conclusion that memories with different temporal dynamics are mutually conflicting and formed in distinct areas, to which different classes of dopamine neurons project. Independent processing of STM and LTM might be a conserved strategy among animals.

21In addition to the function in the recurrent reward circuit, MBON-α1 mediates retrieval of the LTM trace in the MB (Figure 8). MBON-α1 samples from the MB-α1 and terminates in the SIP and the SLP. Another type of MBONs also mediates appetitive LTM retrieval: MBON-α3 (also known as MB-V3). In contrast to MBON-α1, blocking MBON-α3 during conditioning does not impair appetitive LTM acquisition. MBON-α3 arborizes dendrites in the tip of the α lobe and projects to the SMP and the SIP where the terminals of MBON-α1 and MBON-α3 are intermingled. Therefore, appetitive LTM formed in the α/β KCs is read out through at least two different classes of MBONs. These MBONs converge in the SIP and might activate shared downstream targets to guide conditioned odor approach.

22The recent comprehensive connectivity map of the afferent and efferent neurons of the Drosophila MB revealed many potential feedback connections outside the MB. Considering the central role of the MB in sensory integration and memory-guided behavior, recurrent computation between learned information and reinforcement signals might be a fundamental principle of memory processing.

23In both vertebrates and invertebrates, the activity of reinforcement neurons is modified upon associative learning. Our results delineated such a feedback circuit, which was predicted by a physiological study in Drosophila. In this circuit, MBON-α1 mediates monosynaptic feedback to the dopamine neurons. In contrast, there is no monosynaptic feedback to the dopamine neurons found in mammals, rather anatomical studies suggest such a feedback to be polysynaptic. Further comparative studies should assess how the recurrent circuit motif of reinforcement is implemented in different animal brains.

24Canton-S was used as a wild-type strain. Generation and basic characterization of the split-GAL4 drivers (MB008B-GAL4, MB299B-GAL4, MB310C-GAL4, and MB323B-GAL4) are described in. R72D01-LexA was constructed using the methods described in and inserted into attP40. c739-GAL4 is an enhancer-trap GAL4 driver described in. Flies were raised at 60% relative humidity at 24°C. Flies for the NMDAR knock down (Figure 4) were incubated at 30°C for 4–5 days prior to the experiment to facilitate the GAL4 activity.

25The synaptic blockade experiments (Figures 3, 7-9) used F1 progeny of crosses between females of w;;UAS-shits1 (single copy of UAS-shits1 from Thomas Preat laboratory, CNRS, France) or w and males of the GAL4 drivers. The thermoactivation experiment (Figure 5) used F1 progeny of crosses between females of w; UAS-dTrpA1 or w and males of MB310C-GAL4 or w. The receptor knock down experiments (Figures 4, 6) used F1 progeny of crosses between females of UAS-Nmdar1RNAi (P(TRiP.HMS02200) attP40; Bloomington stock center #41667), UAS-Nmdar2RNAi (P(TRiP.HMS02012) attP2; Bloomington stock center #40846), UAS-DopR1RNAi (P(TRiP.HMC02344) attP2; Bloomington stock center #55239) or CS and males of the GAL4 drivers. The dumb2 rescue experiments (Figure 6B) used F1 progeny of crosses between females of UAS-DopR1;;dumb2 or CS and males of MB247-GAL4; dumb2 or dumb2. CS flies were used as wild type (Figures 6B, 10).

26For immunohistochemistry, the following reporter strains were crossed to the respective GAL4 or LexA drivers: pJFRC2-10xUAS-mCD8GFP in VK00005 (Figure 1A–D), pJFRC200-10XUAS-IVS-myr::smGFP-HA in attP18 (Figure 2A–D; Figure 2—figure supplement 1; Figure 3B,F; Figure 6—figure supplement 1A) and pJFRC216-13XLexAop2-IVS-myr::smGFP-V5 in su(Hw)attP8 (Figure 2A-E; Figure 2—figure supplement 1), pJFRC206-5xUAS-IVS-myr-smGFP-FLAG in VK00005 (Figure 1E,F; Figure 1—figure supplement 1; Figure 3—figure supplement 1A), pJFRC51-3xUAS-Syt-smHA in su(Hw)attP1 (presynaptic marker; Figure 1E), UAS-CD4::spGFP1-10 and LexAop-CD4::spGFP11 (Figure 2F).

27pJFRC2 is described in. pJFRC200 and pJFRC206 are described in and pJFRC216 in. pJFRC51 was generated by standard methods described in. UAS-CD4::spGFP1-10 and LexAop-CD4::spGFP11 are described in.

28Multicolor flp-out (MCFO; Figure 1G) is a stochastic method that labels individual cells in different colors using a set of three UAS-STOP-epitope constructs that each expresses a different epitope when the STOP cassette is removed. The STOP cassettes in these constructs are each flanked by FRT sites that are removed in a stochastic way by limited expression of flp recombinase. Reagents for MCFO are described in.

29Dissection and immunohistochemistry of fly brains were done as previously described. The following antibodies were used: rabbit-anti-GFP (1:1000; A11122; Invitrogen), mouse-anti-GFP (1:200; UC Davis/NIH Neuromab Facility; clone N86/38; for GRASP), mouse anti-nc82 (1:33.3; Developmental Studies Hybridoma Bank, Univ. Iowa, Iowa City, IA, United States), rat-anti-nCad (1:200; DN-Ex #8; Developmental Studies Hybridoma Bank, University of Iowa), rabbit anti-HA (1:300; Cell Signaling Technology, MA, United States), rat anti-FLAG (1:200; Novus Biologicals, Littleton, CO, United States), mouse anti-Drosophila ChAT (ChAT4B1; 1: 100; Developmental Studies Hybridoma Bank, Univ. Iowa), rabbit anti-Drosophila GAD1 (1:1000; a gift from Dr FR Jackson), rabbit anti-DvGluT (1:5000; a gift from Dr A DiAntonio) as primary antibodies, and cross-adsorbed secondary antibodies to IgG (H+L): AlexaFluor-488 donkey anti-mouse (1:400; Jackson Labs), Cy3 donkey anti-rabbit (1:500; Jackson Labs), AlexaFluor-647 donkey anti-rat (1:300; Jackson Labs), AlexaFluor-488 goat anti-rabbit (1:800; A11034; Invitrogen, Carlsbad, CA, United States), and AlexaFluor-568 goat anti-mouse (1:400; A11031; Invitrogen).

30Confocal images were obtained by using LSM710 (Zeiss, Oberkochen, Germany) (Figures 1A–F, 2A,E, 3B,F, Figure 1—figure supplement 1, Figure 2—figure supplement 1, Figure 3—figure supplement 1, Figure 6—figure supplement 1) and FV1200 (Olympus, Tokyo, Japan) (Figure 2F) and processed by using ImageJ. The magnified images in SIP and SLP (Figure 2B–D) were obtained by using LSM880 (Zeiss) with Airyscan and were deconvoluted using the Fiji plugin provided by Bob Dougherty (http://fiji.sc/Deconvolution). Airyscan is equipped with 32 GaAsP detectors that collect light from one illumination spot. It can resolve 140 nm (lateral) or 400 nm (axial) at 488-nm wavelength. For the 3D rendering video, confocal images from LSM710 were deconvoluted using the Fiji plugin and then were subjected to 3D image rendering software (Volocity: from PerkinElmer, Waltham, MA, United States).

31The conditioning and testing protocol was as described previously with minor modifications. Briefly, a group of approximately 50 flies in a training tube alternately received octan-3-ol (3-OCT; Merck) and 4-methylcyclohexanol (4-MCH; Sigma–Aldrich, St. Louis, MO, United States) for 1 min in a constant air stream with or without dried sucrose paper. 3-OCT and 4-MCH were diluted 1% and 2%, respectively, in paraffin oil (Sigma–Aldrich), placed in a cup with a diameter of 14 mm and presented to flies. For thermoactivation experiments (Figure 5), 3-OCT and 4-MCH were diluted 10% in paraffin oil, placed in a cup with a diameter of 3 mm and 5, respectively, and presented to flies. For the generalization experiments (Figure 10), 1% 1-octen-3-ol (Sigma–Aldrich) and 2% 2-methylcyclohexanol (Sigma–Aldrich) were also utilized. These diluted odors were mixed with the respective ratios. The restrictive temperature for the experiments with UAS-shits1 was 32°C and the permissive temperature was 24°C, measured with Venta-Hygrometer (Venta Luftwäscher GmbH, Weingarten, Germany). At the test, the trained flies were allowed to choose between MCH and OCT for 2 min in a custom-made transparent Plexiglas T-maze, illuminated by infrared LEDs (Osram 720-SFH487P, Mouser electronics, Mansfield, TX, United States), and the walls were covered with Fluon (Insect-a-Slip, PTFE30, BioQuip Products, Inc., Rancho Dominguez, CA, United States) and the distribution of flies was imaged by cameras (FFMV-03M2M, Point Grey, Richmond, Canada). Captured images were sent to the computer and the fly number in each tube was counted by a custom-made ImageJ macro. The preference index was calculated by taking the mean indices of the last 60 s in the 2-min choice and the learning index was calculated by taking the mean of preference indices of the two reciprocally trained groups. Half of the trained groups received reinforcement together with the first presented odor and the other half with the second odor to cancel the effect of the order of reinforcement.

32For thermoactivation with dTrpA1 (Figure 5), flies were trained by being transferred from a background temperature of 24°C to a prewarmed tube in a climate box (30°C) and presented with the training odorant for 1 min.

33Flies were aged 3–10 days after eclosion and were starved so that the mortality rate reaches 5–15% at the test. Between the training and the test, flies were kept without food. Conditioning was performed in dim red light and testing was in darkness.

34Statistical analyses were performed with StatPlus and Prism5 (GraphPad, La Jolla, CA, United States). Data did not violate the assumption of normal distribution (Shapiro–Wilk test) and homogeneity of variance (Bartlett test). Therefore, the data were analyzed with parametric statistics: one-sample t-test or one-way analysis of variance followed by the planned pairwise multiple comparisons (Bonferroni). The significance level of statistical tests was set to 0.05.