Dopamine modulation of learning and memory in the prefrontal cortex: insights from studies in primates, rodents, and birds

1A major function of executive control is the flexible adaptation to our ever-changing environment. The executive circuits of the brain must, therefore, not only monitor and maintain current behavioral goals but also incorporate new goals and rules. This updating can come in the form of a quick integration of previously acquired knowledge when, for example, a well-known stimulus informs an animal of a change in reward contingencies. In many cases, however, such updating requires new learning, for example when a new stimulus is encountered for the first time. Executive functions are commonly ascribed to the prefrontal cortex (PFC) and frontostriatal networks. The function of these circuits relies heavily on neuromodulation, in particular on dopamine (DA). The aim of this review is to outline the contribution of DA and its receptors in the PFC to learning and memory processes across different species.

2We will first introduce studies in the mammalian brain in the sections on humans, non-human primates, and rodents. Due to the challenges of investigating the role of DA transmission in human PFC, we focus the human section on studies utilizing systemic injections of DA agents and impairments of DA transmission in patients with a variety of neurological and psychiatric disorders. The non-human primate and rodent sections review behavioral studies conducted during local manipulations of the DA system in the PFC. While the dopaminergic system in different mammalian species follows largely the same organization, some conceptual and terminological differences can make a comparison of data across species difficult (Box 1). For a comparative perspective, we will then outline behavioral studies conducted in birds where local manipulations of the DA system were implemented in a structure equivalent to the mammal PFC, the nidopallium caudolaterale (NCL; ). Such a comparison is of particular interest given the large evolutionary gap between these species. The lines of birds and mammals separated around 300 million years ago, long before many of the cognitive functions attributed to the PFC evolved. In spite of this distance, birds and mammals (with the exception of humans and apes) are largely on par when it comes to cognitive abilities. This implies a parallel or convergent evolution of cognition between the species. As a result of this independent evolution, we see stark differences in brain organization between birds and mammals. Most notably, the avian telencephalon does not show the laminar organization of the mammalian cortex. However, other organizational principles were preserved or evolved independently in both lines. This can be taken as a hint of narrow neurobiological constraints in the evolution of a given cognitive ability.

3When comparing the function of prefrontal DA across species it is important to clarify the terminology used in the different fields of research. As reviewed here, prefrontal DA plays an important role in learning and memory and an extensive body of literature is concerned with its role particularly in working memory (WM). In general, the term WM is strongly associated with its original definition by ), who famously proposed that systems for sensory storage (phonological loop, visuospatial sketchpad, and more recently, an episodic buffer) are governed by a central executive. The gist of this definition is that an interconnected neural system allows the brief storage of information and, importantly, its manipulation.

4In primates, a seminal contribution to the understanding of this system was the discovery of of “delay cells” in the PFC. These neurons show increased activity during the delay period of WM tasks maintaining the memory of a stimulus. Consequently, in primates including humans, WM is often modeled as “active memory”, a system that holds information in memory by sustaining neural activity for a few critical seconds.

5Research in rodents commonly uses a broader definition of WM, that refers to “a collection of processes that include the temporary storage of information, as well as executive functions that mediate the manipulation and retrieval of trial-unique information to guide action after both short (seconds) and longer (minutes to hours) delays” (; see also:; ). Importantly, this definition includes a much larger range of delays (seconds to many hours) compared to what is typically used in humans and non-human primates (seconds). Consequently, in rodents, the definition of WM does not necessarily refer to active memory maintenance by delay cells but might rely on different mechanisms that could be classified as learning mechanisms in primates. Thus, it is important to pay attention to the specific paradigms and definitions used when comparing results across species.

6The definition of WM typically used in avian research was developed in parallel to the definition in humans. Both concepts are largely comparable with the exception that no phonological loop is conceptualized in birds. The delay durations in avian research are largely comparable to those in the primate literature and active information maintenance by delay activity is generally assumed to be the key mechanism of WM.

7Taken together, there are fundamental terminological differences between species and it is important to keep these in mind when comparing results across species. In particular, the vast differences in delay duration used in different paradigms could potentially engage distinct neural mechanisms – what is called WM in one species might be viewed as a learning mechanism in another.

8The anatomy of the dopaminergic system is very similar between all mammals and birds (for extensive review, see,; ). DA neurons can be identified by the expression of several catecholamine-synthesizing enzymes, tyrosine hydroxylase (TH), aromatic amino acid decarboxylase (AADC), and dopamine-b-hydroxylase (DBH). With modern immunohistochemical techniques it has been possible to map out in detail the location of DA neurons and their specific projections. DA neurons originate in several neighboring midbrain nuclei, being the substantia nigra pars compacta (SNc; A9) and the ventral tegmental area (VTA; A10) the ones projecting to the forebrain. The total number of TH-positive cells in VTA and SNc (bilateral count) is ∼20.000–30.000 in mice and ∼40.000–45.000 in rats. This number increases considerably in primates, 160.000–320.000 in monkeys and 400.000–600.000 in young humans. DA neurons send afferents to many target areas, including the several regions of the frontal cortex, with the striatum being the most densely innervated target (; Figure 1). PFC-projecting DA neurons are intermingled in VTA and SNc both in primates and in rodents. However, the PFC in primates is much more extensively innervated by midbrain DA afferents than in rodents.

9Postsynaptically, DA exerts its actions within the PFC/NCL via receptors grouped in two major families, D1-like receptors (D1 and D5 in mammals; D1A and D1B in birds) and D2-like receptors (D2, D3, and D4 in mammals and birds), but D1-like receptors are expressed to a greater extent than D2-like receptors. In birds, the D1-like family is extended to include an additional receptor (D1D;; ). Both families are G-protein-coupled receptors that exert slow changes of activity in the cells and act as functional neuromodulators. D1-like receptors show low affinity for DA, whereas D2-like receptors show higher affinity. For the sake of clarity, we will abbreviate D1-like and D2-like receptors as D1R and D2R, respectively, and will point to a specific receptor subtype whenever necessary.

10Interestingly, dopaminergic signaling in the PFC depends on brain maturation and the PFC is the brain structure that matures last. Analyses of human postmortem brain tissue reveal that the levels of mRNA expression of the D2R and D5R subtypes in PFC are highest in neonates and infants and decrease with age, whereas the D1R subtype mRNA expression and protein levels increase with age and are highest in adulthood. By contrast, both in rats and non-human primates, densities of the D1R and D2R subtypes peak during adolescence and decrease in adulthood. In songbirds, D1R and D2R subtypes in the song nuclei increase with age and peak during adolescence. The developmental patterns of related brain regions in non-songbirds are still unclear.

11“Classic” DA neurons show phasic activations (short duration bursts of action potentials) following unpredicted reward coding a quantitative “prediction error” signal, namely the difference between received and predicted reward value. A reward that is better than predicted elicits an activation (positive prediction error response), a fully predicted reward draws no response, and a reward that is worse than predicted induces a decrease in activity (negative error response;;, ). These prediction error responses of DA cells have been closely related to reinforcement learning models which assign a functional role of DA in modulating cortico-striatal inputs through a reward-prediction error teaching signal. In fact, fast DA release consistent with these reward predicting signals of DA neurons has been measured in nucleus accumbens during associative learning. Besides “classic” reward-prediction error responses, phasic DA cell firing patterns also include responses to salient and aversive sensory stimuli.

12Dopamine neurons also exhibit tonic firing driven by pacemaker-like membrane currents. The functional relevance of this tonic DA release is unknown. Transient suppression of tonic spiking in DA neurons follows the omission of expected reward, somehow implicating this spiking pattern in reward-based learning. Recent work has shown that DA release in the striatum increases gradually (ramps up) as rats expect distant reward, perhaps providing motivational drive. However, these types of signals have not been described in PFC.

13Which of these DA signals reaches the PFC remains currently unclear. While phasic DA prediction error signals could be used as a signal to transiently boost working memory (WM) of the corresponding stimuli, it has also been argued that mostly slower, tonic DA signals are relevant in PFC. Moreover, the phasic components of DA cell firing might be transmitted via co-release of glutamate. For computational models of DA function in PFC this has two main consequences. Firstly, the timescales of tonic DA would constrain functional roles to rather general cognitive states such as arousal or attention. Secondly, DA function in PFC circuits should be carefully contrasted with known features of the putatively fast, phasic, signals of the nigrostriatal system.

14In general, heterogeneity among DA cells points to additional functional aspects that are not covered by classic reinforcement learning descriptions. While functional roles of VTA and SNc neurons share common properties, overall evidence for different functional groups among DA cells has been emerging. Moreover, the heterogeneity in DA cell activity patterns is probably related to heterogeneity in the anatomical pathways; DA neurons contribute to reward or aversion depending on whether they are activated from the laterodorsal tegmentum or the lateral habenula, respectively. For these reasons, it has been difficult to dissociate the behavioral correlates of DA release between the projection pathways to the striatum and PFC.

15Investigating the direct role of DA signaling in human PFC during learning and memory brings quite a few challenges and, consequently, only few studies address this question. DA receptor agonists and antagonists cannot be injected locally, restricted to the PFC, and have to be administered systemically in humans. Our knowledge about the role of DA transmission in the human PFC, therefore, comes from studies combining imaging of the brain with other manipulations such as systemic pharmacology or transcranial magnetic stimulation, genetic profiling, and from work in patients with neurological and psychiatric disorders.

16For instance, a recent fMRI study has revealed a connection between context dependent WM and dopaminergic signaling in human PFC. The authors first identified by fMRI that the dorsolateral PFC was involved in the encoding of the context. Selective disruption of activity in this region with transcranial magnetic stimulation adversely impacted performance of the participants, causally implicating PFC in context encoding. PFC activity during the task was then found to correlate with phasic responses in the VTA and SNc. Based on these results, the authors suggest that phasic DA signals regulate the encoding and updating of context representations in the PFC.

17In the 1970s, it was postulated that hypofrontality (i.e., decreased blood flow in the PFC) underlies mental disorders and impaired cognitive function. In the context of schizophrenia, it was proposed that an excess of DA in the mesolimbic system causes the positive symptoms via hyperstimulation of D2R in the basal ganglia, whereas the cognitive and negative symptoms follow insufficient D1R activation in the frontal cortex. We now know that DA hypofrontality by itself cannot fully explain schizophrenia or other complex mental disorders. Impairments in PFC dopaminergic signaling and genetic profiling in these patients, however, have provided valuable information about the role of PFC DA in learning and memory. For example, schizophrenia patients exhibit imbalances in PFC dopaminergic signaling as determined by imaging approaches, and show deficits in learning and WM that correlate with genetic variations in DA related genes. In Parkinson’s disease (PD) patients, degeneration of neurons in the SNc results in decreased phasic and tonic PFC DA levels, which could explain the cognitive impairments present along with the motor deficits. A more direct involvement of DA in PFC-dependent memory processes was established in PD patients with and without DA medication. In a spatial WM task, subjects had to find tokens in boxes presented on a screen. Subjects that were off the DA precursor levodopa (L-DOPA) made more errors (checking boxes that had already been opened) compared to when they had received L-DOPA, indicating that DA is required for proper spatial WM performance. Surprisingly, visual learning and memory was not affected by L-DOPA in this task. Similarly, L-DOPA withdrawal did not affect the performance of PD patients in an N-back task, where WM is assessed when subjects are presented with a series of stimuli and have to indicate when a stimulus is the same as the one n steps back. However, in PD patients undergoing deep brain stimulation surgery, microstimulation of the SN disrupts reinforcement learning in a two-alternative probability learning task. Furthermore, research conducted in attention deficit hyperactivity disorder (ADHD) patients, who also display learning and memory deficits, have also provided some insight into the role of DA in learning and memory. In these patients, the size of the PFC is reduced, and genes involved in dopaminergic pathways are altered. Taken together, the results from work in schizophrenia, PD, and ADHD patients point to an abnormal DA transmission as being responsible for behavioral deficiencies in some learning and memory tasks that depend heavily on PFC function.

18Genetic studies have also provided valuable insight into the contribution of the DA system in learning and memory. Individuals with the Val/Val catechol-O-methyltransferase (COMT, enzyme that deactivates catecholamines) polymorphism [Val(108/158)Met] exhibit higher COMT activity that correlates with lower DA levels in the PFC, and have a slightly higher risk of developing schizophrenia. Moreover, Val/Val carriers perform worse in the Wisconsin card sorting test (WCST) compared to carriers of the Met allele. The WCST consists of a battery of cognitive tasks that include WM, sensitivity to reinforcement, and behavioral flexibility. In addition, brain imaging studies indicate that Val/Val carriers need greater PFC activity to perform WM tasks. Stress may be another factor that should be taken into consideration. Healthy human subjects under stress perform poorly in WM tasks and exhibit exacerbated levels of PFC DA measured by positron emission tomography (PET; ). In line with this finding, subjects with the above mentioned Val/Val COMT alleles and corresponding reduced levels of PFC DA perform better under stress during WM.

19Early evidence for the involvement of D1R in WM processes comes from work by that showed that systemic injections of pergolide, a combined D1R/D2R agonist, but not bromocriptine, a D2R agonist, facilitated WM performance in a delayed matching task with delays of 2–16 s. These results implicated D1R and not D2R in WM modulation. The important role of D1R on WM is also suggested by the correlation between the decrease of D1R binding in the lateral PFC and the decrease in WM performance with age. However, in another study, bromocriptine was shown to improve spatial WM while the D2R antagonist haloperidol (a typical antipsychotic drug) impaired it. Other experiments, though, did not report a general effect of bromocriptine on spatial memory nor binding of the D2R agonist [11C]FLB457 correlated with performance on the WCST.

20Positron emission tomography studies in humans with the radioactively marked D1R agonist [11C]SCH23390 have revealed an inverted-U relationship between D1R binding in the PFC and performance on the WCST. An inverted-U relationship means that an optimal level of D1R activation is required for best performance and, thus, levels below and above this optimum impair performance. These experiments were meant to confirm results provided by experimentation in monkeys (see below). Further support for an inverted-U relationship between D1R density and WM comes from patients with schizophrenia. Deficits in WM have been associated with both decreased and increased densities of PFC D1R in these patients. Taken together, receptor studies in humans point to an important role of PFC D1R in WM with an optimal level of activation needed for best performance. By contrast, the involvement of D2R needs further elucidation.

21The use of invasive approaches in monkeys has provided valuable insights into the crucial role of PFC DA and its receptors in several higher-order executive functions. In fact, global 6-hydroxydopamine (6-OHDA) induced depletions of DA in the lateral PFC of monkeys allowed to establish early on the critical role of DA in WM. Later, a series of studies showed that there is an increase of extracellular DA in the PFC during WM tasks that exerts its actions via local D1R. More specifically, local injections of D1R antagonists, but not D2R antagonists, into the lateral PFC of monkeys caused deficits in oculomotor delayed-response tasks; monkeys were less accurate in making memory-guided saccades to remembered locations on the screen. We note that the WM component of the task in these studies was in the order of 1.5 to 6 s, comparable to the human literature. More recent work has evidenced that an optimal level of D1R tone is required for adequate WM performance, and this may be particularly vulnerable to changes in arousal state such as fatigue or stress. Thus, either too much (under stress) or too little (during fatigue) D1R stimulation impairs performance following an inverted-U shaped curve. These reports in monkeys agree well with both the deleterious effects of stress on WM performance and the inverted-U relationship between D1R binding and cognitive capabilities reported in human subjects. This inverted-U modulation of D1R also occurs at the level of single PFC neurons engaged in WM. A D1R agonist modulates persistent activity during memory delays following an inverted-U response, whereby low levels of D1R stimulation enhance spatial tuning whereas high levels reduce it. By contrast, D2R have little effect on delay activity and instead modulate the motor component of the task, suggesting some contribution of PFC D2R to motor control function. Systemic injections of D1R agonists and antagonists also alter the performance of monkeys during WM tasks, but these studies have been reviewed elsewhere.

22One general question is why detrimental effects of the “wrong” DA concentration are present in the system in the first place. In other words, what could be functional reasons for decreasing WM performance? Speculatively, these could occur in situations in which the contribution of PFC to behavior is reduced anyway. For example, in high stress, fight or flight mode, behavioral control could be directed to subcortical areas to emphasize speed. Alternatively, the fine-tuning of DA concentration could be used to control the “randomness” of behavior to emphasize exploitation or exploration of certain behaviors. Specifically, D1R activation might push the PFC toward an exploitation mode by protecting the WM content against distractors. In contrast, based on both computational and experimental approaches, D2R activation has been proposed to support behavioral flexibility (exploration;;; ). As in physiological situations selective stimulation of D1R or D2R seems problematic, differences in receptor affinities may produce D2R dominated states (very low and very high DA) and D1R dominated states (intermediate DA). While these properties are also well-suited to support the on- and offset of WM-related persistent activity (Box 2), it remains unclear whether the timescales of DA modulation of the PFC firing are fast enough.

23Models of DA effects in the PFC can be categorized based on their biophysical details of description and their assumed DA release patterns. Furthermore, while the neuropsychological definitions of WM seem not always to be consistent across species (Box 1), computational studies often focus on the mechanisms underlying persistent activity during delay periods.

24An influential early model of DA action in the PFC (; see also: ), bridged the gap between DA-induced conductance changes and functional roles. In small networks of multi-compartment models of pyramidal cells and interneurons, increased DA levels changed various intrinsic ionic as well as synaptic conductances. Through a differential effect on cells in high and low activity states, these changes lead to a better separation of the network response to target and distractor patterns. In particular, the network ability to maintain a robust representation of the target pattern for more than one second was improved by increased levels of DA. This feature could be a central function of DA release in PFC, to support persistent activity related to WM.

25In a similar approach, increasing the dominance of feedback inhibition in the network resulted in an inverted-U shape function of DA concentration and persistent activity, suggesting a close relation to well-known inverted-U shape relations between DA levels and behavioral performance. Overall, the ability of DA to enhance persistent activity has been verified on different modeling levels, ranging from detailed Hodgkin-Huxley-like compartmental models, over extended integrate-and-fire type descriptions, to more abstract rate models. However, it remains unclear which level of model detail is necessary to capture all relevant factors of the extremely complex cellular and synaptic effects of DA in the PFC. It has been argued that the fundamental underlying principle of changing the signal-to-noise ratio is the strengthening of both excitatory and inhibitory transmission; in some cases this is achieved through changes in ionic and synaptic conductance, and in others through simple changes in the gain of the neural activation function. Mechanistically, D1R and D2R have been argued to be essential for changing the dynamics of PFC networks during WM. In the state space of PFC pyramidal and interneuron firing rates, baseline and persistent WM activity form two separate attractors. The level of DA controls the distance between these attractors as well as the structure of the underlying energy landscape, and thereby also the probability of noise to cause a switch between the two regimes. Still, besides the support of persistent activity, there are other aspects of DA function in PFC that might not be captured by the same principles.

26While most previous modeling studies focused on the role of prefrontal DA on WM, a recent study emphasized that DA also affects long-term plasticity in the PFC. Through a multi-compartment model of a PFC neuron (modified from ) they demonstrated that DA can control both the sign and amplitude of long-term plasticity. Potential functional roles of DA-mediated long-term plasticity in PFC could lie in the learning of complex high-dimensional representation of task rules and context. This would also expand the functional role from WM to a more fundamental role in shaping cognitive processes. The interaction of such structural changes with the other roles of DA in changing PFC activity and oscillatory patterns during WM remains one important direction for future computational approaches.

27The monkey lateral PFC has also been implicated in associative stimulus-response learning. Reward-prediction error responses of DA cells might be critically involved in these learning processes (,,; see above). Consistent with this role in reward prediction, phasic DA release occurs in nucleus accumbens that is dynamically modified by associative learning. Thus, it is plausible that these DA signals also play a role in modulating PFC-dependent learning. Indeed,, ) have recently shown that PFC D1R and D2R contribute to stimulus-response learning. Monkeys performed an oculomotor delayed response task where they learned by trial and error associations between visual cues and saccades to a right or left target (Figure 2A). Local microinjections of both D1R and D2R antagonists (SCH23390 and eticlopride, respectively) impaired the learning performance of the monkeys, who made more errors and needed more correct trials to learn the associations. The learning impairments correlated with a decrease of neural information about the associations in single prefrontal neurons during both the cue and memory delay (1 s) epochs of the trial. Noteworthy, blocking D1R impaired learning more than blocking D2R, whereas blocking D2R led to more perseverative errors (Figures 2B,C). This suggests that PFC D1R contribute to learning more than D2R, whereas the latter are more involved in cognitive flexibility. These complementary roles of D1R and D2R in PFC function agree well with the computational models mentioned earlier that propose that D1R activation helps stabilize new representations once an effective strategy has been identified (exploitation) whereas D2R activation destabilizes PFC network states favoring the exploration of new strategies (i.e., flexible processing;;;; ).

28Contrary to the prominent role of DA in WM and associative learning, PFC DA does not influence familiar associations. Blockade of D1R and D2R in the lateral PFC does not cause any behavioral deficit in monkeys remembering highly familiar stimulus-response associations (,; Figures 2A,D). This agrees with the hypothesis that DA is essential for the early stages of learning, but with extended training DA appears to play a decreasing role. So there may be a transition from goal-directed to habit-based instrumental performance likely orchestrated by the basal ganglia.

29A series of investigations carried out by the groups of AC Roberts and TW Robbins have shown in monkeys that DA depletions in another region of the PFC, the orbitofrontal cortex (OFC), disrupt conditioned reinforcement (i.e., when previously neutral stimuli in the environment become associated with reward). After DA depletions restricted to the OFC monkeys were insensitive to conditioned reinforcers and persisted responding in the absence of reward, resembling the compulsive responding of drug addicts. The OFC is also critical for reversal learning, the ability to switch responding to a previously non-reinforced stimulus upon learning. After excitotoxic lesions of the OFC monkeys were able to learn novel stimulus-reward associations, but showed marked perseverative deficits in their ability to reverse the associations. Interestingly, this was sensitive to serotonin but not DA depletions. In contrast, DA, but not serotonin, depletions in the caudate nucleus disrupt reversal learning, revealing striking neurochemical dissociations between the DAergic and serotonergic neuromodulatory systems in fronto-striatal circuits. The role of specific DA receptors in these effects have not been explored, so this important piece of information is missing. In this regard, one study showed that systemic blockade of D2R, but not D1R, impairs reversal learning in monkeys without affecting new leaning. However, administration of drugs in this study was systemic, making the specific contribution of PFC D1R and D2R to the reported effects unclear.

30Separate populations of PFC pyramidal neurons with unique morphological and physiological properties have been identified in mice that express only D1R or D2R. This is similar to the well-established direct and indirect pathways in the basal ganglia, that express D1R and D2R, respectively. In fact, a recent study has demonstrated that selective (optogenetic) activation of D1R-expressing neurons in the striatum (direct pathway) promotes reinforcement learning, whereas selective activation of D2R-expressing neurons (indirect pathway) induces transient punishment. However, the specific contribution of D1R- and D2R-expressing neurons in the PFC to learning has yet to be elucidated.

31Early work in rats demonstrated, as in monkeys, that elevating or depleting DA in the PFC impaired spatial WM performance. In keeping with studies in monkeys, there is a phasic release of DA into the PFC during delayed response tasks, the magnitude of DA efflux being predictive of memory accuracy. Moreover, these DA actions are mediated by D1R. reported that overstimulation of PFC D1R with a D1R agonist induced deleterious effects in spatial WM of rats performing a delayed alternation task, an effect reversed by pretreatment with a D1R antagonist. Rats were required to alternate between two arms to obtain a reward, with a delay between trials of 5–30 s. Another study using a comparable range of delays (0–16 s) found that intra-PFC infusions of a D1R agonist, but not a D2R antagonist, could disrupt or facilitate performance in a task designed to account for the contribution of attention to WM. Importantly, this work suggested that different levels of DA may be required for different cognitive processes. Seamans and Floresco used a delayed response variant of the radial-arm maze task to demonstrate, also in rats, that other types of “WM” with comparatively longer delays (in the order of 30 min to several hours) are also sensitive to manipulations of PFC D1, but not D2, receptors. We note that some of these studies aimed at directly testing whether inadequate activation of PFC D1R in rodents caused the same detrimental effects on WM previously reported in monkeys, where memory delays were in the order of few seconds. Thus, and as pointed out previously (Box 1), it seems like studies across species have not reached a consensus in defining what “WM” is. However, altogether, these studies implicate PFC D1R in different types of “short-term” memory.

32Also on par with primate studies, insufficient or excessive activation of PFC D1R impairs the performance of rats in WM tasks following an inverted-U shaped curve. Interestingly, this has been recently extended to a more holistic view of the role of D1R/D2R in cortico-striatal circuits. Transgenic mice with selective and reversible overexpression of D2R in the striatum exhibit poor WM abilities that correlate with exacerbated PFC D1R activation. In contrast with the monkey literature, though, rodent work has suggested that PFC D2R could play a role in WM. reported that intra-PFC infusions of a D2R agonist disrupt performance of rats in a delayed-response task and that this D2R modulation of WM may be linear (i.e., lower/higher levels of D2R activation are associated with better/poorer performance). Thus, PFC D2R could also contribute to WM but following distinct principles of operation than D1R (i.e., linear vs. an inverted-U modulation;; ). So, perhaps the effects of the D2R agonist bromocriptine observed in human studies can be attributed in part to PFC D2R.

33Furthermore, D4R may be key for emotional learning. In rats, activation of D4, but not D1, receptor subtypes in the medial PFC strongly potentiates the salience of emotional associative fear memories. Furthermore, individual neurons in the medial PFC actively encode emotional learning, and this depends on D4R activation. Conversely, stimulation of D1R and not D4R blocks the recall of previously learned emotionally relevant information suggesting, again, that D1R help shape memories. So, PFC D1R and D4R may play discrete roles (memory vs. learning) in the acquisition of emotional associations.

34D1R and D2R exert complex modulatory actions on the activity of PFC neurons, as shown by in vitro recordings in PFC slices of rodents (see for an extensive review ). Briefly, DA tends to enhance spiking via D1R through Na+, K+, and Ca2+ currents, an effect also observed in PFC slices of monkeys. Conversely, DA decreases spiking via D2R, possibly through modulation of glutamatergic receptors and Na+ conductances. Moreover, stimulation of PFC D2R can also induce an afterdepolarization mediated by L-type Ca2+ channels and NMDA receptors. Besides these contributions of DA to the modulation of PFC activity, several rodent studies have also provided evidence that PFC neurons shape the activity of DA neurons. For example, found that OFC inactivation impaired state-value representations in VTA DA cell activity, in particular the effect of the animals own action plan on the state value. Furthermore, showed that PFC inactivation increases the DA response to reward-predicting stimuli. This matches a series of computational modeling studies in which PFC becomes part of the system that determines the value of the current state and propagates this information to the DA system (e.g.,;; ). Although this supports a general role of PFC in shaping DA cell activity, the specific contribution during behavior depends on the corresponding firing patterns of the PFC neurons that affect DA cells.

35Higher cognitive abilities evolved largely independently in birds and mammals. This parallel evolution gave rise to several crucial differences in neural organization. While avian and mammalian striatum and pallium are homolog (derived from a common ancestor), there are considerable differences in the organization of the pallium. For instance, the avian telencephalon does not have a pallial commissure comparable to the mammalian corpus callosum. The most notable difference, however, is the lack of the typical cortical lamination in the avian pallium. In other words, in spite of a shared evolutionary ancestry and a similar functionality, the avian and mammalian “cortex” look entirely different: what has evolved into layers in the mammalian brain might have evolved into different regions in the avian brain. Other organizational principles were preserved or independently evolved. For instance, a recent analysis of the avian connectome revealed a very similar network organization between birds and mammals. Using graph theory, the authors found that the telencephalon of both species has a comparable organization into modular, small-world networks with a connective core of hub nodes. The most relevant here is the “prefrontal” hub. While the avian brain has no homolog of the mammalian PFC, it has a functional analog (structure with comparable functionality) – the NCL. A detailed comparison between both structures has been provided elsewhere. Briefly, PFC and NCL are centers of multimodal integration that are closely connected to all secondary sensory and motor regions.

36Much like the PFC, the NCL is involved in WM as revealed by lesion studies and single cell recordings in pigeons during Go/Nogo tasks. Recently, an elegant study demonstrated that single neurons in the NCL of crows maintain memory information in two versions of a delayed match to sample task (DMS; ), the classical paradigm of WM research in primates. The animals were trained to view a sample image and indicate this image among similar images following a short delay (1–2.3 s). Similar experiments revealed an involvement of NCL in other cognitive functions such as categorization, the integration of time-to-reward with reward amount, and executive control over what information is maintained in WM. Another hallmark of prefrontal function, the processing of rules that guide behavior, was recently reported in the NCL of crows. The authors used the same paradigm that was used in the original demonstration of such processes in primate PFC, a modified DMS task. They report that single neurons in the NCL represent behavioral rules that instruct the animals how to respond to subsequent stimuli, a result that mirrors the original findings in the PFC.

37The NCL, as the PFC, is the prime cortical (pallial) target of dopaminergic innervation. As in mammals, these projections arise in VTA and SNc (; Figure 1). Dopaminergic projections to the avian telencephalon show two distinct anatomical features. One type, “en passant” projections, are also found in the mammalian brain. These axons travel through the telencephalon, contacting a large number of dendrites and somata of predominantly smaller target neurons. The other type, “baskets,” has not been reported in the mammalian brain. Here, individual fibers densely wrap around the somata and initial dendrites of predominantly larger cells. Interestingly, this type of innervation might be functionally comparable to the pattern of innervation in the mammalian cortex. In mammals, large pyramidal neurons lie mainly in deeper layers and are targeted by DA terminals through their proximal (in primates also distal) dendrites. The basket structures might be a way to generate a similar innervation of larger cells in the absence of cortical organization. Compared to the mammalian PFC, the avian NCL contains members of both DA receptor families, with a considerably lower density of D2 compared to D1 receptors.

38Overall, the role of DA in the avian brain is largely comparable to its role in the mammalian brain. DA is involved in motor control and learning, and in birds it also contributes to the acquisition and control of birdsong. Even though birdsong is a major focus of avian research, here we will only briefly refer to this work. It has been reviewed extensively elsewhere and the main focus of the song literature is the role of DA in basal ganglia circuits. To our knowledge, no study has recorded avian dopaminergic neurons during learning, so there is no direct evidence for reward prediction error coding in avian DA neurons. However, several studies provide indirect evidence for temporal discounting (TD)-learning in birds. The only study that recorded from single DA neurons in the VTA of songbirds showed that DA neurons are strongly modulated by social context. The authors interpret this result in the light of “approval” – positive feedback of the females that the male subjects sang to. Later work confirmed that such social context activity is involved in modulating the singing-related activation of the song system. Further evidence comes from behavioral studies. Pigeons learn a simple discrimination task faster if they receive a larger reward for correct discrimination than with a smaller contingent reward. This difference in learning rate can be predicted by different reward prediction errors due to the different reward magnitudes. Furthermore, injections of D1R antagonists in the striatum abolish this effect. Interestingly, the birds are still able to learn the discrimination but the learning rate is no longer modulated by the contingent reward magnitude. Learning shows an average rate with a slight decrease in performance on a large reward and a slight increase in performance with a small reward.

39As in the mammalian PFC, DA in the avian NCL is critically involved in mechanisms of learning and memory. DA levels in the PFC of monkeys increase during WM tasks and, consistently, microdialysis in the NCL of pigeons show an increase in DA during a DMS task with a delay (4 s) compared to the same task without a delay. Furthermore, injections of a D1R agonist (SKF81297) into the NCL and striatum improve performance on a DMS task. Interestingly, these injections were only beneficial on days with low performance; if the animals performed well, agonist injections disrupted performance. These findings are in line with the mammalian literature showing that DA modulates performance following an inverted-U shaped curve, where too much or too little D1R activation is detrimental to performance. It also complements nicely the reports showing that humans with genetically lower levels of DA in PFC are less susceptible to the detrimental effects of stress on WM (see Human Studies). In addition, and again in line with the mammalian literature on WM, injections of a D1R antagonist (SCH23390) into the NCL disrupt the ability of pigeons to focus their attention over longer periods of time and to ignore distracting stimuli.

40In a recent study, assessed the expression of different DA receptor types in the NCL of pigeons trained on different cognitive tasks. This approach allowed the dissociation of changes in receptor expression due to WM (using a DMS task), stimulus selection (a stimulus-response task), or general task components such as reward and response selection. It is noteworthy that the mammalian D1R family is extended in the avian brain. In addition to D1A (D1) and D1B (D5) receptors, the avian brain also contains the receptor D1D. The authors report that general task components have no influence on D1R expression in the NCL. However, WM components increase expression of D1B and stimulus-response learning increases expression of D1A and D1D receptors. None of the task components affected the expression of D2R. These results demonstrate an involvement of DA receptors in the NCL not only in WM but also in learning mechanisms. In line with these results, microinjections of a D1R antagonist (SCH23390) to the NCL of pigeons resulted in severe disruptions of discrimination reversal learning. This result is in contrast to the finding that DA in caudate nucleus, but not the OFC, of monkeys is required for reversal learning (see Non-Human Primate Studies;,,, ).

41Despite decades of intense research, we are only now starting to comprehend the specific roles of DA in several PFC-dependent learning and memory processes. A main obstacle in understanding the complex DA modulation of PFC function, both at anatomical and physiological levels, is the outstanding heterogeneity and specificity of the DA system itself. Therefore, a cross-species comparison may contribute to identify general principles of DA function in the PFC. Each model species discussed here provides its unique advantages and challenges. Certainly, one of the main goals of studying the dopaminergic system is to expand our understanding of the healthy and diseased human brain in order to develop better treatments for neurological and psychiatric disorders with abnormal DA transmission. Since this research poses many technical constraints, non-human primates offer an alternative to study complex behavior and higher cognitive functions. In contrast, rodents can be manipulated vastly with a variety of genetic/optogenetic approaches, but their cognitive abilities might not be sufficient to address higher cognitive functions of humans. Finally, studying the avian brain offers an evolutionary perspective that might help identify crucial features and constraints of the dopaminergic system. Indeed, the crucial role of DA in executive function is highlighted by the fact that the independent evolution of higher-order cognition in birds gave rise to a largely comparable DA function – even in the absence of cortical layering.

42Some major findings have been consistently replicated in different species, establishing their robustness. First, elevating or depleting DA levels in PFC impair performance in WM tasks. Second, PFC DA modulates WM via D1R. The potential involvement of D2R in WM is more controversial. Third, PFC D1R modulate WM following an inverted-U shaped curve. That is, an optimal level of D1R activation is required for adequate WM performance, and this is sensitive to changes in arousal state such as fatigue or stress. Recent studies in monkeys point to interesting extensions of these findings, but still need to be confirmed in other species and in other paradigms. They showed that the inverted-U curve modulation of WM may also occur at the level of spiking in PFC neurons, and that both PFC D1R and D2R play relevant roles in associative learning but not associative memory.

43Clearly, more work will be necessary to fully understand the role of different receptor subtypes present in the avian and mammalian brains in learning and memory processes. In order to succeed, and as underscored in this review, researchers working on different disciplines and with different species will need to reach a consensus in how to define different types of learning and memory processes, paying particular attention to WM-related concepts and terminology (Box 1). Computational modeling could provide such unified definitions and hypotheses that are testable across species.

44Importantly, recent investigations conducted in rodents have highlighted the close interaction between D1 and D2 receptors present in cortico-striatal circuits. In addition, separate populations of pyramidal neurons have been identified in the rat PFC that preferentially express only D1R or D2R, similarly to the D1R-expressing direct and D2R-expressing indirect pathways of the basal ganglia. Although the specific contribution of these PFC neuron populations to learning and memory has yet to be elucidated, the use of genetic and invasive approaches in rodents is proving to be an excellent source of information. However, non-human primate models are better suited to gain deeper insights into the role of DA in more sophisticated tasks that are closer to the human cognitive repertoire. Unfortunately, genetic manipulations and invasive approaches such as optogenetics are just beginning to be developed in primates. A rapid advancement in the development of techniques applicable to humans is especially necessary, since human studies on the role DA in learning and memory have been particularly scarce. In this regard, it is now possible to measure DA release with accurate timescales by molecular fMRI. We hope that these emerging technical advances in primates will allow a more detailed understanding of the roles of D1R and D2R in higher-order executive function. This will be particularly important for the development of adequate drug therapies for patients with disorders that show disrupted prefrontal DA signaling such as schizophrenia, PD, and ADHD.

45The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.