Serotonin, Inhibition, and Negative Mood

1Serotonin (5-hydroxytryptamine [5-HT]) is a neuromodulator that appears to play a critical role in a wealth of psychiatric conditions, including depression, anxiety, panic, and obsessive compulsions. However, despite the importance of serotonergic pharmacotherapies, notably selective serotonin reuptake inhibitors (SSRIs), the roles that serotonin plays in normal and abnormal function are still mysterious. We start from three particular findings. First, 5-HT is involved in the prediction of aversive events, possibly as a form of opponent to dopamine. Second, 5-HT is involved in behavioral inhibition, preventing or curtailing ongoing actions in light of predictions of aversive outcomes. The third finding is the collection of psychopharmacological data implicating 5-HT in animal models of depression and anxiety, together with the fact that depleting 5-HT (by dietary depletion of its precursor, tryptophan) in human subjects who have recovered from depression, can reinstate an acute, at times fulminant, re-experience of subjective symptoms of the disease, as assessed by various rating scales. Furthermore, while SSRIs are used in the treatment of depression, genetically induced, constitutive decreases in the efficiency of 5-HT reuptake are a risk factor for depression. These findings are hard to connect: the second fact seems orthogonal to the first and third, which are themselves in apparent contradiction. If 5-HT is really involved in predicting aversive outcomes, then depleting it should surely have positive rather than negative affective consequences.

2We suggest that the missing link comes from considering the interactions between Pavlovian predictions and ongoing action selection. The interaction is seen in conditioned suppression, a standard workhorse test for aversive predictions. Animals are trained to emit appetitive instrumental actions (such as pressing a lever for reward), and to associate (by classical conditioning) a light with a shock. Presentation of the light during instrumental performance reduces the rate at which animals emit those responses. Neither the theoretical nor the neurobiological status of this interaction is completely resolved, though there is some evidence of the involvement of 5-HT in the nucleus accumbens in its realization.

3Here, we treat a subset of the inhibitory processes associated with Gray's behavioral inhibition system (BIS) in terms of what might be called a preparatory Pavlovian response. Consummatory Pavlovian responses are (evolutionarily) pre-programmed reactions to the presence of affectively significant outcomes such as food, water, or threats. Preparatory Pavlovian responses are similarly pre-programmed responses to predictions of those outcomes. Even though the predictions are learned, the responses are not, and may therefore be behaviorally inappropriate in certain circumstances. For our purposes, and as long noted by Deakin and Graeff, the most important preparatory Pavlovian response to a prediction of a (sufficiently distant) threat is inhibition, in the form of withdrawal or disengagement. This explicitly links the first two findings discussed above, as the inhibition is directly associated with aversive predictions.

4To explore the consequences of reflexive, direct inhibition of action for learning in affective settings, together with the repercussions when 5-HT is compromised, we built a highly simplified model that sought to isolate these effects from more general learning effects. More specifically, we built a model of trains of thoughts. In our treatment, we considered thoughts as actions that lead from one belief state to the next. Trains of thought gained value through their connections with a group of terminal states that were preassigned either positive or negative affective values. 5-HT directly inhibited chains of thought predicted to lead toward negative terminal states. Our model can be seen in terms of 5-HT's pruning of a decision tree of outcome states and choices.

5We argue that the results on tryptophan depletion (TrD) above now emerge when considering the consequences of this reflexive behavioral inhibition on ongoing learning about the world, and on subsequent action choice and predictions. The most notable effect in the model is a critical bias toward optimistic valuation. That is, states and actions with potentially negative consequences are under-explored and incorrectly (over)-valued because of the reflexive inhibition. When inhibition fails, though, which is the last of the three issues mentioned above, there are two adverse consequences. First, the inhibition is no longer a crutch for instrumental action choice, so subjects have to learn to avoid potentially bad situations rather than being able to rely on this reflexive mechanism. Second, due to a mismatch between policy and value function, characteristic inconsistencies between the predicted and actual values arise, with the actual values encountered being more negative than predicted, though also actually more realistic. This mismatch between policy and value function also leads to an overall reduction in rewards obtained. Boosting 5-HT in the model again restores the status quo. Of course, this highly simplified model cannot possibly, by itself, accommodate all the diverse and confusing roles of 5-HT. Nevertheless, it replicates some prominent behavioral and pharmacological facets of depression and anxiety in humans and animal models, which we return to in the Discussion.

6The next section defines the model of trains of thought more formally. The Results section considers normal (hence biased) learning, and the consequences of impairments to 5-HT processing. We save for the Discussion a broader discussion of data and theories pertaining to 5-HT.

7Figure 1 illustrates our underlying model of trains of thought. It is intended to emphasize a role for 5-HT in behavioral inhibition, and is therefore couched at an abstract level. Throughout, we will equate thoughts with actions, and revisit the more general action setting later. We initially focus on the effect of one inhibitory reflexive action in the context of otherwise fixed actions (a fixed policy).

8A train of thoughts starts at one of a set of internal belief states (I℘ +, I℘ −), may proceed through more such states, and ends in one of set of terminal outcome states (O℘ +, O℘ −). The connectivity between belief states is sparse, with states I℘ + leading preferentially to other states in I℘ + and outcome states O℘ + with positive values; and states I℘ − leading preferentially to other states in I℘ − and outcome states O℘ − with negative values (red arrows), though each could also lead to states of opposite “sign” (black arrows in Figure 1). In addition, trains of thought can be inhibited by 5-HT (see below). In this simple model, the value of an internal state is the average value of the terminal states to which it ultimately leads.

9More formally, the model is a form of Markov decision process (see), with four sets of sparsely interconnected states (I℘ ±, O℘ ±). Two sets, O℘ + and O℘ − (each with 100 elements in the simulation) are associated respectively with positive (r(s) ≥ 0, s ∈ O℘ +) and negative affective values (r(s) ≤ 0, s ∈ O℘ −); both are drawn from suitably truncated 0-mean, unit variance, Gaussian distributions (see inset histograms in Figure 1) and are terminal states. The other sets, I℘ + and I℘ − (each with 400 elements), contain internal states and are not associated with immediate affective values (r(s) = 0, ∀s ∈ I℘ − ∪ I℘ +).

10A policy is a (probabilistic) mapping from states to actions a ← π(s) and defines the transition matrix between the states in the model. For simplicity, we consider a fixed, basic, policy π 0. In this, each element of I℘ + effectively has eight outgoing connections: three to other (randomly chosen) elements in I℘ +; three to randomly chosen elements in O℘ +; and one each to randomly chosen elements in I℘ − and O℘ −. Similarly, each element of I℘ − has eight outgoing connections: three to other (randomly chosen) elements in I℘ −; three to randomly chosen elements in O℘ −; and one each to randomly chosen elements in I℘ + and O℘ +. Thoughts are modelled as actions a following these connections, labelled by the identities of the states to which they lead. Text S1 gives details of a more complex environment in which we explicitly explore effects of impulsivity.

11To isolate the effect of 5-HT in inhibiting actions in aversive situations, we consider the highly simplified proposal that serotonin stochastically terminates trains of thoughts when these reach aversive states. More specifically, under serotonergic influence the transition probabilities are modified in a manner that depends on states' values. We let the probability of continuing a train of thought (of continuing along the fixed policy π 0) be dependent (and inversely related to) the value V(s) of a state: where α 5HT is a multiplicative factor that scales the impact of 5-HT (see Figure 2). When thoughts are not continued (inhibited), they stop and restart in a randomly chosen state I℘ (though see below for relaxations of this). The more disastrous the potential sequelæ of state s, the more negative Vπ(s), and so the less likely the chain was to be continued. On the other hand, even slightly positive values would essentially veto any termination. This introduces an asymmetry into the model defined by the simple base policy. Other possibilities for the information reported by 5-HT and for the dynamic interaction between 5-HT and dopamine are considered in the discussion, and the fixed base policy π 0 is relaxed below.

12The value of each state represents the expected reward obtainable from that state when following a particular policy. Under the fixed policy π 0, dynamic programming techniques allow the value function Vπ(s) over states s to be written, and solved for, concisely as: Vπ(s) = r(s), s ∈ O℘±, and where γ is a discount factor (γ = 0.9 in our simulations). Dynamic programming also uses a Q*(s,α) function over states and thoughts defined for those actions that exist by

13Optimal values V *(s) and Q*(s,α) are those value functions associated with any policy that maximizes the long-run affective outcomes of the train.

14While it is not possible to use these techniques directly to evaluate the value function under serotonergic influence (the inhibition depends on the value function itself and thus represents a nonlinear interaction), the temporal difference learning rule can be used to acquire estimates of the values of states under serotonergically modified policies. The temporal difference learning rule specifies an online learning rule for which the change in the estimated value based on taking action α at state s and therefore arriving deterministically at s′ = α(s) is: where the learning rate ɛ = 0.05. A slightly simpler alternative rule suggests that learning of is itself prevented by termination:

15That does not change under this rule given termination implies that learning is only slowed for these states, rather than being biased toward zero. We generally report results from this variant.

16In the sequel, we show values after substantial learning (20,000 trains), plus the consequences of manipulating serotonin (by manipulating α 5HT) once the values are already acquired.

17Given the values learned under a policy π(α 5HT) determined by α 5HT = 20, the steady-state transitions probabilities can be calculated for any new α 5HT ≠ 20 simply by working out the probability of inhibition for each state. In particular, this allows α 5HT to be reduced to model a pharmacological or psychiatric reduction in serotonin function. To separate the effect of this reduction from that of learning, we only learn up to the reduction and then look at the behavior after the reduction in the absence of further learning.

18To account for the effect of recall biases often seen in depression, we will additionally consider the effect of biased resampling after behavioral inhibition. A simple way of achieving this in a manner that relates to the affective value of states is to let whereby values of β < 0 will bias resampling toward states with lower values V(s) (i.e., states in I℘ −).

19So far, only serotonergically determined inhibitory responses have been considered. Mirror to these are dopaminergically controlled approach responses, which actually favour actions α with positive state-action values Q℘(s,α) (under a policy). The combined effect can be incorporated in a straightforward manner by choosing action α in state s according to a softmax where θ controls the degree of influence of the Q℘ value. Note that, in this simple model, instrumental and Pavlovian control are essentially indistinguishable.

20By construction, the environment in Figure 1 is symmetric with respect to rewards and punishments, and so the overall statistics of the values of states are balanced about zero. Indeed, Figure 3A shows that for the base policy, 20,000 learning steps are ample to acquire a reasonable value V est(s) for the states (the remaining discrepancies from V true(s), here defined for α 5HT = 0, arise from the stochasticity in the choice of action together with the fixed learning rate). Critically, there is no bias in either V est(s) or V true(s).

21By contrast, Figure 3D shows the substantial bias in consequent on setting a large value of α 5HT = 20. In this case, low-valued states are much less well visited and explored. The bias comes despite the use of learning rule, which only slows down learning for low-value states rather than also distorting it. Of course, in this case, the extent of the bias depends on the initial values for the states (all of which are set to zero in the simulation).

22Figure 3E shows how frequently each of the outcome states was reached in a run (as a function of its outcome r(s)). Since behavioral inhibition terminates trains on their way to potential disaster, aversive terminal states are sampled less (shown by the red regression line), which is consistent with the bias of the estimated value. Figures 3C and 3F show these effects as a function of α 5HT. The greater the inhibition, the worse estimated the values are (Figure 3C), particularly for aversive states; however, the more benign is the exploration (Figure 3F). Learning with greater inhibition leads to a more optimistic set of values; however, this is coupled with a more aggressive rejection of all actions even mildly associated with negative outcomes.

23Reducing the value of α 5HT after learning a value function under its influence can be expected to have various consequences, as it introduces a mismatch between policy and value function. The most obvious one is a more negative average affective outcome (the average value of trains of thought) in the model. This is because choices are less biased against actions that are predicted to have aversive consequences, and so the latter occur more frequently. A second consequence is that there will be substantial adverse surprises associated with transitions that previously were inhibited. The surprise at reaching an actual outcome can be measured using the prediction error for the last transition of a chain from state to a state. We may expect negative prediction errors to be of special importance, because of substantial evidence that aversive outcomes whose magnitudes and timing are expected so they can be prepared for, have substantially less disutility than outcomes that are more aversive than expected (at least for physiological pains; see).

24Figure 4 shows the consequences of learning under full inhibition and then wandering through state space with reduced inhibition. The change in the average terminal affective value as a percentage of the case during learning that α 5HT = 20 is shown in Figure 4A. As was already apparent in Figure 3F (which averages over the whole course of learning), large costs are incurred for large reductions in inhibition. For α 5HT = 0, the average reward is actually negative, which is why the curve dips below −100%. This value is relevant, since the internal environment is approximately symmetric in terms of the appetitive and aversive outcomes it affords. Subjects normally experience an optimistic or rosy view of it, by terminating any unfortunate trains of thought (indeed, 55% of their state occupancy is in I℘ + compared with I℘ −). Under reduced 5-HT, subjects see it more the way it really is (the ratio becomes 50%).

25Figure 4B and 4C show comparative scatter plots of the terminal prediction errors. Here, we consider just the last transition from an internal state to an outcome state. Prediction errors here that are large and negative, with substantially more aversive outcomes than expected, may be particularly damaging. Figure 4C compares the average terminal prediction errors for all transitions into states in O℘ − with no serotonergic inhibition α 5HT = 0, to those for the value α 5HT = 20 that were used during learning. For the case that α 5HT = 20, the negative prediction errors are on average very small (partly since the probability of receiving one is very low). With reduced inhibition, the errors become dramatically larger, potentially leading to enhanced global aversion. By comparison, as one might expect, the positive prediction errors resulting from transitions into O℘ + are not greatly affected by the inhibition (Figure 4B).

26Two additional effects enrich this partial picture. One, which plays a particularly important role in the cognitive behavioral therapy literature, is that depressed patients have a tendency to prefer to recall aversive states or memories. Figure 5A shows the consequence of doing this according to a simple softmax (see Methods). These curves, as in Figure 4A, show the percentage average utility compared with α 5HT = 20, β = 0 across values of α 5HT, and for β = −10, −9,…,10. As might be expected, biasing the starting point to I℘ −, and, even worse, to those particular states in I℘ − that are most deleterious, has a big negative impact on average utility. For α 5HT = 0; β = −10, occupancy of I℘ + relative to I℘ − became a paltry 27% as subjects ruminate negatively.

27The second factor is our restriction to just inhibition of trains of thought rather than a more fine-scale manipulation of the relative probabilities of different thoughts. We now relax this and explore the effect of additionally allowing preferential transitions toward certain states. In Equation 6, for positive values, the parameter θ biases action choice toward actions leading to positively valued states, whereas for negative values it does the opposite (i.e., subjects prefer to transition to negatively valued states). Figure 5B shows the effects of θ. It is apparent that rather extreme values of θ can both significantly aggravate or suppress the effect of α 5HT. For the highest positive values of θ the curves reverse shape, showing that it can be beneficial not to inhibit trains of thought. This arises since the model of Figure 1 was chosen to have the extreme property that there is always the possibility of avoidance (in that all the states in I℘ − admit at least one action that leads to I℘ +), and inhibiting trains of thought removes this outcome. A different, and rather counterintuitive, interaction between inhibition and reward seeking obtains in environments where rewards are hidden behind punishments (see Text S1 and Figure S1).

28We studied a very simple Markov decision process model of affectively charged thoughts, and showed various aspects of the influence of behavioral inhibition on the experience of appetitive and aversive outcomes, predictions, and prediction errors. The model formalises behavioral inhibition as a Pavlovian control process that arrests internally directed thoughts (and likewise externally directed actions) that are predicted to lead to aversive consequences. Overall this is favourable, leads to enhanced average rewards, and is related to adaptive pruning. However, the consequences can also be deleterious. Compromising inhibition in the model has two related consequences. First, the values of states are revealed to be overly optimistic. Second, control is disturbed, with aversive chains being insufficiently deselected.

29While this work shows how several prominent aspects of serotonin's manifold putative functions and effects can be reconciled within a unifying framework, we acknowledge that we have neglected a wide range of other issues, and certainly do not claim that this is an exhaustive account of the data. There is also an interesting alternative view of 5-HT, such as that due to who suggested that it is involved in controlling the appropriate timescale of behavior by determining the discount factor for future affective outcomes (parameter γ in Equation 2). In this theory, 5-HT depletion reduces the effective value of γ, making subjects appear more impulsive. Our model captures impulsivity through reduced 5-HT more directly, suggesting that actions that are comparatively worse lose direct inhibition that was previously restraining them, and are therefore more likely to be executed.

30We suggested that this form of behavioral inhibition arises through predictions of aversive outcomes, tied to serotonin's putative role in reporting aversive prediction errors as an opponent to dopamine. This comes directly from the original notions of behavioral inhibition and serotonergic effects from Gray, Deakin, Graeff, and their colleagues; however, it is perhaps best seen as a subset of the current version of Gray's BIS. One salient difference is that BIS is suggested as being primarily engaged by conflict, rather than ongoing predictions of future aversive outcomes. Of course, a main source of conflict is that between approach and avoidance, with the latter coming from these aversive predictions. An interesting consequence of dividing the prediction of the value of future outcomes between two separate opponent systems is that it is indeed possible to have simultaneous appetitive and aversive expectations, as opposed to just one combined net prediction. Although we used the net prediction to control inhibition, it would be interesting to explore other possibilities associated with the BIS view, such as that any aversive prediction could arrest ongoing action, even if outweighed by appetitive predictions.

31Further, rather than have the aversive predictive values of states lead to termination of trains of thought, it is possible that the negative prediction error (δ − from Equation 8), which Daw et al. suggested is being reported by phasic serotonin, could be responsible instead. Alternatively, in the mirror reflection of the proposal that a tonic dopaminergic signal reports average reward (and controllable/avoidable punishment) and energises behavior, it could be that a more tonic serotonergic signal, averaging aversion over longer time horizons and favoring quiescence, could be responsible.

32Another difference between our account and the full BIS is that, in the latter, although actions are indeed inhibited in the face of conflict, the BIS is then suggested as initiating a set of behaviors (such as exploration or risk assessment) to resolve that conflict. The set of preparatory Pavlovian actions associated with aversive predictions appears to be more refined than that associated with appetitive predictions (mostly just approach), with a wide range of different defensive possibilities being selected between according to the nature and proximity of the threat. One class of these is even laid out along columns of the dorsal periacqueductal gray (PAG). Nevertheless, any of these defensive manoeuvres would interrupt the ongoing chain of actions, and this is what we modelled. Risk assessment and exploration are of most obvious use in the face of uncertainty and ignorance, whereas conditioned suppression, and thus the sort of inhibition that we consider, remains even after substantial learning. It would certainly be worth going one stage further, modelling the interruption in terms of a switch between different Markov decision problems, with new information changing the transition and payoff structures.

33One of our central results is the effect of an acute reduction in α 5HT after learning with elevated α 5HT has taken place. In our model, this leads to a decrease in behavioral inhibition of actions leading to negative states. Although specific effects might arise from local manipulations of 5-HT concentrations or receptor responsivity, key data come from the systemic manipulation associated with acute TrD, in which plasma levels of tryptophan and, at least in animals, central nervous system levels of serotonin, are drastically reduced (by up to 90%). Although the particular chains of thoughts analysed here have not been the subject of experimental scrutiny, there is by now a considerable body of literature on the effects of TrD on normal human functioning. In broad agreement with our results, various effects have been related to decreased reward processing, decreased behavioral inhibition, rumination, facial fear recognition, and, more indirectly, increased aggressiveness.

34Perhaps of most direct relevance to our implementation are the results of a recent study which decoupled rewards from correct performance of an action from the outcomes of the actions. This study actually involved a sophisticated assessment of the effects of TrD on reversal learning. However, one way of viewing a portion of the results stems from an abstract representation of the task. Subjects had to press one of two buttons (A or B) in response to one of two stimuli (also called A and B), with presses associated with A leading to a symbolic reward and presses associated with B leading to a symbolic punishment. Critically, these outcomes were independent of the rectitude of the subjects' responses, so they couldn't avoid the punishment by making errors. In this case, subjects more often failed to press button B correctly than button A, and this difference disappeared after TrD. This is directly consistent with the present interpretation of serotoninergic inhibition of actions that lead to aversive outcomes.

35Famously, TrD does not have a uniform effect on all subjects. There is an important genetic polymorphism in the 5-HT reuptake mechanism, with subjects having the less efficient version generally showing greater effects. For this to be consistent with our formulation, the difference in functional 5-HT levels before and after TrD has to be greater in the subjects with less efficient reuptake. This in turn might most simply be due to increased levels of 5-HT (and behavioral inhibition) throughout development in carriers of the short 5HTTLPR allele. Perhaps related to this is the finding that TrD produces a dose-dependent relapse of depressive symptomatology in formerly depressed patients, or in patients with risk factors such as a family history of depression (although the three-way interaction between TrD, 5HTTLPR, and past depression is hard to fit into this framework).

36There is a significant body of work on the effects of serotonergic manipulations on affective processing, particularly on processing of facial expressions. It is difficult to interpret this work in our context for several reasons: first, there have often been effects on recognition of specific aversive facial expressions (e.g., fear) but not others (e.g., disgust). Our model does not speak to these distinctions. Second, in these tasks, subjects identify stimuli by pressing a button. Thus, there is a Pavlovian association between certain buttons and the aversive stimuli, and, interpreting these tasks in the same framework as we interpreted the work of Cools et al., one might predict that TrD would increase rather than decrease accuracy. The precise effect, however, would depend on the relative strength of the instructed and the reflexive Pavlovian response, and on the antagonism between the responses. Indeed, both aspects have been found: acute manipulation of serotonin increased recognition accuracy of fearful faces with increasing serotonin, whereas a more chronic increase in 5-HT (via SSRIs) yields a decrease in recollection of negative memories. Furthermore, while the exact relationship between behavioral inhibition and amygdala activation still needs clarification, it is additionally possible that increased amygdala activation may relate to perceptual mood congruency effects: after disinhibition, thoughts often visit negative states, and it is possible that this may affect prior expectations about stimulus which in turn could speed up processing of negatively valenced information.

37TrD (or indeed SSRIs) have not previously been used in tasks like the Markov decision problem of the type we discussed. A direct prediction of the model is that subjects trained under TrD would explore states less when tested in a normal regime, while those trained under SSRIs would do so more (assuming that SSRIs indeed elevate 5-HT levels). Similar predictions hold for subjects with short or long alleles of the 5-HT reuptake mechanism on these tasks. This would essentially represent a generalisation of the findings by Cools et al. to the domain of sequential decisions. The tasks could use external, observable actions; more directly, it would also be useful to monitor the execution of affective trains of thought, and study the perturbation of this under serotonergic manipulations. In designing such studies, it is important to bear in mind the potentially opponent instrumental and Pavlovian effects, in just the same way that boosting dopamine and monitoring the effects on negative automaintenance may be confusing. Note that although there are various important datasets as to the effects of TrD on simple probabilistic and delay-discounting tasks, these studies do not encompass the sorts of behavioral chains that we propose 5-HT to be able to halt.

38One of the backdrops for the present theory was the extensive modeling of phasic dopamine as a prediction error for future reward, and the results that (1) the baseline firing rates of dopamine cells are insufficient to report prediction errors for negative rewards (i.e., punishments); (2) the ample psychological evidence for the existence of a pair of systems, one associated with appetitive outcomes and the other with aversive outcomes; and (3) the evidence that at least some aspects of 5-HT and dopamine are in mutual opposition. Indeed, based on these data and the theories of Deakin and Graeff, Daw et al. suggested that serotonin rather than dopamine reports negative prediction errors based on an antagonism between serotonin and dopamine at both a behavioral and pharmacological level. For example, in rodents, 5-HT antagonises the general excitatory effects of dopamine, the self-administration of amphetamine and intracranial self-stimulation, the effects of dopamine on appetitive learning, and the potentiation of appetitive learning by amphetamine.

39However, pure opponency is far too simple. For instance, there is by now extensive evidence that 5-HT modulates dopaminergic activity both through receptors in the ventral tegmental area and by modulation of distal release sites, and that this modulation can occur in both inhibitory and excitatory directions. Even the rise in 5-HT due to SSRIs has overall pro-dopaminergic effects, both at behavioral and physiological levels, and there is one report that DA antagonists reverse the antidepressant effect of SSRIs. Further, there is evidence that DA itself is released in many aversive circumstances, and is involved in aversively motivated behaviors like avoidance.

40In our terms, apart from the aspects of the interaction of dopamine and 5-HT that were explored in Figures 5B and S2, there are a couple of other effects. First, inhibition in our model has the consequence of increasing the average expected reward. As such, tonic dopamine, which has been suggested to report such a quantity, would be increased when 5-HT is boosted, and potentially vice versa. This would compete with the more direct effect that 5-HT inhibits actions, and particularly inhibits actions supported by dopaminergic predictions or rewards, and thus high levels of 5-HT might also depress levels of tonic dopamine, more in line with accounts that stress the opponent role of dopamine and serotonin.

41The second complexity (Boureau, personal communication) is that active defense (such as active avoidance) requires energizing, and indeed appears to be controlled by the (presumably dopaminergically reported) appetitive outcome of reaching a state of safety rather than the (presumably serotonergically mediated) outcome of leaving a state of fear. That is, it appears that dopamine reports the rewards reaped from avoiding or controlling aversive outcomes.

42We mentioned the mirror notion that the relationship between 5-HT and inhibition arising through aversive predictions is parallel to the obverse relationship for dopamine and engagement/approach through appetitive predictions. In this case, appetitively directed chains of thoughts would be favored. Indeed, Smith et al., in their work on the conditioned avoidance model of schizophrenia, suggested something rather like this. In their account, dopamine controls the extent of search through a forward model, although they did not couple this to dopamine's involvement in appetitive prediction.

43In all, disentangling and elucidating these varied relations between dopamine and serotonin is a pressing task.

44It would be reasonable to argue that the present model is more relevant to anxiety than depression. There is at best a somewhat fuzzy distinction between the two in terms of risk factors and pharmacology, and they are extraordinarily co-morbid. There is also no complete definition of either disease in terms of the sort of reinforcement mechanisms that we have been considering.

45While depressive (but not anxious) symptoms can be reinduced by TrD in a subpopulation of patients, TrD it is not the only such manipulation, and it is not effective in all patients. Patients who are responsive to seratonin–norepinephrine reuptake inhibitors (SNRIs) are more sensitive to catecholamine depletion by α-methyl-tyrosine than TrD, and a recent report with a DA antagonist successfully re-induced depressive symptoms in formerly depressed people. The latter authors suggest that DA may be a “final common path” for depression, and may relate more to the depressive state than serotonin, which in turn may be more important in defining a trait. In addition, only 50% of formerly depressed subjects do respond to TrD, and a pooled analysis of 71 formerly depressed subjects found that previous response to SSRIs had less predictive power for TrD response than chronicity of the depressive disorder and sex. As mentioned, resolving the actual relative contribution of serotoninergic inhibition will be tricky.

46In sum, the findings in this study argue for an involvement of the serotonin reuptake mechanism in mood disorders such as anxiety and depression in the following manner: due to a decreased efficiency of the transporter, increased behavioral inhibition results in acquisition of overly optimistic values. Such value functions are adaptive, but only in conjunction with strong behavioral inhibition. On the other hand, they do render the individual highly sensitive to large decreases in average experienced rewards when serotonin function is reduced. This might underlie a (controversially) larger sensitivity to TrD and SSRIs of persons with the short 5HTTLPR allele (see). Returning to the sequential decision-making tasks suggested above, this study would predict that the short 5HTTLPR allele would be associated with more reflexive avoidance of states predictive of punishment, and it may be possible to assess this with differential effects of TrD on carriers with the short and long 5HTTLPR allele.

47A further, more involved conjecture, which returns to the fact that serotonin is not the sole causative agent in depression, is that it is the effects of reduced 5-HT on affective experience that leads to the various symptoms of depression, acting via the otherwise normative operation of the multiple systems involved in behavioral control. For instance, we have argued that the consequences of 5-HT reduction include unexpected punishments, large negative prediction errors, and a drop in average reward. These changes in the statistics of reward demand explanation, for example in terms of a shift in the characteristics of the environment, and should cause normative behavioral responses. In particular, the unsignalled aversion that comes independent of the subject's actions can be seen as a form of uncontrollable punishment. Uncontrollability lies at the heart of an important characterization of depression centred around learned helplessness.

48We concentrated on the effects of reduced 5-HT rather than on the reasons for this reduction. The obvious option is that it is a pathological result from processes operating at a purely cellular level. However, it could also arise as a normative meta-adaptation to the statistics of experienced punishments and rewards. Formalizing this fully would require a more general theory of inhibition—what level of inhibition is optimal? Tools for the characterisation of the trade-off between accurate knowledge about a state's value and the cost incurred in learning about it are already in existence and might be applicable to aspects of the present case.