Mood as Representation of Momentum

1The enormous and disruptive impact of mood disorders in society, might suggest that mood (see Glossary) is an evolutionary relic that may have been advantageous for early humans but impedes adaptive behavior in the modern world. Indeed, we often attribute irrational behavior to the emotional state of a person,,,. Our language also reflects this view, with expressions such as ‘moody’ and ‘being in a mood’ carrying negative connotations. We argue that moods serve an important role in adaptive behavior, even in the modern world. We elucidate this role by considering recent findings regarding the dynamics of mood, as well as its interaction with the processes of learning and decision making. Based on these findings, we propose that moods benefit ‘moody’ agents by mitigating inefficiencies that can arise in the process of learning about the natural environment.

2Advances in computational modeling have greatly facilitated an understanding of how humans learn from outcomes to make better decisions,,. Recently, scientists have begun to utilize the same computational framework to study the dynamics of human emotional states in health and in mental disorders, focusing on how these states affect and are affected by learning and decision-making processes,,. In particular, two burgeoning lines of research have sought to characterize precisely, on the one hand, the causes of moods, and on the other the consequences of mood states for learning and decision making. We first review these two largely separate strands of research and then integrate them within a coherent theoretical framework. We propose that mood represents the overall momentum of reward in the environment, and that this representation serves to facilitate efficient learning by accounting for statistical dependencies in the availability of rewards that are prevalent in nature.

3To understand the function of mood, we first need to consider its causes. A vast psychological literature demonstrates that mood can be manipulated via a range of techniques. Presentation of a film or story with emotional content is a common and effective mood-induction technique. Other stimuli that reliably affect mood include music, self-referential statements, observed social interactions, and facial expressions. While these stimuli are easy to present in laboratory experiments, they are not readily quantifiable and are typically applied categorically, without variation in either quantity or intensity. Monetary outcomes, by contrast, can be precisely controlled and have also been shown to affect mood,.

4Another line of research, originating primarily in an economics literature, considers real-world circumstances that covary with subjective well-being. Such research is inherently correlational, but has identified various factors that impact on mood, including outcomes of sporting events and levels of sunshine,. Moreover, to measure the dynamics of emotional state that are relevant to understanding adaptive behavior, well-being researchers have developed experience-sampling techniques that probe participants as to their current subjective state while they go about their daily lives,. These techniques, which involve periodically asking participants about their current emotional state and what they are doing, are considered the ‘gold standard’ for investigating real-world emotion. Experience-sampling and related methods, such as the day-reconstruction method,, show that in typical individuals some activities (e.g., conversation, eating) are consistently related to higher happiness ratings, while other activities (e.g., work, commuting) are consistently related to lower happiness ratings. Some studies have also applied these methods to study differences in well-being across individuals, showing greater mood instability in bipolar disorder, and greater negative affect in depression compared to healthy subjects.

5Recent research has used experience sampling to examine momentary mood fluctuations during a laboratory-based probabilistic reward task in which monetary rewards varied from trial to trial. The main conclusion of the study was that happiness depends not on how well things are going (in terms of cumulative earnings) but whether they are going better than expected. In particular, self-reported happiness depended on ‘reward prediction errors’ (RPEs; Box 1), that is, the difference between expected outcomes and obtained outcomes. The laboratory results were also replicated in a large-scale smartphone-based experiment with 18420 participants. In addition, blood-oxygen-level dependent (BOLD) activity measured using functional magnetic resonance imaging (fMRI) in the ventral striatum, a target area for dopamine neurons that represent RPEs,,,,,,, correlated with RPEs and with subsequent happiness ratings. This is consistent with a possible role for dopaminergic RPE signals in determining mood. Indeed, pharmacologically boosting dopamine levels has recently been shown to increase the happiness that results from particular types of reward.

6It has long been thought that happiness induces a ‘rosy’ perspective, whereas a depressed mood engenders negative judgments,,. More recently, researchers have used computational methods in laboratory experiments to precisely quantify the effects of emotional state on behavior. In one study, mood was manipulated using a wheel-of-fortune draw in which participants either won or lost a relatively large sum of money. In participants independently identified as being less emotionally stable, winning the draw increased self-reported happiness and the effect of subsequent rewards on subsequent choices. By contrast, losing the draw reduced happiness, as well as neural responses to subsequent rewards, and the effect of those rewards on choices (Figure 1). Manipulating mood by viewing emotional facial expressions is also known to induce a bias in both neural responses to rewards and learning from rewards. Moreover, a depressed mood is associated with a reduced effect of rewards on subsequent choices,, an effect that is better explained by reduced valuation of reward than by a reduced rate of learning. A similar relationship may also hold between an anxious emotional state and perception of aversive outcomes: stressed humans and rats respond, neurally and behaviorally, to aversive outcomes and ambiguous stimuli as if they are worse than they actually are,,.

7Other studies have explored additional effects of mood on decision making, many of which can be similarly understood as reflecting a biased perception of reward or of stimuli indicating reward availability. For example, positive mood induces risk-taking in laboratory experiments, and in real financial markets,, possibly by biasing upwards the perceived probability of future positive outcomes. In addition, repeated positive RPEs, which should improve mood, invigorate reward-seeking behavior,,,, possibly reflecting an implicit belief in greater reward availability. Furthermore, a positive emotional state reinforces, and a negative emotional state inhibits, one's current mode of thought, presumably by biasing perception of how well that mode of thought is functioning,,. Finally, many studies suggest that a depressed mood is associated with greater attention or sensitivity to negative information, an effect that may underlie biased perception of outcomes. Notably, both effects can be seen to reflect an implicit belief that things are worse than the objective evidence suggests,.

8The upshot of this research is that mood induced by a stimulus can affect judgment about other, potentially unrelated, stimuli. Indeed, this property may have given mood its reputation as a rich fountain for irrational behavior. Any attempt to rationalize moods must therefore explain how such biased judgments, which in some cases may reinforce irrelevant actions, nevertheless promote adaptive behavior.

9According to current theories, agents can maximize reward by keeping track of how much reward is obtained in each experienced state of the environment, and then choosing actions that return them to the states in which such reward has been most abundant,. For example, an animal using such a mechanism can learn which specific trees bear more fruit and focus its foraging efforts accordingly. This type of ‘reinforcement learning’ algorithm constitutes a powerful way to learn about the environment and converges upon optimal behavioral policies (e.g., ). However, there are many real-world situations for which such an algorithm may be poorly equipped. We propose that the information represented by mood is used to mitigate problems that arise in the application of reinforcement learning to such real-world problems.

10One such learning inefficiency arises when changes in reward in different states are correlated. For instance, increased rainfall or sunshine may cause fruit to become more abundant in all trees simultaneously. In this situation, it makes little sense to update expectations for each tree independently, and a more efficient learning algorithm would instead infer a general increase in reward and update expectations for all related trees accordingly. We suggest this is the function of mood. If fruit becomes more abundant in all trees, a foraging animal will be positively surprised multiple times as it visits adjacent trees and, as a result, its mood will improve. Improved mood will bias the subjective reward for each subsequent fruit upwards, and because these observations are used to update expectations, expectations associated with these trees will be adjusted upwards more rapidly than they would be otherwise. In essence, the effect of positive surprises will be enhanced as more positive surprises are encountered.

11Through the existence of mood, as an animal learns from experience, its expectations come to reflect not only the reward associated with each particular state (e.g., each tree), but also recent overall changes in the availability of reward in its environment. In this way, learning can account, albeit approximately, for the impact of multiple general environmental factors without having to directly infer the number of factors or the extent of their impact (Box 2). We have described one scenario in which this can be beneficial, but such a generalization mechanism can improve the efficiency of learning in any environment in which different sources of reward are interdependent. Indeed, such interdependencies may be the rule rather than the exception, for both animals and humans, because success in acquiring skills, material resources, social status, and even mating partners can be tightly correlated.

12Mood can also be useful for learning in another common scenario in which current changes in reward predict later changes in reward. Many processes in the natural world have such momentum. For instance, initial increases in fruit availability may indicate that spring is coming and that further increases are probable. In such a case, a positive mood would represent inference of a positive momentum – which would, in turn, bias perception of subsequent rewards upwards. Because rewards would then be perceived as better than they really are, expectations would be updated upwards quickly and would catch up with rising rewards. Similarly, if reward availability is decreasing in an environment (e.g., winter is coming), then a negative mood leads to rewards being perceived as less good than they actually are (even though increasingly rare rewards still result in positive RPEs) and expectations will catch up with declining rewards, allowing behavior to be quickly adjusted (e.g., hibernate). In accordance with this idea, the relationship between mood and reward perception suggested by the recent literature can be formally derived as statistical inference of average reward and its momentum (Box 2).

13Identifying the function of mood points to how it might be compromised, potentially leading to maladaptive behavior. The proper function of mood, as we delineate, increases the efficiency of learning about the environment when emotional reactions to changes in reward are appropriate in intensity and duration. Positive or negative moods maximize their usefulness by persisting only until expectations are fully updated in accordance with changes in rewards. Indeed, happiness eventually returns to a baseline level even following highly significant changes in circumstances, including winning the lottery, whereas excessive happiness can induce maladaptive behavior,. This homeostasis crucially depends on appropriate updating of expectations, that is, on the integrity of learning processes. If, for instance, expectations are not updated downwards following outcomes that are worse than expected, encountering the same outcomes again would continue to generate negative surprises indefinitely, inducing a negative mood. In fact, in environments with even modest amounts of variability or randomness, it suffices that the rate of learning (ηt in Box 2) is lower for negative than for positive surprises in order for overly optimistic expectations to develop. As a result, the frequency and magnitude of negative surprises would increase, leading to low mood (Figure 2A). Indeed, low serotonin function, which has been associated with impaired learning from negative outcomes, is linked to both depression and risk-taking behavior, two co-occurring conditions,,, that may stem from lower negative learning rates and consequent overly optimistic expectations. Interestingly, in the general population, positive mood and risk aversion predominate,, possibly indicating higher learning rates for negative than for positive surprises, which could reflect the greater importance to survival of avoiding negative outcomes.

14More generally, if a negative mood is too intense or persists for too long, positive feedback dynamics can exacerbate the situation. Bad mood will result in subsequent outcomes being perceived as worse than they really are, leading to further negative surprises that induce further decreases in mood, which in turn will make outcomes seem even worse, and so on (Figure 2B). As expectations are updated to match biased perception of outcomes, overly pessimistic expectations can develop. Only if expectations catch up with perceived outcomes will the escalatory dynamics abate and de-escalation begin. Empirical findings indicate that an affective perceptual bias precedes ostensible changes in mood in response to treatment with serotonergic drug in major depressive disorder, an observation that supports a possible role for such a feedback cycle in the unfolding of depressive episodes.

15If mood does eventually return to baseline levels, the pessimistic expectations that developed when mood was lower may now lead to increased positive surprises and improved mood. In some individuals, good mood may also persist and a positive feedback cycle may develop in the opposite direction, with good mood biasing perception of outcomes upwards, thereby increasing positive surprises, which further improve mood (Figure 2B). Overly optimistic expectations will develop, setting the stage again for negative surprises, which decrease mood, and potentially turning the cycle in the negative direction again. The overall result could be oscillatory dynamics, as observed in bipolar disorder, in which expectations and mood cyclically fluctuate even in the absence of objective changes in the external environment.

16Thus, while learning makes outcomes more predictable and promotes habituation to these outcomes, the biasing effect of mood on the perception of outcomes has the opposite sensitizing effect of increasing responsivity to outcomes. A predisposition to emotional instability could therefore result from any factor that strengthens the sensitizing effect of mood or that weakens the habituating effects of learning (e.g., ηt << and high ft in Box 2). Laboratory evidence suggests that such sensitization may indeed underlie emotional instability. Specifically, participants who report being more emotionally unstable show stronger effects of outcomes on their feelings, as well as on their evaluation of subsequent outcomes. It is notable that clinically pathological escalation in the direction of negative mood seems to be more prevalent than escalation of positive moods. Negative moods might escalate more frequently because of a stronger biasing effect, possibly reflecting the greater overall adaptive significance of reacting quickly to negative changes in momentum.

17We have outlined a normative perspective on mood, according to which mood serves as a representation of the momentum of changes in reward. This momentum signal can be used to adjust learning to account for dependencies between different states and across time. How this momentum is represented in the brain is an open question (see Outstanding Questions), although some studies implicate the neuromodulators serotonin and dopamine,,,,. Our approach suggests different ways in which the function of mood might be disrupted, and we have described two specific dysfunctions that might contribute to the emergence of depression and mood instability. The proper function of mood might also lead to maladaptive behavior in particular scenarios. Thus, moods can reflect inference of momentum even when there is none in the environment, leading to excessive optimism or pessimism. However, the ubiquity of moods and the extent of their impact on our lives tells us that, throughout the course of evolution, our moodiness must have conferred a significant competitive advantage. Being moody at times may be a small price to pay for the ability to adapt quickly when facing momentous environmental changes.How is mood represented in the brain?How do long-lasting moods interact with and relate to more short-lasting emotions?Can an anxious mood be understood as a representation of momentum in aversive outcomes?How can our model, which was derived from studies of healthy individuals, be utilized to explain the dynamics of mood in psychiatric mood disorders?How do antidepressants, mood stabilizers, and other therapeutic interventions affect the dynamics of mood?

18How is mood represented in the brain?

19How do long-lasting moods interact with and relate to more short-lasting emotions?

20Can an anxious mood be understood as a representation of momentum in aversive outcomes?

21How can our model, which was derived from studies of healthy individuals, be utilized to explain the dynamics of mood in psychiatric mood disorders?

22How do antidepressants, mood stabilizers, and other therapeutic interventions affect the dynamics of mood?