Adaptive History Biases Result from Confidence-Weighted Accumulation of past Choices

Abstract

Perceptual decision-making is biased by previous events, including the history of preceding choices: observers tend to repeat (or alternate) their judgments of the sensory environment more often than expected by chance. Computational models postulate that these so-called choice history biases result from the accumulation of internal decision signals across trials. Here, we provide psychophysical evidence for such a mechanism and its adaptive utility. Male and female human observers performed different variants of a challenging visual motion discrimination task near psychophysical threshold. In a first experiment, we decoupled categorical perceptual choices and motor responses on a trial-by-trial basis. Choice history bias was explained by previous perceptual choices, not motor responses, highlighting the importance of internal decision signals in action-independent formats. In a second experiment, observers performed the task in stimulus environments containing different levels of autocorrelation and providing no external feedback about choice correctness. Despite performing under overall high levels of uncertainty, observers adjusted both the strength and the sign of their choice history biases to these environments. When stimulus sequences were dominated by either repetitions or alternations, the individual degree of this adjustment of history bias was about as good a predictor of individual performance as individual perceptual sensitivity. The history bias adjustment scaled with two proxies for observers' confidence about their previous choices (accuracy and reaction time). Together, our results are consistent with the idea that action-independent, confidence-modulated decision variables are accumulated across choices in a flexible manner that depends on decision-makers' model of their environment. SIGNIFICANCE STATEMENT Decisions based on sensory input are often influenced by the history of one's preceding choices, manifesting as a bias to systematically repeat (or alternate) choices. We here provide support for the idea that such choice history biases arise from the context-dependent accumulation of a quantity referred to as the decision variable: the variable's sign dictates the choice and its magnitude the confidence about choice correctness. We show that choices are accumulated in an action-independent format and a context-dependent manner, weighted by the confidence about their correctness. This confidence-weighted accumulation of choices enables decision-makers to flexibly adjust their behavior to different sensory environments. The bias adjustment can be as important for optimizing performance as one's sensitivity to the momentary sensory input.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 54

1. 1.

   Abrahamyan A, Silva LL, Dakin SC, Carandini M, Gardner JL (2016 ) Adaptable history biases in human perceptual decisions. Proc Natl Acad Sci U S A 113:E3548–E3557.

   Not in our library10.1073/pnas.1518786113

2. 2.

   Akaishi R, Umeda K, Nagase A, Sakai K (2014 ) Autonomous mechanism of internal choice estimate underlies decision inertia. Neuron 81:195–206.

   Not in our library10.1016/j.neuron.2013.10.018

3. 3.

   Allefeld C, Soon CS, Bogler C, Heinzle J, Haynes J-D (2013 ) Sequential dependencies between trials in free choice tasks. arXiv arXiv:1311.0753.

   Not in our library

4. 4.

   Benjamini Y, Hochberg Y (1995 ) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 57:289–300.

   Not in our library

5. 5.

   Bennur S, Gold JI (2011 ) Distinct representations of a perceptual decision and the associated oculomotor plan in the monkey lateral intraparietal area. J Neurosci 31:913–921.

   Not in our library10.1523/jneurosci.4417-10.2011

6. 6.

   Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD (2006 ) The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol Rev 113:700–765.

   Not in our library10.1037/0033-295x.113.4.700

7. 7.
   Response repetition biases in human perceptual decisions are explained by activity decay in competitive attractor models · 2016

   Not checked
8. 8.

   Brody CD, Hanks TD (2016 ) Neural underpinnings of the evidence accumulator. Curr Opin Neurobiol 37:149–157.

   Not in our library10.1016/j.conb.2016.01.003

9. 9.

   Busse L, Ayaz A, Dhruv NT, Katzner S, Saleem AB, Schölvinck ML, Zaharia AD, Carandini M (2011 ) The detection of visual contrast in the behaving mouse. J Neurosci 31:11351–11361.

   Not in our library10.1523/jneurosci.6689-10.2011

10. 10.

    de Lange FP, Rahnev DA, Donner TH, Lau H (2013 ) Prestimulus oscillatory activity over motor cortex reflects perceptual expectations. J Neurosci 33:1400–1410.

    Not in our library10.1523/jneurosci.1094-12.2013

11. 11.

    Donner TH, Siegel M, Fries P, Engel AK (2009 ) Buildup of choice-predictive activity in human motor cortex during perceptual decision making. Curr Biol 19:1581–1585.

    Not in our library10.1016/j.cub.2009.07.066

12. 12.

    Efron B, Tibshirani R (1998 ) An introduction to the bootstrap. Boca Raton, FL: Chapman and Hall/CRC.

    Not in our library

13. 13.

    Fernberger SW (1920 ) Interdependence of judgments within the series for the method of constant stimuli. J Exp Psychol 3:126–150.

    Not in our library10.1037/h0065212

14. 14.

    Fischer J, Whitney D (2014 ) Serial dependence in visual perception. Nat Neurosci 17:738–743.

    Not in our library10.1038/nn.3689

15. 15.

    Fritsche M, Mostert P, de Lange FP (2017 ) Opposite effects of recent history on perception and decision. Curr Biol 27:590–595.

    Not in our library10.1016/j.cub.2017.01.006

16. 16.

    Fründ I, Wichmann FA, Macke JH (2014 ) Quantifying the effect of intertrial dependence on perceptual decisions. J Vis 14(7):9 1–6.

    Not in our library10.1167/14.7.9

17. 17.
    Normative evidence accumulation in unpredictable environments · 2015

    No notice
18. 18.

    Glimcher PW (2011 ) Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci U S A 108:15647–15654.

    Not in our library10.1073/pnas.1014269108

19. 19.

    Gold JI, Shadlen MN (2007 ) The neural basis of decision making. Annu Rev Neurosci 30:535–574.

    Not in our library10.1146/annurev.neuro.29.051605.113038

20. 20.

    Gold JI, Law CT, Connolly P, Bennur S (2008 ) The relative influences of priors and sensory evidence on an oculomotor decision variable during perceptual learning. J Neurophysiol 100:2653–2668.

    Not in our library10.1152/jn.90629.2008

21. 21.

    Goldfarb S, Wong-Lin K, Schwemmer M, Leonard NE, Holmes P (2012 ) Can post-error dynamics explain sequential reaction time patterns? Front Psychol 3:213.

    Not in our library10.3389/fpsyg.2012.00213

22. 22.

    Hebart MN, Donner TH, Haynes JD (2012 ) Human visual and parietal cortex encode visual choices independent of motor plans. Neuroimage 63:1393–1403.

    Not in our library10.1016/j.neuroimage.2012.08.027

23. 23.

    Hebart MN, Schriever Y, Donner TH, Haynes JD (2016 ) The relationship between perceptual decision variables and confidence in the human brain. Cereb Cortex 26:118–130.

    Not in our library10.1093/cercor/bhu181

24. 24.
    History-based action selection bias in posterior parietal cortex · 2017

    No notice
25. 25.

    Kelly SP, O'Connell RG (2015 ) The neural processes underlying perceptual decision making in humans: recent progress and future directions. J Physiol-Paris 109:27–37.

    Not in our library10.1016/j.jphysparis.2014.08.003

26. 26.

    Kepecs A, Uchida N, Zariwala HA, Mainen ZF (2008 ) Neural correlates, computation and behavioural impact of decision confidence. Nature 455:227–231.

    Not in our library10.1038/nature07200

27. 27.

    Kiani R, Shadlen MN (2009 ) Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324:759–764.

    Not in our library10.1126/science.1169405

28. 28.

    Kiani R, Hanks TD, Shadlen MN (2008 ) Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. J Neurosci 28:3017–3029.

    Not in our library10.1523/jneurosci.4761-07.2008

29. 29.

    Kim TD, Kabir M, Gold JI (2017 ) Coupled decision processes update and maintain saccadic priors in a dynamic environment. J Neurosci 37:3632–3645.

    Not in our library10.1523/jneurosci.3078-16.2017

30. 30.

    Kohn A (2007 ) Visual adaptation: physiology, mechanisms, and functional benefits. J Neurophysiol 97:3155–3164.

    Not in our library10.1152/jn.00086.2007

31. 31.

    Lak A, Costa GM, Romberg E, Koulakov AA, Mainen ZF, Kepecs A (2014 ) Orbitofrontal Cortex Is Required for Optimal Waiting Based on Decision Confidence. Neuron 84:190–201.

    Not in our library10.1016/j.neuron.2014.08.039

32. 32.

    Leopold DA, Wilke M, Maier A, Logothetis NK (2002 ) Stable perception of visually ambiguous patterns. Nat Neurosci 5:605–609.

    Not in our library10.1038/nn0602-851

33. 33.

    Meyniel F, Maheu M, Dehaene S (2016 ) Human inferences about sequences: a minimal transition probability model. PLoS Comput Biol 12:e1005260.

    Not in our library10.1371/journal.pcbi.1005260

34. 34.

    Morcos AS, Harvey CD (2016 ) History-dependent variability in population dynamics during evidence accumulation in cortex. Nat Neurosci 19:1672–1681.

    Not in our library10.1038/nn.4403

35. 35.

    Mulder MJ, Wagenmakers EJ, Ratcliff R, Boekel W, Forstmann BU (2012 ) Bias in the brain: a diffusion model analysis of prior probability and potential payoff. J Neurosci 32:2335–2343.

    Not in our library10.1523/jneurosci.4156-11.2012

36. 36.

    O'Connell RG, Dockree PM, Kelly SP (2012 ) A supramodal accumulation-to-bound signal that determines perceptual decisions in humans. Nat Neurosci 15:1729–1735.

    Not in our library10.1038/nn.3248

37. 37.

    Ossmy O, Moran R, Pfeffer T, Tsetsos K, Usher M, Donner TH (2013 ) The timescale of perceptual evidence integration can be adapted to the environment. Curr Biol 23:981–986.

    Not in our library10.1016/j.cub.2013.04.039

38. 38.

    Padoa-Schioppa C (2013 ) Neuronal origins of choice variability in economic decisions. Neuron 80:1322–1336.

    Not in our library10.1016/j.neuron.2013.09.013

39. 39.

    Pape AA, Siegel M (2016 ) Motor cortex activity predicts response alternation during sensorimotor decisions. Nat Commun 7:13098.

    Not in our library10.1038/ncomms13098

40. 40.

    Pouget A, Drugowitsch J, Kepecs A (2016 ) Confidence and certainty: distinct probabilistic quantities for different goals. Nat Neurosci 19:366–374.

    Not in our library10.1038/nn.4240

41. 41.

    Ratcliff R, McKoon G (2008 ) The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput 20:873–922.

    Not in our library10.1162/neco.2008.12-06-420

42. 42.

    Rouder JN, Speckman PL, Sun D, Morey RD, Iverson G (2009 ) Bayesian t tests for accepting and rejecting the null hypothesis. Psychon Bull Rev 16:225–237.

    Not in our library10.3758/pbr.16.2.225

43. 43.

    Sanders JI, Hangya B, Kepecs A (2016 ) Signatures of a statistical computation in the human sense of confidence. Neuron 90:499–506.

    Not in our library10.1016/j.neuron.2016.03.025

44. 44.
    Cortical Network Dynamics of Perceptual Decision-Making in the Human Brain · 2011

    Not checked
45. 45.

    St. John-Saaltink E, Kok P, Lau HC, de Lange FP (2016 ) Serial dependence in perceptual decisions is reflected in activity patterns in primary visual cortex. J Neurosci 36:6186–6192.

    Not in our library10.1523/jneurosci.4390-15.2016

46. 46.

    Steiger JH (1980 ) Tests for comparing elements of a correlation matrix. Psychol Bull 87:245–251.

    Not in our library10.1037/0033-2909.87.2.245

47. 47.

    Sugrue LP, Corrado GS, Newsome WT (2004 ) Matching behavior and the representation of value in the parietal cortex. Science 304:1782–1787.

    Not in our library10.1126/science.1094765

48. 48.

    Sutton RS, Barto AG (1998 ) Reinforcement learning: an introduction (adaptive computation and machine learning). Cambridge, MA: MIT.

    Not in our library

49. 49.

    Tsetsos K, Pfeffer T, Jentgens P, Donner TH (2015 ) Action planning and the timescale of evidence accumulation. PLoS One 10:e0129473.

    Not in our library10.1371/journal.pone.0129473

50. 50.
    Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias · 2017

    No notice
51. 51.

    van den Brink RL, Wynn SC, Nieuwenhuis S (2014 ) Post-error slowing as a consequence of disturbed low-frequency oscillatory phase entrainment. J Neurosci 34:11096–11105.

    Not in our library10.1523/jneurosci.4991-13.2014

52. 52.

    Wang XJ (2002 ) Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36:955–968.

    Not in our library10.1016/s0896-6273(02)01092-9

53. 53.

    Wang XJ (2008 ) Decision making in recurrent neuronal circuits. Neuron 60:215–234.

    Not in our library10.1016/j.neuron.2008.09.034

54. 54.

    Yu AJ, Cohen JD (2008 ) Sequential effects: superstition or rational behavior. Adv Neural Inf Process Syst 21:1873–1880.

    Not in our library

Cited by papers we hold 2

• A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception · 2020
  No notice
• Choice history biases subsequent evidence accumulation · 2019
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.