• PNAS Sustainability Science
  • Sign-up for PNAS eTOC Alerts

Dopamine reward prediction error signal codes the temporal evaluation of a perceptual decision report

  1. Néstor Pargaa,b
  1. aDepartamento de Física Teórica, Universidad Autónoma de Madrid, Cantoblanco 28049, Madrid, Spain;
  2. bCentro de Investigación Avanzada en Física Fundamental, Universidad Autónoma de Madrid, Cantoblanco 28049, Madrid, Spain;
  3. cInstituto de Neurobiología, Universidad Nacional Autónoma de México, 76230 Querétaro, México;
  4. dEl Colegio Nacional, 06020 México DF, México;
  5. eInstituto de Fisiología Celular-Neurociencias, Universidad Nacional Autónoma de México, 04510 México DF, México
  1. Contributed by Ranulfo Romo, October 14, 2017 (sent for review July 13, 2017; reviewed by Stefano Panzeri and Joe Paton)

Significance

How do animals learn to take correct actions based on uncertain observations? Although dopamine neurons can guide learning in conditioning experiments, their role in decision-making tasks is poorly understood. How can they code reward prediction errors and simultaneously exhibit decision-making processes and beliefs about the state of the environment? Using modeling work and analysis of data recorded from monkeys detecting weak stimuli delivered at uncertain times, we propose some answers to these questions. Specifically, we explain how the certainty about the presence of a stimulus is communicated to midbrain dopamine neurons through transient cortical events and why that certainty becomes visible in their response to a relevant task event.

Abstract

Learning to associate unambiguous sensory cues with rewarded choices is known to be mediated by dopamine (DA) neurons. However, little is known about how these neurons behave when choices rely on uncertain reward-predicting stimuli. To study this issue we reanalyzed DA recordings from monkeys engaged in the detection of weak tactile stimuli delivered at random times and formulated a reinforcement learning model based on belief states. Specifically, we investigated how the firing activity of DA neurons should behave if they were coding the error in the prediction of the total future reward when animals made decisions relying on uncertain sensory and temporal information. Our results show that the same signal that codes for reward prediction errors also codes the animal’s certainty about the presence of the stimulus and the temporal expectation of sensory cues.

Footnotes

  • ?1To whom correspondence should be addressed. Email: rromo{at}ifc.unam.mx.
  • Author contributions: R.R. and N.P. designed research; V.d.L. and R.R. performed research; S.S. and N.P. analyzed data; S.S. and N.P. wrote the paper; S.S. performed data analysis and modeling; and R.R. and N.P. supervised all the stages of the project.

  • Reviewers: S.P., Italian Institute of Technology; and J.P., Champalimaud Foundation.

  • The authors declare no conflict of interest.

  • This article contains supporting information online at www.danielhellerman.com/lookup/suppl/doi:10.1073/pnas.1712479114/-/DCSupplemental.

Published under the PNAS license.

Online Impact

                                      1. 613261309 2018-02-21
                                      2. 6972481308 2018-02-21
                                      3. 2758991307 2018-02-21
                                      4. 5213301306 2018-02-21
                                      5. 6402651305 2018-02-21
                                      6. 975701304 2018-02-20
                                      7. 619701303 2018-02-20
                                      8. 6291841302 2018-02-20
                                      9. 8182271301 2018-02-20
                                      10. 7717531300 2018-02-20
                                      11. 2811781299 2018-02-20
                                      12. 9132041298 2018-02-20
                                      13. 285331297 2018-02-20
                                      14. 2838721296 2018-02-20
                                      15. 274321295 2018-02-20
                                      16. 2027431294 2018-02-20
                                      17. 2738641293 2018-02-20
                                      18. 9584601292 2018-02-20
                                      19. 9002021291 2018-02-20
                                      20. 7995901290 2018-02-20