Even in the face of disappointment resulting from a lack of expected reward, it is important not to give up. Instead, one must learn to actively overcome the absence of the expected reward and adjust behavior to increase the likelihood of obtaining it again.
The ability to cope with the lack of expected reward is key to pursuing uncertain rewards and ultimately obtaining more rewards. Failure to develop this ability can lead to depressive states, while excessive pursuit of a particular reward despite negative consequences may lead to addiction.
In animal behavior, the inability to actively cope with negative outcomes in foraging and courtship behaviors, which are often uncertain and offer limited choice options, can significantly affect the survival of the species.
Adaptive Behavior and Partial Reinforcement Extinction Effect
When animals are required to pursue a probabilistic reward by performing a specific action without any choice options, they exhibit the ability to actively switch their behaviors towards pursuing the next opportunity to obtain the reward, even after it is not presented by chance.
This phenomenon is known as the partial reinforcement extinction effect and represents a paradoxical reward effect, as the expected value of a 50% reward is lower than that of a 100% reward.
The ability to pursue the next reward after a lack of reward is not solely based on the instantaneous change in reward value, as the expected value of the reward decreases after experiencing no reward. Instead, this ability is believed to be based on the animal’s experience of eventually obtaining a reward following a period of no reward.
Through learning the relationship between occasional lack of expected reward and antecedent cues, animals can predict the absence of reward in advance and cope more adaptively with the negative outcome. In situations where the likelihood of not receiving a reward is high, animals can learn to disengage from the situation and actively switch their focus towards the next opportunity earlier.
Dopamine Neurons and Reward Prediction Error
One potential candidate for the neural basis of coping with the lack of expected reward is the midbrain dopamine (DA) neurons. Midbrain DA neurons are well-known for providing an error signal known as the reward prediction error (RPE), which reflects the disparity between obtained reward and expected reward. RPE-type DA neurons play a crucial role in moment-by-moment computation of values and value-based learning.
However, the activity of RPE neurons decreases when faced with unexpected lack of reward, leading to a decrease in reward value and supporting negative learning. Thus, RPE neurons alone do not directly support the ability to cope with the lack of expected reward and adjust behavior towards the next opportunity to obtain the reward. Additionally, recent studies have revealed that DA neurons are heterogeneous and signal more than just RPE. However, the specific role of DA neurons in actively coping with the lack of expected reward remains unknown.
Study Objective and Methodology
To investigate the role of DA neurons in actively coping with the lack of expected reward, researchers developed a task that required head-restrained rats to continuously pursue a probabilistic reward by repeating a specific sequence of actions without any choice options. This task allowed for the monitoring of active behavioral switching towards the next opportunity to obtain a reward after its omission. The researchers employed various techniques, including in vivo single-unit recording and single-cell calcium imaging from DA neurons in the ventral tegmental area (VTA), DA measurement in the nucleus accumbens (NAc), and optogenetics.
Key Findings
The study revealed that a subset of DA neurons in the lateral VTA and DA levels in a particular region of the NAc exhibited increased responses to unexpected reward omission and decreased responses to unexpected reward. These responses were interpreted as signaling an error that enables active coping with the lack of reward based on the expected reward. Notably, these responses were slower compared to the opposite responses of RPE-type DA neurons.
Further experiments involving reward extinction tasks and the transition to a Pavlovian task demonstrated that the DA error signal in the NAc primarily facilitated the learning process, enabling the adjustment of behavior to actively overcome unexpected lack of reward.
Conclusion
The findings of this study provide valuable insights into the role of dopamine in pursuing uncertain rewards and coping with the lack of expected reward. The ability to actively adjust behavior in the face of unexpected outcomes is crucial for efficient reward-seeking and optimal decision-making. By cooperating with RPE-type DA neurons, the DA error signal enables animals to pursue uncertain rewards adaptively, ultimately leading to the attainment of more rewards.
Understanding the neural mechanisms underlying the active coping with lack of expected reward may have implications for human conditions such as depression and addiction, where disturbances in reward processing and decision-making are often observed. Further research in this area could contribute to the development of therapeutic interventions aimed at enhancing adaptive behavior in the pursuit of uncertain rewards.
