As our brains take in information about the world and use it to steer our actions, two key principles guide our choices: seek pleasure and avoid pain. Researchers at Cold Spring Harbor Laboratory (CSHL) have zeroed in on an information-processing hub in the brains of mice to discover how neurons there divide the labor to handle these opposing behavioral motivations.
Their work, reported December 31, 2019 in the journal Neuron, reveals that different classes of neurons control positive and negative motivation, sending opposing signals along a shared motivation-processing brain circuit.
Ultimately, the balance of activity between these two groups of cells may determine whether a person acts to seek out pleasurable experiences or avoid negative ones, says CSHL Professor Bo Li, who led the study.
Li wants to understand the brain’s motivation-processing circuits because the behaviors they control are often disrupted in people with mental illness.
People suffering from depression may stop doing things that once gave them pleasure, for example, whereas people with anxiety disorders may go to greater lengths to avoid potential threats.
The ability to recognize and respond to potential rewards or punishments depends in part on a part of the brain called the ventral pallidum. Researchers have observed activity in this brain region when animals seek rewards, such as a sip of water, or avoid punishments, such as an annoying puff of air.
What Li wanted to understand was how the different types of neurons that reside in this part of the brain ensure an animal responds appropriately to signals associated with both types of motivation.
To investigate, his team took advantage of research tools that allowed them to monitor the activity of individual brain cells and to confirm those cells’ identities with a flash of light. After training mice to associate certain sounds with either a sip of water or a puff of air, Li and his colleagues used the technique to monitor neural activity in the ventral pallidum.
They found that neurons that used the neurotransmitter known as GABA to dampen activity in the circuit influencing motivation were important in motivating the mice to seek a water reward.
The neurons that used the neurotransmitter known as glutamate to excite the brain circuit, on the other hand, were essential for avoiding the air-puff punishment.
Researchers have zeroed in on the ventral pallidum (see VP in brain diagram), an information-processing hub in the brains of mice to discover how neurons there influence the animals’ motivation to either seek pleasurable experiences or avoid negative ones. Image is credited to Li lab/CSHL, 2019.
In more complex situations, where animals were presented with the potential for both punishment and reward, both sets of neurons responded. Mice made different choices in response to the combined stimuli:
Thirsty animals, for example, were more willing to risk an air puff to obtain a sip of water than animals that had just drunk their fill. But if the team artificially shifted the balance of activity in the ventral pallidum by manipulating one class of neurons or the other, they could alter the animals’ behavior.
That balance between signals that either inhibit or excite neurons in the ventral pallidum appears critical in controlling which motivation an animal acts on, Li says. Now, he is eager to find out whether it is disrupted in people with psychiatric disorders.
“Behavioral changes in people with depression or stress-induced anxiety may be caused by changes in this circuit,” he says. With the new findings, his team has important leads about how to investigate the causes and symptoms of these disorders more deeply.
Motivational control over the purposeful action, or goal-directed behavior, is essential for gaining reward from an environment through knowledge about the association between action and its consequences (Dickinson, 1985; Dickinson and Balleine, 1994). Impairment of motivational control of goal-directed behavior promotes autonomic (habitual) control of actions and this is evident in addictive disorders (Everitt and Robbins, 2005; Ersche et al., 2016).
Goal-directed behavior is regulated by two factors: the incentive value of the goal (reward) and the internal drive (physiological state) of an agent (Berridge, 2004; Zhang et al., 2009). Accordingly, motivational processes are governed by the signals related to these two factors (i.e., motivational value), although their neural mechanism has remained largely unknown.
The ventral pallidum (VP), an output nucleus of ventral basal ganglia, is posited in the heart of the limbic system (Haber et al., 1985; Groenewegen et al., 1993; Ray and Price, 1993; Mai and Paxinos, 2011) and has been strongly implicated in reward processing (Smith et al., 2009; Castro et al., 2015; Root et al., 2015). Neuronal activity in VP has been shown to reflect the incentive value of reward cue in rodents (Tindell et al., 2004; Ahrens et al., 2016) and monkeys (Tachibana and Hikosaka, 2012; Saga et al., 2017). Pharmacological manipulation of VP disrupted normal reward-based behavior in monkeys (Tachibana and Hikosaka, 2012; Saga et al., 2017). Dysregulation of its neuronal activity induces addiction-like behavior in mice (Mahler et al., 2014; Faget et al., 2018). Collectively, these results suggest a significant contribution of VP to goal-directed behavior.
Other studies have also focused on the rostromedial part of the caudate nucleus (rmCD), one of the upstream structures of VP (Haber et al., 1990), as making a significant contribution to goal-directed behavior. The rmCD receives projections from the lateral orbitofrontal cortex (OFC) (Haber and Knutson, 2010; Averbeck et al., 2014) and attenuation of OFC-striatal activity promotes habitual control of action over the goal-directed action in rodents (Yin et al., 2005; Gremel and Costa, 2013; Gremel et al., 2016). In monkeys, neuronal activity in the middle caudate including the rmCD reflected reward size (Nakamura et al., 2012) and silencing of rmCD neurons induced a loss of reward size sensitivity and disrupted goal-directed performance (Nagai et al., 2016).
Considering the direct anatomical connection from rmCD to VP, although they may interplay and contribute to a goal-directed control of action, the exact mechanism still remains unclear. One possible mechanism is that rmCD relays the signal derived from the OFC, which reflects the incentive value and internal drive (Rolls, 2006) and is further strengthened through the convergence projection to the VP (Haber and Knutson, 2010; Averbeck et al., 2014).
Other mechanisms are also conceivable, for example, that incentive value and internal drive are processed outside of the striatum, such as in the amygdala and hypothalamus (Burton et al., 1976; Paton et al., 2006), which project to VP to compute motivational value through its downstream structures. Therefore, to clarify the neural mechanism of motivational control of action, a comparison of the neuronal coding of VP and rmCD in terms of incentive and drive is essential.
In this study, we analyzed the single-unit activities of these two areas while macaque monkeys performed an instrumental lever release task in which a visual cue indicated the forthcoming reward size (Minamimoto et al., 2009). Because this task design permits us to infer the impact of incentive value (i.e., reward size) and internal drive (i.e., satiation level of monkeys) on performance, we assessed the neuronal correlate of the two factors and compared neuronal coding between the two areas.
With a population-level comparison, we found that the coding of reward size and satiation level in VP was greater than that in rmCD. Pharmacological inactivation of VP further examined the causal contribution of the neuronal activity to goal-directed action. Our results suggest a central role of VP in motivational control of goal-directed behavior and may have implications for the neural mechanism of addictive disorders.
Discussion
In the present study, we examined the activity of VP and rmCD neurons during goal-directed behavior controlled by both incentive value (i.e., reward size) and internal drive (i.e., satiation level). We found that reward size coding after a reward size cue was stronger and earlier in VP neurons than in rmCD neurons. We also found that satiation level coding was observed throughout a trial and appeared more frequently in VP than in rmCD neurons. In both areas, information regarding reward size and satiation level was not systematically integrated into a single neuron, but was independently signaled in the population.
Inactivation of the bilateral VP disrupted normal goal-directed control of action, suggesting a causal role of VP in signaling incentive and drive for motivational control of goal-directed behavior.
Past studies demonstrated that neurons in VP and rmCD encode the incentive value of cue during performing the task offering binary outcomes (e.g., large or small reward) (Tindell et al., 2004; Nakamura et al., 2012; Tachibana and Hikosaka, 2012; Saga et al., 2017). Instead, the reward size task used in the present study offered four reward sizes that enabled us to quantify the linearity of value coding in the activity of single neurons.
With this paradigm, we found that some rmCD neurons exhibited the activity reflecting reward size mainly during cue periods. The activity is likely to mediate goal-directed behavior based on expected reward size because inactivation of this brain area impaired the task performance by the loss of reward size sensitivity (Nagai et al., 2016).
Our data are also consistent with previous research proposing that the projection from the OFC to the striatum is the critical pathway of carrying incentive information and performing a goal-directed behavior (Yin et al., 2005; Gremel and Costa, 2013; Gremel et al., 2016); we thereby confirmed the role of rmCD for mediating goal-directed behavior.
The present results, however, highlight the more prominent role of VP in signaling incentive information for goal-directed behavior. We found that neuronal modulation by expected value was stronger and more frequent in VP neurons than in rmCD neurons.
The strong signal in VP could not be accounted for solely by the convergence input from the rmCD for two reasons. First, the coding latency of VP neurons was significantly shorter than that of rmCD projection neurons (PANs).
Second, if this were the case, then response polarity would be opposite between the two areas, considering that the projection from rmCD to VP is GABAergic (Haber et al., 1990). However, a majority of the neurons in both areas showed excitatory response to cue. One might argue that the reward size coding in rmCD neurons would be non-monotonic (e.g., neurons exclusively respond to a certain reward size), so the coding strength of rmCD would have been underestimated.
This was not likely, however, because we did not find such neurons when we checked the SDF of each neuron and because we observed only a few neurons that showed significance by one-way ANOVA but not by linear regression. Together, our results suggest that VP signals incentive value that does not primarily originate from rmCD. This suggestion may also extend to the limbic striatum given the previous finding of similar earlier incentive signaling in VP than in the nucleus accumbens in rats (Richard et al., 2016).
A remaining question is as follows: where is such rich and rapid incentive-value information derived from? One possible source is the basolateral amygdala (BLA), which has a reciprocal connection to VP (Mitrovic and Napier, 1998; Root et al., 2015) and is known to contain neurons reflecting incentive value of cue with short latency (Paton et al., 2006; Belova et al., 2008; Jenison et al., 2011).
Recent studies demonstrated that amygdala lesion impaired reward-based learning more severely than VS lesion in monkeys (Averbeck et al., 2014; Costa et al., 2016), supporting the contribution of BLA-VP projection in incentive-value processing. Another candidate is the projection from the subthalamic nucleus (STN). VP has a reciprocal connection with the medial STN that receives projections from limbic cortical areas (Haynes and Haber, 2013), composing the limbic cortico-subthalamo-pallidal “hyperdirect” pathway (Nambu et al., 2002).
It has been shown that STN neurons respond to cues predicting rewards in monkeys (Matsumura et al., 1992; Darbaky et al., 2005; Espinosa-Parrilla et al., 2015). Future studies should identify the source of the incentive value information in terms of the latency, strength, and linearity of the coding.
In addition to reward size coding, VP neurons also encoded the internal drive (satiation level) of monkeys, although there could have been other confounding factors (e.g., time elapsed, fatigue, etc.). This satiation level coding was prominent even in the ITI phase, suggesting that this is indeed a reflection of motivational state rather than task structure per se. A similar type of state coding has been reported in agouti-related peptide (AgRP)-producing neurons in the arcuate nucleus of the hypothalamus (ARH); hunger/satiety state modulates the firing rate of ARH-AgRP neurons (Chen et al., 2015), which regulates feeding behavior together with the lateral hypothalamus (Burton et al., 1976; Petrovich, 2018).
Given that VP receives direct input from the hypothalamus (Castro et al., 2015; Root et al., 2015), the satiation coding in VP might reflect the state-dependent activity originating from the hypothalamus. Although the current results indicate that neither incentive value nor internal drive is systematically integrated at the single-neuron level, VP may play a pivotal role in representing the two factors, which may be integrated in downstream structures such as the mediodorsal (MD) thalamus (Haber and Knutson, 2010), one of the brain regions responsible for “reinforcer devaluation”; that is, appropriate action selection according to the satiation of specific needs (Mitchell et al., 2007; Izquierdo and Murray, 2010), so motivational value could be formulated in this area.
By definition, motivational value refers to the value that is directly linked to goal-directed action. A similar value system, the subjective value that is inferred from choice behavior, also refers to the subject’s internal state (Bernoulli, 1954; Stephens and Krebs, 1986). A crucial role for subject-centered decision has been demonstrated in the ventromedial prefrontal cortex (vmPFC), where neural activity encodes subjective value (de Araujo et al., 2003; Kable and Glimcher, 2007; Bouret and Richmond, 2010; O’Doherty, 2011).
Because vmPFC is one of the projection targets from MD (Goldman-Rakic and Porrino, 1985; McFarland and Haber, 2002), its value coding could be indirectly modulated by signal originating from VP. Therefore, the current results may provide a subcortical mechanism of value coding that contributes to value-based decision making as well as goal-directed control of action.
The causal contribution of value coding in VP was examined by an inactivation study. We found that bilateral inactivation of VP increased premature errors regardless of incentive conditions. VP inactivation had little effect on the general sensory-motor process, motivation, or arousal because the inactivation did not change RT or decrease the number of trials performed. VP inactivation attenuated the satiation effect on lever grip time, which is considered to reflect the motivational state (Kobayashi et al., 2002).
Failure of model fitting after VP inactivation also suggests impairment of motivational control based on incentive and drive. The present results, together with a previous study, support the view that the value coding of VP contributes to the motivational control of goal-directed behavior (Tachibana and Hikosaka, 2012). Another mechanism is also possible: that suppressing general high neuronal activity in VP (cf. Fig. 3) would promote a premature response.
Inactivation of VP would activate its efferent target neurons including dopamine neurons by a disinhibition mechanism and thereby abnormally invigorate current actions (Niv et al., 2007; Tachibana and Hikosaka, 2012). However, the situation is not so simple because injection of the GABAA receptor antagonist bicuculline into bilateral VP also increased premature responses in monkeys (Saga et al., 2017). Therefore, disruption of value coding in VP may be the fundamental mechanism underlying the observed abnormal behavior.
The loss of information regarding motivational value could promote a shift from goal-directed to habitual control of action (Dickinson, 1985; Dickinson and Balleine, 1994), which is implicated in the hallmark of addictive disorders (Everitt and Robbins, 2005; Ersche et al., 2016). Our results therefore emphasize the importance of future investigations into the exact neuronal mechanisms of motivational value formulation and control of actions in both the normal and abnormal state.
In conclusion, our data highlight the critical contribution of VP in goal-directed action. VP neurons independently encode information regarding incentive and drive that are essential for motivational control of goal-directed behavior.
Therefore, VP may gain access to motor-related processes and adjust the motivation of action based on the expected reward value in accordance with the current needs.
Source:
Cold Spring Harbor Laboratory
Media Contacts:
Sara Roncero-Menendez – Cold Spring Harbor Laboratory
Image Source:
The image is credited to Li lab/CSHL, 2019.
