Sample records for learning reward cues
-
Morphine Reward Promotes Cue-Sensitive Learning: Implication of Dorsal Striatal CREB Activity
Directory of Open Access Journals (Sweden)
Mathieu Baudonnat
2017-05-01
Full Text Available Different parallel neural circuits interact and may even compete to process and store information: whereas stimulus–response (S–R learning critically depends on the dorsal striatum (DS, spatial memory relies on the hippocampus (HPC. Strikingly, despite its potential importance for our understanding of addictive behaviors, the impact of drug rewards on memory systems dynamics has not been extensively studied. Here, we assessed long-term effects of drug- vs food reinforcement on the subsequent use of S–R vs spatial learning strategies and their neural substrates. Mice were trained in a Y-maze cue-guided task, during which either food or morphine injections into the ventral tegmental area (VTA were used as rewards. Although drug- and food-reinforced mice learned the Y-maze task equally well, drug-reinforced mice exhibited a preferential use of an S–R learning strategy when tested in a water-maze competition task designed to dissociate cue-based and spatial learning. This cognitive bias was associated with a persistent increase in the phosphorylated form of cAMP response element-binding protein phosphorylation (pCREB within the DS, and a decrease of pCREB expression in the HPC. Pharmacological inhibition of striatal PKA pathway in drug-rewarded mice limited the morphine-induced increase in levels of pCREB in DS and restored a balanced use of spatial vs cue-based learning. Our findings suggest that drug (opiate reward biases the engagement of separate memory systems toward a predominant use of the cue-dependent system via an increase in learning-related striatal pCREB activity. Persistent functional imbalance between striatal and hippocampal activity could contribute to the persistence of addictive behaviors, or counteract the efficiency of pharmacological or psychotherapeutic treatments.
-
De Tommaso, Matteo; Mastropasqua, Tommaso; Turatto, Massimo
2018-01-25
Reward-predicting cues attract attention because of their motivational value. A debated question regards the conditions under which the cue's attentional salience is governed more by reward expectancy rather than by reward uncertainty. To help shedding light on this relevant issue, here, we manipulated expectancy and uncertainty using three levels of reward-cue contingency, so that, for example, a high level of reward expectancy (p = .8) was compared with the highest level of reward uncertainty (p = .5). In Experiment 1, the best reward-cue during conditioning was preferentially attended in a subsequent visual search task. This result was replicated in Experiment 2, in which the cues were matched in terms of response history. In Experiment 3, we implemented a hybrid procedure consisting of two phases: an omission contingency procedure during conditioning, followed by a visual search task as in the previous experiments. Crucially, during both phases, the reward-cues were never task relevant. Results confirmed that, when multiple reward-cue contingencies are explored by a human observer, expectancy is the major factor controlling both the attentional and the oculomotor salience of the reward-cue.
-
Anderson, Brian A
2017-03-01
Through associative reward learning, arbitrary cues acquire the ability to automatically capture visual attention. Previous studies have examined the neural correlates of value-driven attentional orienting, revealing elevated activity within a network of brain regions encompassing the visual corticostriatal loop [caudate tail, lateral occipital complex (LOC) and early visual cortex] and intraparietal sulcus (IPS). Such attentional priority signals raise a broader question concerning how visual signals are combined with reward signals during learning to create a representation that is sensitive to the confluence of the two. This study examines reward signals during the cued reward training phase commonly used to generate value-driven attentional biases. High, compared with low, reward feedback preferentially activated the value-driven attention network, in addition to regions typically implicated in reward processing. Further examination of these reward signals within the visual system revealed information about the identity of the preceding cue in the caudate tail and LOC, and information about the location of the preceding cue in IPS, while early visual cortex represented both location and identity. The results reveal teaching signals within the value-driven attention network during associative reward learning, and further suggest functional specialization within different regions of this network during the acquisition of an integrated representation of stimulus value. © The Author (2016). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Global cue inconsistency diminishes learning of cue validity
Directory of Open Access Journals (Sweden)
Tony Wang
2016-11-01
Full Text Available We present a novel two-stage probabilistic learning task that examines the participants’ ability to learn and utilize valid cues across several levels of probabilistic feedback. In the first stage, participants sample from one of three cues that gives predictive information about the outcome of the second stage. Participants are rewarded for correct prediction of the outcome in stage two. Only one of the three cues gives valid predictive information and thus participants can maximise their reward by learning to sample from the valid cue. The validity of this predictive information, however, is reinforced across several levels of probabilistic feedback. A second manipulation involved changing the consistency of the predictive information in stage one and the outcome in stage two. The results show that participants, with higher probabilistic feedback, learned to utilise the valid cue. In inconsistent task conditions, however, participants were significantly less successful in utilising higher validity cues. We interpret this result as implying that learning in probabilistic categorization is based on developing a representation of the task that allows for goal-directed action.
-
Extinction and renewal of cue-elicited reward-seeking.
Bezzina, Louise; Lee, Jessica C; Lovibond, Peter F; Colagiuri, Ben
2016-12-01
Reward cues can contribute to overconsumption of food and drugs and can relapse. The failure of exposure therapies to reduce overconsumption and relapse is generally attributed to the context-specificity of extinction. However, no previous study has examined whether cue-elicited reward-seeking (as opposed to cue-reactivity) is sensitive to context renewal. We tested this possibility in 160 healthy volunteers using a Pavlovian-instrumental transfer (PIT) design involving voluntary responding for a high value natural reward (chocolate). One reward cue underwent Pavlovian extinction in the same (Group AAA) or different context (Group ABA) to all other phases. This cue was compared with a second non-extinguished reward cue and an unpaired control cue. There was a significant overall PIT effect with both reward cues eliciting reward-seeking on test relative to the unpaired cue. Pavlovian extinction substantially reduced this effect, with the extinguished reward cue eliciting less reward-seeking than the non-extinguished reward cue. Most interestingly, extinction of cue-elicited reward-seeking was sensitive to renewal, with extinction less effective for reducing PIT when conducted in a different context. These findings have important implications for extinction-based interventions for reducing maladaptive reward-seeking in practice. Copyright © 2016 Elsevier Ltd. All rights reserved.
-
The habenula governs the attribution of incentive salience to reward predictive cues
Danna, Carey L.; Shepard, Paul D.; Elmer, Greg I.
2013-01-01
The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr), the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms. PMID:24368898
-
The habenula governs the attribution of incentive salience to reward predictive cues.
Directory of Open Access Journals (Sweden)
Carey L. Danna
2013-12-01
Full Text Available The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb, a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr, the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms.
-
Roselino, Ana Carolina; Rodrigues, André Vieira; Hrncir, Michael
2016-10-01
Foraging insects leave chemical footprints on flowers that subsequent foragers may use as indicators for recent flower visits and, thus, potential resource depletion. Accordingly, foragers should reject food sources presenting these chemical cues. Contrasting this assumption, experimental studies in stingless bees (Apidae, Meliponini), so far, demonstrated an attractive effect of footprints. These findings lead to doubts about the meaning of these chemical cues in natural foraging contexts. Here, we asked whether foragers of stingless bees (Melipona scutellaris) use footprints according to the previously experienced reward level of visited food sources. Bees were trained to artificial flower patches, at which the reward of a flower either decreased or, alternatively, increased after a visit by a forager. Individuals were allowed a total of nine foraging bouts to the patch, after which their preference for visited or unvisited flowers was tested. In the choice tests, bees trained under the decreasing reward context preferred unvisited flowers, whereas individuals trained under the increasing reward context preferred visited flowers. Foragers without experience chose randomly between visited and unvisited flowers. These results demonstrate that M. scutellaris learns to associate unspecific footprint cues at food sources with differential, specific reward contexts, and uses these chemical cues accordingly for their foraging decisions.
-
'Proactive' use of cue-context congruence for building reinforcement learning's reward function.
Zsuga, Judit; Biro, Klara; Tajti, Gabor; Szilasi, Magdolna Emma; Papp, Csaba; Juhasz, Bela; Gesztelyi, Rudolf
2016-10-28
Reinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent's knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent's control either using, or not using a model. In the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively. Based on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.
-
Facilitation of voluntary goal-directed action by reward cues.
Lovibond, Peter F; Colagiuri, Ben
2013-10-01
Reward-associated cues are known to influence motivation to approach both natural and man-made rewards, such as food and drugs. However, the mechanisms underlying these effects are not well understood. To model these processes in the laboratory with humans, we developed an appetitive Pavlovian-instrumental transfer procedure with a chocolate reward. We used a single unconstrained response that led to an actual rather than symbolic reward to assess the strength of reward motivation. Presentation of a chocolate-paired cue, but not an unpaired cue, markedly enhanced instrumental responding over a 30-s period. The same pattern was observed with 10-s and 30-s cues, showing that close cue-reward contiguity is not necessary for facilitation of reward-directed action. The results confirm that reward-related cues can instigate voluntary action to obtain that reward. The effectiveness of long-duration cues suggests that in clinical settings, attention should be directed to both proximal and distal cues for reward.
-
Extinction Can Reduce the Impact of Reward Cues on Reward-Seeking Behavior.
Lovibond, Peter F; Satkunarajah, Michelle; Colagiuri, Ben
2015-07-01
Reward-associated cues are thought to promote relapse after treatment of appetitive disorders such as drug-taking, binge eating, and gambling. This process has been modelled in the laboratory using a Pavlovian-instrumental transfer (PIT) design in which Pavlovian cues facilitate instrumental reward-directed action. Attempts to reduce facilitation by cue exposure (extinction) have produced mixed results. We tested the effect of extinction in a recently developed PIT procedure using a natural reward, chocolate, in human participants. Facilitation of instrumental responding was only observed in participants who were aware of the Pavlovian contingencies. Pavlovian extinction successfully reduced, but did not completely eliminate, expectancy of reward and facilitation of instrumental responding. The results indicate that exposure can reduce the ability of cues to promote reward-directed behavior in the laboratory. However, the residual potency of extinguished cues means that additional active strategies may be needed in clinical practice to train patients to resist the impact of these cues in their environment. Copyright © 2015. Published by Elsevier Ltd.
-
Watanabe, Noriya; Sakagami, Masamichi; Haruno, Masahiko
2013-03-06
Learning does not only depend on rationality, because real-life learning cannot be isolated from emotion or social factors. Therefore, it is intriguing to determine how emotion changes learning, and to identify which neural substrates underlie this interaction. Here, we show that the task-independent presentation of an emotional face before a reward-predicting cue increases the speed of cue-reward association learning in human subjects compared with trials in which a neutral face is presented. This phenomenon was attributable to an increase in the learning rate, which regulates reward prediction errors. Parallel to these behavioral findings, functional magnetic resonance imaging demonstrated that presentation of an emotional face enhanced reward prediction error (RPE) signal in the ventral striatum. In addition, we also found a functional link between this enhanced RPE signal and increased activity in the amygdala following presentation of an emotional face. Thus, this study revealed an acceleration of cue-reward association learning by emotion, and underscored a role of striatum-amygdala interactions in the modulation of the reward prediction errors by emotion.
-
Mahler, Stephen V.; Berridge, Kent C.
2009-01-01
The central nucleus of the amygdala (CeA) helps translate learning into motivation, and here we show that opioid stimulation of CeA magnifies and focuses learned incentive salience onto a specific reward cue (Pavlovian conditioned stimulus, or CS). This motivation enhancement makes that cue more attractive, noticeable, and liable to elicit appetitive and consummatory behaviors. To reveal the focusing of incentive salience, we exploited individual differences in an autoshaping paradigm in which a rat prefers to approach, nibble and sniff one of two reward-associated stimuli (its prepotent stimulus). The individually-prepotent cue is either a predictive CS+ that signals reward (8sec metal lever insertion), or instead the metal cup that delivers sucrose pellets (the reward source). Results indicated that CeA opioid activation by microinjection of the μ agonist DAMGO (0.1μg) selectively and reversibly enhanced the attractiveness of whichever reward CS was that rat's prepotent cue. CeA DAMGO microinjections made rats more vigorously approach their particular prepotent CS, and to energetically sniff and nibble it in a nearly frenzied consummatory fashion. Only the prepotent cue was enhanced as an incentive target, and alternative cues were not enhanced. Conversely, inactivation of CeA by muscimol microinjection (0.25μg) suppressed approach, nibbles and sniffs of the prepotent CS. Confirming modulation of incentive salience, unconditioned food intake was similarly increased by DAMGO microinjection and decreased by muscimol in CeA. We conclude that opioid neurotransmission in central amygdala helps determine which environmental stimuli become most ‘wanted,’ and how ‘wanted’ they become. This may powerfully guide reward-seeking behavior. PMID:19458221
-
Brenes, Juan C; Schwarting, Rainer K W
2015-10-01
Reward-related stimuli come to acquire incentive salience through Pavlovian learning and become capable of controlling reward-oriented behaviors. Here, we examined individual differences in anticipatory activity elicited by reward-related cues as indicative of how animals attribute incentive salience to otherwise neutral stimuli. Since adult rats can signal incentive motivation states through ultrasonic vocalizations (USVs) at around 50-kHz, such calls were recorded in food-deprived rats trained to associate cues with food rewards, which were subsequently devalued by satiation.We found that the extent to which animals developed conditioned anticipatory activity to food cues while food deprived determined the level of cue-induced appetitive USVs while sated. Re-exposure to reward cues after a free-testing period reinstated USVs, invigorated reward seeking and consumption, and again, increases in calling occurred only in animals with high levels of cue-induced anticipatory activity. Reward-experienced rats systemically challenged with the catecholamine agonist amphetamine or with the dopamine receptor antagonist flupenthixol showed attenuated responses to these drugs, especially for USVs and in subjects with high levels of cue-induced anticipatory activity. Our results suggest that individuals prone to attribute incentive salience to reward cues showed heightened reward-induced USVs which were reliably expressed over time and persisted despite physiological needs being fulfilled. Also, prone subjects seemed to undergo particular adaptations in their dopaminergic system related with incentive learning. Our findings may have translational relevance in preclinical research modeling compulsive disorders, which may be due to excessive attribution of incentive salience to reward cues, such as overeating, pathological gambling, and drug addiction.
-
DiFeliceantonio, Alexandra G.; Berridge, Kent C.
2016-01-01
Pavlovian cues for rewards can become attractive incentives: approached and ‘wanted’ as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here we report that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individu...
-
Flagel, Shelly B; Akil, Huda; Robinson, Terry E
2009-01-01
Drugs of abuse acquire different degrees of control over thoughts and actions based not only on the effects of drugs themselves, but also on predispositions of the individual. Those individuals who become addicted are unable to shift their thoughts and actions away from drugs and drug-associated stimuli. Thus in addicts, exposure to places or things (cues) that has been previously associated with drug-taking often instigates renewed drug-taking. We and others have postulated that drug-associated cues acquire the ability to maintain and instigate drug-taking behavior in part because they acquire incentive motivational properties through Pavlovian (stimulus-stimulus) learning. In the case of compulsive behavioral disorders, including addiction, such cues may be attributed with pathological incentive value ("incentive salience"). For this reason, we have recently begun to explore individual differences in the tendency to attribute incentive salience to cues that predict rewards. When discrete cues are associated with the non-contingent delivery of food or drug rewards some animals come to quickly approach and engage the cue even if it is located at a distance from where the reward will be delivered. In these animals the reward-predictive cue itself becomes attractive, eliciting approach towards it, presumably because it is attributed with incentive salience. Animals that develop this type of conditional response are called "sign-trackers". Other animals, "goal-trackers", do not approach the reward-predictive cue, but upon cue presentation they immediately go to the location where food will be delivered (the "goal"). For goal-trackers the reward-predictive cue is not attractive, presumably because it is not attributed with incentive salience. We review here preliminary data suggesting that these individual differences in the tendency to attribute incentive salience to cues predictive of reward may confer vulnerability or resistance to compulsive behavioral disorders
-
San Martín, René; Appelbaum, Lawrence G; Huettel, Scott A; Woldorff, Marty G
2016-01-01
Adaptive choice behavior depends critically on identifying and learning from outcome-predicting cues. We hypothesized that attention may be preferentially directed toward certain outcome-predicting cues. We studied this possibility by analyzing event-related potential (ERP) responses in humans during a probabilistic decision-making task. Participants viewed pairs of outcome-predicting visual cues and then chose to wager either a small (i.e., loss-minimizing) or large (i.e., gain-maximizing) amount of money. The cues were bilaterally presented, which allowed us to extract the relative neural responses to each cue by using a contralateral-versus-ipsilateral ERP contrast. We found an early lateralized ERP response, whose features matched the attention-shift-related N2pc component and whose amplitude scaled with the learned reward-predicting value of the cues as predicted by an attention-for-reward model. Consistently, we found a double dissociation involving the N2pc. Across participants, gain-maximization positively correlated with the N2pc amplitude to the most reliable gain-predicting cue, suggesting an attentional bias toward such cues. Conversely, loss-minimization was negatively correlated with the N2pc amplitude to the most reliable loss-predicting cue, suggesting an attentional avoidance toward such stimuli. These results indicate that learned stimulus-reward associations can influence rapid attention allocation, and that differences in this process are associated with individual differences in economic decision-making performance. © The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
-
DiFeliceantonio, Alexandra G; Berridge, Kent C
2016-05-01
Pavlovian cues for rewards can become attractive incentives: approached and 'wanted' as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here it was reported that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individualized fashion, at the expense of a competing cue. In an autoshaping (sign-tracking vs. goal-tracking) paradigm, microinjection of the mu opioid receptor agonist (DAMGO) or dopamine indirect agonist (amphetamine) in the DLS of sign-tracker individuals selectively enhanced their sign-tracking attraction toward the reward-predictive lever cue. By contrast, DAMGO or amphetamine in the DLS of goal-trackers selectively enhanced prepotent attraction toward the reward-proximal cue of sucrose dish. Amphetamine also enhanced goal-tracking in some sign-tracker individuals (if they ever defected to the dish even once). That DLS enhancement of cue attraction was due to stronger motivation, not stronger habits, was suggested by: (i) sign-trackers flexibly followed their cue to a new location when the lever was suddenly moved after DLS DAMGO microinjection; and (ii) DAMGO in the DLS also made sign-trackers work harder on a new instrumental nose-poke response required to earn presentations of their Pavlovian lever cue (instrumental conditioned reinforcement). Altogether, the current results suggest that DLS circuitry can enhance the incentive salience of a Pavlovian reward cue, selectively making that cue a stronger motivational magnet. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
-
Reichelt, A C; Westbrook, R F; Morris, M J
2015-11-01
Understanding the neurobiological substrates that encode learning about food-associated cues and how those signals are modulated is of great clinical importance especially in light of the worldwide obesity problem. Inappropriate or maladaptive responses to food-associated cues can promote over-consumption, leading to excessive energy intake and weight gain. Chronic exposure to foods rich in fat and sugar alters the reinforcing value of foods and weakens inhibitory neural control, triggering learned, but maladaptive, associations between environmental cues and food rewards. Thus, responses to food-associated cues can promote cravings and food-seeking by activating mesocorticolimbic dopamine neurocircuitry, and exert physiological effects including salivation. These responses may be analogous to the cravings experienced by abstaining drug addicts that can trigger relapse into drug self-administration. Preventing cue-triggered eating may therefore reduce the over-consumption seen in obesity and binge-eating disorder. In this review we discuss recent research examining how cues associated with palatable foods can promote reward-based feeding behaviours and the potential involvement of appetite-regulating peptides including leptin, ghrelin, orexin and melanin concentrating hormone. These peptide signals interface with mesolimbic dopaminergic regions including the ventral tegmental area to modulate reactivity to cues associated with palatable foods. Thus, a novel target for anti-obesity therapeutics is to reduce non-homeostatic, reward driven eating behaviour, which can be triggered by environmental cues associated with highly palatable, fat and sugar rich foods. © 2015 The British Pharmacological Society.
-
On the motivational properties of reward cues: Individual differences.
Robinson, Terry E; Yager, Lindsay M; Cogan, Elizabeth S; Saunders, Benjamin T
2014-01-01
Cues associated with rewards, such as food or drugs of abuse, can themselves acquire motivational properties. Acting as incentive stimuli, such cues can exert powerful control over motivated behavior, and in the case of cues associated with drugs, they can goad continued drug-seeking behavior and relapse. However, recent studies reviewed here suggest that there are large individual differences in the extent to which food and drug cues are attributed with incentive salience. Rats prone to approach reward cues (sign-trackers) attribute greater motivational value to discrete localizable cues and interoceptive cues than do rats less prone to approach reward cues (goal-trackers). In contrast, contextual cues appear to exert greater control over motivated behavior in goal-trackers than sign-trackers. It is possible to predict, therefore, before any experience with drugs, in which animals specific classes of drug cues will most likely reinstate drug-seeking behavior. The finding that different individuals may be sensitive to different triggers capable of motivating behavior and producing relapse suggests there may be different pathways to addiction, and has implications for thinking about individualized treatment. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'. Copyright © 2013 Elsevier Ltd. All rights reserved.
-
Incidental rewarding cues influence economic decision-making in obesity
Directory of Open Access Journals (Sweden)
Jakob eSimmank
2015-10-01
Full Text Available Recent research suggests that obesity is linked to prominent alterations in learning and decision-making. This general difference may also underlie the preference for immediately consumable, highly palatable but unhealthy and high-calorie foods. Such poor food-related inter-temporal decision-making can explain weight gain; however, it is not yet clear whether this deficit can be generalized to other domains of inter-temporal decision-making, for example financial decisions. Further, little is known about the stability of decision-making behavior in obesity, especially in the presence of rewarding cues. To answer these questions, obese and lean participants (n=52 completed two sessions of a novel priming paradigm including a computerized monetary delay discounting task. In the first session, general differences between groups in financial delay discounting were measured. In the second session, we tested the general stability of discounting rates. Additionally, participants were primed by affective visual cues of different contextual categories before the financial decision. We found that the obese group showed stronger discounting of future monetary rewards than the lean group, but groups did not differ in their general stability between sessions nor in their sensitivity towards changes in reward magnitude. In the obese group, a fast decrease of subjective value over time was directly related to a higher tendency for opportunistic eating. Obese in contrast to lean people were primed by the affective cues, showing a sex-specific pattern of priming direction. Our findings demonstrate that environments rich of cues, aiming at inducing unhealthy consumer decisions, can be highly detrimental for obese people. It also underscores that obesity is not merely a medical condition but has a strong cognitive component, meaning that current dietary and medical treatment strategies may fall too short.
-
Improved memory for reward cues following acute buprenorphine administration in humans
Syal, Supriya; Ipser, Jonathan; Terburg, David|info:eu-repo/dai/nl/32304087X; Solms, Mark; Panksepp, Jaak; Malcolm-Smith, Susan; Bos, Peter A.|info:eu-repo/dai/nl/337018995; Montoya, Estrella R.|info:eu-repo/dai/nl/34141347X; Stein, Dan J.; van Honk, Jack|info:eu-repo/dai/nl/188602801
2015-01-01
In rodents, there is abundant evidence for the involvement of the opioid system in the processing of reward cues, but this system has remained understudied in humans. In humans, the happy facial expression is a pivotal reward cue. Happy facial expressions activate the brain's reward system and are
-
Lee, Maan-Gee; Jun, Gayoung; Choi, Hyo-Soon; Jang, Hwan Soo; Bae, Yong Chul; Suk, Kyoungho; Jang, Il-Sung; Choi, Byung-Ju
2010-07-01
Operant conditioning is often used to train a desired behavior in an animal. The contingency between a specific behavior and a reward is required for successful training. Here, we compared the effectiveness of two different mazes for training turning behaviors in response to directional cues in Sprague-Dawley rats. Forty-three rats were implanted with electrodes into the medial forebrain bundle and the left and right somatosensory cortices for reward and cues. Among them, thirteen rats discriminated between the left and right somatosensory stimulations to obtain rewards. They were trained to learn ipsilateral turning response to the stimulation of the left or right somatosensory cortex in either the T-maze (Group T) or the E| maze (Group W). Performance was measured by the navigation speed in the mazes. Performances of rats in Group T were enhanced faster than those in Group W. A significant correlation between performances during training and performance in final testing was observed in Group T starting with the fifth training session while such a correlation was not observed in Group W until the tenth training session. The training mazes did not however affect the performances in the final test. These results suggest that a simple maze is better than a complicated maze for training animals to learn directions and direct cortical stimulation can be used as a cue for direction training. Copyright (c) 2010 Elsevier B.V. All rights reserved.
-
Incidental rewarding cues influence economic decisions in people with obesity
Simmank, Jakob; Murawski, Carsten; Bode, Stefan; Horstmann, Annette
2015-01-01
Recent research suggests that obesity is linked to prominent alterations in learning and decision-making. This general difference may also underlie the preference for immediately consumable, highly palatable but unhealthy and high-calorie foods. Such poor food-related inter-temporal decision-making can explain weight gain; however, it is not yet clear whether this deficit can be generalized to other domains of inter-temporal decision-making, for example financial decisions. Further, little is known about the stability of decision-making behavior in obesity, especially in the presence of rewarding cues. To answer these questions, obese and lean participants (n = 52) completed two sessions of a novel priming paradigm including a computerized monetary delay discounting task. In the first session, general differences between groups in financial delay discounting were measured. In the second session, we tested the general stability of discount rates. Additionally, participants were primed by affective visual cues of different contextual categories before making financial decisions. We found that the obese group showed stronger discounting of future monetary rewards than the lean group, but groups did not differ in their general stability between sessions nor in their sensitivity toward changes in reward magnitude. In the obese group, a fast decrease of subjective value over time was directly related to a higher tendency for opportunistic eating. Obese in contrast to lean people were primed by the affective cues, showing a sex-specific pattern of priming direction. Our findings demonstrate that environments rich of cues, aiming at inducing unhealthy consumer decisions, can be highly detrimental for obese people. It also underscores that obesity is not merely a medical condition but has a strong cognitive component, meaning that current dietary and medical treatment strategies may fall too short. PMID:26528158
-
Murayama, Kou; Kitagami, Shinji
2014-02-01
Recent research suggests that extrinsic rewards promote memory consolidation through dopaminergic modulation processes. However, no conclusive behavioral evidence exists given that the influence of extrinsic reward on attention and motivation during encoding and consolidation processes are inherently confounded. The present study provides behavioral evidence that extrinsic rewards (i.e., monetary incentives) enhance human memory consolidation independently of attention and motivation. Participants saw neutral pictures, followed by a reward or control cue in an unrelated context. Our results (and a direct replication study) demonstrated that the reward cue predicted a retrograde enhancement of memory for the preceding neutral pictures. This retrograde effect was observed only after a delay, not immediately upon testing. An additional experiment showed that emotional arousal or unconscious resource mobilization cannot explain the retrograde enhancement effect. These results provide support for the notion that the dopaminergic memory consolidation effect can result from extrinsic reward.
-
Flagel, Shelly B; Robinson, Terry E; Clark, Jeremy J; Clinton, Sarah M; Watson, Stanley J; Seeman, Phillip; Phillips, Paul E M; Akil, Huda
2009-01-01
Rats selectively bred based on high or low reactivity to a novel environment were characterized for other behavioral and neurobiological traits thought to be relevant to addiction vulnerability. The two lines of animals, which differ in their propensity to self-administer drugs, also differ in the value they attribute to cues associated with reward, in impulsive behavior, and in their dopamine system. When a cue was paired with food or cocaine reward bred high-responder rats (bHRs) learned to...
-
Impaired associative learning with food rewards in obese women.
Zhang, Zhihao; Manson, Kirk F; Schiller, Daniela; Levy, Ifat
2014-08-04
Obesity is a major epidemic in many parts of the world. One of the main factors contributing to obesity is overconsumption of high-fat and high-calorie food, which is driven by the rewarding properties of these types of food. Previous studies have suggested that dysfunction in reward circuits may be associated with overeating and obesity. The nature of this dysfunction, however, is still unknown. Here, we demonstrate impairment in reward-based associative learning specific to food in obese women. Normal-weight and obese participants performed an appetitive reversal learning task in which they had to learn and modify cue-reward associations. To test whether any learning deficits were specific to food reward or were more general, we used a between-subject design in which half of the participants received food reward and the other half received money reward. Our results reveal a marked difference in associative learning between normal-weight and obese women when food was used as reward. Importantly, no learning deficits were observed with money reward. Multiple regression analyses also established a robust negative association between body mass index and learning performance in the food domain in female participants. Interestingly, such impairment was not observed in obese men. These findings suggest that obesity may be linked to impaired reward-based associative learning and that this impairment may be specific to the food domain. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Quantifying individual variation in the propensity to attribute incentive salience to reward cues.
Meyer, Paul J; Lovic, Vedran; Saunders, Benjamin T; Yager, Lindsay M; Flagel, Shelly B; Morrow, Jonathan D; Robinson, Terry E
2012-01-01
If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.
-
Improved memory for reward cues following acute buprenorphine administration in humans.
Syal, Supriya; Ipser, Jonathan; Terburg, David; Solms, Mark; Panksepp, Jaak; Malcolm-Smith, Susan; Bos, Peter A; Montoya, Estrella R; Stein, Dan J; van Honk, Jack
2015-03-01
In rodents, there is abundant evidence for the involvement of the opioid system in the processing of reward cues, but this system has remained understudied in humans. In humans, the happy facial expression is a pivotal reward cue. Happy facial expressions activate the brain's reward system and are disregarded by subjects scoring high on depressive mood who are low in reward drive. We investigated whether a single 0.2mg administration of the mixed mu-opioid agonist/kappa-antagonist, buprenorphine, would influence short-term memory for happy, angry or fearful expressions relative to neutral faces. Healthy human subjects (n38) participated in a randomized placebo-controlled within-subject design, and performed an emotional face relocation task after administration of buprenorphine and placebo. We show that, compared to placebo, buprenorphine administration results in a significant improvement of memory for happy faces. Our data demonstrate that acute manipulation of the opioid system by buprenorphine increases short-term memory for social reward cues. Copyright © 2015. Published by Elsevier Ltd.
-
Quantifying individual variation in the propensity to attribute incentive salience to reward cues.
Directory of Open Access Journals (Sweden)
Paul J Meyer
Full Text Available If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience" to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878 subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience, to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers" vs. others that approached the location of reward delivery ("goal-trackers". This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.
-
Ostlund, Sean B; Maidment, Nigel T
2012-01-01
Environmental cues affect our behavior in a variety of ways. Despite playing an invaluable role in guiding our daily activities, such cues also appear to trigger the harmful, compulsive behaviors that characterize addiction and other disorders of behavioral control. In instrumental conditioning, rewards and reward-paired cues bias action selection and invigorate reward-seeking behaviors, and appear to do so through distinct neurobehavioral processes. Although reward-paired cues are known to invigorate performance through a dopamine-dependent incentive motivational process, it is not known if dopamine also mediates the influence of rewards and reward-paired cues over action selection. The current study contrasted the effects of systemic administration of the nonspecific dopamine receptor antagonist flupentixol on response invigoration and action bias in Pavlovian-instrumental transfer, a test of cue-elicited responding, and in instrumental reinstatement, a test of noncontingent reward-elicited responding. Hungry rats were trained on two different stimulus-outcome relationships (eg, tone-grain pellets and noise-sucrose solution) and two different action-outcome relationships (eg, left press-grain and right press-sucrose). At test, we found that flupentixol pretreatment blocked the response invigoration generated by the cues but spared their ability to bias action selection to favor the action whose outcome was signaled by the cue being presented. The response-biasing influence of noncontingent reward deliveries was also unaffected by flupentixol. Interestingly, although flupentixol had a modest effect on the immediate response invigoration produced by those rewards, it was particularly potent in countering the lingering enhancement of responding produced by multiple reward deliveries. These findings indicate that dopamine mediates the general incentive motivational effects of noncontingent rewards and reward-paired cues but does not support their ability to bias
-
Do cattle (Bos taurus) retain an association of a visual cue with a food reward for a year?
Hirata, Masahiko; Takeno, Nozomi
2014-06-01
Use of visual cues to locate specific food resources from a distance is a critical ability of animals foraging in a spatially heterogeneous environment. However, relatively little is known about how long animals can retain the learned cue-reward association without reinforcement. We compared feeding behavior of experienced and naive Japanese Black cows (Bos taurus) in discovering food locations in a pasture. Experienced animals had been trained to respond to a visual cue (plastic washtub) for a preferred food (grain-based concentrate) 1 year prior to the experiment, while naive animals had no exposure to the cue. Cows were tested individually in a test arena including tubs filled with the concentrate on three successive days (Days 1-3). Experienced cows located the first tub more quickly and visited more tubs than naive cows on Day 1 (usually P visual cue with a food reward within a day and retain the association for 1 year despite a slight decay. © 2014 Japanese Society of Animal Science.
-
Directory of Open Access Journals (Sweden)
Mateusz Gola
2016-08-01
Full Text Available There is an increasing number of neuroimaging studies using visual sexual stimuli (VSS for human sexuality studies, including emerging field of research on compulsive sexual behaviors. A central question in this field is whether behaviors such as extensive pornography consumption share common brain mechanisms with widely studied substance and behavioral addictions. Depending on how VSS are conceptualized, different predictions can be formulated within the frameworks of Reinforcement Learning or Incentive Salience Theory, where a crucial distinction is made between conditioned (cue and unconditioned (reward stimuli (related to reward anticipation vs reward consumption, respectively. Surveying 40 recent human neuroimaging studies we show existing ambiguity about the conceptualization of VSS. Therefore, we feel that it is important to address the question of whether VSS should be considered as cues (conditioned stimuli or rewards (unconditioned stimuli. Here we present our own perspective, which is that in most laboratory settings VSS play a role of reward (unconditioned stimuli, as evidenced by: 1. experience of pleasure while watching VSS, possibly accompanied by genital reaction 2. reward-related brain activity correlated with these pleasurable feelings in response to VSS, 3. a willingness to exert effort to view VSS similarly as for other rewarding stimuli such as money, and/or 4. conditioning for cues (CS predictive for. We hope that this perspective paper will initiate a scientific discussion on this important and overlooked topic and increase attention for appropriate interpretations of results of human neuroimaging studies using VSS.
-
Reward dependence moderates smoking-cue- and stress-induced cigarette cravings.
Michalowski, Alexandra; Erblich, Joel
2014-12-01
Cigarette cravings following exposure to smoking cues in a smoker's environment are thought to play an important role in cessation failure. The possibility that dispositional factors may impact cue-induced cravings, though intriguing, has received little attention. According to Cloninger's Tridimensional Personality Theory, factors such as reward dependence (RD), harm avoidance (HA), and novelty seeking (NS) may figure prominently in risk for addiction, as well as relapse, in individuals attempting to abstain from drug and alcohol use. Particularly interesting in this regard is the possibility that smokers with higher levels of RD, who are especially sensitive to reward signals, will have heightened craving reactions to smoking cues. To that end, non-treatment-seeking nicotine dependent smokers (n=96, mean age=41.1, 47% African American, 17% Caucasian, 22% Hispanic, 19.3cigs/day, FTND=7.5) underwent a classic experimental cue-induction, during which they were exposed to imagery of: (1) smoking, (2) neutral, and (3) stress cues, and reported their cigarette cravings (0-100) before and after each exposure. Participants also completed the Tridimensional Personality Questionnaire. Not surprisingly, smoking and stress cues (but not neutral cues) elicited significant elevations in craving (p'scues (pcues (pcues. Furthermore, the similar effects of RD on stress-induced craving suggest that both cue-and stress-induced cravings may be influenced by a common underlying disposition. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Reward-based spatial learning in unmedicated adults with obsessive-compulsive disorder.
Marsh, Rachel; Tau, Gregory Z; Wang, Zhishun; Huo, Yuankai; Liu, Ge; Hao, Xuejun; Packard, Mark G; Peterson, Bradley S; Simpson, H Blair
2015-04-01
The authors assessed the functioning of mesolimbic and striatal areas involved in reward-based spatial learning in unmedicated adults with obsessive-compulsive disorder (OCD). Functional MRI blood-oxygen-level-dependent response was compared in 33 unmedicated adults with OCD and 33 healthy, age-matched comparison subjects during a reward-based learning task that required learning to use extramaze cues to navigate a virtual eight-arm radial maze to find hidden rewards. The groups were compared in their patterns of brain activation associated with reward-based spatial learning versus a control condition in which rewards were unexpected because they were allotted pseudorandomly to experimentally prevent learning. Both groups learned to navigate the maze to find hidden rewards, but group differences in neural activity during navigation and reward processing were detected in mesolimbic and striatal areas. During navigation, the OCD group, unlike the healthy comparison group, exhibited activation in the left posterior hippocampus. Unlike healthy subjects, participants in the OCD group did not show activation in the left ventral putamen and amygdala when anticipating rewards or in the left hippocampus, amygdala, and ventral putamen when receiving unexpected rewards (control condition). Signal in these regions decreased relative to baseline during unexpected reward receipt among those in the OCD group, and the degree of activation was inversely associated with doubt/checking symptoms. Participants in the OCD group displayed abnormal recruitment of mesolimbic and ventral striatal circuitry during reward-based spatial learning. Whereas healthy comparison subjects exhibited activation in this circuitry in response to the violation of reward expectations, unmedicated OCD participants did not and instead over-relied on the posterior hippocampus during learning. Thus, dopaminergic innervation of reward circuitry may be altered, and future study of anterior/posterior hippocampal
-
Neural correlates of water reward in thirsty Drosophila
Lin, Suewei; Owald, David; Chandra, Vikram; Talbot, Clifford; Huetteroth, Wolf; Waddell, Scott
2014-01-01
Drinking water is innately rewarding to thirsty animals. In addition, the consumed value can be assigned to behavioral actions and predictive sensory cues by associative learning. Here we show that thirst converts water avoidance into water-seeking in naive Drosophila melanogaster. Thirst also permitted flies to learn olfactory cues paired with water reward. Water learning required water taste and
-
Peciña, Susana; Schulkin, Jay; Berridge, Kent C
2006-04-13
Corticotropin-releasing factor (CRF) is typically considered to mediate aversive aspects of stress, fear and anxiety. However, CRF release in the brain is also elicited by natural rewards and incentive cues, raising the possibility that some CRF systems in the brain mediate an independent function of positive incentive motivation, such as amplifying incentive salience. Here we asked whether activation of a limbic CRF subsystem magnifies the increase in positive motivation for reward elicited by incentive cues previously associated with that reward, in a way that might exacerbate cue-triggered binge pursuit of food or other incentives? We assessed the impact of CRF microinjections into the medial shell of nucleus accumbens using a pure incentive version of Pavlovian-Instrumental transfer, a measure specifically sensitive to the incentive salience of reward cues (which it separates from influences of aversive stress, stress reduction, frustration and other traditional explanations for stress-increased behavior). Rats were first trained to press one of two levers to obtain sucrose pellets, and then separately conditioned to associate a Pavlovian cue with free sucrose pellets. On test days, rats received microinjections of vehicle, CRF (250 or 500 ng/0.2 microl) or amphetamine (20 microg/0.2 microl). Lever pressing was assessed in the presence or absence of the Pavlovian cues during a half-hour test. Microinjections of the highest dose of CRF (500 ng) or amphetamine (20 microg) selectively enhanced the ability of Pavlovian reward cues to trigger phasic peaks of increased instrumental performance for a sucrose reward, each peak lasting a minute or so before decaying after the cue. Lever pressing was not enhanced by CRF microinjections in the baseline absence of the Pavlovian cue or during the presentation without a cue, showing that the CRF enhancement could not be explained as a result of generalized motor arousal, frustration or stress, or by persistent attempts to
-
Anderson, Rachel I; Bush, Peter C; Spear, Linda P
2013-11-15
Cues repeatedly paired with rewards often themselves become imbued with enhanced motivational value, or incentive salience. During Pavlovian conditioned approach procedures, a cue repeatedly preceding reward delivery often elicits conditioned responses at either the reward delivery location ("goal-tracking") or the cue itself ("sign-tracking"). Sign-tracking behavior is thought to reflect the individual differences in attribution of incentive salience to reward-paired cues that may contribute to addiction vulnerability. Adolescent rats typically demonstrate less sign-tracking behavior than adult rats, a surprising finding given that adolescence is hypothesized to be a time of heightened addiction vulnerability. Given evidence that adult sign-tracking behavior can be influenced by environmental conditions, the present study compared the effects of isolate housing and food deprivation on expression of sign-tacking and goal-tracking behavior in adolescent and adult male rats across eight days of a Pavlovian conditioned approach procedure. Pair-housed adults exhibited more sign-tracking behavior than pair-housed adolescents; however, this age difference was not apparent in isolate-housed subjects. Adolescents often appeared more sensitive than adults to both food restriction- and isolate housing-induced changes in behavior, with food restriction promoting an increase in sign-tracking among isolate-housed adolescents and an increase in goal-tracking among pair-housed adolescents. For adults, food restriction resulted in a modest increase in overall expression of both sign- and goal-tracking behavior. To the extent that sign-tracking behavior reflects attribution of incentive salience to reward-paired cues, results from the present study provide evidence that reactivity to rewards during adolescence is strongly related to the nature of the surrounding environment. Copyright © 2013 Elsevier B.V. All rights reserved.
-
Directory of Open Access Journals (Sweden)
Schulkin Jay
2006-04-01
Full Text Available Abstract Background Corticotropin-releasing factor (CRF is typically considered to mediate aversive aspects of stress, fear and anxiety. However, CRF release in the brain is also elicited by natural rewards and incentive cues, raising the possibility that some CRF systems in the brain mediate an independent function of positive incentive motivation, such as amplifying incentive salience. Here we asked whether activation of a limbic CRF subsystem magnifies the increase in positive motivation for reward elicited by incentive cues previously associated with that reward, in a way that might exacerbate cue-triggered binge pursuit of food or other incentives? We assessed the impact of CRF microinjections into the medial shell of nucleus accumbens using a pure incentive version of Pavlovian-Instrumental transfer, a measure specifically sensitive to the incentive salience of reward cues (which it separates from influences of aversive stress, stress reduction, frustration and other traditional explanations for stress-increased behavior. Rats were first trained to press one of two levers to obtain sucrose pellets, and then separately conditioned to associate a Pavlovian cue with free sucrose pellets. On test days, rats received microinjections of vehicle, CRF (250 or 500 ng/0.2 μl or amphetamine (20 μg/0.2 μl. Lever pressing was assessed in the presence or absence of the Pavlovian cues during a half-hour test. Results Microinjections of the highest dose of CRF (500 ng or amphetamine (20 μg selectively enhanced the ability of Pavlovian reward cues to trigger phasic peaks of increased instrumental performance for a sucrose reward, each peak lasting a minute or so before decaying after the cue. Lever pressing was not enhanced by CRF microinjections in the baseline absence of the Pavlovian cue or during the presentation without a cue, showing that the CRF enhancement could not be explained as a result of generalized motor arousal, frustration or stress
-
Gola, Mateusz; Wordecha, Małgorzata; Marchewka, Artur; Sescousse, Guillaume
2016-01-01
There is an increasing number of neuroimaging studies using visual sexual stimuli (VSS), especially within the emerging field of research on compulsive sexual behaviors (CSB). A central question in this field is whether behaviors such as excessive pornography consumption share common brain mechanisms with widely studied substance and behavioral addictions. Depending on how VSS are conceptualized, different predictions can be formulated within the frameworks of Reinforcement Learning or Incentive Salience Theory, where a crucial distinction is made between conditioned and unconditioned stimuli (related to reward anticipation vs. reward consumption, respectively). Surveying 40 recent human neuroimaging studies we show existing ambiguity about the conceptualization of VSS. Therefore, we feel that it is important to address the question of whether VSS should be considered as conditioned stimuli (cue) or unconditioned stimuli (reward). Here we present our own perspective, which is that in most laboratory settings VSS play a role of reward, as evidenced by: (1) experience of pleasure while watching VSS, possibly accompanied by genital reaction; (2) reward-related brain activity correlated with these pleasurable feelings in response to VSS; (3) a willingness to exert effort to view VSS similarly as for other rewarding stimuli such as money; and (4) conditioning for cues predictive of VSS. We hope that this perspective article will initiate a scientific discussion on this important and overlooked topic and increase attention for appropriate interpretations of results of human neuroimaging studies using VSS.
-
Gola, Mateusz; Wordecha, Małgorzata; Marchewka, Artur; Sescousse, Guillaume
2016-01-01
There is an increasing number of neuroimaging studies using visual sexual stimuli (VSS), especially within the emerging field of research on compulsive sexual behaviors (CSB). A central question in this field is whether behaviors such as excessive pornography consumption share common brain mechanisms with widely studied substance and behavioral addictions. Depending on how VSS are conceptualized, different predictions can be formulated within the frameworks of Reinforcement Learning or Incentive Salience Theory, where a crucial distinction is made between conditioned and unconditioned stimuli (related to reward anticipation vs. reward consumption, respectively). Surveying 40 recent human neuroimaging studies we show existing ambiguity about the conceptualization of VSS. Therefore, we feel that it is important to address the question of whether VSS should be considered as conditioned stimuli (cue) or unconditioned stimuli (reward). Here we present our own perspective, which is that in most laboratory settings VSS play a role of reward, as evidenced by: (1) experience of pleasure while watching VSS, possibly accompanied by genital reaction; (2) reward-related brain activity correlated with these pleasurable feelings in response to VSS; (3) a willingness to exert effort to view VSS similarly as for other rewarding stimuli such as money; and (4) conditioning for cues predictive of VSS. We hope that this perspective article will initiate a scientific discussion on this important and overlooked topic and increase attention for appropriate interpretations of results of human neuroimaging studies using VSS. PMID:27574507
-
Neural correlates of reward-based spatial learning in persons with cocaine dependence.
Tau, Gregory Z; Marsh, Rachel; Wang, Zhishun; Torres-Sanchez, Tania; Graniello, Barbara; Hao, Xuejun; Xu, Dongrong; Packard, Mark G; Duan, Yunsuo; Kangarlu, Alayar; Martinez, Diana; Peterson, Bradley S
2014-02-01
Dysfunctional learning systems are thought to be central to the pathogenesis of and impair recovery from addictions. The functioning of the brain circuits for episodic memory or learning that support goal-directed behavior has not been studied previously in persons with cocaine dependence (CD). Thirteen abstinent CD and 13 healthy participants underwent MRI scanning while performing a task that requires the use of spatial cues to navigate a virtual-reality environment and find monetary rewards, allowing the functional assessment of the brain systems for spatial learning, a form of episodic memory. Whereas both groups performed similarly on the reward-based spatial learning task, we identified disturbances in brain regions involved in learning and reward in CD participants. In particular, CD was associated with impaired functioning of medial temporal lobe (MTL), a brain region that is crucial for spatial learning (and episodic memory) with concomitant recruitment of striatum (which normally participates in stimulus-response, or habit, learning), and prefrontal cortex. CD was also associated with enhanced sensitivity of the ventral striatum to unexpected rewards but not to expected rewards earned during spatial learning. We provide evidence that spatial learning in CD is characterized by disturbances in functioning of an MTL-based system for episodic memory and a striatum-based system for stimulus-response learning and reward. We have found additional abnormalities in distributed cortical regions. Consistent with findings from animal studies, we provide the first evidence in humans describing the disruptive effects of cocaine on the coordinated functioning of multiple neural systems for learning and memory.
-
From prediction error to incentive salience: mesolimbic computation of reward motivation
Berridge, Kent C.
2011-01-01
Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I will discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g., drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously-learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus a consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To comprehend these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience. PMID:22487042
-
Weber, S C; Beck-Schimmer, B; Kajdi, M-E; Müller, D; Tobler, P N; Quednow, B B
2016-07-05
Increased responding to drug-associated stimuli (cue reactivity) and an inability to tolerate delayed gratification (reward impulsivity) have been implicated in the development and maintenance of drug addiction. Whereas data from animal studies suggest that both the dopamine and opioid system are involved in these two reward-related processes, their role in humans is less clear. Moreover, dopaminergic and opioidergic drugs have not been directly compared with regard to these functions, even though a deeper understanding of the underlying mechanisms might inform the development of specific treatments for elevated cue reactivity and reward impulsivity. In a randomized, double-blind, between-subject design we administered the selective dopamine D2/D3 receptor antagonist amisulpride (400 mg, n=41), the unspecific opioid receptor antagonist naltrexone (50 mg, n=40) or placebo (n=40) to healthy humans and measured cue-induced responding with a Pavlovian-instrumental transfer task and reward impulsivity with a delay discounting task. Mood was assessed using a visual analogue scale. Compared with placebo, amisulpride significantly suppressed cue-induced responding and reward impulsivity. The effects of naltrexone were similar, although less pronounced. Both amisulpride and naltrexone decreased average mood ratings compared with placebo. Our results demonstrate that a selective blockade of dopamine D2/D3 receptors reduces cue-induced responding and reward impulsivity in healthy humans. Antagonizing μ-opioid receptors has similar effects for cue-induced responding and to a lesser extent for reward impulsivity.
-
Amygdala mu-opioid receptors mediate the motivating influence of cue-triggered reward expectations.
Lichtenberg, Nina T; Wassum, Kate M
2017-02-01
Environmental reward-predictive stimuli can retrieve from memory a specific reward expectation that allows them to motivate action and guide choice. This process requires the basolateral amygdala (BLA), but little is known about the signaling systems necessary within this structure. Here we examined the role of the neuromodulatory opioid receptor system in the BLA in such cue-directed action using the outcome-specific Pavlovian-to-instrumental transfer (PIT) test in rats. Inactivation of BLA mu-, but not delta-opioid receptors was found to dose-dependently attenuate the ability of a reward-predictive cue to selectively invigorate the performance of actions directed at the same unique predicted reward (i.e. to express outcome-specific PIT). BLA mu-opioid receptor inactivation did not affect the ability of a reward itself to similarly motivate action (outcome-specific reinstatement), suggesting a more selective role for the BLA mu-opioid receptor in the motivating influence of currently unobservable rewarding events. These data reveal a new role for BLA mu-opioid receptor activation in the cued recall of precise reward memories and the use of this information to motivate specific action plans. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
-
From prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, Kent C
2012-04-01
Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience. © 2012 The Author. European Journal of Neuroscience © 2012 Federation of European Neuroscience Societies and Blackwell Publishing Ltd.
-
Ihssen, Niklas; Mussweiler, Thomas; Linden, David E J
2016-08-01
Reward properties of stimuli can undergo sudden changes, and the detection of these 'reversals' is often made difficult by the probabilistic nature of rewards/punishments. Here we tested whether and how humans use social information (someone else's choices) to overcome uncertainty during reversal learning. We show a substantial social influence during reversal learning, which was modulated by the type of observed behavior. Participants frequently followed observed conservative choices (no switches after punishment) made by the (fictitious) other player but ignored impulsive choices (switches), even though the experiment was set up so that both types of response behavior would be similarly beneficial/detrimental (Study 1). Computational modeling showed that participants integrated the observed choices as a 'social prediction error' instead of ignoring or blindly following the other player. Modeling also confirmed higher learning rates for 'conservative' versus 'impulsive' social prediction errors. Importantly, this 'conservative bias' was boosted by interpersonal similarity, which in conjunction with the lack of effects observed in a non-social control experiment (Study 2) confirmed its social nature. A third study suggested that relative weighting of observed impulsive responses increased with increased volatility (frequency of reversals). Finally, simulations showed that in the present paradigm integrating social and reward information was not necessarily more adaptive to maximize earnings than learning from reward alone. Moreover, integrating social information increased accuracy only when conservative and impulsive choices were weighted similarly during learning. These findings suggest that to guide decisions in choice contexts that involve reward reversals humans utilize social cues conforming with their preconceptions more strongly than cues conflicting with them, especially when the other is similar. Copyright © 2016 The Authors. Published by Elsevier B
-
Monetary rewards influence retrieval orientations.
Halsband, Teresa M; Ferdinand, Nicola K; Bridger, Emma K; Mecklinger, Axel
2012-09-01
Reward anticipation during learning is known to support memory formation, but its role in retrieval processes is so far unclear. Retrieval orientations, as a reflection of controlled retrieval processing, are one aspect of retrieval that might be modulated by reward. These processes can be measured using the event-related potentials (ERPs) elicited by retrieval cues from tasks with different retrieval requirements, such as via changes in the class of targeted memory information. To determine whether retrieval orientations of this kind are modulated by reward during learning, we investigated the effects of high and low reward expectancy on the ERP correlates of retrieval orientation in two separate experiments. The reward manipulation at study in Experiment 1 was associated with later memory performance, whereas in Experiment 2, reward was directly linked to accuracy in the study task. In both studies, the participants encoded mixed lists of pictures and words preceded by high- or low-reward cues. After 24 h, they performed a recognition memory exclusion task, with words as the test items. In addition to a previously reported material-specific effect of retrieval orientation, a frontally distributed, reward-associated retrieval orientation effect was found in both experiments. These findings suggest that reward motivation during learning leads to the adoption of a reward-associated retrieval orientation to support the retrieval of highly motivational information. Thus, ERP retrieval orientation effects not only reflect retrieval processes related to the sought-for materials, but also relate to the reward conditions with which items were combined during encoding.
-
The habenula governs the attribution of incentive salience to reward predictive cues
Danna, Carey L.; Shepard, Paul D.; Elmer, Greg I.
2013-01-01
The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral...
-
Activation of dopamine D3 receptors inhibits reward-related learning induced by cocaine.
Kong, H; Kuang, W; Li, S; Xu, M
2011-03-10
Memories of learned associations between the rewarding properties of drugs and environmental cues contribute to craving and relapse in humans. The mesocorticolimbic dopamine (DA) system is involved in reward-related learning induced by drugs of abuse. DA D3 receptors are preferentially expressed in mesocorticolimbic DA projection areas. Genetic and pharmacological studies have shown that DA D3 receptors suppress locomotor-stimulant effects of cocaine and reinstatement of cocaine-seeking behaviors. Activation of the extracellular signal-regulated kinase (ERK) induced by acute cocaine administration is also inhibited by D3 receptors. How D3 receptors modulate cocaine-induced reward-related learning and associated changes in cell signaling in reward circuits in the brain, however, have not been fully investigated. In the present study, we show that D3 receptor mutant mice exhibit potentiated acquisition of conditioned place preference (CPP) at low doses of cocaine compared to wild-type mice. Activation of ERK and CaMKIIα, but not the c-Jun N-terminal kinase and p38, in the nucleus accumbens, amygdala and prefrontal cortex is also potentiated in D3 receptor mutant mice compared to that in wild-type mice following CPP expression. These results support a model in which D3 receptors modulate reward-related learning induced by low doses of cocaine by inhibiting activation of ERK and CaMKIIα in reward circuits in the brain. Copyright © 2011 IBRO. Published by Elsevier Ltd. All rights reserved.
-
A causal link between prediction errors, dopamine neurons and learning.
Steinberg, Elizabeth E; Keiflin, Ronald; Boivin, Josiah R; Witten, Ilana B; Deisseroth, Karl; Janak, Patricia H
2013-07-01
Situations in which rewards are unexpectedly obtained or withheld represent opportunities for new learning. Often, this learning includes identifying cues that predict reward availability. Unexpected rewards strongly activate midbrain dopamine neurons. This phasic signal is proposed to support learning about antecedent cues by signaling discrepancies between actual and expected outcomes, termed a reward prediction error. However, it is unknown whether dopamine neuron prediction error signaling and cue-reward learning are causally linked. To test this hypothesis, we manipulated dopamine neuron activity in rats in two behavioral procedures, associative blocking and extinction, that illustrate the essential function of prediction errors in learning. We observed that optogenetic activation of dopamine neurons concurrent with reward delivery, mimicking a prediction error, was sufficient to cause long-lasting increases in cue-elicited reward-seeking behavior. Our findings establish a causal role for temporally precise dopamine neuron signaling in cue-reward learning, bridging a critical gap between experimental evidence and influential theoretical frameworks.
-
Directory of Open Access Journals (Sweden)
Stephan Koenig
2017-07-01
Full Text Available Stimuli in our sensory environment differ with respect to their physical salience but moreover may acquire motivational salience by association with reward. If we repeatedly observed that reward is available in the context of a particular cue but absent in the context of another cue the former typically attracts more attention than the latter. However, we also may encounter cues uncorrelated with reward. A cue with 50% reward contingency may induce an average reward expectancy but at the same time induces high reward uncertainty. In the current experiment we examined how both values, reward expectancy and uncertainty, affected overt attention. Two different colors were established as predictive cues for low reward and high reward respectively. A third color was followed by high reward on 50% of the trials and thus induced uncertainty. Colors then were introduced as distractors during search for a shape target, and we examined the relative potential of the color distractors to capture and hold the first fixation. We observed that capture frequency corresponded to reward expectancy while capture duration corresponded to uncertainty. The results may suggest that within trial reward expectancy is represented at an earlier time window than uncertainty.
-
Bédard, Anne-Marie; Maheux, Jérôme; Lévesque, Daniel; Samaha, Anne-Noël
2011-05-01
Chronic exposure to antipsychotic medications can persistently change brain dopamine systems. Most studies on the functional significance of these neural changes have focused on motor behavior and few have addressed how long-term antipsychotic treatment might influence dopamine-mediated reward function. We asked, therefore, whether a clinically relevant antipsychotic treatment regimen would alter the incentive motivational properties of a reward cue. We assessed the ability of a Pavlovian-conditioned stimulus to function as a conditioned reward, as well as to elicit approach behavior in rats treated with haloperidol, either continuously (achieved via subcutaneous osmotic minipump) or intermittently (achieved via daily subcutaneous injections). Continuous, but not intermittent, treatment enhanced the ability of amphetamine to potentiate the conditioned reinforcing effects of a cue associated with water. This effect was not related to differences in the ability to attribute predictive value to a conditioned stimulus (as measured by conditioned approach behavior), but was potentially linked to the development of behavioral supersensitivity to amphetamine and to augmented amphetamine-induced immediate early-gene expression (c-fos and Nur77) in dorsal striatopallidal and striatonigral cells. By enhancing the ability of reward cues to control behavior and by intensifying dopamine-mediated striatopallidal and striatonigral cell activity, standard (ie, continuous) antipsychotic treatment regimens might exacerbate drug-seeking and drug-taking behavior in schizophrenia. Achieving regular but transiently high antipsychotic levels in the brain (as modeled in the intermittent condition) might be a viable option to prevent these changes. This possibility should be explored in the clinic.
-
de Clauser, Larissa; Kasper, Hansjörg; Schwab, Martin E.
2016-01-01
Motor skills represent high-precision movements performed at optimal speed and accuracy. Such motor skills are learned with practice over time. Besides practice, effects of motivation have also been shown to influence speed and accuracy of movements, suggesting that fast movements are performed to maximize gained reward over time as noted in previous studies. In rodents, skilled motor performance has been successfully modeled with the skilled grasping task, in which animals use their forepaw to grasp for sugar pellet rewards through a narrow window. Using sugar pellets, the skilled grasping task is inherently tied to motivation processes. In the present study, we performed three experiments modulating animals’ motivation during skilled grasping by changing the motivational state, presenting different reward value ratios, and displaying Pavlovian stimuli. We found in all three studies that motivation affected the speed of skilled grasping movements, with the strongest effects seen due to motivational state and reward value. Furthermore, accuracy of the movement, measured in success rate, showed a strong dependence on motivational state as well. Pavlovian cues had only minor effects on skilled grasping, but results indicate an inverse Pavlovian-instrumental transfer effect on movement speed. These findings have broad implications considering the increasing use of skilled grasping in studies of motor system structure, function, and recovery after injuries. PMID:27194796
-
Floresco, Stan B; Montes, David R; Tse, Maric M T; van Holstein, Mieke
2018-02-21
The nucleus accumbens (NAc) is a key node within corticolimbic circuitry for guiding action selection and cost/benefit decision making in situations involving reward uncertainty. Preclinical studies have typically assessed risk/reward decision making using assays where decisions are guided by internally generated representations of choice-outcome contingencies. Yet, real-life decisions are often influenced by external stimuli that inform about likelihoods of obtaining rewards. How different subregions of the NAc mediate decision making in such situations is unclear. Here, we used a novel assay colloquially termed the "Blackjack" task that models these types of situations. Male Long-Evans rats were trained to choose between one lever that always delivered a one-pellet reward and another that delivered four pellets with different probabilities [either 50% (good-odds) or 12.5% (poor-odds)], which were signaled by one of two auditory cues. Under control conditions, rats selected the large/risky option more often on good-odds versus poor-odds trials. Inactivation of the NAc core caused indiscriminate choice patterns. In contrast, NAc shell inactivation increased risky choice, more prominently on poor-odds trials. Additional experiments revealed that both subregions contribute to auditory conditional discrimination. NAc core or shell inactivation reduced Pavlovian approach elicited by an auditory CS+, yet shell inactivation also increased responding during presentation of a CS-. These data highlight distinct contributions for NAc subregions in decision making and reward seeking guided by discriminative stimuli. The core is crucial for implementation of conditional rules, whereas the shell refines reward seeking by mitigating the allure of larger, unlikely rewards and reducing expression of inappropriate or non-rewarded actions. SIGNIFICANCE STATEMENT Using external cues to guide decision making is crucial for adaptive behavior. Deficits in cue-guided behavior have been
-
Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior.
Garofalo, Sara; di Pellegrino, Giuseppe
2015-01-01
Pavlovian-to-instrumental transfer (PIT) refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking). In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses (CR) during Pavlovian conditioning: Sign-Trackers (ST) and Goal-Trackers (GT). Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect). This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction.
-
Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior
Directory of Open Access Journals (Sweden)
Sara eGarofalo
2015-06-01
Full Text Available Pavlovian-to-instrumental transfer (PIT refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking. In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses during Pavlovian conditioning: Sign-Trackers (ST and Goal-Trackers (GT. Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect. This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction.
-
Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior
Garofalo, Sara; di Pellegrino, Giuseppe
2015-01-01
Pavlovian-to-instrumental transfer (PIT) refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking). In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses (CR) during Pavlovian conditioning: Sign-Trackers (ST) and Goal-Trackers (GT). Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect). This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction. PMID:26157371
-
Colour cues facilitate learning flower refill schedules in wild hummingbirds.
Samuels, Michael; Hurly, T Andrew; Healy, Susan D
2014-11-01
Free-living hummingbirds can learn the refill schedules of individual experimental flowers but little is known about what information they use to do this. Colour cues, in particular, may be important to hummingbirds when learning about rewarded flower properties. We investigated, therefore, whether colour cues facilitated the learning of flower refill schedules in wild, free-living rufous hummingbirds (Selasphorus rufus). In the Cued condition, we presented birds with an array of six flowers, three of one colour, each of which were refilled 10min after being emptied by the bird and three of a different colour, which were refilled 20min after being emptied. In the Uncued condition we presented birds with six flowers of the same colour, three of which were refilled after 10min and three of which were refilled after 20min as for the birds in the Cued condition. In the second part of the experiment, we moved the array 2m and changed the shape of the array. Across both phases, birds in the Cued condition learned to discriminate between 10 and 20-min flowers more quickly than did the birds in the Uncued condition. The Cued birds were also better at discriminating between the two distinct refill intervals. Colour cues can, therefore, facilitate learning the refill schedules of experimental flowers in these birds. This article is part of a Special Issue entitled: Cognition in the wild. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Hefer, Carmen; Dreisbach, Gesine
2017-01-01
Growing evidence suggests that reward prospect promotes cognitive stability in terms of increased context or cue maintenance. In 3 Experiments, using different versions of the AX-continuous performance task, we investigated whether this reward effect comes at the cost of decreased cognitive flexibility. Experiment 1 shows that the reward induced…
-
The Role of Extrinsic Rewards and Cue-Intention Association in Prospective Memory in Young Children.
Sheppard, Daniel Patrick; Kretschmer, Anett; Knispel, Elisa; Vollert, Bianka; Altgassen, Mareike
2015-01-01
The current study examined, for the first time, the effect of cue-intention association, as well as the effects of promised extrinsic rewards, on prospective memory in young children, aged 5-years-old (n = 39) and 7-years-old (n = 40). Children were asked to name pictures for a toy mole, whilst also having to remember to respond differently to certain target pictures (prospective memory task). The level to which the target picture was associated with the intention was manipulated across two conditions (low- or high-association) for all participants, whilst half of the participants were promised a reward for good prospective memory performance. Results showed a main effect of age, with the 7-year-olds outperforming the 5-year-olds. Furthermore, there was a main effect of reward, with those promised a reward performing better than those who were not. No effect was found for cue-association, with the participants of both age groups performing equally well in both association conditions. No significant interactions were found between any of the variables. The potentially important role of reward in young children's everyday prospective memory tasks, and possible reasons for the lack of a reflexive-associative effect, are discussed.
-
Vernetti, Angélina; Smith, Tim J; Senju, Atsushi
2017-03-15
While numerous studies have demonstrated that infants and adults preferentially orient to social stimuli, it remains unclear as to what drives such preferential orienting. It has been suggested that the learned association between social cues and subsequent reward delivery might shape such social orienting. Using a novel, spontaneous indication of reinforcement learning (with the use of a gaze contingent reward-learning task), we investigated whether children and adults' orienting towards social and non-social visual cues can be elicited by the association between participants' visual attention and a rewarding outcome. Critically, we assessed whether the engaging nature of the social cues influences the process of reinforcement learning. Both children and adults learned to orient more often to the visual cues associated with reward delivery, demonstrating that cue-reward association reinforced visual orienting. More importantly, when the reward-predictive cue was social and engaging, both children and adults learned the cue-reward association faster and more efficiently than when the reward-predictive cue was social but non-engaging. These new findings indicate that social engaging cues have a positive incentive value. This could possibly be because they usually coincide with positive outcomes in real life, which could partly drive the development of social orienting. © 2017 The Authors.
-
Belief reward shaping in reinforcement learning
CSIR Research Space (South Africa)
Marom, O
2018-02-01
Full Text Available A key challenge in many reinforcement learning problems is delayed rewards, which can significantly slow down learning. Although reward shaping has previously been introduced to accelerate learning by bootstrapping an agent with additional...
-
Valence of Facial Cues Influences Sheep Learning in a Visual Discrimination Task
Directory of Open Access Journals (Sweden)
Lucille G. A. Bellegarde
2017-11-01
Full Text Available Sheep are one of the most studied farm species in terms of their ability to process information from faces, but little is known about their face-based emotion recognition abilities. We investigated (a whether sheep could use images of sheep faces taken in situation of varying valence as cues in a simultaneous discrimination task and (b whether the valence of the situation affects their learning performance. To accomplish this, we photographed faces of sheep in three situations inducing emotional states of neutral (ruminating in the home pen or negative valence (social isolation or aggressive interaction. Sheep (n = 35 first had to learn a discrimination task with colored cards. Animals that reached the learning criterion (n = 16 were then presented with pairs of images of the face of a single individual taken in the neutral situation and in one of the negative situations. Finally, sheep had to generalize what they had learned to new pairs of images of faces taken in the same situation, but of a different conspecific. All sheep that learned the discrimination task with colored cards reached the learning criterion with images of faces. Sheep that had to associate a negative image with a food reward learned faster than sheep that had to associate a neutral image with a reward. With the exception of sheep from the aggression-rewarded group, sheep generalized this discrimination to images of faces of different individuals. Our results suggest that sheep can perceive the emotional valence displayed on faces of conspecifics and that this valence affects learning processes.
-
Response of neural reward regions to food cues in autism spectrum disorders
Directory of Open Access Journals (Sweden)
Cascio Carissa J
2012-05-01
Full Text Available Abstract Background One hypothesis for the social deficits that characterize autism spectrum disorders (ASD is diminished neural reward response to social interaction and attachment. Prior research using established monetary reward paradigms as a test of non-social reward to compare with social reward may involve confounds in the ability of individuals with ASD to utilize symbolic representation of money and the abstraction required to interpret monetary gains. Thus, a useful addition to our understanding of neural reward circuitry in ASD includes a characterization of the neural response to primary rewards. Method We asked 17 children with ASD and 18 children without ASD to abstain from eating for at least four hours before an MRI scan in which they viewed images of high-calorie foods. We assessed the neural reward network for increases in the blood oxygenation level dependent (BOLD signal in response to the food images Results We found very similar patterns of increased BOLD signal to these images in the two groups; both groups showed increased BOLD signal in the bilateral amygdala, as well as in the nucleus accumbens, orbitofrontal cortex, and insula. Direct group comparisons revealed that the ASD group showed a stronger response to food cues in bilateral insula along the anterior-posterior gradient and in the anterior cingulate cortex than the control group, whereas there were no neural reward regions that showed higher activation for controls than for ASD. Conclusion These results suggest that neural response to primary rewards is not diminished but in fact shows an aberrant enhancement in children with ASD.
-
Klein-Flügge, Miriam Cornelia; Barron, Helen Catharine; Brodersen, Kay Henning; Dolan, Raymond J; Behrens, Timothy Edward John
2013-02-13
A dominant focus in studies of learning and decision-making is the neural coding of scalar reward value. This emphasis ignores the fact that choices are strongly shaped by a rich representation of potential rewards. Here, using fMRI adaptation, we demonstrate that responses in the human orbitofrontal cortex (OFC) encode a representation of the specific type of food reward predicted by a visual cue. By controlling for value across rewards and by linking each reward with two distinct stimuli, we could test for representations of reward-identity that were independent of associative information. Our results show reward-identity representations in a medial-caudal region of OFC, independent of the associated predictive stimulus. This contrasts with a more rostro-lateral OFC region encoding reward-identity representations tied to the predicate stimulus. This demonstration of adaptation in OFC to reward specific representations opens an avenue for investigation of more complex decision mechanisms that are not immediately accessible in standard analyses, which focus on correlates of average activity.
-
Robinson, Mike J F; Anselme, Patrick; Fischer, Adam M; Berridge, Kent C
2014-06-01
Uncertainty is a component of many gambling games and may play a role in incentive motivation and cue attraction. Uncertainty can increase the attractiveness for predictors of reward in the Pavlovian procedure of autoshaping, visible as enhanced sign-tracking (or approach and nibbles) by rats of a metal lever whose sudden appearance acts as a conditioned stimulus (CS+) to predict sucrose pellets as an unconditioned stimulus (UCS). Here we examined how reward uncertainty might enhance incentive salience as sign-tracking both in intensity and by broadening the range of attractive CS+s. We also examined whether initially induced uncertainty enhancements of CS+ attraction can endure beyond uncertainty itself, and persist even when Pavlovian prediction becomes 100% certain. Our results show that uncertainty can broaden incentive salience attribution to make CS cues attractive that would otherwise not be (either because they are too distal from reward or too risky to normally attract sign-tracking). In addition, uncertainty enhancement of CS+ incentive salience, once induced by initial exposure, persisted even when Pavlovian CS-UCS correlations later rose toward 100% certainty in prediction. Persistence suggests an enduring incentive motivation enhancement potentially relevant to gambling, which in some ways resembles incentive-sensitization. Higher motivation to uncertain CS+s leads to more potent attraction to these cues when they predict the delivery of uncertain rewards. In humans, those cues might possibly include the sights and sounds associated with gambling, which contribute a major component of the play immersion experienced by problematic gamblers. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Impairment of probabilistic reward-based learning in schizophrenia.
Weiler, Julia A; Bellebaum, Christian; Brüne, Martin; Juckel, Georg; Daum, Irene
2009-09-01
Recent models assume that some symptoms of schizophrenia originate from defective reward processing mechanisms. Understanding the precise nature of reward-based learning impairments might thus make an important contribution to the understanding of schizophrenia and the development of treatment strategies. The present study investigated several features of probabilistic reward-based stimulus association learning, namely the acquisition of initial contingencies, reversal learning, generalization abilities, and the effects of reward magnitude. Compared to healthy controls, individuals with schizophrenia exhibited attenuated overall performance during acquisition, whereas learning rates across blocks were similar to the rates of controls. On the group level, persons with schizophrenia were, however, unable to learn the reversal of the initial reward contingencies. Exploratory analysis of only the subgroup of individuals with schizophrenia who showed significant learning during acquisition yielded deficits in reversal learning with low reward magnitudes only. There was further evidence of a mild generalization impairment of the persons with schizophrenia in an acquired equivalence task. In summary, although there was evidence of intact basic processing of reward magnitudes, individuals with schizophrenia were impaired at using this feedback for the adaptive guidance of behavior.
-
A Molecular Dissociation between Cued and Contextual Appetitive Learning
Kheirbek, Mazen A.; Beeler, Jeff A.; Chi, Wanhao; Ishikawa, Yoshihiro; Zhuang, Xiaoxi
2010-01-01
In appetitive Pavlovian learning, animals learn to associate discrete cues or environmental contexts with rewarding outcomes, and these cues and/or contexts can potentiate an ongoing instrumental response for reward. Although anatomical substrates underlying cued and contextual learning have been proposed, it remains unknown whether specific…
-
Conditioned Object Preference: An Alternative Approach to Measuring Reward Learning in Rats
Kennedy, Bruce C.; Kohli, Maulika; Maertens, Jamie J.; Marell, Paulina S.; Gewirtz, Jonathan C.
2016-01-01
Pavlovian conditioned approach behavior can be directed as much toward discrete cues as it is toward the environmental contexts in which those cues are encountered. The current experiments characterized a tendency of rats to approach object cues whose prior exposure had been paired with reward (conditioned object preference, COP). To demonstrate…
-
Dopamine, reward learning, and active inference.
FitzGerald, Thomas H B; Dolan, Raymond J; Friston, Karl
2015-01-01
Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.
-
Dopamine, reward learning, and active inference
Directory of Open Access Journals (Sweden)
Thomas eFitzgerald
2015-11-01
Full Text Available Temporal difference learning models propose phasic dopamine signalling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behaviour. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.
-
Associations among smoking, anhedonia, and reward learning in depression.
Liverant, Gabrielle I; Sloan, Denise M; Pizzagalli, Diego A; Harte, Christopher B; Kamholz, Barbara W; Rosebrock, Laina E; Cohen, Andrew L; Fava, Maurizio; Kaplan, Gary B
2014-09-01
Depression and cigarette smoking co-occur at high rates. However, the etiological mechanisms that contribute to this relationship remain unclear. Anhedonia and associated impairments in reward learning are key features of depression, which also have been linked to the onset and maintenance of cigarette smoking. However, few studies have investigated differences in anhedonia and reward learning among depressed smokers and depressed nonsmokers. The goal of this study was to examine putative differences in anhedonia and reward learning in depressed smokers (n=36) and depressed nonsmokers (n=44). To this end, participants completed self-report measures of anhedonia and behavioral activation (BAS reward responsiveness scores) and as well as a probabilistic reward task rooted in signal detection theory, which measures reward learning (Pizzagalli, Jahn, & O'Shea, 2005). When considering self-report measures, depressed smokers reported higher trait anhedonia and reduced BAS reward responsiveness scores compared to depressed nonsmokers. In contrast to self-report measures, nicotine-satiated depressed smokers demonstrated greater acquisition of reward-based learning compared to depressed nonsmokers as indexed by the probabilistic reward task. Findings may point to a potential mechanism underlying the frequent co-occurrence of smoking and depression. These results highlight the importance of continued investigation of the role of anhedonia and reward system functioning in the co-occurrence of depression and nicotine abuse. Results also may support the use of treatments targeting reward learning (e.g., behavioral activation) to enhance smoking cessation among individuals with depression. Copyright © 2014. Published by Elsevier Ltd.
-
Vaidya, Avinash R; Fellows, Lesley K
2015-09-16
Adaptively interacting with our environment requires extracting information that will allow us to successfully predict reward. This can be a challenge, particularly when there are many candidate cues, and when rewards are probabilistic. Recent work has demonstrated that visual attention is allocated to stimulus features that have been associated with reward on previous trials. The ventromedial frontal lobe (VMF) has been implicated in learning in dynamic environments of this kind, but the mechanism by which this region influences this process is not clear. Here, we hypothesized that the VMF plays a critical role in guiding attention to reward-predictive stimulus features based on feedback. We tested the effects of VMF damage in human subjects on a visual search task in which subjects were primed to attend to task-irrelevant colors associated with different levels of reward, incidental to the search task. Consistent with previous work, we found that distractors had a greater influence on reaction time when they appeared in colors associated with high reward in the previous trial compared with colors associated with low reward in healthy control subjects and patients with prefrontal damage sparing the VMF. However, this reward modulation of attentional priming was absent in patients with VMF damage. Thus, an intact VMF is necessary for directing attention based on experience with cue-reward associations. We suggest that this region plays a role in selecting reward-predictive cues to facilitate future learning. There has been a swell of interest recently in the ventromedial frontal cortex (VMF), a brain region critical to associative learning. However, the underlying mechanism by which this region guides learning is not well understood. Here, we tested the effects of damage to this region in humans on a task in which rewards were linked incidentally to visual features, resulting in trial-by-trial attentional priming. Controls and subjects with prefrontal damage
-
Reward-Guided Learning with and without Causal Attribution
Jocham, Gerhard; Brodersen, Kay H.; Constantinescu, Alexandra O.; Kahn, Martin C.; Ianni, Angela M.; Walton, Mark E.; Rushworth, Matthew F.S.; Behrens, Timothy E.J.
2016-01-01
Summary When an organism receives a reward, it is crucial to know which of many candidate actions caused this reward. However, recent work suggests that learning is possible even when this most fundamental assumption is not met. We used novel reward-guided learning paradigms in two fMRI studies to show that humans deploy separable learning mechanisms that operate in parallel. While behavior was dominated by precise contingent learning, it also revealed hallmarks of noncontingent learning strategies. These learning mechanisms were separable behaviorally and neurally. Lateral orbitofrontal cortex supported contingent learning and reflected contingencies between outcomes and their causal choices. Amygdala responses around reward times related to statistical patterns of learning. Time-based heuristic mechanisms were related to activity in sensorimotor corticostriatal circuitry. Our data point to the existence of several learning mechanisms in the human brain, of which only one relies on applying known rules about the causal structure of the task. PMID:26971947
-
Information search with situation-specific reward functions
Directory of Open Access Journals (Sweden)
Bjorn Meder
2012-03-01
Full Text Available can strongly conflict with the goal of obtaining information for improving payoffs. Two environments with such a conflict were identified through computer optimization. Three subsequent experiments investigated people's search behavior in these environments. Experiments 1 and 2 used a multiple-cue probabilistic category-learning task to convey environmental probabilities. In a subsequent search task subjects could query only a single feature before making a classification decision. The crucial manipulation concerned the search-task reward structure. The payoffs corresponded either to accuracy, with equal rewards associated with the two categories, or to an asymmetric payoff function, with different rewards associated with each category. In Experiment 1, in which learning-task feedback corresponded to the true category, people later preferentially searched the accuracy-maximizing feature, whether or not this would improve monetary rewards. In Experiment 2, an asymmetric reward structure was used during learning. Subjects searched the reward-maximizing feature when asymmetric payoffs were preserved in the search task. However, if search-task payoffs corresponded to accuracy, subjects preferentially searched a feature that was suboptimal for reward and accuracy alike. Importantly, this feature would have been most useful, under the learning-task payoff structure. Experiment 3 found that, if words and numbers are used to convey environmental probabilities, neither reward nor accuracy consistently predicts search. These findings emphasize the necessity of taking into account people's goals and search-and-decision processes during learning, thereby challenging current models of information search.
-
Working memory and reward association learning impairments in obesity.
Coppin, Géraldine; Nolan-Poupart, Sarah; Jones-Gotman, Marilyn; Small, Dana M
2014-12-01
Obesity has been associated with impaired executive functions including working memory. Less explored is the influence of obesity on learning and memory. In the current study we assessed stimulus reward association learning, explicit learning and memory and working memory in healthy weight, overweight and obese individuals. Explicit learning and memory did not differ as a function of group. In contrast, working memory was significantly and similarly impaired in both overweight and obese individuals compared to the healthy weight group. In the first reward association learning task the obese, but not healthy weight or overweight participants consistently formed paradoxical preferences for a pattern associated with a negative outcome (fewer food rewards). To determine if the deficit was specific to food reward a second experiment was conducted using money. Consistent with Experiment 1, obese individuals selected the pattern associated with a negative outcome (fewer monetary rewards) more frequently than healthy weight individuals and thus failed to develop a significant preference for the most rewarded patterns as was observed in the healthy weight group. Finally, on a probabilistic learning task, obese compared to healthy weight individuals showed deficits in negative, but not positive outcome learning. Taken together, our results demonstrate deficits in working memory and stimulus reward learning in obesity and suggest that obese individuals are impaired in learning to avoid negative outcomes. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Voss, Joel L; O'Neil, Jonathan T; Kharitonova, Maria; Briggs-Gowan, Margaret J; Wakschlag, Lauren S
2015-01-01
Expression of learned stimulus-reward associations based on context is essential for regulation of behavior to meet situational demands. Contextual regulation improves during development, although the developmental progression of relevant neural and cognitive processes is not fully specified. We therefore measured neural correlates of flexible, contextual expression of stimulus-reward associations in pre/early-adolescent children (ages 9-13 years) and young adults (ages 19-22 years). After reinforcement learning using standard parameters, a contextual reversal manipulation was used whereby contextual cues indicated that stimulus-reward associations were the same as previously reinforced for some trials (consistent trials) or were reversed on other trials (inconsistent trials). Subjects were thus required to respond according to original stimulus-reward associations vs. reversed associations based on trial-specific contextual cues. Children and young adults did not differ in reinforcement learning or in relevant functional magnetic resonance imaging (fMRI) correlates. In contrast, adults outperformed children during contextual reversal, with better performance specifically for inconsistent trials. fMRI signals corresponding to this selective advantage included greater activity in lateral prefrontal cortex (LPFC), hippocampus, and dorsal striatum for young adults relative to children. Flexible expression of stimulus-reward associations based on context thus improves via adolescent development, as does recruitment of brain regions involved in reward learning and contextual expression of memory. HighlightsEarly-adolescent children and young adults were equivalent in reinforcement learning.Adults outperformed children in contextual expression of stimulus-reward associations.Adult advantages correlated with increased activity of relevant brain regions.Specific neurocognitive developmental changes support better contextual regulation.
-
Adolescents, adults and rewards: comparing motivational neurocircuitry recruitment using fMRI.
Directory of Open Access Journals (Sweden)
James M Bjork
Full Text Available BACKGROUND: Adolescent risk-taking, including behaviors resulting in injury or death, has been attributed in part to maturational differences in mesolimbic incentive-motivational neurocircuitry, including ostensible oversensitivity of the nucleus accumbens (NAcc to rewards. METHODOLOGY/PRINCIPAL FINDINGS: To test whether adolescents showed increased NAcc activation by cues for rewards, or by delivery of rewards, we scanned 24 adolescents (age 12-17 and 24 adults age (22-42 with functional magnetic resonance imaging while they performed a monetary incentive delay (MID task. The MID task was configured to temporally disentangle potential reward or potential loss anticipation-related brain signal from reward or loss notification-related signal. Subjects saw cues signaling opportunities to win or avoid losing $0, $.50, or $5 for responding quickly to a subsequent target. Subjects then viewed feedback of their trial success after a variable interval from cue presentation of between 6 to 17 s. Adolescents showed reduced NAcc recruitment by reward-predictive cues compared to adult controls in a linear contrast with non-incentive cues, and in a volume-of-interest analysis of signal change in the NAcc. In contrast, adolescents showed little difference in striatal and frontocortical responsiveness to reward deliveries compared to adults. CONCLUSIONS/SIGNIFICANCE: In light of divergent developmental difference findings between neuroimaging incentive paradigms (as well as at different stages within the same task, these data suggest that maturational differences in incentive-motivational neurocircuitry: 1 may be sensitive to nuances of incentive tasks or stimuli, such as behavioral or learning contingencies, and 2 may be specific to the component of the instrumental behavior (such as anticipation versus notification.
-
Adolescent cannabinoid exposure effects on natural reward seeking and learning in rats.
Schoch, H; Huerta, M Y; Ruiz, C M; Farrell, M R; Jung, K M; Huang, J J; Campbell, R R; Piomelli, D; Mahler, S V
2018-01-01
Adolescence is characterized by endocannabinoid (ECB)-dependent refinement of neural circuits underlying emotion, learning, and motivation. As a result, adolescent cannabinoid receptor stimulation (ACRS) with phytocannabinoids or synthetic agonists like "Spice" cause robust and persistent changes in both behavior and circuit architecture in rodents, including in reward-related regions like medial prefrontal cortex and nucleus accumbens (NAc). Here, we examine persistent effects of ACRS with the cannabinoid receptor 1/2 specific agonist WIN55-212,2 (WIN; 1.2 mg/kg/day, postnatal day (PD) 30-43), on natural reward-seeking behaviors and ECB system function in adult male Long Evans rats (PD 60+). WIN ACRS increased palatable food intake, and altered attribution of incentive salience to food cues in a sign-/goal-tracking paradigm. ACRS also blunted hunger-induced sucrose intake, and resulted in increased anandamide and oleoylethanolamide levels in NAc after acute food restriction not seen in controls. ACRS did not affect food neophobia or locomotor response to a novel environment, but did increase preference for exploring a novel environment. These results demonstrate that ACRS causes long-term increases in natural reward-seeking behaviors and ECB system function that persist into adulthood, potentially increasing liability to excessive natural reward seeking later in life.
-
Post-learning hippocampal dynamics promote preferential retention of rewarding events
Gruber, Matthias J.; Ritchey, Maureen; Wang, Shao-Fang; Doss, Manoj K.; Ranganath, Charan
2016-01-01
Reward motivation is known to modulate memory encoding, and this effect depends on interactions between the substantia nigra/ ventral tegmental area complex (SN/VTA) and the hippocampus. It is unknown, however, whether these interactions influence offline neural activity in the human brain that is thought to promote memory consolidation. Here, we used functional magnetic resonance imaging (fMRI) to test the effect of reward motivation on post-learning neural dynamics and subsequent memory for objects that were learned in high- or low-reward motivation contexts. We found that post-learning increases in resting-state functional connectivity between the SN/VTA and hippocampus predicted preferential retention of objects that were learned in high-reward contexts. In addition, multivariate pattern classification revealed that hippocampal representations of high-reward contexts were preferentially reactivated during post-learning rest, and the number of hippocampal reactivations was predictive of preferential retention of items learned in high-reward contexts. These findings indicate that reward motivation alters offline post-learning dynamics between the SN/VTA and hippocampus, providing novel evidence for a potential mechanism by which reward could influence memory consolidation. PMID:26875624
-
Incidental Learning of Rewarded Associations Bolsters Learning on an Associative Task
Freedberg, Michael; Schacherer, Jonathan; Hazeltine, Eliot
2016-01-01
Reward has been shown to change behavior as a result of incentive learning (by motivating the individual to increase their effort) and instrumental learning (by increasing the frequency of a particular behavior). However, Palminteri et al. (2011) demonstrated that reward can also improve the incidental learning of a motor skill even when…
-
A review of reward processing and motivational impairment in schizophrenia.
Strauss, Gregory P; Waltz, James A; Gold, James M
2014-03-01
This article reviews and synthesizes research on reward processing in schizophrenia, which has begun to provide important insights into the cognitive and neural mechanisms associated with motivational impairments. Aberrant cortical-striatal interactions may be involved with multiple reward processing abnormalities, including: (1) dopamine-mediated basal ganglia systems that support reinforcement learning and the ability to predict cues that lead to rewarding outcomes; (2) orbitofrontal cortex-driven deficits in generating, updating, and maintaining value representations; (3) aberrant effort-value computations, which may be mediated by disrupted anterior cingulate cortex and midbrain dopamine functioning; and (4) altered activation of the prefrontal cortex, which is important for generating exploratory behaviors in environments where reward outcomes are uncertain. It will be important for psychosocial interventions targeting negative symptoms to account for abnormalities in each of these reward processes, which may also have important interactions; suggestions for novel behavioral intervention strategies that make use of external cues, reinforcers, and mobile technology are discussed.
-
Reward/Punishment reversal learning in older suicide attempters.
Dombrovski, Alexandre Y; Clark, Luke; Siegle, Greg J; Butters, Meryl A; Ichikawa, Naho; Sahakian, Barbara J; Szanto, Katalin
2010-06-01
Suicide rates are high in old age, and the contribution of cognitive risk factors remains poorly understood. Suicide may be viewed as an outcome of an altered decision process. The authors hypothesized that impairment in reward/punishment-based learning, a component of affective decision making, is associated with attempted suicide in late-life depression. They expected that suicide attempters would discount past reward/punishment history, focusing excessively on the most recent rewards and punishments. The authors further hypothesized that this impairment could be dissociated from executive abilities, such as forward planning. The authors assessed reward/punishment-based learning using the probabilistic reversal learning task in 65 individuals age 60 and older: suicide attempters, suicide ideators, nonsuicidal depressed elderly, and nondepressed comparison subjects. The authors used a reinforcement learning computational model to decompose reward/punishment processing over time. The Stockings of Cambridge test served as a control measure of executive function. Suicide attempters but not suicide ideators showed impaired probabilistic reversal learning compared to both nonsuicidal depressed elderly and nondepressed comparison subjects, after controlling for effects of education, global cognitive function, and substance use. Model-based analyses revealed that suicide attempters discounted previous history to a higher degree relative to comparison subjects, basing their choice largely on reward/punishment received on the last trial. Groups did not differ in their performance on the Stockings of Cambridge test. Older suicide attempters display impaired reward/punishment-based learning. The authors propose a hypothesis that older suicide attempters make overly present-focused decisions, ignoring past experiences. Modification of this "myopia for the past" may have therapeutic potential.
-
Wild, free-living rufous hummingbirds do not use geometric cues in a spatial task.
Hornsby, Mark A W; Hurly, T Andrew; Hamilton, Caitlin E; Pritchard, David J; Healy, Susan D
2014-10-01
In the laboratory, many species orient themselves using the geometric properties of an enclosure or array and geometric information is often preferred over visual cues. Whether animals use geometric cues when relocating rewarded locations in the wild, however, has rarely been investigated. We presented free-living rufous hummingbirds with a rectangular array of four artificial flowers to investigate learning of rewarded locations using geometric cues. In one treatment, we rewarded two of four flowers at diagonally opposite corners. In a second treatment, we provided a visual cue to the rewarded flower by connecting the flowers with "walls" consisting of four dowels (three white, one blue) laid on the ground connecting each of the flowers. Neither treatment elicited classical geometry results; instead, hummingbirds typically chose one particular flower over all others. When we exchanged that flower with another, hummingbirds tended to visit the original flower. These results suggest that (1) hummingbirds did not use geometric cues, but instead may have used a visually derived cue on the flowers themselves, and (2) using geometric cues may have been more difficult than using visual characteristics. Although hummingbirds typically prefer spatial over visual information, we hypothesize that they will not use geometric cues over stable visual features but that they make use of small, flower-specific visual cues. Such cues may play a more important role in foraging decisions than previously thought. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Learning Reward Uncertainty in the Basal Ganglia.
Directory of Open Access Journals (Sweden)
John G Mikhael
2016-09-01
Full Text Available Learning the reliability of different sources of rewards is critical for making optimal choices. However, despite the existence of detailed theory describing how the expected reward is learned in the basal ganglia, it is not known how reward uncertainty is estimated in these circuits. This paper presents a class of models that encode both the mean reward and the spread of the rewards, the former in the difference between the synaptic weights of D1 and D2 neurons, and the latter in their sum. In the models, the tendency to seek (or avoid options with variable reward can be controlled by increasing (or decreasing the tonic level of dopamine. The models are consistent with the physiology of and synaptic plasticity in the basal ganglia, they explain the effects of dopaminergic manipulations on choices involving risks, and they make multiple experimental predictions.
-
The Role of Extrinsic Rewards and Cue-Intention Association in Prospective Memory in Young Children
Sheppard, D.P.; Kretschmer, A.; Knispel, E.; Vollert, B.; Altgassen, A.M.
2015-01-01
The current study examined, for the first time, the effect of cue-intention association, as well as the effects of promised extrinsic rewards, on prospective memory in young children, aged 5-years-old (n = 39) and 7-years-old (n = 40). Children were asked to name pictures for a toy mole, whilst also
-
Prosocial reward learning in children and adolescents
Directory of Open Access Journals (Sweden)
Youngbin Kwak
2016-10-01
Full Text Available Adolescence is a period of increased sensitivity to social contexts. To evaluate how social context sensitivity changes over development – and influences reward learning – we investigated how children and adolescents perceive and integrate rewards for oneself and others during a dynamic risky-decision-making task. Children and adolescents (N=75, 8-16 yrs performed the Social Gambling Task (SGT, (Kwak et al., 2014 and completed a set of questionnaires measuring other-regarding behavior. In the SGT, participants choose amongst four card decks that have different payout structures for oneself and for a charity. We examined patterns of choices, overall decision strategies, and how reward outcomes led to trial-by-trial adjustments in behavior, as estimated using a reinforcement-learning model. Performance of children and adolescents was compared to data from a previously collected sample of adults (N=102 performing the identical task. We found that that children/adolescents were not only more sensitive to rewards directed to the charity than self but also showed greater prosocial tendencies on independent measures of other-regarding behavior. Children and adolescents also showed less use of a strategy that prioritizes rewards for self at the expense of rewards for others. These results support the conclusion that, compared to adults, children and adolescents show greater sensitivity to outcomes for others when making decisions and learning about potential rewards.
-
Kim, Sang Hee; Yoon, HeungSik; Kim, Hackjin; Hamann, Stephan
2015-09-01
In this functional neuroimaging study, we investigated neural activations during the process of learning to gain monetary rewards and to avoid monetary loss, and how these activations are modulated by individual differences in reward and punishment sensitivity. Healthy young volunteers performed a reinforcement learning task where they chose one of two fractal stimuli associated with monetary gain (reward trials) or avoidance of monetary loss (avoidance trials). Trait sensitivity to reward and punishment was assessed using the behavioral inhibition/activation scales (BIS/BAS). Functional neuroimaging results showed activation of the striatum during the anticipation and reception periods of reward trials. During avoidance trials, activation of the dorsal striatum and prefrontal regions was found. As expected, individual differences in reward sensitivity were positively associated with activation in the left and right ventral striatum during reward reception. Individual differences in sensitivity to punishment were negatively associated with activation in the left dorsal striatum during avoidance anticipation and also with activation in the right lateral orbitofrontal cortex during receiving monetary loss. These results suggest that learning to attain reward and learning to avoid loss are dependent on separable sets of neural regions whose activity is modulated by trait sensitivity to reward or punishment. © The Author (2015). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Evidence for a shared representation of sequential cues that engage sign-tracking.
Smedley, Elizabeth B; Smith, Kyle S
2018-06-19
Sign-tracking is a phenomenon whereby cues that predict rewards come to acquire their own motivational value (incentive salience) and attract appetitive behavior. Typically, sign-tracking paradigms have used single auditory, visual, or lever cues presented prior to a reward delivery. Yet, real world examples of events often can be predicted by a sequence of cues. We have shown that animals will sign-track to multiple cues presented in temporal sequence, and with time develop a bias in responding toward a reward distal cue over a reward proximal cue. Further, extinction of responding to the reward proximal cue directly decreases responding to the reward distal cue. One possible explanation of this result is that serial cues become representationally linked with one another. Here we provide further support of this by showing that extinction of responding to a reward distal cue directly reduces responding to a reward proximal cue. We suggest that the incentive salience of one cue can influence the incentive salience of the other cue. Copyright © 2018. Published by Elsevier B.V.
-
Luo, Shan; Monterosso, John R; Sarpelleh, Kayan; Page, Kathleen A
2015-05-19
Prior studies suggest that fructose compared with glucose may be a weaker suppressor of appetite, and neuroimaging research shows that food cues trigger greater brain reward responses in a fasted relative to a fed state. We sought to determine the effects of ingesting fructose versus glucose on brain, hormone, and appetitive responses to food cues and food-approach behavior. Twenty-four healthy volunteers underwent two functional magnetic resonance imaging (fMRI) sessions with ingestion of either fructose or glucose in a double-blinded, random-order cross-over design. fMRI was performed while participants viewed images of high-calorie foods and nonfood items using a block design. After each block, participants rated hunger and desire for food. Participants also performed a decision task in which they chose between immediate food rewards and delayed monetary bonuses. Hormones were measured at baseline and 30 and 60 min after drink ingestion. Ingestion of fructose relative to glucose resulted in smaller increases in plasma insulin levels and greater brain reactivity to food cues in the visual cortex (in whole-brain analysis) and left orbital frontal cortex (in region-of-interest analysis). Parallel to the neuroimaging findings, fructose versus glucose led to greater hunger and desire for food and a greater willingness to give up long-term monetary rewards to obtain immediate high-calorie foods. These findings suggest that ingestion of fructose relative to glucose results in greater activation of brain regions involved in attention and reward processing and may promote feeding behavior.
-
Food approach conditioning and discrimination learning using sound cues in benthic sharks.
Vila Pouca, Catarina; Brown, Culum
2018-07-01
The marine environment is filled with biotic and abiotic sounds. Some of these sounds predict important events that influence fitness while others are unimportant. Individuals can learn specific sound cues and 'soundscapes' and use them for vital activities such as foraging, predator avoidance, communication and orientation. Most research with sounds in elasmobranchs has focused on hearing thresholds and attractiveness to sound sources, but very little is known about their abilities to learn about sounds, especially in benthic species. Here we investigated if juvenile Port Jackson sharks could learn to associate a musical stimulus with a food reward, discriminate between two distinct musical stimuli, and whether individual personality traits were linked to cognitive performance. Five out of eight sharks were successfully conditioned to associate a jazz song with a food reward delivered in a specific corner of the tank. We observed repeatable individual differences in activity and boldness in all eight sharks, but these personality traits were not linked to the learning performance assays we examined. These sharks were later trained in a discrimination task, where they had to distinguish between the same jazz and a novel classical music song, and swim to opposite corners of the tank according to the stimulus played. The sharks' performance to the jazz stimulus declined to chance levels in the discrimination task. Interestingly, some sharks developed a strong side bias to the right, which in some cases was not the correct side for the jazz stimulus.
-
Peciña, Susana; Berridge, Kent C
2013-05-01
Pavlovian cues [conditioned stimulus (CS+)] often trigger intense motivation to pursue and consume related reward [unconditioned stimulus (UCS)]. But cues do not always trigger the same intensity of motivation. Encountering a reward cue can be more tempting on some occasions than on others. What makes the same cue trigger more intense motivation to pursue reward on a particular encounter? The answer may be the level of incentive salience ('wanting') that is dynamically generated by mesocorticolimbic brain systems, influenced especially by dopamine and opioid neurotransmission in the nucleus accumbens (NAc) at that moment. We tested the ability of dopamine stimulation (by amphetamine microinjection) vs. mu opioid stimulation [by d-Ala, nMe-Phe, Glyol-enkephalin (DAMGO) microinjection] of either the core or shell of the NAc to amplify cue-triggered levels of motivation to pursue sucrose reward, measured with a Pavlovian-Instrumental Transfer (PIT) procedure, a relatively pure assay of incentive salience. Cue-triggered 'wanting' in PIT was enhanced by amphetamine or DAMGO microinjections equally, and also equally at nearly all sites throughout the entire core and medial shell (except for a small far-rostral strip of shell). NAc dopamine/opioid stimulations specifically enhanced CS+ ability to trigger phasic peaks of 'wanting' to obtain UCS, without altering baseline efforts when CS+ was absent. We conclude that dopamine/opioid stimulation throughout nearly the entire NAc can causally amplify the reactivity of mesocorticolimbic circuits, and so magnify incentive salience or phasic UCS 'wanting' peaks triggered by a CS+. Mesolimbic amplification of incentive salience may explain why a particular cue encounter can become irresistibly tempting, even when previous encounters were successfully resisted before. © 2013 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
-
Tiger salamanders' (Ambystoma tigrinum) response learning and usage of visual cues.
Kundey, Shannon M A; Millar, Roberto; McPherson, Justin; Gonzalez, Maya; Fitz, Aleyna; Allen, Chadbourne
2016-05-01
We explored tiger salamanders' (Ambystoma tigrinum) learning to execute a response within a maze as proximal visual cue conditions varied. In Experiment 1, salamanders learned to turn consistently in a T-maze for reinforcement before the maze was rotated. All learned the initial task and executed the trained turn during test, suggesting that they learned to demonstrate the reinforced response during training and continued to perform it during test. In a second experiment utilizing a similar procedure, two visual cues were placed consistently at the maze junction. Salamanders were reinforced for turning towards one cue. Cue placement was reversed during test. All learned the initial task, but executed the trained turn rather than turning towards the visual cue during test, evidencing response learning. In Experiment 3, we investigated whether a compound visual cue could control salamanders' behaviour when it was the only cue predictive of reinforcement in a cross-maze by varying start position and cue placement. All learned to turn in the direction indicated by the compound visual cue, indicating that visual cues can come to control their behaviour. Following training, testing revealed that salamanders attended to stimuli foreground over background features. Overall, these results suggest that salamanders learn to execute responses over learning to use visual cues but can use visual cues if required. Our success with this paradigm offers the potential in future studies to explore salamanders' cognition further, as well as to shed light on how features of the tiger salamanders' life history (e.g. hibernation and metamorphosis) impact cognition.
-
DISRUPTION OF CONDITIONED REWARD ASSOCIATION BY TYPICAL AND ATYPICAL ANTIPSYCHOTICS
Danna, C.L.; Elmer, G.I.
2013-01-01
Antipsychotic drugs are broadly classified into typical and atypical compounds; they vary in their pharmacological profile however a common component is their antagonist effects at the D2 dopamine receptors (DRD2). Unfortunately, diminished DRD2 activation is generally thought to be associated with the severity of neuroleptic-induced anhedonia. The purpose of this study was to determine the effect of the atypical antipsychotic olanzapine and typical antipsychotic haloperidol in a paradigm that reflects the learned transfer of incentive motivational properties to previously neutral stimuli, namely autoshaping. In order to provide a dosing comparison to a therapeutically relevant endpoint, both drugs were tested against amphetamine-induced disruption of prepulse inhibition as well. In the autoshaping task, rats were exposed to repeated pairings of stimuli that were differentially predictive of reward delivery. Conditioned approach to the reward predictive cue (sign-tracking) and to the reward (goal-tracking) increased during repeated pairings in the vehicle treated rats. Haloperidol and olanzapine completely abolished this behavior at relatively low doses (100 μg/kg). This same dose was the threshold dose for each drug to antagonize the sensorimotor gating deficits produced by amphetamine. At lower doses (3–30 μg/kg) both drugs produced a dose-dependent decrease in conditioned approach to the reward predictive cue. There was no difference between drugs at this dose range which indicates that olanzapine disrupts autoshaping at a significantly lower proposed DRD2 receptor occupancy. Interestingly, neither drug disrupted conditioned approach to the reward at the same dose range that disrupted conditioned approach to the reward predictive cue. Thus, haloperidol and olanzapine, at doses well below what is considered therapeutically relevant, disrupts the attribution of incentive motivational value to previously neutral cues. Drug effects on this dimension of reward
-
Bialleck, Katharina A.; Schaal, Hans-Peter; Kranz, Thorsten A.; Fell, Juergen; Elger, Christian E.; Axmacher, Nikolai
2011-01-01
During reinforcement learning, dopamine release shifts from the moment of reward consumption to the time point when the reward can be predicted. Previous studies provide consistent evidence that reward-predicting cues enhance long-term memory (LTM) formation of these items via dopaminergic projections to the ventral striatum. However, it is less clear whether memory for items that do not precede a reward but are directly associated with reward consumption is also facilitated. Here, we investigated this question in an fMRI paradigm in which LTM for reward-predicting and neutral cues was compared to LTM for items presented during consumption of reliably predictable as compared to less predictable rewards. We observed activation of the ventral striatum and enhanced memory formation during reward anticipation. During processing of less predictable as compared to reliably predictable rewards, the ventral striatum was activated as well, but items associated with less predictable outcomes were remembered worse than items associated with reliably predictable outcomes. Processing of reliably predictable rewards activated the ventromedial prefrontal cortex (vmPFC), and vmPFC BOLD responses were associated with successful memory formation of these items. Taken together, these findings show that consumption of reliably predictable rewards facilitates LTM formation and is associated with activation of the vmPFC. PMID:21326612
-
Shang, Andrea; Bylipudi, Sooraz; Bieszczad, Kasia M
2018-05-31
Epigenetic mechanisms are key for regulating long-term memory (LTM) and are known to exert control on memory formation in multiple systems of the adult brain, including the sensory cortex. One epigenetic mechanism is chromatin modification by histone acetylation. Blocking the action of histone de-acetylases (HDACs) that normally negatively regulate LTM by repressing transcription, has been shown to enable memory formation. Indeed, HDAC-inhibition appears to facilitate memory by altering the dynamics of gene expression events important for memory consolidation. However less understood are the ways in which molecular-level consolidation processes alter subsequent memory to enhance storage or facilitate retrieval. Here we used a sensory perspective to investigate whether the characteristics of memory formed with HDAC inhibitors are different from naturally-formed memory. One possibility is that HDAC inhibition enables memory to form with greater sensory detail than normal. Because the auditory system undergoes learning-induced remodeling that provides substrates for sound-specific LTM, we aimed to identify behavioral effects of HDAC inhibition on memory for specific sound features using a standard model of auditory associative cue-reward learning, memory, and cortical plasticity. We found that three systemic post-training treatments of an HDAC3-inhibitor (RGPF966, Abcam Inc.) in rats in the early phase of training facilitated auditory discriminative learning, changed auditory cortical tuning, and increased the specificity for acoustic frequency formed in memory of both excitatory (S+) and inhibitory (S-) associations for at least 2 weeks. The findings support that epigenetic mechanisms act on neural and behavioral sensory acuity to increase the precision of associative cue memory, which can be revealed by studying the sensory characteristics of long-term associative memory formation with HDAC inhibitors. Published by Elsevier B.V.
-
Instant transformation of learned repulsion into motivational "wanting".
Robinson, Mike J F; Berridge, Kent C
2013-02-18
Learned cues for pleasant reward often elicit desire, which, in addicts, may become compulsive. According to the dominant view in addiction neuroscience and reinforcement modeling, such desires are the simple products of learning, coming from a past association with reward outcome. We demonstrate that cravings are more than merely the products of accumulated pleasure memories-even a repulsive learned cue for unpleasantness can become suddenly desired via the activation of mesocorticolimbic circuitry. Rats learned repulsion toward a Pavlovian cue (a briefly-inserted metal lever) that always predicted an unpleasant Dead Sea saltiness sensation. Yet, upon first reencounter in a novel sodium-depletion state to promote mesocorticolimbic reactivity (reflected by elevated Fos activation in ventral tegmentum, nucleus accumbens, ventral pallidum, and the orbitofrontal prefrontal cortex), the learned cue was instantly transformed into an attractive and powerful motivational magnet. Rats jumped and gnawed on the suddenly attractive Pavlovian lever cue, despite never having tasted intense saltiness as anything other than disgusting. Instant desire transformation of a learned cue contradicts views that Pavlovian desires are essentially based on previously learned values (e.g., prediction error or temporal difference models). Instead desire is recomputed at reencounter by integrating Pavlovian information with the current brain/physiological state. This powerful brain transformation reverses strong learned revulsion into avid attraction. When applied to addiction, related mesocorticolimbic transformations (e.g., drugs or neural sensitization) of cues for already-pleasant drug experiences could create even more intense cravings. This cue/state transformation helps define what it means to say that addiction hijacks brain limbic circuits of natural reward. Copyright © 2013 Elsevier Ltd. All rights reserved.
-
Neuropsychology of Reward Learning and Negative Symptoms in Schizophrenia
Nestor, Paul G.; Choate, Victoria; Niznikiewicz, Margaret; Levitt, James J.; Shenton, Martha E; McCarley, Robert W.
2014-01-01
We used the Iowa Gambling Test (IGT) to examine the relationship of reward learning to both neuropsychological functioning and symptom formation in 65 individuals with schizophrenia. Results indicated that compared to controls, participants with schizophrenia showed significantly reduced reward learning, which in turn correlated with reduced intelligence, memory and executive function, and increased negative symptoms. The current findings suggested that a disease-related disturbance in reward...
-
Pressure to cooperate: is positive reward interdependence really needed in cooperative learning?
Buchs, Céline; Gilles, Ingrid; Dutrévis, Marion; Butera, Fabrizio
2011-03-01
BACKGROUND. Despite extensive research on cooperative learning, the debate regarding whether or not its effectiveness depends on positive reward interdependence has not yet found clear evidence. AIMS. We tested the hypothesis that positive reward interdependence, as compared to reward independence, enhances cooperative learning only if learners work on a 'routine task'; if the learners work on a 'true group task', positive reward interdependence induces the same level of learning as reward independence. SAMPLE. The study involved 62 psychology students during regular workshops. METHOD. Students worked on two psychology texts in cooperative dyads for three sessions. The type of task was manipulated through resource interdependence: students worked on either identical (routine task) or complementary (true group task) information. Students expected to be assessed with a Multiple Choice Test (MCT) on the two texts. The MCT assessment type was introduced according to two reward interdependence conditions, either individual (reward independence) or common (positive reward interdependence). A follow-up individual test took place 4 weeks after the third session of dyadic work to examine individual learning. RESULTS. The predicted interaction between the two types of interdependence was significant, indicating that students learned more with positive reward interdependence than with reward independence when they worked on identical information (routine task), whereas students who worked on complementary information (group task) learned the same with or without reward interdependence. CONCLUSIONS. This experiment sheds light on the conditions under which positive reward interdependence enhances cooperative learning, and suggests that creating a real group task allows to avoid the need for positive reward interdependence. © 2010 The British Psychological Society.
-
Neuropsychology of reward learning and negative symptoms in schizophrenia.
Nestor, Paul G; Choate, Victoria; Niznikiewicz, Margaret; Levitt, James J; Shenton, Martha E; McCarley, Robert W
2014-11-01
We used the Iowa Gambling Test (IGT) to examine the relationship of reward learning to both neuropsychological functioning and symptom formation in 65 individuals with schizophrenia. Results indicated that compared to controls, participants with schizophrenia showed significantly reduced reward learning, which in turn correlated with reduced intelligence, memory and executive function, and negative symptoms. The current findings suggested that a disease-related disturbance in reward learning may underlie both cognitive and motivation deficits, as expressed by neuropsychological impairment and negative symptoms in schizophrenia. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Schuweiler, D R; Athens, J M; Thompson, J M; Vazhayil, S T; Garris, P A
2018-01-15
Rewarding doses of amphetamine increase the amplitude, duration, and frequency of dopamine transients in the ventral striatum. Debate continues at the behavioral level about which component of reward, learning or incentive salience, is signaled by these dopamine transients and thus altered in addiction. The learning hypothesis proposes that rewarding drugs result in pathological overlearning of drug-predictive cues, while the incentive sensitization hypothesis suggests that rewarding drugs result in sensitized attribution of incentive salience to drug-predictive cues. Therapeutic doses of amphetamine, such as those used to treat attention-deficit hyperactivity disorder, are hypothesized to enhance the ventral striatal dopamine transients that are critical for reward-related learning and to enhance Pavlovian learning. However, the effects of therapeutic doses of amphetamine on Pavlovian learning are poorly understood, and the effects on dopamine transients are completely unknown. We determined the effects of an acute pre-training therapeutic or rewarding amphetamine injection on the acquisition of Pavlovian autoshaping in the intact rat. We also determined the effects of these doses on electrically evoked transient-like dopamine signals using fast-scan cyclic voltammetry in the anesthetized rat. The rewarding dose enhanced the amplitude and duration of DA signals, caused acute task disengagement, impaired learning for several days, and triggered incentive sensitization. The therapeutic dose produced smaller enhancements in DA signals but did not have similar behavioral effects. These results underscore the necessity of more studies using therapeutic doses, and suggest a hybrid learning/incentive sensitization model may be required to explain the development of addiction. Copyright © 2017 Elsevier B.V. All rights reserved.
-
Distinct Motivational Effects of Contingent and Noncontingent Rewards.
Manohar, Sanjay G; Finzi, Rebecca Dawn; Drew, Daniel; Husain, Masud
2017-07-01
When rewards are available, people expend more energy, increasing their motivational vigor. In theory, incentives might drive behavior for two distinct reasons: First, they increase expected reward; second, they increase the difference in subjective value between successful and unsuccessful performance, which increases contingency-the degree to which action determines outcome. Previous studies of motivational vigor have never compared these directly. Here, we indexed motivational vigor by measuring the speed of eye movements toward a target after participants heard a cue indicating how outcomes would be determined. Eye movements were faster when the cue indicated that monetary rewards would be contingent on performance than when the cue indicated that rewards would be random. But even when the cue indicated that a reward was guaranteed regardless of speed, movement was still faster than when no reward was available. Motivation by contingent and certain rewards was uncorrelated across individuals, which suggests that there are two separable, independent components of motivation. Contingent motivation generated autonomic arousal, and unlike noncontingent motivation, was effective with penalties as well as rewards.
-
Bennion, Kelly A; Payne, Jessica D; Kensinger, Elizabeth A
2016-06-01
Prior research has demonstrated that sleep enhances memory for future-relevant information, including memory for information that is salient due to emotion, reward, or knowledge of a later memory test. Although sleep has been shown to prioritize information with any of these characteristics, the present study investigates the novel question of how sleep prioritizes information when multiple salience cues exist. Participants encoded scenes that were future-relevant based on emotion (emotional vs. neutral), reward (rewarded vs. unrewarded), and instructed learning (intentionally vs. incidentally encoded), preceding a delay consisting of a nap, an equivalent time period spent awake, or a nap followed by wakefulness (to control for effects of interference). Recognition testing revealed that when multiple dimensions of future relevance co-occur, sleep prioritizes top-down, goal-directed cues (instructed learning, and to a lesser degree, reward) over bottom-up, stimulus-driven characteristics (emotion). Further, results showed that these factors interact; the effect of a nap on intentionally encoded information was especially strong for neutral (relative to emotional) information, suggesting that once one cue for future relevance is present, there are diminishing returns with additional cues. Sleep may binarize information based on whether it is future-relevant or not, preferentially consolidating memory for the former category. Potential neural mechanisms underlying these selective effects and the implications of this research for educational and vocational domains are discussed. (PsycINFO Database Record (c) 2016 APA, all rights reserved).
-
Reward learning and negative emotion during rapid attentional competition
Directory of Open Access Journals (Sweden)
Takemasa eYokoyama
2015-03-01
Full Text Available Learned stimulus-reward associations influence how attention is allocated, such that stimuli rewarded in the past are favored in situations involving limited resources and competition. At the same time, task-irrelevant, high-arousal negative stimuli capture attention and divert resources away from tasks resulting in poor behavioral performance. Yet, investigations of how reward learning and negative stimuli affect perceptual and attentional processing have been conducted in a largely independent fashion. We have recently reported that performance-based monetary rewards reduce negative stimuli interference during perception. The goal of the present study was to investigate how stimuli associated with past monetary rewards compete with negative stimuli during a subsequent attentional task when, critically, no performance-based rewards were at stake. Across two experiments, we found that target stimuli that were associated with high reward reduced the interference effect of potent, negative distractors. Similar to our recent findings with performance-based rewards, our results demonstrate that reward-associated stimuli reduce the deleterious impact of negative stimuli on behavior.
-
Reminder cues modulate the renewal effect in human predictive learning
Directory of Open Access Journals (Sweden)
Javier Bustamante
2016-12-01
Full Text Available Associative learning refers to our ability to learn about regularities in our environment. When a stimulus is repeatedly followed by a specific outcome, we learn to expect the outcome in the presence of the stimulus. We are also able to modify established expectations in the face of disconfirming information (the stimulus is no longer followed by the outcome. Both the change of environmental regularities and the related processes of adaptation are referred to as extinction. However, extinction does not erase the initially acquired expectations. For instance, following successful extinction, the initially learned expectations can recover when there is a context change – a phenomenon called the renewal effect, which is considered as a model for relapse after exposure therapy. Renewal was found to be modulated by reminder cues of acquisition and extinction. However, the mechanisms underlying the effectiveness of reminder cues are not well understood. The aim of the present study was to investigate the impact of reminder cues on renewal in the field of human predictive learning. Experiment I demonstrated that renewal in human predictive learning is modulated by cues related to acquisition or extinction. Initially, participants received pairings of a stimulus and an outcome in one context. These stimulus-outcome pairings were preceded by presentations of a reminder cue (acquisition cue. Then, participants received extinction in a different context in which presentations of the stimulus were no longer followed by the outcome. These extinction trials were preceded by a second reminder cue (extinction cue. During a final phase conducted in a third context, participants showed stronger expectations of the outcome in the presence of the stimulus when testing was accompanied by the acquisition cue compared to the extinction cue. Experiment II tested an explanation of the reminder cue effect in terms of simple cue-outcome associations. Therefore
-
Wolosin, Sasha M; Zeithamova, Dagmar; Preston, Alison R
2013-11-01
Recent research indicates that reward-based motivation impacts medial temporal lobe (MTL) encoding processes, leading to enhanced memory for rewarded events. In particular, previous functional magnetic resonance imaging (fMRI) studies of motivated learning have shown that MTL activation is greater for highly rewarded events, with the degree of reward-related activation enhancement tracking the corresponding behavioral memory advantage. These studies, however, do not directly address leading theoretical perspectives that propose such reward-based enhancements in MTL encoding activation reflect enhanced discrimination of the motivational context of specific events. In this study, a high-value or low-value monetary cue preceded a pair of objects, indicating the future reward for successfully remembering the pair. Using representational similarity analysis and high-resolution fMRI, we show that MTL activation patterns are more similar for encoding trials preceded by the same versus different reward cues, indicating a distributed code in this region that distinguishes between motivational contexts. Moreover, we show that activation patterns in hippocampus and parahippocampal cortex (PHc) that differentiate reward conditions during anticipatory cues and object pairs relate to successful associative memory. Additionally, the degree to which patterns differentiate reward contexts in dentate gyrus/CA2,3 and PHc is related to individual differences in reward modulation of memory. Collectively, these findings suggest that distributed activation patterns in the human hippocampus and PHc reflect the rewards associated with individual events. Furthermore, we show that these activation patterns-which discriminate between reward conditions--may influence memory through the incorporation of information about motivational contexts into stored memory representations. PsycINFO Database Record (c) 2013 APA, all rights reserved.
-
Autistic Traits Moderate the Impact of Reward Learning on Social Behaviour.
Panasiti, Maria Serena; Puzzo, Ignazio; Chakrabarti, Bhismadev
2016-04-01
A deficit in empathy has been suggested to underlie social behavioural atypicalities in autism. A parallel theoretical account proposes that reduced social motivation (i.e., low responsivity to social rewards) can account for the said atypicalities. Recent evidence suggests that autistic traits modulate the link between reward and proxy metrics related to empathy. Using an evaluative conditioning paradigm to associate high and low rewards with faces, a previous study has shown that individuals high in autistic traits show reduced spontaneous facial mimicry of faces associated with high vs. low reward. This observation raises the possibility that autistic traits modulate the magnitude of evaluative conditioning. To test this, we investigated (a) if autistic traits could modulate the ability to implicitly associate a reward value to a social stimulus (reward learning/conditioning, using the Implicit Association Task, IAT); (b) if the learned association could modulate participants' prosocial behaviour (i.e., social reciprocity, measured using the cyberball task); (c) if the strength of this modulation was influenced by autistic traits. In 43 neurotypical participants, we found that autistic traits moderated the relationship of social reward learning on prosocial behaviour but not reward learning itself. This evidence suggests that while autistic traits do not directly influence social reward learning, they modulate the relationship of social rewards with prosocial behaviour. © 2015 The Authors Autism Research published by Wiley Periodicals, Inc. on behalf of International Society for Autism Research.
-
The neural dynamics of reward value and risk coding in the human orbitofrontal cortex.
Li, Yansong; Vanni-Mercier, Giovanna; Isnard, Jean; Mauguière, François; Dreher, Jean-Claude
2016-04-01
The orbitofrontal cortex is known to carry information regarding expected reward, risk and experienced outcome. Yet, due to inherent limitations in lesion and neuroimaging methods, the neural dynamics of these computations has remained elusive in humans. Here, taking advantage of the high temporal definition of intracranial recordings, we characterize the neurophysiological signatures of the intact orbitofrontal cortex in processing information relevant for risky decisions. Local field potentials were recorded from the intact orbitofrontal cortex of patients suffering from drug-refractory partial epilepsy with implanted depth electrodes as they performed a probabilistic reward learning task that required them to associate visual cues with distinct reward probabilities. We observed three successive signals: (i) around 400 ms after cue presentation, the amplitudes of the local field potentials increased with reward probability; (ii) a risk signal emerged during the late phase of reward anticipation and during the outcome phase; and (iii) an experienced value signal appeared at the time of reward delivery. Both the medial and lateral orbitofrontal cortex encoded risk and reward probability while the lateral orbitofrontal cortex played a dominant role in coding experienced value. The present study provides the first evidence from intracranial recordings that the human orbitofrontal cortex codes reward risk both during late reward anticipation and during the outcome phase at a time scale of milliseconds. Our findings offer insights into the rapid mechanisms underlying the ability to learn structural relationships from the environment. © The Author (2016). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com.
-
Decision Utility, Incentive Salience, and Cue-Triggered "Wanting"
Berridge, Kent C; Aldridge, J Wayne
2009-01-01
This chapter examines brain mechanisms of reward utility operating at particular decision moments in life-moments such as when one encounters an image, sound, scent, or other cue associated in the past with a particular reward or perhaps just when one vividly imagines that cue. Such a cue can often trigger a sudden motivational urge to pursue its reward and sometimes a decision to do so. Drawing on a utility taxonomy that distinguishes among subtypes of reward utility-predicted utility, decision utility, experienced utility, and remembered utility-it is shown how cue-triggered cravings, such as an addict's surrender to relapse, can hang on special transformations by brain mesolimbic systems of one utility subtype, namely, decision utility. The chapter focuses on a particular form of decision utility called incentive salience, a type of "wanting" for rewards that is amplified by brain mesolimbic systems. Sudden peaks of intensity of incentive salience, caused by neurobiological mechanisms, can elevate the decision utility of a particular reward at the moment its cue occurs. An understanding of what happens at such moments leads to a better understanding of the mechanisms at work in decision making in general.
-
Morie, K P; De Sanctis, P; Foxe, J J
2014-07-25
Task execution almost always occurs in the context of reward-seeking or punishment-avoiding behavior. As such, ongoing task-monitoring systems are influenced by reward anticipation systems. In turn, when a task has been executed either successfully or unsuccessfully, future iterations of that task will be re-titrated on the basis of the task outcome. Here, we examined the neural underpinnings of the task-monitoring and reward-evaluation systems to better understand how they govern reward-seeking behavior. Twenty-three healthy adult participants performed a task where they accrued points that equated to real world value (gift cards) by responding as rapidly as possible within an allotted timeframe, while success rate was titrated online by changing the duration of the timeframe dependent on participant performance. Informative cues initiated each trial, indicating the probability of potential reward or loss (four levels from very low to very high). We manipulated feedback by first informing participants of task success/failure, after which a second feedback signal indicated actual magnitude of reward/loss. High-density electroencephalography (EEG) recordings allowed for examination of event-related potentials (ERPs) to the informative cues and in turn, to both feedback signals. Distinct ERP components associated with reward cues, task-preparatory and task-monitoring processes, and reward feedback processes were identified. Unsurprisingly, participants displayed increased ERP amplitudes associated with task-preparatory processes following cues that predicted higher chances of reward. They also rapidly updated reward and loss prediction information dependent on task performance after the first feedback signal. Finally, upon reward receipt, initial reward probability was no longer taken into account. Rather, ERP measures suggested that only the magnitude of actual reward or loss was now processed. Reward and task-monitoring processes are clearly dissociable, but
-
Gnadt, William; Grossberg, Stephen
2008-06-01
How do reactive and planned behaviors interact in real time? How are sequences of such behaviors released at appropriate times during autonomous navigation to realize valued goals? Controllers for both animals and mobile robots, or animats, need reactive mechanisms for exploration, and learned plans to reach goal objects once an environment becomes familiar. The SOVEREIGN (Self-Organizing, Vision, Expectation, Recognition, Emotion, Intelligent, Goal-oriented Navigation) animat model embodies these capabilities, and is tested in a 3D virtual reality environment. SOVEREIGN includes several interacting subsystems which model complementary properties of cortical What and Where processing streams and which clarify similarities between mechanisms for navigation and arm movement control. As the animat explores an environment, visual inputs are processed by networks that are sensitive to visual form and motion in the What and Where streams, respectively. Position-invariant and size-invariant recognition categories are learned by real-time incremental learning in the What stream. Estimates of target position relative to the animat are computed in the Where stream, and can activate approach movements toward the target. Motion cues from animat locomotion can elicit head-orienting movements to bring a new target into view. Approach and orienting movements are alternately performed during animat navigation. Cumulative estimates of each movement are derived from interacting proprioceptive and visual cues. Movement sequences are stored within a motor working memory. Sequences of visual categories are stored in a sensory working memory. These working memories trigger learning of sensory and motor sequence categories, or plans, which together control planned movements. Predictively effective chunk combinations are selectively enhanced via reinforcement learning when the animat is rewarded. Selected planning chunks effect a gradual transition from variable reactive exploratory
-
Reward Learning, Neurocognition, Social Cognition, and Symptomatology in Psychosis.
Lewandowski, Kathryn E; Whitton, Alexis E; Pizzagalli, Diego A; Norris, Lesley A; Ongur, Dost; Hall, Mei-Hua
2016-01-01
Patients with psychosis spectrum disorders exhibit deficits in social and neurocognition, as well as hallmark abnormalities in motivation and reward processing. Aspects of reward processing may overlap behaviorally and neurobiologically with some elements of cognitive functioning, and abnormalities in these processes may share partially overlapping etiologies in patients. However, whether reward processing and cognition are associated across the psychoses and linked to state and trait clinical symptomatology is unclear. The present study examined associations between cognitive functioning, reward learning, and clinical symptomatology in a cross-diagnostic sample. Patients with schizophrenia (SZ; n = 37), bipolar I disorder with psychosis (BD; n = 42), and healthy controls (n = 29) were assessed for clinical symptoms (patients only), neurocognitive functioning using the MATRICS Battery (MCCB) and reward learning using the probabilistic reward task (PRT). Groups were compared on neurocognition and PRT response bias, and associations between PRT response bias and neurocognition or clinical symptoms were examined controlling for demographic variables and PRT task difficulty (discriminability). Patients with SZ performed worse than controls on most measures of neurocognition; patients with BD exhibited deficits in some domains between the level of patients with SZ and controls. The SZ - but not BD - group exhibited deficits in social cognition compared to controls. Patients and controls did not differ on PRT response bias, but did differ on PRT discriminability. Better response bias across the sample was associated with poorer social cognition, but not neurocognition; conversely, discriminability was associated with neurocognition but not social cognition. Symptoms of psychosis, particularly negative symptoms, were associated with poorer response bias across patient groups. Reward learning was associated with symptoms of psychosis - in particular negative
-
Rewards modulate saccade latency but not exogenous spatial attention.
Dunne, Stephen; Ellison, Amanda; Smith, Daniel T
2015-01-01
The eye movement system is sensitive to reward. However, whilst the eye movement system is extremely flexible, the extent to which changes to oculomotor behavior induced by reward paradigms persist beyond the training period or transfer to other oculomotor tasks is unclear. To address these issues we examined the effects of presenting feedback that represented small monetary rewards to spatial locations on the latency of saccadic eye movements, the time-course of learning and extinction of the effects of rewarding saccades on exogenous spatial attention and oculomotor inhibition of return. Reward feedback produced a relative facilitation of saccadic latency in a stimulus driven saccade task which persisted for three blocks of extinction trials. However, this hemifield-specific effect failed to transfer to peripheral cueing tasks. We conclude that rewarding specific spatial locations is unlikely to induce long-term, systemic changes to the human oculomotor or attention systems.
-
Rewards modulate saccade latency but not exogenous spatial attention.
Directory of Open Access Journals (Sweden)
Stephen eDunne
2015-07-01
Full Text Available The eye movement system is sensitive to reward. However, whilst the eye movement system is extremely flexible, the extent to which changes to oculomotor behaviour induced by reward paradigms persist beyond the training period or transfer to other oculomotor tasks is unclear. To address these issues we examined the effects of presenting feedback that represented small monetary rewards to spatial locations on the latency of saccadic eye movements, the time-course of learning and extinction of the effects of rewarding saccades on exogenous spatial attention and oculomotor IOR. Reward feedback produced a relative facilitation of saccadic latency in a stimulus driven saccade task which persisted for 3 blocks of extinction trials. However this hemifield-specific effect failed to transfer to peripheral cueing tasks. We conclude that rewarding specific spatial locations is unlikely to induce long-term, systemic changes to the human oculomotor or attention systems.
-
Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia.
Directory of Open Access Journals (Sweden)
Erin C Dowd
Full Text Available Reward processing abnormalities have been implicated in the pathophysiology of negative symptoms such as anhedonia and avolition in schizophrenia. However, studies examining neural responses to reward anticipation and receipt have largely relied on instrumental tasks, which may confound reward processing abnormalities with deficits in response selection and execution. 25 chronic, medicated outpatients with schizophrenia and 20 healthy controls underwent functional magnetic resonance imaging using a pavlovian reward prediction paradigm with no response requirements. Subjects passively viewed cues that predicted subsequent receipt of monetary reward or non-reward, and blood-oxygen-level-dependent signal was measured at the time of cue presentation and receipt. At the group level, neural responses to both reward anticipation and receipt were largely similar between groups. At the time of cue presentation, striatal anticipatory responses did not differ between patients and controls. Right anterior insula demonstrated greater activation for nonreward than reward cues in controls, and for reward than nonreward cues in patients. At the time of receipt, robust responses to receipt of reward vs. nonreward were seen in striatum, midbrain, and frontal cortex in both groups. Furthermore, both groups demonstrated responses to unexpected versus expected outcomes in cortical areas including bilateral dorsolateral prefrontal cortex. Individual difference analyses in patients revealed an association between physical anhedonia and activity in ventral striatum and ventromedial prefrontal cortex during anticipation of reward, in which greater anhedonia severity was associated with reduced activation to money versus no-money cues. In ventromedial prefrontal cortex, this relationship held among both controls and patients, suggesting a relationship between anticipatory activity and anhedonia irrespective of diagnosis. These findings suggest that in the absence of
-
Scaling prediction errors to reward variability benefits error-driven learning in humans.
Diederen, Kelly M J; Schultz, Wolfram
2015-09-01
Effective error-driven learning requires individuals to adapt learning to environmental reward variability. The adaptive mechanism may involve decays in learning rate across subsequent trials, as shown previously, and rescaling of reward prediction errors. The present study investigated the influence of prediction error scaling and, in particular, the consequences for learning performance. Participants explicitly predicted reward magnitudes that were drawn from different probability distributions with specific standard deviations. By fitting the data with reinforcement learning models, we found scaling of prediction errors, in addition to the learning rate decay shown previously. Importantly, the prediction error scaling was closely related to learning performance, defined as accuracy in predicting the mean of reward distributions, across individual participants. In addition, participants who scaled prediction errors relative to standard deviation also presented with more similar performance for different standard deviations, indicating that increases in standard deviation did not substantially decrease "adapters'" accuracy in predicting the means of reward distributions. However, exaggerated scaling beyond the standard deviation resulted in impaired performance. Thus efficient adaptation makes learning more robust to changing variability. Copyright © 2015 the American Physiological Society.
-
Robinson, Mike J.F.; Anselme, Patrick; Suchomel, Kristen; Berridge, Kent C.
2015-01-01
Amphetamine and stress can sensitize mesolimbic dopamine-related systems. In Pavlovian autoshaping, repeated exposure to uncertainty of reward prediction can enhance motivated sign-tracking or attraction to a discrete reward-predicting cue (lever CS+), as well as produce cross-sensitization to amphetamine. However, it remains unknown how amphetamine-sensitization or repeated restraint stress interact with uncertainty in controlling CS+ incentive salience attribution reflected in sign-tracking. Here rats were tested in three successive phases. First, different groups underwent either induction of amphetamine sensitization or repeated restraint stress, or else were not sensitized or stressed as control groups (either saline injections only, or no stress or injection at all). All next received Pavlovian autoshaping training under either certainty conditions (100% CS-UCS association) or uncertainty conditions (50% CS-UCS association and uncertain reward magnitude). During training, rats were assessed for sign-tracking to the lever CS+ versus goal-tracking to the sucrose dish. Finally, all groups were tested for psychomotor sensitization of locomotion revealed by an amphetamine challenge. Our results confirm that reward uncertainty enhanced sign-tracking attraction toward the predictive CS+ lever, at the expense of goal-tracking. We also report that amphetamine sensitization promoted sign-tracking even in rats trained under CS-UCS certainty conditions, raising them to sign-tracking levels equivalent to the uncertainty group. Combining amphetamine sensitization and uncertainty conditions together did not add together to elevate sign-tracking further above the relatively high levels induced by either manipulation alone. In contrast, repeated restraint stress enhanced subsequent amphetamine-elicited locomotion, but did not enhance CS+ attraction. PMID:26076340
-
Modulation of spatial attention by goals, statistical learning, and monetary reward.
Jiang, Yuhong V; Sha, Li Z; Remington, Roger W
2015-10-01
This study documented the relative strength of task goals, visual statistical learning, and monetary reward in guiding spatial attention. Using a difficult T-among-L search task, we cued spatial attention to one visual quadrant by (i) instructing people to prioritize it (goal-driven attention), (ii) placing the target frequently there (location probability learning), or (iii) associating that quadrant with greater monetary gain (reward-based attention). Results showed that successful goal-driven attention exerted the strongest influence on search RT. Incidental location probability learning yielded a smaller though still robust effect. Incidental reward learning produced negligible guidance for spatial attention. The 95 % confidence intervals of the three effects were largely nonoverlapping. To understand these results, we simulated the role of location repetition priming in probability cuing and reward learning. Repetition priming underestimated the strength of location probability cuing, suggesting that probability cuing involved long-term statistical learning of how to shift attention. Repetition priming provided a reasonable account for the negligible effect of reward on spatial attention. We propose a multiple-systems view of spatial attention that includes task goals, search habit, and priming as primary drivers of top-down attention.
-
Evidence for the negative impact of reward on self-regulated learning.
Wehe, Hillary S; Rhodes, Matthew G; Seger, Carol A
2015-01-01
The undermining effect refers to the detrimental impact rewards can have on intrinsic motivation to engage in a behaviour. The current study tested the hypothesis that participants' self-regulated learning behaviours are susceptible to the undermining effect. Participants were assigned to learn a set of Swahili-English word pairs. Half of the participants were offered a reward for performance, and half were not offered a reward. After the initial study phase, participants were permitted to continue studying the words during a free period. The results were consistent with an undermining effect: Participants who were not offered a reward spent more time studying the words during the free period. The results suggest that rewards may negatively impact self-regulated learning behaviours and provide support for the encouragement of intrinsic motivation.
-
Temporal dynamics of reward anticipation in the human brain.
Zhang, Yuanyuan; Li, Qi; Wang, Zhao; Liu, Xun; Zheng, Ya
2017-09-01
Reward anticipation is a complex process including cue evaluation, motor preparation, and feedback anticipation. The present study investigated whether these psychological processes were dissociable on neural dynamics in terms of incentive valence and approach motivation. We recorded EEG when participants were performing a monetary incentive delay task, and found a cue-P3 during the cue-evaluation stage, a contingent negative variation (CNV) during the motor-preparation stage, and a stimulus-preceding negativity (SPN) during the feedback-anticipation stage. Critically, both the cue-P3 and SPN exhibited an enhanced sensitivity to gain versus loss anticipation, which was not observed for the CNV. Moreover, both the cue-P3 and SPN, instead of the CNV, for gain anticipation selectively predicted the participants' approach motivation as measured in a following effort expenditure for rewards task, particularly when reward uncertainty was maximal. Together, these results indicate that reward anticipation consists of several sub-stages, each with distinct functional significance, thus providing implications for neuropsychiatric diseases characterized by dysfunction in anticipatory reward processing. Copyright © 2017 Elsevier B.V. All rights reserved.
-
Abnormal Striatal BOLD Responses to Reward Anticipation and Reward Delivery in ADHD
Furukawa, Emi; Bado, Patricia; Tripp, Gail; Mattos, Paulo; Wickens, Jeff R.; Bramati, Ivanei E.; Alsop, Brent; Ferreira, Fernanda Meireles; Lima, Debora; Tovar-Moll, Fernanda; Sergeant, Joseph A.; Moll, Jorge
2014-01-01
Altered reward processing has been proposed to contribute to the symptoms of attention deficit hyperactivity disorder (ADHD). The neurobiological mechanism underlying this alteration remains unclear. We hypothesize that the transfer of dopamine release from reward to reward-predicting cues, as normally observed in animal studies, may be deficient in ADHD. Functional magnetic resonance imaging (fMRI) was used to investigate striatal responses to reward-predicting cues and reward delivery in a classical conditioning paradigm. Data from 14 high-functioning and stimulant-naïve young adults with elevated lifetime symptoms of ADHD (8 males, 6 females) and 15 well-matched controls (8 males, 7 females) were included in the analyses. During reward anticipation, increased blood-oxygen-level-dependent (BOLD) responses in the right ventral and left dorsal striatum were observed in controls, but not in the ADHD group. The opposite pattern was observed in response to reward delivery; the ADHD group demonstrated significantly greater BOLD responses in the ventral striatum bilaterally and the left dorsal striatum relative to controls. In the ADHD group, the number of current hyperactivity/impulsivity symptoms was inversely related to ventral striatal responses during reward anticipation and positively associated with responses to reward. The BOLD response patterns observed in the striatum are consistent with impaired predictive dopamine signaling in ADHD, which may explain altered reward-contingent behaviors and symptoms of ADHD. PMID:24586543
-
Signed reward prediction errors drive declarative learning
De Loof, E.; Ergo, K.; Naert, L.; Janssens, C.; Talsma, D.; van Opstal, F.; Verguts, T.
2018-01-01
Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning–a quintessentially human form of learning–remains surprisingly absent. We
-
Toda, Koji; Sugase-Miyamoto, Yasuko; Mizuhiki, Takashi; Inaba, Kiyonori; Richmond, Barry J; Shidara, Munetaka
2012-01-01
The value of a predicted reward can be estimated based on the conjunction of both the intrinsic reward value and the length of time to obtain it. The question we addressed is how the two aspects, reward size and proximity to reward, influence the responses of neurons in rostral anterior cingulate cortex (rACC), a brain region thought to play an important role in reward processing. We recorded from single neurons while two monkeys performed a multi-trial reward schedule task. The monkeys performed 1-4 sequential color discrimination trials to obtain a reward of 1-3 liquid drops. There were two task conditions, a valid cue condition, where the number of trials and reward amount were associated with visual cues, and a random cue condition, where the cue was picked from the cue set at random. In the valid cue condition, the neuronal firing is strongly modulated by the predicted reward proximity during the trials. Information about the predicted reward amount is almost absent at those times. In substantial subpopulations, the neuronal responses decreased or increased gradually through schedule progress to the predicted outcome. These two gradually modulating signals could be used to calculate the effect of time on the perception of reward value. In the random cue condition, little information about the reward proximity or reward amount is encoded during the course of the trial before reward delivery, but when the reward is actually delivered the responses reflect both the reward proximity and reward amount. Our results suggest that the rACC neurons encode information about reward proximity and amount in a manner that is dependent on utility of reward information. The manner in which the information is represented could be used in the moment-to-moment calculation of the effect of time and amount on predicted outcome value.
-
Dopamine selectively remediates 'model-based' reward learning: a computational approach.
Sharp, Madeleine E; Foerde, Karin; Daw, Nathaniel D; Shohamy, Daphna
2016-02-01
Patients with loss of dopamine due to Parkinson's disease are impaired at learning from reward. However, it remains unknown precisely which aspect of learning is impaired. In particular, learning from reward, or reinforcement learning, can be driven by two distinct computational processes. One involves habitual stamping-in of stimulus-response associations, hypothesized to arise computationally from 'model-free' learning. The other, 'model-based' learning, involves learning a model of the world that is believed to support goal-directed behaviour. Much work has pointed to a role for dopamine in model-free learning. But recent work suggests model-based learning may also involve dopamine modulation, raising the possibility that model-based learning may contribute to the learning impairment in Parkinson's disease. To directly test this, we used a two-step reward-learning task which dissociates model-free versus model-based learning. We evaluated learning in patients with Parkinson's disease tested ON versus OFF their dopamine replacement medication and in healthy controls. Surprisingly, we found no effect of disease or medication on model-free learning. Instead, we found that patients tested OFF medication showed a marked impairment in model-based learning, and that this impairment was remediated by dopaminergic medication. Moreover, model-based learning was positively correlated with a separate measure of working memory performance, raising the possibility of common neural substrates. Our results suggest that some learning deficits in Parkinson's disease may be related to an inability to pursue reward based on complete representations of the environment. © The Author (2015). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com.
-
Learned reward association improves visual working memory.
Gong, Mengyuan; Li, Sheng
2014-04-01
Statistical regularities in the natural environment play a central role in adaptive behavior. Among other regularities, reward association is potentially the most prominent factor that influences our daily life. Recent studies have suggested that pre-established reward association yields strong influence on the spatial allocation of attention. Here we show that reward association can also improve visual working memory (VWM) performance when the reward-associated feature is task-irrelevant. We established the reward association during a visual search training session, and investigated the representation of reward-associated features in VWM by the application of a change detection task before and after the training. The results showed that the improvement in VWM was significantly greater for items in the color associated with high reward than for those in low reward-associated or nonrewarded colors. In particular, the results from control experiments demonstrate that the observed reward effect in VWM could not be sufficiently accounted for by attentional capture toward the high reward-associated item. This was further confirmed when the effect of attentional capture was minimized by presenting the items in the sample and test displays of the change detection task with the same color. The results showed significantly larger improvement in VWM performance when the items in a display were in the high reward-associated color than those in the low reward-associated or nonrewarded colors. Our findings suggest that, apart from inducing space-based attentional capture, the learned reward association could also facilitate the perceptual representation of high reward-associated items through feature-based attentional modulation.
-
A neurogenetic dissociation between punishment-, reward- and relief-learning in Drosophila
Directory of Open Access Journals (Sweden)
ayse Yarali
2010-12-01
Full Text Available What is particularly worth remembering about a traumatic experience is what brought it about, and what made it cease. For example, fruit flies avoid an odour which during training had preceded electric shock punishment; on the other hand, if the odour had followed shock during training, it is later on approached as a signal for the relieving end of shock. We provide a neurogenetic analysis of such relief learning. Blocking, using UAS-shibirets1, the output from a particular set of dopaminergic neurons defined by the TH-Gal4 driver partially impaired punishment learning, but left relief learning intact. Thus, with respect to these particular neurons, relief learning differs from punishment learning. Targeting another set of dopaminergic/ serotonergic neurons defined by the DDC-Gal4 driver on the other hand affected neither punishment nor relief learning. As for the octopaminergic system, the tbhM18 mutation, compromising octopamine biosynthesis, partially impaired sugar-reward learning, but not relief learning. Thus, with respect to this particular mutation, relief learning and reward learning are dissociated. Finally, blocking output from the set of octopaminergic/ tyraminergic neurons defined by the TDC2-Gal4 driver affected neither reward, nor relief learning. We conclude that regarding the used genetic tools, relief learning is neurogenetically dissociated from both punishment and reward learning.
-
Robinson, Mike J F; Anselme, Patrick; Suchomel, Kristen; Berridge, Kent C
2015-08-01
Amphetamine and stress can sensitize mesolimbic dopamine-related systems. In Pavlovian autoshaping, repeated exposure to uncertainty of reward prediction can enhance motivated sign-tracking or attraction to a discrete reward-predicting cue (lever-conditioned stimulus; CS+), as well as produce cross-sensitization to amphetamine. However, it remains unknown how amphetamine sensitization or repeated restraint stress interact with uncertainty in controlling CS+ incentive salience attribution reflected in sign-tracking. Here rats were tested in 3 successive phases. First, different groups underwent either induction of amphetamine sensitization or repeated restraint stress, or else were not sensitized or stressed as control groups (either saline injections only, or no stress or injection at all). All next received Pavlovian autoshaping training under either certainty conditions (100% CS-UCS association) or uncertainty conditions (50% CS-UCS association and uncertain reward magnitude). During training, rats were assessed for sign-tracking to the CS+ lever versus goal-tracking to the sucrose dish. Finally, all groups were tested for psychomotor sensitization of locomotion revealed by an amphetamine challenge. Our results confirm that reward uncertainty enhanced sign-tracking attraction toward the predictive CS+ lever, at the expense of goal-tracking. We also reported that amphetamine sensitization promoted sign-tracking even in rats trained under CS-UCS certainty conditions, raising them to sign-tracking levels equivalent to the uncertainty group. Combining amphetamine sensitization and uncertainty conditions did not add together to elevate sign-tracking further above the relatively high levels induced by either manipulation alone. In contrast, repeated restraint stress enhanced subsequent amphetamine-elicited locomotion, but did not enhance CS+ attraction. (c) 2015 APA, all rights reserved).
-
Effort-Reward Imbalance for Learning Is Associated with Fatigue in School Children
Fukuda, Sanae; Yamano, Emi; Joudoi, Takako; Mizuno, Kei; Tanaka, Masaaki; Kawatani, Junko; Takano, Miyuki; Tomoda, Akemi; Imai-Matsumura, Kyoko; Miike, Teruhisa; Watanabe, Yasuyoshi
2010-01-01
We examined relationships among fatigue, sleep quality, and effort-reward imbalance for learning in school children. We developed an effort-reward for learning scale in school students and examined its reliability and validity. Self-administered surveys, including the effort reward for leaning scale and fatigue scale, were completed by 1,023…
-
Fasting for 24 hours heightens reward from food and food-related cues.
Cameron, Jameason D; Goldfield, Gary S; Finlayson, Graham; Blundell, John E; Doucet, Eric
2014-01-01
We examined the impact of a 24 hour complete fast (vs. fed state) on two measures of food reward: 1) 'wanting', as measured by response to food images and by the relative-reinforcing value of food (RRV), and 2) 'liking', as measured by response to food images and the hedonic evaluation of foods consumed. Utilizing a randomized crossover design, 15 subjects (9 male; 6 female) aged 28.6±4.5 yrs with body mass index 25.3±1.4 kg/m(2) were randomized and counterbalanced to normal feeding (FED) and 24-hour fast (FASTED) conditions. Trait characteristics were measured with the Three Factor Eating Questionnaire. Two computer tasks measured food reward: 1) RRV progressive ratio task, 2) explicit 'liking' and 'wanting' (Leeds Food Preference Questionnaire, LFPQ). Also measured were ad libitum energy intake (EI; buffet) and food 'liking' (visual analogue scale) of personalized stimuli. There were no significant anthropometric changes between conditions. Appetite scores, hedonic ratings of 'liking', and ad libitum EI all significantly increased under the FASTED condition (pFASTED condition there were significant increases in the RRV of snack foods; similarly, explicit 'wanting' and 'liking' significantly increased for all food categories. 'Liking' of sweet foods remained high across-meals under FASTED, but savory foods decreased in hedonic saliency. Relative to a fed state, we observed an increase in hedonic ratings of food, the rewarding value of food, and food intake after a 24 hr fast. Alliesthesia to food and food cues is suggested by heightened hedonic ratings under the FASTED condition relative to FED.
-
Episodic Memory Encoding Interferes with Reward Learning and Decreases Striatal Prediction Errors
Braun, Erin Kendall; Daw, Nathaniel D.
2014-01-01
Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making. PMID:25378157
-
O'Hara, Caitlin B; Campbell, Iain C; Schmidt, Ulrike
2015-05-01
This focussed narrative review examines neurobiological and psychophysiological evidence supporting a role for altered reward processes in the development and maintenance of anorexia nervosa (AN). In AN, there does not appear to be a generalised inability to experience reward. Rather, data suggest that a reluctance to gain weight leads to an aversive appraisal of food- and taste-related stimuli. As a result, cues compatible with this aberrant mode of thinking become rewarding for the individual. Evidence also suggests that attribution of motivational salience to such cues promotes anorectic behaviours. These findings are consistent with models in which interactions between cognition and reward are important in eliciting the anorectic "habit". A model is proposed which is consistent with elements of other theoretical frameworks, but differs in that its emphasis is towards neural overlaps between AN and addiction. It is consistent with AN being a reward-based learned behaviour in which aberrant cognitions related to eating and shape alter functioning of central reward systems. It proposes that the primary neural problem responsible for the development, maintenance, and treatment resistance is centred in the striatal reward system. This helps shift the emphasis of aetiological models towards reward processing, particularly in the context of illness-compatible cues. Furthermore, it suggests that continuing to explore the utility and valued nature of AN in the patient's life would be a useful inclusion in treatment and prevention models. Copyright © 2015. Published by Elsevier Ltd.
-
Fouragnan, Elsa; Queirazza, Filippo; Retzler, Chris; Mullinger, Karen J; Philiastides, Marios G
2017-07-06
Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.
-
Takikawa, Yoriko; Kawagoe, Reiko; Hikosaka, Okihide
2004-10-01
Dopamine (DA) neurons respond to sensory stimuli that predict reward. To understand how DA neurons acquire such ability, we trained monkeys on a one-direction-rewarded version of memory-guided saccade task (1DR) only when we recorded from single DA neurons. In 1DR, position-reward mapping was changed across blocks of trials. In the early stage of training of 1DR, DA neurons responded to reward delivery; in the later stages, they responded predominantly to the visual cue that predicted reward or no reward (reward predictor) differentially. We found that such a shift of activity from reward to reward predictor also occurred within a block of trials after position-reward mapping was altered. A main effect of long-term training was to accelerate the within-block reward-to-predictor shift of DA neuronal responses. The within-block shift appeared first in the intermediate stage, but was slow, and DA neurons often responded to the cue that indicated reward in the preceding block. In the advanced stage, the reward-to-predictor shift occurred quickly such that the DA neurons' responses to visual cues faithfully matched the current position-reward mapping. Changes in the DA neuronal responses co-varied with the reward-predictive differentiation of saccade latency both in short-term (within-block) and long-term adaptation. DA neurons' response to the fixation point also underwent long-term changes until it occurred predominantly in the first trial within a block. This might trigger a switch between the learned sets. These results suggest that midbrain DA neurons play an essential role in adapting oculomotor behavior to frequent switches in position-reward mapping.
-
Boosting Vocabulary Learning by Verbal Cueing During Sleep.
Schreiner, Thomas; Rasch, Björn
2015-11-01
Reactivating memories during sleep by re-exposure to associated memory cues (e.g., odors or sounds) improves memory consolidation. Here, we tested for the first time whether verbal cueing during sleep can improve vocabulary learning. We cued prior learned Dutch words either during non-rapid eye movement sleep (NonREM) or during active or passive waking. Re-exposure to Dutch words during sleep improved later memory for the German translation of the cued words when compared with uncued words. Recall of uncued words was similar to an additional group receiving no verbal cues during sleep. Furthermore, verbal cueing failed to improve memory during active and passive waking. High-density electroencephalographic recordings revealed that successful verbal cueing during NonREM sleep is associated with a pronounced frontal negativity in event-related potentials, a higher frequency of frontal slow waves as well as a cueing-related increase in right frontal and left parietal oscillatory theta power. Our results indicate that verbal cues presented during NonREM sleep reactivate associated memories, and facilitate later recall of foreign vocabulary without impairing ongoing consolidation processes. Likewise, our oscillatory analysis suggests that both sleep-specific slow waves as well as theta oscillations (typically associated with successful memory encoding during wakefulness) might be involved in strengthening memories by cueing during sleep. © The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
-
The role of within-compound associations in learning about absent cues.
Witnauer, James E; Miller, Ralph R
2011-05-01
When two cues are reinforced together (in compound), most associative models assume that animals learn an associative network that includes direct cue-outcome associations and a within-compound association. All models of associative learning subscribe to the importance of cue-outcome associations, but most models assume that within-compound associations are irrelevant to each cue's subsequent behavioral control. In the present article, we present an extension of Van Hamme and Wasserman's (Learning and Motivation 25:127-151, 1994) model of retrospective revaluation based on learning about absent cues that are retrieved through within-compound associations. The model was compared with a model lacking retrieval through within-compound associations. Simulations showed that within-compound associations are necessary for the model to explain higher-order retrospective revaluation and the observed greater retrospective revaluation after partial reinforcement than after continuous reinforcement alone. These simulations suggest that the associability of an absent stimulus is determined by the extent to which the stimulus is activated through the within-compound association.
-
Model Appreciative Learning Untuk Perancangan Reward Pada Game Pendidikan
Haryanto, Hanny; Kardianawati, Acun; Rosyidah, Umi
2017-01-01
Reward adalah elemen dari game yang sangat penting untuk membentuk pengalaman positif dan motivasi bagi pemain. Reward dalam suatu game pendidikan memegang peranan penting dalam menjaga motivasi pembelajar dan memberikan evaluasi dari apa yang dikerjakan. Namun perancangan reward seringkali masih tidak terkonsep dengan baik, acak dan bersifat subjektif. Penelitian ini menggunakan model Appreciative Learning, yang terdiri dari tahapan Discovery, Dream, Design dan Destiny, untuk mel...
-
Episodic memory encoding interferes with reward learning and decreases striatal prediction errors.
Wimmer, G Elliott; Braun, Erin Kendall; Daw, Nathaniel D; Shohamy, Daphna
2014-11-05
Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making. Copyright © 2014 the authors 0270-6474/14/3414901-12$15.00/0.
-
Learning to Cooperate: The Evolution of Social Rewards in Repeated Interactions.
Dridi, Slimane; Akçay, Erol
2018-01-01
Understanding the behavioral and psychological mechanisms underlying social behaviors is one of the major goals of social evolutionary theory. In particular, a persistent question about animal cooperation is to what extent it is supported by other-regarding preferences-the motivation to increase the welfare of others. In many situations, animals adjust their behaviors through learning by responding to the rewards they experience as a consequence of their actions. Therefore, we may ask whether learning in social situations can be driven by evolved other-regarding rewards. Here we develop a mathematical model in order to ask whether the mere act of cooperating with a social partner will evolve to be inherently rewarding. Individuals interact repeatedly in pairs and adjust their behaviors through reinforcement learning. We assume that individuals associate with each game outcome an internal reward value. These perceived rewards are genetically evolving traits. We find that conditionally cooperative rewards that value mutual cooperation positively but the sucker's outcome negatively tend to be evolutionarily stable. Purely other-regarding rewards can evolve only under special parameter combinations. On the other hand, selfish rewards that always lead to pure defection are also evolutionarily successful. These findings are consistent with empirical observations showing that humans tend to display conditionally cooperative behavior and also exhibit a diversity of preferences. Our model also demonstrates the need to further integrate multiple levels of biological causation of behavior.
-
Monitoring and regulation of learning in medical education: the need for predictive cues.
de Bruin, Anique B H; Dunlosky, John; Cavalcanti, Rodrigo B
2017-06-01
Being able to accurately monitor learning activities is a key element in self-regulated learning in all settings, including medical schools. Yet students' ability to monitor their progress is often limited, leading to inefficient use of study time. Interventions that improve the accuracy of students' monitoring can optimise self-regulated learning, leading to higher achievement. This paper reviews findings from cognitive psychology and explores potential applications in medical education, as well as areas for future research. Effective monitoring depends on students' ability to generate information ('cues') that accurately reflects their knowledge and skills. The ability of these 'cues' to predict achievement is referred to as 'cue diagnosticity'. Interventions that improve the ability of students to elicit predictive cues typically fall into two categories: (i) self-generation of cues and (ii) generation of cues that is delayed after self-study. Providing feedback and support is useful when cues are predictive but may be too complex to be readily used. Limited evidence exists about interventions to improve the accuracy of self-monitoring among medical students or trainees. Developing interventions that foster use of predictive cues can enhance the accuracy of self-monitoring, thereby improving self-study and clinical reasoning. First, insight should be gained into the characteristics of predictive cues used by medical students and trainees. Next, predictive cue prompts should be designed and tested to improve monitoring and regulation of learning. Finally, the use of predictive cues should be explored in relation to teaching and learning clinical reasoning. Improving self-regulated learning is important to help medical students and trainees efficiently acquire knowledge and skills necessary for clinical practice. Interventions that help students generate and use predictive cues hold the promise of improved self-regulated learning and achievement. This framework is
-
Curiosity and reward: Valence predicts choice and information prediction errors enhance learning.
Marvin, Caroline B; Shohamy, Daphna
2016-03-01
Curiosity drives many of our daily pursuits and interactions; yet, we know surprisingly little about how it works. Here, we harness an idea implied in many conceptualizations of curiosity: that information has value in and of itself. Reframing curiosity as the motivation to obtain reward-where the reward is information-allows one to leverage major advances in theoretical and computational mechanisms of reward-motivated learning. We provide new evidence supporting 2 predictions that emerge from this framework. First, we find an asymmetric effect of positive versus negative information, with positive information enhancing both curiosity and long-term memory for information. Second, we find that it is not the absolute value of information that drives learning but, rather, the gap between the reward expected and reward received, an "information prediction error." These results support the idea that information functions as a reward, much like money or food, guiding choices and driving learning in systematic ways. (c) 2016 APA, all rights reserved).
-
Prelude to passion: limbic activation by "unseen" drug and sexual cues.
Directory of Open Access Journals (Sweden)
Anna Rose Childress
2008-01-01
Full Text Available The human brain responds to recognizable signals for sex and for rewarding drugs of abuse by activation of limbic reward circuitry. Does the brain respond in similar way to such reward signals even when they are "unseen", i.e., presented in a way that prevents their conscious recognition? Can the brain response to "unseen" reward cues predict the future affective response to recognizable versions of such cues, revealing a link between affective/motivational processes inside and outside awareness?We exploited the fast temporal resolution of event-related functional magnetic resonance imaging (fMRI to test the brain response to "unseen" (backward-masked cocaine, sexual, aversive and neutral cues of 33 milliseconds duration in male cocaine patients (n = 22. Two days after scanning, the affective valence for visible versions of each cue type was determined using an affective bias (priming task. We demonstrate, for the first time, limbic brain activation by "unseen" drug and sexual cues of only 33 msec duration. Importantly, increased activity in an large interconnected ventral pallidum/amygdala cluster to the "unseen" cocaine cues strongly predicted future positive affect to visible versions of the same cues in subsequent off-magnet testing, pointing both to the functional significance of the rapid brain response, and to shared brain substrates for appetitive motivation within and outside awareness.These findings represent the first evidence that brain reward circuitry responds to drug and sexual cues presented outside awareness. The results underscore the sensitivity of the brain to "unseen" reward signals and may represent the brain's primordial signature for desire. The limbic brain response to reward cues outside awareness may represent a potential vulnerability in disorders (e.g., the addictions for whom poorly-controlled appetitive motivation is a central feature.
-
Amygdala subsystems and control of feeding behavior by learned cues.
Petrovich, Gorica D; Gallagher, Michela
2003-04-01
A combination of behavioral studies and a neural systems analysis approach has proven fruitful in defining the role of the amygdala complex and associated circuits in fear conditioning. The evidence presented in this chapter suggests that this approach is also informative in the study of other adaptive functions that involve the amygdala. In this chapter we present a novel model to study learning in an appetitive context. Furthermore, we demonstrate that long-recognized connections between the amygdala and the hypothalamus play a crucial role in allowing learning to modulate feeding behavior. In the first part we describe a behavioral model for motivational learning. In this model a cue that acquires motivational properties through pairings with food delivery when an animal is hungry can override satiety and promote eating in sated rats. Next, we present evidence that a specific amygdala subsystem (basolateral area) is responsible for allowing such learned cues to control eating (override satiety and promote eating in sated rats). We also show that basolateral amygdala mediates these actions via connectivity with the lateral hypothalamus. Lastly, we present evidence that the amygdalohypothalamic system is specific for the control of eating by learned motivational cues, as it does not mediate another function that depends on intact basolateral amygdala, namely, the ability of a conditioned cue to support new learning based on its acquired value. Knowledge about neural systems through which food-associated cues specifically control feeding behavior provides a defined model for the study of learning. In addition, this model may be informative for understanding mechanisms of maladaptive aspects of learned control of eating that contribute to eating disorders and more moderate forms of overeating.
-
Studte, Sara; Bridger, Emma; Mecklinger, Axel
2017-04-01
The consolidation of new associations is thought to depend in part on physiological processes engaged during non-REM (NREM) sleep, such as slow oscillations and sleep spindles. Moreover, NREM sleep is thought to selectively benefit associations that are adaptive for the future. In line with this, the current study investigated whether different reward cues at encoding are associated with changes in sleep physiology and memory retention. Participants' associative memory was tested after learning a list of arbitrarily paired words both before and after taking a 90-min nap. During learning, word-pairs were preceded by a cue indicating either a high or a low reward for correct memory performance at test. The motivation manipulation successfully impacted retention such that memory declined to a greater extent from pre- to post sleep for low rewarded than for high rewarded word-pairs. In line with previous studies, positive correlations between spindle density during NREM sleep and general memory performance pre- and post-sleep were found. In addition to this, however, a selective positive relationship between memory performance for highly rewarded word-pairs at posttest and spindle density during NREM sleep was also observed. These results support the view that motivationally salient memories are preferentially consolidated and that sleep spindles may be an important underlying mechanism for selective consolidation. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Dose Dependent Dopaminergic Modulation of Reward-Based Learning in Parkinson's Disease
van Wouwe, N. C.; Ridderinkhof, K. R.; Band, G. P. H.; van den Wildenberg, W. P. M.; Wylie, S. A.
2012-01-01
Learning to select optimal behavior in new and uncertain situations is a crucial aspect of living and requires the ability to quickly associate stimuli with actions that lead to rewarding outcomes. Mathematical models of reinforcement-based learning to select rewarding actions distinguish between (1) the formation of stimulus-action-reward…
-
Expected reward value and reward uncertainty have temporally dissociable effects on memory formation
Adcock, R; Clement, Nathaniel; Chiew, Kimberly; Dickerson, Kathryn; Stanek, Jessica
2018-01-01
Anticipating rewards has been shown to enhance memory formation. While substantial evidence implicates dopamine in this behavioral effect, the precise mechanisms remain ambiguous. Because dopamine nuclei show two distinct physiological signatures of reward prediction, we hypothesized two dissociable effects on memory formation. These two signatures are a phasic dopamine response immediately following a reward cue that encodes its expected value, and a sustained, ramping dopamine response that...
-
HolmesParker, Chris; Taylor, Mathew E.; Tumer, Kagan; Agogino, Adrian
2014-01-01
Learning in multiagent systems can be slow because agents must learn both how to behave in a complex environment and how to account for the actions of other agents. The inability of an agent to distinguish between the true environmental dynamics and those caused by the stochastic exploratory actions of other agents creates noise in each agent's reward signal. This learning noise can have unforeseen and often undesirable effects on the resultant system performance. We define such noise as exploratory action noise, demonstrate the critical impact it can have on the learning process in multiagent settings, and introduce a reward structure to effectively remove such noise from each agent's reward signal. In particular, we introduce Coordinated Learning without Exploratory Action Noise (CLEAN) rewards and empirically demonstrate their benefits
-
Errorful and errorless learning: The impact of cue-target constraint in learning from errors.
Bridger, Emma K; Mecklinger, Axel
2014-08-01
The benefits of testing on learning are well described, and attention has recently turned to what happens when errors are elicited during learning: Is testing nonetheless beneficial, or can errors hinder learning? Whilst recent findings have indicated that tests boost learning even if errors are made on every trial, other reports, emphasizing the benefits of errorless learning, have indicated that errors lead to poorer later memory performance. The possibility that this discrepancy is a function of the materials that must be learned-in particular, the relationship between the cues and targets-was addressed here. Cued recall after either a study-only errorless condition or an errorful learning condition was contrasted across cue-target associations, for which the extent to which the target was constrained by the cue was either high or low. Experiment 1 showed that whereas errorful learning led to greater recall for low-constraint stimuli, it led to a significant decrease in recall for high-constraint stimuli. This interaction is thought to reflect the extent to which retrieval is constrained by the cue-target association, as well as by the presence of preexisting semantic associations. The advantage of errorful retrieval for low-constraint stimuli was replicated in Experiment 2, and the interaction with stimulus type was replicated in Experiment 3, even when guesses were randomly designated as being either correct or incorrect. This pattern provides support for inferences derived from reports in which participants made errors on all learning trials, whilst highlighting the impact of material characteristics on the benefits and disadvantages that accrue from errorful learning in episodic memory.
-
COMT Val158Met genotype is associated with reward learning: A replication study and meta-analysis
Corral-Frías, Nadia S.; Pizzagalli, Diego A.; Carré, Justin; Michalski, Lindsay J; Nikolova, Yuliya S.; Perlis, Roy H.; Fagerness, Jesen; Lee, Mary R.; Conley, Emily Drabant; Lancaster, Thomas M.; Haddad, Stephen; Wolf, Aaron; Smoller, Jordan W.; Hariri, Ahmad R.; Bogdan, Ryan
2016-01-01
Identifying mechanisms through which individual differences in reward learning emerge offers an opportunity to understand both a fundamental form of adaptive responding as well as etiological pathways through which aberrant reward learning may contribute to maladaptive behaviors and psychopathology. One candidate mechanism through which individual differences in reward learning may emerge is variability in dopaminergic reinforcement signaling. A common functional polymorphism within the catechol-O-methyl transferase gene (COMT; rs4680, Val158Met) has been linked to reward learning where homozygosity for the Met allele (associated with heightened prefrontal dopamine function and decreased dopamine synthesis in the midbrain) has been associated with relatively increased reward learning. Here, we used a probabilistic reward learning task to asses response bias, a behavioral form of reward learning, across 3 separate samples that were combined for analyses (age: 21.80 ± 3.95; n=392; 268 female; European-American, n=208). We replicate prior reports that COMT rs4680 Met allele homozygosity is associated with increased reward learning in European-American participants (β=0.20, t= 2.75, p< 0.01; ΔR2= 0.04). Moreover, a meta-analysis of 4 studies, including the current one, confirmed the association between COMT rs4680 genotype and reward learning (95% CI −0.11 to −0.03; z=3.2; p<0.01). These results suggest that variability in dopamine signaling associated with COMT rs4680 influences individual differences in reward which may potentially contribute to psychopathology characterized by reward dysfunction. PMID:27138112
-
Dual mechanisms governing reward-driven perceptual learning [version 1; referees: 2 approved
Directory of Open Access Journals (Sweden)
Dongho Kim
2015-09-01
Full Text Available In this review, we explore how reward signals shape perceptual learning in animals and humans. Perceptual learning is the well-established phenomenon by which extensive practice elicits selective improvement in one’s perceptual discrimination of basic visual features, such as oriented lines or moving stimuli. While perceptual learning has long been thought to rely on ‘top-down’ processes, such as attention and decision-making, a wave of recent findings suggests that these higher-level processes are, in fact, not necessary. Rather, these recent findings indicate that reward signals alone, in the absence of the contribution of higher-level cognitive processes, are sufficient to drive the benefits of perceptual learning. Here, we will review the literature tying reward signals to perceptual learning. Based on these findings, we propose dual underlying mechanisms that give rise to perceptual learning: one mechanism that operates ‘automatically’ and is tied directly to reward signals, and another mechanism that involves more ‘top-down’, goal-directed computations.
-
Examining the durability of incidentally learned trust from gaze cues.
Strachan, James W A; Tipper, Steven P
2017-10-01
In everyday interactions we find our attention follows the eye gaze of faces around us. As this cueing is so powerful and difficult to inhibit, gaze can therefore be used to facilitate or disrupt visual processing of the environment, and when we experience this we infer information about the trustworthiness of the cueing face. However, to date no studies have investigated how long these impressions last. To explore this we used a gaze-cueing paradigm where faces consistently demonstrated either valid or invalid cueing behaviours. Previous experiments show that valid faces are subsequently rated as more trustworthy than invalid faces. We replicate this effect (Experiment 1) and then include a brief interference task in Experiment 2 between gaze cueing and trustworthiness rating, which weakens but does not completely eliminate the effect. In Experiment 3, we explore whether greater familiarity with the faces improves the durability of trust learning and find that the effect is more resilient with familiar faces. Finally, in Experiment 4, we push this further and show that evidence of trust learning can be seen up to an hour after cueing has ended. Taken together, our results suggest that incidentally learned trust can be durable, especially for faces that deceive.
-
The role of reward in word learning and its implications for language acquisition.
Ripollés, Pablo; Marco-Pallarés, Josep; Hielscher, Ulrike; Mestres-Missé, Anna; Tempelmann, Claus; Heinze, Hans-Jochen; Rodríguez-Fornells, Antoni; Noesselt, Toemme
2014-11-03
The exact neural processes behind humans' drive to acquire a new language--first as infants and later as second-language learners--are yet to be established. Recent theoretical models have proposed that during human evolution, emerging language-learning mechanisms might have been glued to phylogenetically older subcortical reward systems, reinforcing human motivation to learn a new language. Supporting this hypothesis, our results showed that adult participants exhibited robust fMRI activation in the ventral striatum (VS)--a core region of reward processing--when successfully learning the meaning of new words. This activation was similar to the VS recruitment elicited using an independent reward task. Moreover, the VS showed enhanced functional and structural connectivity with neocortical language areas during successful word learning. Together, our results provide evidence for the neural substrate of reward and motivation during word learning. We suggest that this strong functional and anatomical coupling between neocortical language regions and the subcortical reward system provided a crucial advantage in humans that eventually enabled our lineage to successfully acquire linguistic skills. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Changes in expression of c-Fos protein following cocaine-cue extinction learning.
Nic Dhonnchadha, B Á; Lovascio, B F; Shrestha, N; Lin, A; Leite-Morris, K A; Man, H Y; Kaplan, G B; Kantak, K M
2012-09-01
Extinguishing abnormally strengthened learned responses to cues associated with drugs of abuse remains a key tactic for alleviating addiction. To assist in developing pharmacotherapies to augment exposure therapy for relapse prevention, investigation into neurobiological underpinnings of drug-cue extinction learning is needed. We used regional analyses of c-Fos and GluR2 protein expression to delineate neural activity and plasticity that may be associated with cocaine-cue extinction learning. Rats were trained to self-administer cocaine paired with a light cue, and later underwent a single 2h extinction session for which cocaine was withheld but response-contingent cues were presented (cocaine-cue extinction). Control groups consisted of rats yoked to animals self-administering cocaine and receiving saline non-contingently followed by an extinction session, or rats trained to self-administer cocaine followed by a no-extinction session for which levers were retracted, and cocaine and cues were withheld. Among 11 brain sites examined, extinction training increased c-Fos expression in basolateral amygdala and prelimbic prefrontal cortex of cocaine-cue extinguished rats relative to both control conditions. In dorsal subiculum and infralimbic prefrontal cortex, extinction training increased c-Fos expression in both cocaine-cue and saline-cue extinguished rats relative to the no-extinction control condition. GluR2 protein expression was not altered in any site examined after extinction or control training. Findings suggest that basolateral amygdala and prelimbic prefrontal cortex neurons are activated during acquisition of cocaine-cue extinction learning, a process that is independent of changes in GluR2 abundance. Other sites are implicated in processing the significance of cues that are present early in extinction training. Copyright © 2012 Elsevier B.V. All rights reserved.
-
Directory of Open Access Journals (Sweden)
Badri Munir Sukoco
2015-06-01
Full Text Available Managers could elicit customers’ repeat purchase behavior through a well-designed reward program. This study examines two extrinsic cues - business longevity and timing effects of reward – to determine the consumers’ perceived risk and intention to participate in this kind of program. Moreover, this study discusses how different levels of involvement might interact with these two cues. An experiment with a 2 (business longevity: long vs. short x 2 (timing of reward: delayed vs. immediate x 2 (involvement: high vs. low between-subject factorial design is conducted to validate the proposed research hypotheses. The results show that an immediate reward offered by an older, more established, firm for a highly-involved product, make loyalty programs less risky and consequently attract consumers to participate. Interestingly, immediate rewards that are offered by older firms for a product that customers are less involved in has the opposite effects. Managerial and academic implications are further presented in this study.
-
Macaque monkeys can learn token values from human models through vicarious reward.
Bevacqua, Sara; Cerasti, Erika; Falcone, Rossella; Cervelloni, Milena; Brunamonti, Emiliano; Ferraina, Stefano; Genovesio, Aldo
2013-01-01
Monkeys can learn the symbolic meaning of tokens, and exchange them to get a reward. Monkeys can also learn the symbolic value of a token by observing conspecifics but it is not clear if they can learn passively by observing other actors, e.g., humans. To answer this question, we tested two monkeys in a token exchange paradigm in three experiments. Monkeys learned token values through observation of human models exchanging them. We used, after a phase of object familiarization, different sets of tokens. One token of each set was rewarded with a bit of apple. Other tokens had zero value (neutral tokens). Each token was presented only in one set. During the observation phase, monkeys watched the human model exchange tokens and watched them consume rewards (vicarious rewards). In the test phase, the monkeys were asked to exchange one of the tokens for food reward. Sets of three tokens were used in the first experiment and sets of two tokens were used in the second and third experiments. The valuable token was presented with different probabilities in the observation phase during the first and second experiments in which the monkeys exchanged the valuable token more frequently than any of the neutral tokens. The third experiments examined the effect of unequal probabilities. Our results support the view that monkeys can learn from non-conspecific actors through vicarious reward, even a symbolic task like the token-exchange task.
-
First-Pass Processing of Value Cues in the Ventral Visual Pathway.
Sasikumar, Dennis; Emeric, Erik; Stuphorn, Veit; Connor, Charles E
2018-02-19
Real-world value often depends on subtle, continuously variable visual cues specific to particular object categories, like the tailoring of a suit, the condition of an automobile, or the construction of a house. Here, we used microelectrode recording in behaving monkeys to test two possible mechanisms for category-specific value-cue processing: (1) previous findings suggest that prefrontal cortex (PFC) identifies object categories, and based on category identity, PFC could use top-down attentional modulation to enhance visual processing of category-specific value cues, providing signals to PFC for calculating value, and (2) a faster mechanism would be first-pass visual processing of category-specific value cues, immediately providing the necessary visual information to PFC. This, however, would require learned mechanisms for processing the appropriate cues in a given object category. To test these hypotheses, we trained monkeys to discriminate value in four letter-like stimulus categories. Each category had a different, continuously variable shape cue that signified value (liquid reward amount) as well as other cues that were irrelevant. Monkeys chose between stimuli of different reward values. Consistent with the first-pass hypothesis, we found early signals for category-specific value cues in area TE (the final stage in monkey ventral visual pathway) beginning 81 ms after stimulus onset-essentially at the start of TE responses. Task-related activity emerged in lateral PFC approximately 40 ms later and consisted mainly of category-invariant value tuning. Our results show that, for familiar, behaviorally relevant object categories, high-level ventral pathway cortex can implement rapid, first-pass processing of category-specific value cues. Copyright © 2018 Elsevier Ltd. All rights reserved.
-
Learning Grammatical Categories from Distributional Cues: Flexible Frames for Language Acquisition
St. Clair, Michelle C.; Monaghan, Padraic; Christiansen, Morten H.
2010-01-01
Numerous distributional cues in the child's environment may potentially assist in language learning, but what cues are useful to the child and when are these cues utilised? We propose that the most useful source of distributional cue is a flexible frame surrounding the word, where the language learner integrates information from the preceding and…
-
A configural dominant account of contextual cueing: Configural cues are stronger than colour cues.
Kunar, Melina A; John, Rebecca; Sweetman, Hollie
2014-01-01
Previous work has shown that reaction times to find a target in displays that have been repeated are faster than those for displays that have never been seen before. This learning effect, termed "contextual cueing" (CC), has been shown using contexts such as the configuration of the distractors in the display and the background colour. However, it is not clear how these two contexts interact to facilitate search. We investigated this here by comparing the strengths of these two cues when they appeared together. In Experiment 1, participants searched for a target that was cued by both colour and distractor configural cues, compared with when the target was only predicted by configural information. The results showed that the addition of a colour cue did not increase contextual cueing. In Experiment 2, participants searched for a target that was cued by both colour and distractor configuration compared with when the target was only cued by colour. The results showed that adding a predictive configural cue led to a stronger CC benefit. Experiments 3 and 4 tested the disruptive effects of removing either a learned colour cue or a learned configural cue and whether there was cue competition when colour and configural cues were presented together. Removing the configural cue was more disruptive to CC than removing colour, and configural learning was shown to overshadow the learning of colour cues. The data support a configural dominant account of CC, where configural cues act as the stronger cue in comparison to colour when they are presented together.
-
Reward-related learning via multiple memory systems.
Delgado, Mauricio R; Dickerson, Kathryn C
2012-07-15
The application of a neuroeconomic approach to the study of reward-related processes has provided significant insights in our understanding of human learning and decision making. Much of this research has focused primarily on the contributions of the corticostriatal circuitry, involved in trial-and-error reward learning. As a result, less consideration has been allotted to the potential influence of different neural mechanisms such as the hippocampus or to more common ways in human society in which information is acquired and utilized to reach a decision, such as through explicit instruction rather than trial-and-error learning. This review examines the individual contributions of multiple learning and memory neural systems and their interactions during human decision making in both normal and neuropsychiatric populations. Specifically, the anatomical and functional connectivity across multiple memory systems are highlighted to suggest that probing the role of the hippocampus and its interactions with the corticostriatal circuitry via the application of model-based neuroeconomic approaches may provide novel insights into neuropsychiatric populations that suffer from damage to one of these structures and as a consequence have deficits in learning, memory, or decision making. Copyright © 2012 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.
-
de Haas, Elske N; Lee, Caroline; Hernandez, Carlos E; Naguib, Marc; Rodenburg, T Bas
2017-01-01
Personality can influence how animals perceive and learn cues. The behaviour and physiological responses animals show during stressful events is indicative of their personality. Acute induced stress prior to a cognitive test are known to affect the judgement of a stimulus, but personality of an individual could also affect learning of a specific cognitive paradigm. Here, we assessed if adult laying hens' behaviour and physiological responses, as indicators of their personality, were related to their cognitive performance. We assessed their behavioural responses to a tonic immobility test, an open field test, and a manual restraint test, and measured plasma corticosterone levels after manual restraint. After that, hens (n=20) were trained in a pre-set training schedule to associate a colour-cue with a reward. In a two-choice go-go test, hens needed to choose between a baited or non-baited food container displayed randomly on the left or right side of an arena. Success in learning was related to personality, with better performance of hens which showed a reactive personality type by a long latency to walk, struggle or vocalize during the tests. Only eight out of 20 hens reached the training criteria. The non-learners showed a strong side preference during all training days. Side preferences were strong in hens with high levels of plasma corticosterone and with a long duration of tonic immobility, indicating that fearful, stress-sensitive hens are more prone to develop side biases. Our results show that learning can be hindered by side biases, and fearful animals with a more proactive personality type are more sensitive to develop such biases. Copyright © 2016 Elsevier B.V. All rights reserved.
-
Cánovas, Rosa; García, Rubén Fernández; Cimadevilla, Jose Manuel
2011-01-01
The aim of this study was to examine the influence of the number of cues and cue location in human spatial learning. To assess their importance, subjects performed variants of a virtual task called "The Boxes Room". Participants were trained to locate, in a computer-generated environment with 16 boxes, the rewarded boxes through 8 trials. In experiment I, the number of distal cues available was zero, one, two or the standard arrangement (seven cues). In experiment II, place navigation was compared based on distal landmarks (extra-maze cues placed on the walls) and proximal landmarks (proximal cues placed between the boxes). The results of experiment I demonstrated that one cue in the room is enough to obtain a good performance in the task. Experiment II showed that groups using proximal cues were slower and less accurate than groups using distal cues. In addition, our data suggest that men are better navigators than women, as they found the rewarded boxes sooner and committed fewer errors in both studies. These results indicate that performance can change depending on the number and location of available cues. Copyright © 2010 Elsevier B.V. All rights reserved.
-
Noori, Hamid R; Cosa Linan, Alejandro; Spanagel, Rainer
2016-09-01
Cue reactivity to natural and social rewards is essential for motivational behavior. However, cue reactivity to drug rewards can also elicit craving in addicted subjects. The degree to which drug and natural rewards share neural substrates is not known. The objective of this study is to conduct a comprehensive meta-analysis of neuroimaging studies on drug, gambling and natural stimuli (food and sex) to identify the common and distinct neural substrates of cue reactivity to drug and natural rewards. Neural cue reactivity studies were selected for the meta-analysis by means of activation likelihood estimations, followed by sensitivity and clustering analyses of averaged neuronal response patterns. Data from 176 studies (5573 individuals) suggests largely overlapping neural response patterns towards all tested reward modalities. Common cue reactivity to natural and drug rewards was expressed by bilateral neural responses within anterior cingulate gyrus, insula, caudate head, inferior frontal gyrus, middle frontal gyrus and cerebellum. However, drug cues also generated distinct activation patterns in medial frontal gyrus, middle temporal gyrus, posterior cingulate gyrus, caudate body and putamen. Natural (sexual) reward cues induced unique activation of the pulvinar in thalamus. Neural substrates of cue reactivity to alcohol, drugs of abuse, food, sex and gambling are largely overlapping and comprise a network that processes reward, emotional responses and habit formation. This suggests that cue-mediated craving involves mechanisms that are not exclusive for addictive disorders but rather resemble the intersection of information pathways for processing reward, emotional responses, non-declarative memory and obsessive-compulsive behavior. Copyright © 2016 Elsevier B.V. and ECNP. All rights reserved.
-
Data from ‘Placebo Enhances Reward Learning in Healthy Individuals’
Directory of Open Access Journals (Sweden)
Zsolt Turi
2018-04-01
Full Text Available This dataset contains three repeated measures of a standard reward-based reinforcement-learning task from 29 healthy male individuals who participated in three experimental sessions exploring cognitive placebo effects on reward learning. The dataset includes behavioural data (accuracy, reaction times during learning and transfer, estimates of model-free computational analysis, self-reported arousal values, and expectations about the interventions’ efficacy. The data were collected in 2014 at the Department of Clinical Neurophysiology, University Medical Center Goettingen, Germany. The data collection and formal analysis used a triple-blind study design as participants, operator and analyst were unaware of conditions. A github repository contains data and analyses for the paper “Placebo Intervention Enhances Reward Learning in Healthy Individuals”. The dataset can be used for further analysis, reference, validation studies, teaching purposes, and collaborative research. Funding statement: This study was supported by the DFG (PA 419/15-1 awarded to WP. The preparation of this manuscript was supported by the “Research program, University Medical Center, University of Goettingen” awarded to Z.T.
-
Goldman, Marina; Szucs-Reed, Regina P; Jagannathan, Kanchana; Ehrman, Ronald N; Wang, Ze; Li, Yin; Suh, Jesse J; Kampman, Kyle; O'Brien, Charles P; Childress, Anna Rose; Franklin, Teresa R
2013-01-01
: Determining the brain substrates underlying the motivation to abuse addictive drugs is critical for understanding and treating addictive disorders. Laboratory neuroimaging studies have demonstrated differential activation of limbic and motivational circuitry (eg, amygdala, hippocampus, ventral striatum, insula, and orbitofrontal cortex) triggered by cocaine, heroin, nicotine, and alcohol cues. The literature on neural responses to marijuana cues is sparse. Thus, the goals of this study were to characterize the brain's response to marijuana cues, a major motivator underlying drug use and relapse, and determine whether these responses are linked to self-reported craving in a clinically relevant population of treatment-seeking marijuana-dependent subjects. : Marijuana craving was assessed in 12 marijuana-dependent subjects using the Marijuana Craving Questionnaire-Short Form. Subsequently, blood oxygen level dependent functional magnetic resonance imaging data were acquired during exposure to alternating 20-second blocks of marijuana-related versus matched nondrug visual cues. : Brain activation during marijuana cue exposure was significantly greater in the bilateral amygdala and the hippocampus. Significant positive correlations between craving scores and brain activation were found in the ventral striatum and the medial and lateral orbitofrontal cortex (P cues and craving and extends the current literature on marijuana cue reactivity. Furthermore, the correlative relationship between craving and brain activity in reward-related regions was observed in a clinically relevant sample (treatment-seeking marijuana-dependent subjects). Results are consistent with prior findings in cocaine, heroin, nicotine, and alcohol cue studies, indicating that the brain substrates of cue-triggered drug motivation are shared across abused substances.
-
Averbeck, Bruno B.
2017-01-01
Orbitofrontal cortex (OFC), medial frontal cortex (MFC), and amygdala mediate stimulus–reward learning, but the mechanisms through which they interact are unclear. Here, we investigated how neurons in macaque OFC and MFC signaled rewards and the stimuli that predicted them during learning with and without amygdala input. Macaques performed a task that required them to evaluate two stimuli and then choose one to receive the reward associated with that option. Four main findings emerged. First, amygdala lesions slowed the acquisition and use of stimulus–reward associations. Further analyses indicated that this impairment was due, at least in part, to ineffective use of negative feedback to guide subsequent decisions. Second, the activity of neurons in OFC and MFC rapidly evolved to encode the amount of reward associated with each stimulus. Third, amygdalectomy reduced encoding of stimulus–reward associations during the evaluation of different stimuli. Reward encoding of anticipated and received reward after choices were made was not altered. Fourth, amygdala lesions led to an increase in the proportion of neurons in MFC, but not OFC, that encoded the instrumental response that monkeys made on each trial. These correlated changes in behavior and neural activity after amygdala lesions strongly suggest that the amygdala contributes to the ability to learn stimulus–reward associations rapidly by shaping encoding within OFC and MFC. SIGNIFICANCE STATEMENT Altered functional interactions among orbital frontal cortex (OFC), medial frontal cortex (MFC), and amygdala are thought to underlie several psychiatric conditions, many related to reward learning. Here, we investigated the causal contribution of the amygdala to the development of neuronal activity in macaque OFC and MFC related to rewards and the stimuli that predict them during learning. Without amygdala inputs, neurons in both OFC and MFC showed decreased encoding of stimulus–reward associations. MFC also
-
Stimulus homogeneity enhances implicit learning: evidence from contextual cueing.
Feldmann-Wüstefeld, Tobias; Schubö, Anna
2014-04-01
Visual search for a target object is faster if the target is embedded in a repeatedly presented invariant configuration of distractors ('contextual cueing'). It has also been shown that the homogeneity of a context affects the efficiency of visual search: targets receive prioritized processing when presented in a homogeneous context compared to a heterogeneous context, presumably due to grouping processes at early stages of visual processing. The present study investigated in three Experiments whether context homogeneity also affects contextual cueing. In Experiment 1, context homogeneity varied on three levels of the task-relevant dimension (orientation) and contextual cueing was most pronounced for context configurations with high orientation homogeneity. When context homogeneity varied on three levels of the task-irrelevant dimension (color) and orientation homogeneity was fixed, no modulation of contextual cueing was observed: high orientation homogeneity led to large contextual cueing effects (Experiment 2) and low orientation homogeneity led to low contextual cueing effects (Experiment 3), irrespective of color homogeneity. Enhanced contextual cueing for homogeneous context configurations suggest that grouping processes do not only affect visual search but also implicit learning. We conclude that memory representation of context configurations are more easily acquired when context configurations can be processed as larger, grouped perceptual units. However, this form of implicit perceptual learning is only improved by stimulus homogeneity when stimulus homogeneity facilitates grouping processes on a dimension that is currently relevant in the task. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Directory of Open Access Journals (Sweden)
K. Richard eRidderinkhof
2012-06-01
Full Text Available Reward-based decision-learning refers to the process of learning to select those actions that lead to rewards while avoiding actions that lead to punishments. This process, known to rely on dopaminergic activity in striatal brain regions, is compromised in Parkinson’s disease (PD. We hypothesized that such decision-learning deficits are alleviated by induced positive affect, which is thought to incur transient boosts in midbrain and striatal dopaminergic activity. Computational measures of probabilistic reward-based decision-learning were determined for 51 patients diagnosed with PD. Previous work has shown these measures to rely on the nucleus caudatus (outcome evaluation during the early phases of learning and the putamen (reward prediction during later phases of learning. We observed that induced positive affect facilitated learning, through its effects on reward prediction rather than outcome evaluation. Viewing a few minutes of comedy clips served to remedy dopamine-related problems in putamen-based frontostriatal circuitry and, consequently, in learning to predict which actions will yield reward.
-
Audiovisual Cues and Perceptual Learning of Spectrally Distorted Speech
Pilling, Michael; Thomas, Sharon
2011-01-01
Two experiments investigate the effectiveness of audiovisual (AV) speech cues (cues derived from both seeing and hearing a talker speak) in facilitating perceptual learning of spectrally distorted speech. Speech was distorted through an eight channel noise-vocoder which shifted the spectral envelope of the speech signal to simulate the properties…
-
Finding intrinsic rewards by embodied evolution and constrained reinforcement learning.
Uchibe, Eiji; Doya, Kenji
2008-12-01
Understanding the design principle of reward functions is a substantial challenge both in artificial intelligence and neuroscience. Successful acquisition of a task usually requires not only rewards for goals, but also for intermediate states to promote effective exploration. This paper proposes a method for designing 'intrinsic' rewards of autonomous agents by combining constrained policy gradient reinforcement learning and embodied evolution. To validate the method, we use Cyber Rodent robots, in which collision avoidance, recharging from battery packs, and 'mating' by software reproduction are three major 'extrinsic' rewards. We show in hardware experiments that the robots can find appropriate 'intrinsic' rewards for the vision of battery packs and other robots to promote approach behaviors.
-
Mixed signals: The effect of conflicting reward- and goal-driven biases on selective attention.
Preciado, Daniel; Munneke, Jaap; Theeuwes, Jan
2017-07-01
Attentional selection depends on the interaction between exogenous (stimulus-driven), endogenous (goal-driven), and selection history (experience-driven) factors. While endogenous and exogenous biases have been widely investigated, less is known about their interplay with value-driven attention. The present study investigated the interaction between reward-history and goal-driven biases on perceptual sensitivity (d') and response time (RT) in a modified cueing paradigm presenting two coloured cues, followed by sinusoidal gratings. Participants responded to the orientation of one of these gratings. In Experiment 1, one cue signalled reward availability but was otherwise task irrelevant. In Experiment 2, the same cue signalled reward, and indicated the target's most likely location at the opposite side of the display. This design introduced a conflict between reward-driven biases attracting attention and goal-driven biases directing it away. Attentional effects were examined comparing trials in which cue and target appeared at the same versus opposite locations. Two interstimulus interval (ISI) levels were used to probe the time course of attentional effects. Experiment 1 showed performance benefits at the location of the reward-signalling cue and costs at the opposite for both ISIs, indicating value-driven capture. Experiment 2 showed performance benefits only for the long ISI when the target was at the opposite to the reward-associated cue. At the short ISI, only performance costs were observed. These results reveal the time course of these biases, indicating that reward-driven effects influence attention early but can be overcome later by goal-driven control. This suggests that reward-driven biases are integrated as attentional priorities, just as exogenous and endogenous factors.
-
No two cues are alike: Depth of learning during infancy is dependent on what orients attention.
Wu, Rachel; Kirkham, Natasha Z
2010-10-01
Human infants develop a variety of attentional mechanisms that allow them to extract relevant information from a cluttered multimodal world. We know that both social and nonsocial cues shift infants' attention, but not how these cues differentially affect learning of multimodal events. Experiment 1 used social cues to direct 8- and 4-month-olds' attention to two audiovisual events (i.e., animations of a cat or dog accompanied by particular sounds) while identical distractor events played in another location. Experiment 2 directed 8-month-olds' attention with colorful flashes to the same events. Experiment 3 measured baseline learning without attention cues both with the familiarization and test trials (no cue condition) and with only the test trials (test control condition). The 8-month-olds exposed to social cues showed specific learning of audiovisual events. The 4-month-olds displayed only general spatial learning from social cues, suggesting that specific learning of audiovisual events from social cues may be a function of experience. Infants cued with the colorful flashes looked indiscriminately to both cued locations during test (similar to the 4-month-olds learning from social cues) despite attending for equal duration to the training trials as the 8-month-olds with the social cues. Results from Experiment 3 indicated that the learning effects in Experiments 1 and 2 resulted from exposure to the different cues and multimodal events. We discuss these findings in terms of the perceptual differences and relevance of the cues. Copyright 2010 Elsevier Inc. All rights reserved.
-
Memory for location and visual cues in white-eared hummingbirds Hylocharis leucotis
Directory of Open Access Journals (Sweden)
Guillermo PÉREZ, Carlos LARA, José VICCON-PALE, Martha SIGNORET-POILLON
2011-08-01
Full Text Available In nature hummingbirds face floral resources whose availability, quality and quantity can vary spatially and temporally. Thus, they must constantly make foraging decisions about which patches, plants and flowers to visit, partly as a function of the nectar reward. The uncertainty of these decisions would possibly be reduced if an individual could remember locations or use visual cues to avoid revisiting recently depleted flowers. In the present study, we carried out field experiments with white-eared hummingbirds Hylocharis leucotis, to evaluate their use of locations or visual cues when foraging on natural flowers Penstemon roseus. We evaluated the use of spatial memory by observing birds while they were foraging between two plants and within a single plant. Our results showed that hummingbirds prefer to use location when foraging in two plants, but they also use visual cues to efficiently locate unvisited rewarded flowers when they feed on a single plant. However, in absence of visual cues, in both experiments birds mainly used the location of previously visited flowers to make subsequent visits. Our data suggest that hummingbirds are capable of learning and employing this flexibility depending on the faced environmental conditions and the information acquired in previous visits [Current Zoology 57 (4: 468–476, 2011].
-
Multi-layer network utilizing rewarded spike time dependent plasticity to learn a foraging task.
Directory of Open Access Journals (Sweden)
Pavel Sanda
2017-09-01
Full Text Available Neural networks with a single plastic layer employing reward modulated spike time dependent plasticity (STDP are capable of learning simple foraging tasks. Here we demonstrate advanced pattern discrimination and continuous learning in a network of spiking neurons with multiple plastic layers. The network utilized both reward modulated and non-reward modulated STDP and implemented multiple mechanisms for homeostatic regulation of synaptic efficacy, including heterosynaptic plasticity, gain control, output balancing, activity normalization of rewarded STDP and hard limits on synaptic strength. We found that addition of a hidden layer of neurons employing non-rewarded STDP created neurons that responded to the specific combinations of inputs and thus performed basic classification of the input patterns. When combined with a following layer of neurons implementing rewarded STDP, the network was able to learn, despite the absence of labeled training data, discrimination between rewarding patterns and the patterns designated as punishing. Synaptic noise allowed for trial-and-error learning that helped to identify the goal-oriented strategies which were effective in task solving. The study predicts a critical set of properties of the spiking neuronal network with STDP that was sufficient to solve a complex foraging task involving pattern classification and decision making.
-
Effects of motivation on reward and attentional networks: an fMRI study.
Ivanov, Iliyan; Liu, Xun; Clerkin, Suzanne; Schulz, Kurt; Friston, Karl; Newcorn, Jeffrey H; Fan, Jin
2012-11-01
Existing evidence suggests that reward and attentional networks function in concert and that activation in one system influences the other in a reciprocal fashion; however, the nature of these influences remains poorly understood. We therefore developed a three-component task to assess the interaction effects of reward anticipation and conflict resolution on the behavioral performance and the activation of brain reward and attentional systems. Sixteen healthy adult volunteers aged 21-45 years were scanned with functional magnetic resonance imaging (fMRI) while performing the task. A two-way repeated measures analysis of variance (ANOVA) with cue (reward vs. non-reward) and target (congruent vs. incongruent) as within-subjects factors was used to test for main and interaction effects. Neural responses to anticipation, conflict, and reward outcomes were tested. Behaviorally there were main effects of both reward cue and target congruency on reaction time. Neuroimaging results showed that reward anticipation and expected reward outcomes activated components of the attentional networks, including the inferior parietal and occipital cortices, whereas surprising non-rewards activated the frontoinsular cortex bilaterally and deactivated the ventral striatum. In turn, conflict activated a broad network associated with cognitive control and motor functions. Interaction effects showed decreased activity in the thalamus, anterior cingulated gyrus, and middle frontal gyrus bilaterally when difficult conflict trials (e.g., incongruent targets) were preceded by reward cues; in contrast, the ventral striatum and orbitofrontal cortex showed greater activation during congruent targets preceded by reward cues. These results suggest that reward anticipation is associated with lower activation in attentional networks, possibly due to increased processing efficiency, whereas more difficult, conflict trials are associated with lower activity in regions of the reward system, possibly
-
Attentional Bias for Reward and Punishment in Overweight and Obesity: The TRAILS Study.
Jonker, Nienke C; Glashouwer, Klaske A; Ostafin, Brian D; van Hemel-Ruiter, Madelon E; Smink, Frédérique R E; Hoek, Hans W; de Jong, Peter J
2016-01-01
More than 80% of obese adolescents will become obese adults, and it is therefore important to enhance insight into characteristics that underlie the development and maintenance of overweight and obesity at a young age. The current study is the first to focus on attentional biases towards rewarding and punishing cues as potentially important factors. Participants were young adolescents (N = 607) who were followed from the age of 13 until the age of 19, and completed a motivational game indexing the attentional bias to general cues of reward and punishment. Additionally, self-reported reward and punishment sensitivity was measured. This study showed that attentional biases to cues that signal reward or punishment and self-reported reward and punishment sensitivity were not related to body mass index or the change in body mass index over six years in adolescents. Thus, attentional bias to cues of reward and cues of punishment, and self-reported reward and punishment sensitivity, do not seem to be crucial factors in the development and maintenance of overweight and obesity in adolescents. Exploratory analyses of the current study suggest that the amount of effort to gain reward and to avoid punishment may play a role in the development and maintenance of overweight and obesity. However, since the effort measure was a construct based on face validity and has not been properly validated, more studies are necessary before firm conclusions can be drawn.
-
Wu, Kit; Politis, Marios; O'Sullivan, Sean S; Lawrence, Andrew D; Warsi, Sarah; Bose, Subrata; Lees, Andrew J; Piccini, Paola
2015-06-01
Impulse control disorders (ICDs) are reported in Parkinson's disease (PD) in association with dopaminergic treatment. Approximately 25 % of patients with ICDs have multiple co-occurring ICDs (i.e. more than one diagnosed ICD). The extent to which dopaminergic neurotransmission in PD patients with multiple ICDs differs from those with only one diagnosed ICD is unknown. The aims of this study are: (1) to investigate dopamine neurotransmission in PD patients diagnosed with multiple ICDs, single ICDs and non-ICD controls in response to reward-related visual cues using positron emission tomography with (11)C-raclopride. (2) to compare clinical features of the above three groups. PD individuals with mulitple ICDs (n = 10), single ICD (n = 7) and no ICDs (n = 9) were recruited and underwent two positron emission tomography (PET) scans with (11)C-raclopride: one where they viewed neutral visual cues and the other where they viewed a range of visual cues related to different rewards. Individuals with both multiple ICDs and single ICDs showed significantly greater ventral striatal dopamine release compared to non-ICD PD individuals in response to reward cues, but the two ICD groups did not differ from each other in the extent of dopamine release. Subjects with multiple ICDs were, however, significantly more depressed, and had higher levels of impulsive sensation-seeking compared to subjects with single ICDs and without ICDs. This is the first study to compare dopamine neurotransmission using PET neuroimaging in PD subjects with multiple vs. single ICDs. Our results suggest that striatal dopamine neurotransmission is not directly related to the co-occurrence of ICDs in PD, potentially implicating non-dopaminergic mechanisms linked to depression; and suggest that physicians should be vigilant in managing depression in PD patients with ICDs.
-
Signed reward prediction errors drive declarative learning.
Directory of Open Access Journals (Sweden)
Esther De Loof
Full Text Available Reward prediction errors (RPEs are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning. However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.
-
Signed reward prediction errors drive declarative learning.
De Loof, Esther; Ergo, Kate; Naert, Lien; Janssens, Clio; Talsma, Durk; Van Opstal, Filip; Verguts, Tom
2018-01-01
Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals) during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.
-
Directory of Open Access Journals (Sweden)
Yannick eBoddez
2014-11-01
Full Text Available Blocking is the most important phenomenon in the history of associative learning theory: For over 40 years, blocking has inspired a whole generation of learning models. Blocking is part of a family of effects that are typically termed cue competition effects. Common amongst all cue competition effects is that a cue-outcome relation is poorly learned or poorly expressed because the cue is trained in the presence of an alternative predictor or cause of the outcome. We provide an overview of the cognitive processes involved in cue competition effects in humans and propose a stage framework that brings these processes together. The framework contends that the behavioral display of cue competition is cognitively construed following three stages that include (1 an encoding stage, (2 a retention stage, and (3 a performance stage. We argue that the stage framework supports a comprehensive understanding of cue competition effects.
-
Aversive counterconditioning attenuates reward signalling in the ventral striatum
Directory of Open Access Journals (Sweden)
Anne Marije Kaag
2016-08-01
Full Text Available Appetitive conditioning refers to the process of learning cue-reward associations and is mediated by the mesocorticolimbic system. Appetitive conditioned responses are difficult to extinguish, especially for highly salient rewards such as food and drugs. We investigate whether aversive counterconditioning can alter reward reinstatement in the ventral striatum in healthy volunteers using functional Magnetic Resonance Imaging (fMRI. In the initial conditioning phase, two different stimuli were reinforced with a monetary reward. In the subsequent counterconditioning phase, one of these stimuli was paired with an aversive shock to the wrist. In the following extinction phase, none of the stimuli were reinforced. In the final reinstatement phase, reward was reinstated by informing the participants that the monetary gain could be doubled. Our fMRI data revealed that reward signalling in the ventral striatum and ventral tegmental area following reinstatement was smaller for the stimulus that was counterconditioned with an electrical shock, compared to the non-counterconditioned stimulus. A functional connectivity analysis showed that aversive counterconditioning strengthened striatal connectivity with the hippocampus and insula. These results suggest that reward signalling in the ventral striatum can be attenuated through aversive counterconditioning, possibly by concurrent retrieval of the aversive association through enhanced connectivity with hippocampus and insula.
-
Value and probability coding in a feedback-based learning task utilizing food rewards.
Tricomi, Elizabeth; Lempert, Karolina M
2015-01-01
For the consequences of our actions to guide behavior, the brain must represent different types of outcome-related information. For example, an outcome can be construed as negative because an expected reward was not delivered or because an outcome of low value was delivered. Thus behavioral consequences can differ in terms of the information they provide about outcome probability and value. We investigated the role of the striatum in processing probability-based and value-based negative feedback by training participants to associate cues with food rewards and then employing a selective satiety procedure to devalue one food outcome. Using functional magnetic resonance imaging, we examined brain activity related to receipt of expected rewards, receipt of devalued outcomes, omission of expected rewards, omission of devalued outcomes, and expected omissions of an outcome. Nucleus accumbens activation was greater for rewarding outcomes than devalued outcomes, but activity in this region did not correlate with the probability of reward receipt. Activation of the right caudate and putamen, however, was largest in response to rewarding outcomes relative to expected omissions of reward. The dorsal striatum (caudate and putamen) at the time of feedback also showed a parametric increase correlating with the trialwise probability of reward receipt. Our results suggest that the ventral striatum is sensitive to the motivational relevance, or subjective value, of the outcome, while the dorsal striatum codes for a more complex signal that incorporates reward probability. Value and probability information may be integrated in the dorsal striatum, to facilitate action planning and allocation of effort. Copyright © 2015 the American Physiological Society.
-
Reward sensitivity predicts ice cream-related attentional bias assessed by inattentional blindness.
Li, Xiaoming; Tao, Qian; Fang, Ya; Cheng, Chen; Hao, Yangyang; Qi, Jianjun; Li, Yu; Zhang, Wei; Wang, Ying; Zhang, Xiaochu
2015-06-01
The cognitive mechanism underlying the association between individual differences in reward sensitivity and food craving is unknown. The present study explored the mechanism by examining the role of reward sensitivity in attentional bias toward ice cream cues. Forty-nine college students who displayed high level of ice cream craving (HICs) and 46 who displayed low level of ice cream craving (LICs) performed an inattentional blindness (IB) task which was used to assess attentional bias for ice cream. In addition, reward sensitivity and coping style were assessed by the Behavior Inhibition System/Behavior Activation System Scales and Simplified Coping Style Questionnaire. Results showed significant higher identification rate of the critical stimulus in the HICs than LICs, suggesting greater attentional bias for ice cream in the HICs. It was indicated that attentional bias for food cues persisted even under inattentional condition. Furthermore, a significant correlation was found between the attentional bias and reward sensitivity after controlling for coping style, and reward sensitivity predicted attentional bias for food cues. The mediation analyses showed that attentional bias mediated the relationship between reward sensitivity and food craving. Those findings suggest that the association between individual differences in reward sensitivity and food craving may be attributed to attentional bias for food-related cues. Copyright © 2015 Elsevier Ltd. All rights reserved.
-
Caffrey, Martha K; Febo, Marcelo
2014-01-01
Cue triggered relapse during the postpartum period can negatively impact maternal care. Given the high reward value of pups in maternal rats, we designed an fMRI experiment to test whether offspring presence reduces the neural response to a cocaine associated olfactory cue. Cocaine conditioned place preference was carried out before pregnancy in the presence of two distinct odors that were paired with cocaine or saline (+Cue and -Cue). The BOLD response to +Cue and -Cue was measured in dams on postpartum days 2-4. Odor cues were delivered to dams in the absence and then the presence of pups. Our data indicate that several limbic and cognitive regions of the maternal rat brain show a greater BOLD signal response to a +Cue versus -Cue. These include dorsal striatum, prelimbic cortex, parietal cortex, habenula, bed nucleus of stria terminalis, lateral septum and the mediodorsal and the anterior thalamic nucleus. Of the aforementioned brain regions, only the parietal cortex of cocaine treated dams showed a significant modulatory effect of pup presence. In this area of the cortex, cocaine exposed maternal rats showed a greater BOLD activation in response to the +Cue in the presence than in the absence of pups. Specific regions of the cocaine exposed maternal rat brain are strongly reactive to drug associated cues. The regions implicated in cue reactivity have been previously reported in clinical imaging work, and previous work supports their role in various motivational and cognitive functions. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.
-
Caffrey, Martha K.; Febo, Marcelo
2013-01-01
BACKGROUND Cue triggered relapse during the postpartum period can negatively impact maternal care. Given the high reward value of pups in maternal rats, we designed an fMRI experiment to test whether offspring presence reduces the neural response to a cocaine associated olfactory cue. METHODS Cocaine conditioned place preference was carried out before pregnancy in the presence of two distinct odors that were paired with cocaine or saline (+Cue and −Cue). The BOLD response to +Cue and −Cue was measured in dams on postpartum days 2–4. Odor cues were delivered to dams in the absence and then the presence of pups. RESULTS Our data indicate that several limbic and cognitive regions of the maternal rat brain show a greater BOLD signal response to a +Cue versus −Cue. These include dorsal striatum, prelimbic cortex, parietal cortex, habenula, bed nucleus of stria terminalis, lateral septum and the mediodorsal and the anterior thalamic nucleus. Of the aforementioned brain regions, only the parietal cortex of cocaine treated dams showed a significant modulatory effect of pup presence. In this area of the cortex, cocaine exposed maternal rats showed a greater BOLD activation in response to the +Cue in the presence than in the absence of pups. CONCLUSIONS Specific regions of the cocaine exposed maternal rat brain are strongly reactive to drug associated cues. The regions implicated in cue reactivity have been previously reported in clinical imaging work, and previous work supports their role in various motivational and cognitive functions. PMID:24183499
-
Two spatiotemporally distinct value systems shape reward-based learning in the human brain.
Fouragnan, Elsa; Retzler, Chris; Mullinger, Karen; Philiastides, Marios G
2015-09-08
Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions. Yet, the neural underpinnings of the value systems that encode different decision-outcomes remain elusive. Here coupling single-trial electroencephalography with simultaneously acquired functional magnetic resonance imaging, we uncover the spatiotemporal dynamics of two separate but interacting value systems encoding decision-outcomes. Consistent with a role in regulating alertness and switching behaviours, an early system is activated only by negative outcomes and engages arousal-related and motor-preparatory brain structures. Consistent with a role in reward-based learning, a later system differentially suppresses or activates regions of the human reward network in response to negative and positive outcomes, respectively. Following negative outcomes, the early system interacts and downregulates the late system, through a thalamic interaction with the ventral striatum. Critically, the strength of this coupling predicts participants' switching behaviour and avoidance learning, directly implicating the thalamostriatal pathway in reward-based learning.
-
A Plant Control Technology Using Reinforcement Learning Method with Automatic Reward Adjustment
Eguchi, Toru; Sekiai, Takaaki; Yamada, Akihiro; Shimizu, Satoru; Fukai, Masayuki
A control technology using Reinforcement Learning (RL) and Radial Basis Function (RBF) Network has been developed to reduce environmental load substances exhausted from power and industrial plants. This technology consists of the statistic model using RBF Network, which estimates characteristics of plants with respect to environmental load substances, and RL agent, which learns the control logic for the plants using the statistic model. In this technology, it is necessary to design an appropriate reward function given to the agent immediately according to operation conditions and control goals to control plants flexibly. Therefore, we propose an automatic reward adjusting method of RL for plant control. This method adjusts the reward function automatically using information of the statistic model obtained in its learning process. In the simulations, it is confirmed that the proposed method can adjust the reward function adaptively for several test functions, and executes robust control toward the thermal power plant considering the change of operation conditions and control goals.
-
Cingulate neglect in humans: disruption of contralesional reward learning in right brain damage.
Lecce, Francesca; Rotondaro, Francesca; Bonnì, Sonia; Carlesimo, Augusto; Thiebaut de Schotten, Michel; Tomaiuolo, Francesco; Doricchi, Fabrizio
2015-01-01
Motivational valence plays a key role in orienting spatial attention. Nonetheless, clinical documentation and understanding of motivationally based deficits of spatial orienting in the human is limited. Here in a series of one group-study and two single-case studies, we have examined right brain damaged patients (RBD) with and without left spatial neglect in a spatial reward-learning task, in which the motivational valence of the left contralesional and the right ipsilesional space was contrasted. In each trial two visual boxes were presented, one to the left and one to the right of central fixation. In one session monetary rewards were released more frequently in the box on the left side (75% of trials) whereas in another session they were released more frequently on the right side. In each trial patients were required to: 1) point to each one of the two boxes; 2) choose one of the boxes for obtaining monetary reward; 3) report explicitly the position of reward and whether this position matched or not the original choice. Despite defective spontaneous allocation of attention toward the contralesional space, RBD patients with left spatial neglect showed preserved contralesional reward learning, i.e., comparable to ipsilesional learning and to reward learning displayed by patients without neglect. A notable exception in the group of neglect patients was L.R., who showed no sign of contralesional reward learning in a series of 120 consecutive trials despite being able of reaching learning criterion in only 20 trials in the ipsilesional space. L.R. suffered a cortical-subcortical brain damage affecting the anterior components of the parietal-frontal attentional network and, compared with all other neglect and non-neglect patients, had additional lesion involvement of the medial anterior cingulate cortex (ACC) and of the adjacent sectors of the corpus callosum. In contrast to his lateralized motivational learning deficit, L.R. had no lateral bias in the early phases of
-
Reward reduces conflict by enhancing attentional control and biasing visual cortical processing.
Padmala, Srikanth; Pessoa, Luiz
2011-11-01
How does motivation interact with cognitive control during challenging behavioral conditions? Here, we investigated the interactions between motivation and cognition during a response conflict task and tested a specific model of the effect of reward on cognitive processing. Behaviorally, participants exhibited reduced conflict during the reward versus no-reward condition. Brain imaging results revealed that a group of subcortical and fronto-parietal regions was robustly influenced by reward at cue processing and, importantly, that cue-related responses in fronto-parietal attentional regions were predictive of reduced conflict-related signals in the medial pFC (MPFC)/ACC during the upcoming target phase. Path analysis revealed that the relationship between cue responses in the right intraparietal sulcus (IPS) and interference-related responses in the MPFC during the subsequent target phase was mediated via signals in the left fusiform gyrus, which we linked to distractor-related processing. Finally, reward increased functional connectivity between the right IPS and both bilateral putamen and bilateral nucleus accumbens during the cue phase, a relationship that covaried with across-individual sensitivity to reward in the case of the right nucleus accumbens. Taken together, our findings are consistent with a model in which motivationally salient cues are employed to upregulate top-down control processes that bias the selection of visual information, thereby leading to more efficient stimulus processing during conflict conditions.
-
Fernández-Lamo, Iván; Sánchez-Campusano, Raudel; Gruart, Agnès; Delgado-García, José M
2016-11-21
Proper performance of acquired abilities can be disturbed by the unexpected occurrence of external changes. Rats trained with an operant conditioning task (to press a lever in order to obtain a food pellet) using a fixed-ratio (1:1) schedule were subsequently placed in a Skinner box in which the lever could be removed randomly. Field postsynaptic potentials (fPSPs) were chronically evoked in perforant pathway-hippocampal CA1 (PP-CA1), CA1-subiculum (CA1-SUB), CA1-medial prefrontal cortex (CA1-mPFC), mPFC-nucleus accumbens (mPFC-NAc), and mPFC-basolateral amygdala (mPFC-BLA) synapses during lever IN and lever OUT situations. While lever presses were accompanied by a significant increase in fPSP slopes at the five synapses, the unpredictable absence of the lever were accompanied by decreased fPSP slopes in all, except PP-CA1 synapses. Spectral analysis of local field potentials (LFPs) recorded when the animal approached the corresponding area in the lever OUT situation presented lower spectral powers than during lever IN occasions for all recording sites, apart from CA1. Thus, the unpredictable availability of a reward-related cue modified the activity of cortical and subcortical areas related with the acquisition of operant learning tasks, suggesting an immediate functional reorganization of these neural circuits to address the changed situation and to modify ongoing behaviors accordingly.
-
Rudebeck, Peter H; Ripple, Joshua A; Mitz, Andrew R; Averbeck, Bruno B; Murray, Elisabeth A
2017-02-22
Orbitofrontal cortex (OFC), medial frontal cortex (MFC), and amygdala mediate stimulus-reward learning, but the mechanisms through which they interact are unclear. Here, we investigated how neurons in macaque OFC and MFC signaled rewards and the stimuli that predicted them during learning with and without amygdala input. Macaques performed a task that required them to evaluate two stimuli and then choose one to receive the reward associated with that option. Four main findings emerged. First, amygdala lesions slowed the acquisition and use of stimulus-reward associations. Further analyses indicated that this impairment was due, at least in part, to ineffective use of negative feedback to guide subsequent decisions. Second, the activity of neurons in OFC and MFC rapidly evolved to encode the amount of reward associated with each stimulus. Third, amygdalectomy reduced encoding of stimulus-reward associations during the evaluation of different stimuli. Reward encoding of anticipated and received reward after choices were made was not altered. Fourth, amygdala lesions led to an increase in the proportion of neurons in MFC, but not OFC, that encoded the instrumental response that monkeys made on each trial. These correlated changes in behavior and neural activity after amygdala lesions strongly suggest that the amygdala contributes to the ability to learn stimulus-reward associations rapidly by shaping encoding within OFC and MFC. SIGNIFICANCE STATEMENT Altered functional interactions among orbital frontal cortex (OFC), medial frontal cortex (MFC), and amygdala are thought to underlie several psychiatric conditions, many related to reward learning. Here, we investigated the causal contribution of the amygdala to the development of neuronal activity in macaque OFC and MFC related to rewards and the stimuli that predict them during learning. Without amygdala inputs, neurons in both OFC and MFC showed decreased encoding of stimulus-reward associations. MFC also showed
-
Reward components of feeding behavior are preserved during mouse aging.
Harb, Mazen R; Sousa, Nuno; Zihl, Joseph; Almeida, Osborne F X
2014-01-01
Eating behavior depends on associations between the sensory and energetic properties of foods. Healthful balance of these factors is a challenge for industrialized societies that have an abundance of food, food choices and food-related cues. Here, we were interested in whether appetitive conditioning changes as a function of age. Operant and pavlovian conditioning experiments (rewarding stimulus was a palatable food) in male mice (aged 3, 6, and 15 months) showed that implicit (non-declarative) memory remains intact during aging. Two other essential components of eating behavior, motivation and hedonic preference for rewarding foods, were also found not to be altered in aging mice. Specifically, hedonic responding by satiated mice to isocaloric foods of differing sensory properties (sucrose, milk) was similar in all age groups; importantly, however, this paradigm disclosed that older animals adjust their energy intake according to energetic need. Based on the assumption that the mechanisms that control feeding are conserved across species, it would appear that overeating and obesity in humans reflects a mismatch between ancient physiological mechanisms and today's cue-laden environment. The implication of the present results showing that aging does not impair the ability to learn stimulus-food associations is that the risk of overeating in response to food cues is maintained through to old age.
-
Hall, Philip S.
2009-01-01
Using rewards to impact students' behavior has long been common practice. However, using reward systems to enhance student learning conveniently masks the larger and admittedly more difficult task of finding and implementing the structure and techniques that children with special needs require to learn. More important, rewarding the child for good…
-
Implicit Sequence Learning and Contextual Cueing Do Not Compete for Central Cognitive Resources
Jimenez, Luis; Vazquez, Gustavo A.
2011-01-01
Sequence learning and contextual cueing explore different forms of implicit learning, arising from practice with a structured serial task, or with a search task with informative contexts. We assess whether these two learning effects arise simultaneously when both remain implicit. Experiments 1 and 2 confirm that a cueing effect can be observed…
-
Blunted Striatal Responses to Favorite Food Cues in Smokers*
Jastreboff, Ania M.; Sinha, Rajita; Lacadie, Cheryl M.; Balodis, Iris M.; Sherwin, Robert; Potenza, Marc N.
2014-01-01
Background Although tobacco-smoking is associated with relatively leaner body mass and smoking cessation with weight gain, the brain mechanisms underlying these relationships are not well understood. Smokers compared to non-smokers have shown diminished neural responses to non-tobacco rewarding stimuli (e.g., monetary rewards), but brain responses to favorite-food cues have not been investigated relative to smoking status. We hypothesized that smokers would exhibit diminished neural responses compared to non-smokers in response to favorite-food cues in motivation-reward and emotion-regulating regions of the brain. Methods Twenty-three smokers and 23 non-smokers matched based on body mass index (BMI), age, and gender listened to personalized favorite-food-cue, stress, and neutral-relaxing audiotapes during fMRI. Results During favorite-food-cue exposure, smokers versus non-smokers exhibited diminished activations in the caudate, putamen, insula, and thalamus. Neural responses during stress and neutral-relaxing conditions were similar across groups. Subjective food-craving ratings were similar across groups. Conclusions The relatively diminished neural responses to favorite-food cues in smokers may contribute to lower BMI. PMID:25444233
-
Castellanos, E H; Charboneau, E; Dietrich, M S; Park, S; Bradley, B P; Mogg, K; Cowan, R L
2009-09-01
The major aim of this study was to investigate whether the motivational salience of food cues (as reflected by their attention-grabbing properties) differs between obese and normal-weight subjects in a manner consistent with altered reward system function in obesity. A total of 18 obese and 18 normal-weight, otherwise healthy, adult women between the ages of 18 and 35 participated in an eye-tracking paradigm in combination with a visual probe task. Eye movements and reaction time to food and non-food images were recorded during both fasted and fed conditions in a counterbalanced design. Eating behavior and hunger level were assessed by self-report measures. Obese individuals had higher scores than normal-weight individuals on self-report measures of responsiveness to external food cues and vulnerability to disruptions in control of eating behavior. Both obese and normal-weight individuals demonstrated increased gaze duration for food compared to non-food images in the fasted condition. In the fed condition, however, despite reduced hunger in both groups, obese individuals maintained the increased attention to food images, whereas normal-weight individuals had similar gaze duration for food and non-food images. Additionally, obese individuals had preferential orienting toward food images at the onset of each image. Obese and normal-weight individuals did not differ in reaction time measures in the fasted or fed condition. Food cue incentive salience is elevated equally in normal-weight and obese individuals during fasting. Obese individuals retain incentive salience for food cues despite feeding and decreased self-report of hunger. Sensitization to food cues in the environment and their dysregulation in obese individuals may play a role in the development and/or maintenance of obesity.
-
Zhang, Chen; Sun, Chao; Gao, Liqiang; Zheng, Nenggan; Chen, Weidong; Zheng, Xiaoxiang
2013-01-01
Bio-robots based on brain computer interface (BCI) suffer from the lack of considering the characteristic of the animals in navigation. This paper proposed a new method for bio-robots' automatic navigation combining the reward generating algorithm base on Reinforcement Learning (RL) with the learning intelligence of animals together. Given the graded electrical reward, the animal e.g. the rat, intends to seek the maximum reward while exploring an unknown environment. Since the rat has excellent spatial recognition, the rat-robot and the RL algorithm can convergent to an optimal route by co-learning. This work has significant inspiration for the practical development of bio-robots' navigation with hybrid intelligence.
-
Schindler, Abigail G; Li, Shuang; Chavkin, Charles
2010-08-01
Stress exposure increases the risk of addictive drug use in human and animal models of drug addiction by mechanisms that are not completely understood. Mice subjected to repeated forced swim stress (FSS) before cocaine develop significantly greater conditioned place preference (CPP) for the drug-paired chamber than unstressed mice. Analysis of the dose dependency showed that FSS increased both the maximal CPP response and sensitivity to cocaine. To determine whether FSS potentiated CPP by enhancing associative learning mechanisms, mice were conditioned with cocaine in the absence of stress, then challenged after association was complete with the kappa-opioid receptor (KOR) agonist U50,488 or repeated FSS, before preference testing. Mice challenged with U50,488 60 min before CPP preference testing expressed significantly greater cocaine-CPP than saline-challenged mice. Potentiation by U50,488 was dose and time dependent and blocked by the KOR antagonist norbinaltorphimine (norBNI). Similarly, mice subjected to repeated FSS before the final preference test expressed significantly greater cocaine-CPP than unstressed controls, and FSS-induced potentiation was blocked by norBNI. Novel object recognition (NOR) performance was not affected by U50,488 given 60 min before assay, but was impaired when given 15 min before NOR assay, suggesting that KOR activation did not potentiate CPP by facilitating memory retrieval or expression. The results from this study show that the potentiation of cocaine-CPP by KOR activation does not result from an enhancement of associative learning mechanisms and that stress may instead enhance the rewarding valence of cocaine-associated cues by a dynorphin-dependent mechanism.
-
On Principle of Rewards in English Learning
Institute of Scientific and Technical Information of China (English)
熊莉芸
2004-01-01
There is no question that learning a foreign language like English is different from learning other subjects, mainly be cause it is new to us Chinese and there is no enough enviroment. But that doesn't mean we have no way to learn it and do it well .If asked to identify the most powerful influences on learning, motivation would probably be high on most teachers' and learners' lists. It seens only sensible to assume that English learning is most likely to occur when the learners want to learn. That is, when motivation such as interest, curiosity, or a desire achieves, the learners would be engaged in learning. However, how do we teachers motivate our students to like learning and learn well? Here, rewards both extrinsic and intrinsic are of great value and play a vital role in English learning.
-
Hedging Your Bets by Learning Reward Correlations in the Human Brain
Wunderlich, Klaus; Symmonds, Mkael; Bossaerts, Peter; Dolan, Raymond J.
2011-01-01
Summary Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling. PMID:21943609
-
Expected reward modulates encoding-related theta activity before an event.
Gruber, Matthias J; Watrous, Andrew J; Ekstrom, Arne D; Ranganath, Charan; Otten, Leun J
2013-01-01
Oscillatory brain activity in the theta frequency range (4-8 Hz) before the onset of an event has been shown to affect the likelihood of successfully encoding the event into memory. Recent work has also indicated that frontal theta activity might be modulated by reward, but it is not clear how reward expectancy, anticipatory theta activity, and memory formation might be related. Here, we used scalp electroencephalography (EEG) to assess the relationship between these factors. EEG was recorded from healthy adults while they memorized a series of words. Each word was preceded by a cue that indicated whether a high or low monetary reward would be earned if the word was successfully remembered in a later recognition test. Frontal theta power between the presentation of the reward cue and the onset of a word was predictive of later memory for the word, but only in the high reward condition. No theta differences were observed before word onset following low reward cues. The magnitude of prestimulus encoding-related theta activity in the high reward condition was correlated with the number of high reward words that were later confidently recognized. These findings provide strong evidence for a link between reward expectancy, theta activity, and memory encoding. Theta activity before event onset seems to be especially important for the encoding of motivationally significant stimuli. One possibility is that dopaminergic activity during reward anticipation mediates frontal theta activity related to memory. Copyright © 2012 Elsevier Inc. All rights reserved.
-
The amygdala, reward and emotion.
Murray, Elisabeth A
2007-11-01
Recent research provides new insights into amygdala contributions to positive emotion and reward. Studies of neuronal activity in the monkey amygdala and of autonomic responses mediated by the monkey amygdala show that, contrary to a widely held view, the amygdala is just as important for processing positive reward and reinforcement as it is for negative. In addition, neuropsychological studies reveal that the amygdala is essential for only a fraction of what might be considered 'stimulus-reward processing', and that the neural substrates for emotion and reward are partially nonoverlapping. Finally, evidence suggests that two systems within the amygdala, operating in parallel, enable reward-predicting cues to influence behavior; one mediates a general, arousing effect of reward and the other links the sensory properties of reward to emotion.
-
Byrne, Jamie E M; Murray, Greg
2017-01-01
A range of evidence suggests that human reward functioning is partly driven by the endogenous circadian system, generating 24-hour rhythms in behavioural measures of reward activation. Reward functioning is multifaceted but literature to date is largely limited to measures of self-reported positive mood states. The aim of this study was to advance the field by testing for hypothesised diurnal variation in previously unexplored components of psychological reward: 'wanting', liking, and learning using subjective and behavioural measures. Risky decision making (automatic Balloon Analogue Risk Task), affective responsivity to positive images (International Affective Pictures System), uncued self-reported discrete emotions, and learning-contingent reward (Iowa Gambling Task) were measured at 10.00 hours, 14.00 hours, and 19.00 hours in a counterbalanced repeated measures design with 50 healthy male participants (aged 18-30). As hypothesised, risky decision making (unconscious 'wanting') and ratings of arousal towards positive images (conscious wanting) exhibited a diurnal waveform with indices highest at 14.00 hours. No diurnal rhythm was observed for liking (pleasure ratings to positive images, discrete uncued positive emotions) or in a learning-contingent reward task. Findings reaffirm that diurnal variation in human reward functioning is most pronounced in the motivational 'wanting' components of reward.
-
Shinya, Ishii; Munetaka, Shidara; Katsunari, Shibata
2006-01-01
In an experiment of multi-trial task to obtain a reward, reward expectancy neurons,###which responded only in the non-reward trials that are necessary to advance###toward the reward, have been observed in the anterior cingulate cortex of monkeys.###In this paper, to explain the emergence of the reward expectancy neuron in###terms of reinforcement learning theory, a model that consists of a recurrent neural###network trained based on reinforcement learning is proposed. The analysis of the###hi...
-
Enhancing a Multi-body Mechanism with Learning-Aided Cues in an Augmented Reality Environment
International Nuclear Information System (INIS)
Sidhu, Manjit Singh
2013-01-01
Augmented Reality (AR) is a potential area of research for education, covering issues such as tracking and calibration, and realistic rendering of virtual objects. The ability to augment real world with virtual information has opened the possibility of using AR technology in areas such as education and training as well. In the domain of Computer Aided Learning (CAL), researchers have long been looking into enhancing the effectiveness of the teaching and learning process by providing cues that could assist learners to better comprehend the materials presented. Although a number of works were done looking into the effectiveness of learning-aided cues, but none has really addressed this issue for AR-based learning solutions. This paper discusses the design and model of an AR based software that uses visual cues to enhance the learning process and the outcome perception results of the cues.
-
Enhancing a Multi-body Mechanism with Learning-Aided Cues in an Augmented Reality Environment
Singh Sidhu, Manjit
2013-06-01
Augmented Reality (AR) is a potential area of research for education, covering issues such as tracking and calibration, and realistic rendering of virtual objects. The ability to augment real world with virtual information has opened the possibility of using AR technology in areas such as education and training as well. In the domain of Computer Aided Learning (CAL), researchers have long been looking into enhancing the effectiveness of the teaching and learning process by providing cues that could assist learners to better comprehend the materials presented. Although a number of works were done looking into the effectiveness of learning-aided cues, but none has really addressed this issue for AR-based learning solutions. This paper discusses the design and model of an AR based software that uses visual cues to enhance the learning process and the outcome perception results of the cues.
-
Conci, Markus; Müller, Hermann J; von Mühlenen, Adrian
2013-07-09
In visual search, detection of a target is faster when it is presented within a spatial layout of repeatedly encountered nontarget items, indicating that contextual invariances can guide selective attention (contextual cueing; Chun & Jiang, 1998). However, perceptual regularities may interfere with contextual learning; for instance, no contextual facilitation occurs when four nontargets form a square-shaped grouping, even though the square location predicts the target location (Conci & von Mühlenen, 2009). Here, we further investigated potential causes for this interference-effect: We show that contextual cueing can reliably occur for targets located within the region of a segmented object, but not for targets presented outside of the object's boundaries. Four experiments demonstrate an object-based facilitation in contextual cueing, with a modulation of context-based learning by relatively subtle grouping cues including closure, symmetry, and spatial regularity. Moreover, the lack of contextual cueing for targets located outside the segmented region was due to an absence of (latent) learning of contextual layouts, rather than due to an attentional bias towards the grouped region. Taken together, these results indicate that perceptual segmentation provides a basic structure within which contextual scene regularities are acquired. This in turn argues that contextual learning is constrained by object-based selection.
-
The impact of effort-reward imbalance and learning motivation on teachers' sickness absence.
Derycke, Hanne; Vlerick, Peter; Van de Ven, Bart; Rots, Isabel; Clays, Els
2013-02-01
The aim of this study was to analyse the impact of the effort-reward imbalance and learning motivation on sickness absence duration and sickness absence frequency among beginning teachers in Flanders (Belgium). A total of 603 teachers, who recently graduated, participated in this study. Effort-reward imbalance and learning motivation were assessed by means of self-administered questionnaires. Prospective data of registered sickness absence during 12 months follow-up were collected. Multivariate logistic regression analyses were performed. An imbalance between high efforts and low rewards (extrinsic hypothesis) was associated with longer sickness absence duration and more frequent absences. A low level of learning motivation (intrinsic hypothesis) was not associated with longer sickness absence duration but was significantly positively associated with sickness absence frequency. No significant results were obtained for the interaction hypothesis between imbalance and learning motivation. Further research is needed to deepen our understanding of the impact of psychosocial work conditions and personal resources on both sickness absence duration and frequency. Specifically, attention could be given to optimizing or reducing efforts spent at work, increasing rewards and stimulating learning motivation to influence sickness absence. Copyright © 2012 John Wiley & Sons, Ltd.
-
Daniel, Reka; Pollmann, Stefan
2010-01-06
The dopaminergic system is known to play a central role in reward-based learning (Schultz, 2006), yet it was also observed to be involved when only cognitive feedback is given (Aron et al., 2004). Within the domain of information-integration category learning, in which information from several stimulus dimensions has to be integrated predecisionally (Ashby and Maddox, 2005), the importance of contingent feedback is well established (Maddox et al., 2003). We examined the common neural correlates of reward anticipation and prediction error in this task. Sixteen subjects performed two parallel information-integration tasks within a single event-related functional magnetic resonance imaging session but received a monetary reward only for one of them. Similar functional areas including basal ganglia structures were activated in both task versions. In contrast, a single structure, the nucleus accumbens, showed higher activation during monetary reward anticipation compared with the anticipation of cognitive feedback in information-integration learning. Additionally, this activation was predicted by measures of intrinsic motivation in the cognitive feedback task and by measures of extrinsic motivation in the rewarded task. Our results indicate that, although all other structures implicated in category learning are not significantly affected by altering the type of reward, the nucleus accumbens responds to the positive incentive properties of an expected reward depending on the specific type of the reward.
-
De Decker, Annelies; Verbeken, Sandra; Sioen, Isabelle; Van Lippevelde, Wendy; Braet, Caroline; Eiben, Gabriele; Pala, Valeria; Reisch, Lucia A; De Henauw, Stefaan
2017-04-01
To understand the importance of the home food environment on unhealthy food consumption in children high in reward sensitivity, this study tested the hypothesis that the home availability of unhealthy food moderates the effect of reward sensitivity on children's fast-food consumption frequency, exerted via food cue responsiveness. Children between 7.5 and 14 years (n = 174, 50.6% boys) reported on reward sensitivity and food cue responsiveness (by means of the subscale 'external eating'). Their height and weight were measured. Parents reported on their children's fast-food consumption frequency, food cue responsiveness (by means of the subscale 'food responsiveness'), and on the home availability of unhealthy foods. Two moderated mediation models were conducted, one with the parent- and one with the child-reported food cue responsiveness as mediator. Findings suggested that with a high home availability of unhealthy foods, (a) a higher fast-food consumption frequency was found in children high in reward sensitivity and (b) the relation between reward sensitivity and the fast-food consumption frequency was mediated by external eating. The findings point at the importance of the home food environment in children high in reward sensitivity. They suggest to limit the home availability of unhealthy foods. What is Known: • Reward sensitivity (RS) is positively associated with children's palatable food consumption • In adolescents, this effect is mediated by food cue responsiveness, which determines the strength of an individual's motivation to obtain food when perceiving food cues What is New: • Children high in RS may be more vulnerable to palatable food cues in their everyday food environment because of a higher food cue responsiveness • The home food environment may be an important determining factor of the palatable food consumption of these children.
-
Garland, Eric L; Froeliger, Brett; Howard, Matthew O
2015-02-01
Chronic pain patients who misuse prescription opioids may suffer from allostatic dysregulation of natural reward processing. Hence, this study examined whether prescription opioid misusers with chronic pain (n=72) evidenced decreased natural reward responsiveness relative to non-misusers with chronic pain (n=26). Subjects completed a dot probe task containing pain-related, opioid-related, and natural reward stimuli while attentional bias (AB) scores and heart rate variability (HRV) responses were assessed. Compared to non-misusers, misusers evidenced significantly more attenuated HRV responses to opioid, pain, and natural reward cues presented during the dot probe task. These significant between-groups differences in HRV were largest during attention to natural reward cues, but became non-significant in a sensitivity analysis controlling for opioid dosing. In addition, non-misusers evidenced an AB toward natural reward cues, whereas misusers did not. Findings suggest that opioid misusers exhibit attentional and autonomic deficits during reward processing. Copyright © 2015 Elsevier B.V. All rights reserved.
-
Huber, Robert; Panksepp, Jules B; Nathaniel, Thomas; Alcaro, Antonio; Panksepp, Jaak
2011-10-01
In mammals, rewarding properties of drugs depend on their capacity to activate appetitive motivational states. With the underlying mechanisms strongly conserved in evolution, invertebrates have recently emerged as a powerful new model in addiction research. In crayfish natural reward has proven surprisingly sensitive to human drugs of abuse, opening an unlikely avenue of research into the basic biological mechanisms of drug addiction. In a series of studies we first examined the presence of natural reward systems in crayfish, then characterized its sensitivity to a wide range of human drugs of abuse. A conditioned place preference (CPP) paradigm was used to demonstrate that crayfish seek out those environments that had previously been paired with the psychostimulants cocaine and amphetamine, and the opioid morphine. The administration of amphetamine exerted its effects at a number of sites, including the stimulation of circuits for active exploratory behaviors (i.e., SEEKING). A further study examined morphine-induced reward, extinction and reinstatement in crayfish. Repeated intra-circulatory infusions of morphine served as a reward when paired with distinct visual or tactile cues. Morphine-induced CPP was extinguished after repeated saline injections. Following this extinction phase, morphine-experienced crayfish were once again challenged with the drug. The priming injections of morphine reinstated CPP at all tested doses, suggesting that morphine-induced CPP is unrelenting. In an exploration of drug-associated behavioral sensitization in crayfish we concurrently mapped measures of locomotion and rewarding properties of morphine. Single and repeated intra-circulatory infusions of morphine resulted in persistent locomotory sensitization, even 5 days following the infusion. Moreover, a single dose of morphine was sufficient to induce long-term behavioral sensitization. CPP for morphine and context-dependent cues could not be disrupted over a drug free period of 5
-
Monetary reward modulates task-irrelevant perceptual learning for invisible stimuli.
Pascucci, David; Mastropasqua, Tommaso; Turatto, Massimo
2015-01-01
Task Irrelevant Perceptual Learning (TIPL) shows that the brain's discriminative capacity can improve also for invisible and unattended visual stimuli. It has been hypothesized that this form of "unconscious" neural plasticity is mediated by an endogenous reward mechanism triggered by the correct task performance. Although this result has challenged the mandatory role of attention in perceptual learning, no direct evidence exists of the hypothesized link between target recognition, reward and TIPL. Here, we manipulated the reward value associated with a target to demonstrate the involvement of reinforcement mechanisms in sensory plasticity for invisible inputs. Participants were trained in a central task associated with either high or low monetary incentives, provided only at the end of the experiment, while subliminal stimuli were presented peripherally. Our results showed that high incentive-value targets induced a greater degree of perceptual improvement for the subliminal stimuli, supporting the role of reinforcement mechanisms in TIPL.
-
Mosberger, Alice C.; de Clauser, Larissa; Kasper, Hansjörg; Schwab, Martin E.
2016-01-01
Motor skills represent high-precision movements performed at optimal speed and accuracy. Such motor skills are learned with practice over time. Besides practice, effects of motivation have also been shown to influence speed and accuracy of movements, suggesting that fast movements are performed to maximize gained reward over time as noted in…
-
Combining Correlation-Based and Reward-Based Learning in Neural Control for Policy Improvement
DEFF Research Database (Denmark)
Manoonpong, Poramate; Kolodziejski, Christoph; Wörgötter, Florentin
2013-01-01
Classical conditioning (conventionally modeled as correlation-based learning) and operant conditioning (conventionally modeled as reinforcement learning or reward-based learning) have been found in biological systems. Evidence shows that these two mechanisms strongly involve learning about...... associations. Based on these biological findings, we propose a new learning model to achieve successful control policies for artificial systems. This model combines correlation-based learning using input correlation learning (ICO learning) and reward-based learning using continuous actor–critic reinforcement...... learning (RL), thereby working as a dual learner system. The model performance is evaluated by simulations of a cart-pole system as a dynamic motion control problem and a mobile robot system as a goal-directed behavior control problem. Results show that the model can strongly improve pole balancing control...
-
Construction of a Learning Agent Handling Its Rewards According to Environmental Situations
Moriyama, Koichi; Numao, Masayuki
The authors aim at constructing an agent which learns appropriate actions in a Multi-Agent environment with and without social dilemmas. For this aim, the agent must have nonrationality that makes it give up its own profit when it should do that. Since there are many studies on rational learning that brings more and more profit, it is desirable to utilize them for constructing the agent. Therefore, we use a reward-handling manner that makes internal evaluation from the agent's rewards, and then the agent learns actions by a rational learning method with the internal evaluation. If the agent has only a fixed manner, however, it does not act well in the environment with and without dilemmas. Thus, the authors equip the agent with several reward-handling manners and criteria for selecting an effective one for the environmental situation. In the case of humans, what generates the internal evaluation is usually called emotion. Hence, this study also aims at throwing light on emotional activities of humans from a constructive view. In this paper, we divide a Multi-Agent environment into three situations and construct an agent having the reward-handling manners and the criteria. We observe that the agent acts well in all the three Multi-Agent situations composed of homogeneous agents.
-
Blunted striatal responses to favorite-food cues in smokers.
Jastreboff, Ania M; Sinha, Rajita; Lacadie, Cheryl M; Balodis, Iris M; Sherwin, Robert; Potenza, Marc N
2015-01-01
Although tobacco-smoking is associated with relatively leaner body mass and smoking cessation with weight gain, the brain mechanisms underlying these relationships are not well understood. Smokers compared to non-smokers have shown diminished neural responses to non-tobacco rewarding stimuli (e.g., monetary rewards), but brain responses to favorite-food cues have not been investigated relative to smoking status. We hypothesized that smokers would exhibit diminished neural responses compared to non-smokers in response to favorite-food cues in motivation-reward and emotion-regulating regions of the brain. Twenty-three smokers and 23 non-smokers matched based on body mass index (BMI), age, and gender listened to personalized favorite-food cue, stress, and neutral-relaxing audiotapes during fMRI. During favorite-food cue exposure, smokers versus non-smokers exhibited diminished activations in the caudate, putamen, insula, and thalamus. Neural responses during stress and neutral-relaxing conditions were similar across groups. Subjective food-craving ratings were similar across groups. The relatively diminished neural responses to favorite-food cues in smokers may contribute to lower BMI. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.
-
Reward and punishment learning in daily life : A replication study
Heininga, Vera E; van Roekel, G.H.; Wichers, Marieke; Oldehinkel, Albertine J
2017-01-01
Day-to-day experiences are accompanied by feelings of Positive Affect (PA) and Negative Affect (NA). Implicitly, without conscious processing, individuals learn about the reward and punishment value of each context and activity. These associative learning processes, in turn, affect the probability
-
Directory of Open Access Journals (Sweden)
Natália Kocsel
2017-05-01
Full Text Available Cumulative evidence suggests that trait rumination can be defined as an abstract information processing mode, which leads people to constantly anticipate the likely impact of present events on future events and experiences. A previous study with remitted depressed patients suggested that enhanced rumination tendencies distort brain mechanisms of anticipatory processes associated with reward and loss cues. In the present study, we explored the impact of trait rumination on neural activity during reward and loss anticipation among never-depressed people. We analyzed the data of 37 healthy controls, who performed the monetary incentive delay (MID task which was designed for the simultaneous measurement of the anticipation (motivational and consumption (hedonic phase of reward processing, during functional magnetic resonance imaging (fMRI. Our results show that rumination—after controlling for age, gender, and current mood—significantly influenced neural responses to reward (win cues compared to loss cues. Blood-oxygenation-level-dependent (BOLD activity in the left inferior frontal gyrus (IFG triangularis, left anterior insula, and left rolandic operculum was positively related to Ruminative Response Scale (RRS scores. We did not detect any significant rumination-related activations associated with win-neutral or loss-neutral cues and with reward or loss consumption. Our results highlight the influence of trait rumination on reward anticipation in a non-depressed sample. They also suggest that for never-depressed ruminators rewarding cues are more salient than loss cues. BOLD response during reward consumption did not relate to rumination, suggesting that rumination mainly relates to processing of the motivational (wanting aspect of reward rather than the hedonic (liking aspect, at least in the absence of pathological mood.
-
Altered neural processing of reward and punishment in adolescents with Major Depressive Disorder.
Landes, I; Bakos, S; Kohls, G; Bartling, J; Schulte-Körne, G; Greimel, E
2018-05-01
Altered reward and punishment function has been suggested as an important vulnerability factor for the development of Major Depressive Disorder (MDD). Prior ERP studies found evidence for neurophysiological dysfunctions in reinforcement processes in adults with MDD. To date, only few ERP studies have examined the neural underpinnings of reinforcement processing in adolescents diagnosed with MDD. The present event-related potential (ERP) study aimed to investigate neurophysiological mechanisms of anticipation and consumption of reward and punishment in adolescents with MDD in one comprehensive paradigm. During ERP recording, 25 adolescents with MDD and 29 healthy controls (12-17 years) completed a Monetary Incentive Delay Task comprising both a monetary reward and a monetary punishment condition. During anticipation, the cue-P3 signaling attentional allocation was recorded. During consumption, the feedback-P3 and Reward Positivity (RewP) were recorded to capture attentional allocation and outcome evaluation, respectively. Compared to controls, adolescents with MDD showed prolonged cue-P3 latencies to reward cues. Furthermore, unlike controls, adolescents with MDD displayed shorter feedback-P3 latencies in the reward versus punishment condition. RewPs did not differ between groups. It remains unanswered whether the observed alterations in adolescent MDD represent a state or trait. Delayed neural processing of reward cues corresponds to the clinical presentation of adolescent MDD with reduced motivational tendencies to obtain rewards. Relatively shorter feedback-P3 latencies in the reward versus punishment condition could indicate a high salience of performance-contingent reward. Frequent exposure of negatively biased adolescents with MDD to performance-contingent rewards might constitute a promising intervention approach. Copyright © 2018 Elsevier B.V. All rights reserved.
-
Acupuncture inhibits cue-induced heroin craving and brain activation.
Cai, Xinghui; Song, Xiaoge; Li, Chuanfu; Xu, Chunsheng; Li, Xiliang; Lu, Qi
2012-11-25
Previous research using functional MRI has shown that specific brain regions associated with drug dependence and cue-elicited heroin craving are activated by environmental cues. Craving is an important trigger of heroin relapse, and acupuncture may inhibit craving. In this study, we performed functional MRI in heroin addicts and control subjects. We compared differences in brain activation between the two groups during heroin cue exposure, heroin cue exposure plus acupuncture at the Zusanli point (ST36) without twirling of the needle, and heroin cue exposure plus acupuncture at the Zusanli point with twirling of the needle. Heroin cue exposure elicited significant activation in craving-related brain regions mainly in the frontal lobes and callosal gyri. Acupuncture without twirling did not significantly affect the range of brain activation induced by heroin cue exposure, but significantly changed the extent of the activation in the heroin addicts group. Acupuncture at the Zusanli point with twirling of the needle significantly decreased both the range and extent of activation induced by heroin cue exposure compared with heroin cue exposure plus acupuncture without twirling of the needle. These experimental findings indicate that presentation of heroin cues can induce activation in craving-related brain regions, which are involved in reward, learning and memory, cognition and emotion. Acupuncture at the Zusanli point can rapidly suppress the activation of specific brain regions related to craving, supporting its potential as an intervention for drug craving.
-
Repeated nicotine exposure enhances reward-related learning in the rat.
Olausson, Peter; Jentsch, J David; Taylor, Jane R
2003-07-01
Repeated exposure to addictive drugs causes neuroadaptive changes in cortico-limbic-striatal circuits that may underlie alterations in incentive-motivational processes and reward-related learning. Such drug-induced alterations may be relevant to drug addiction because enhanced incentive motivation and increased control over behavior by drug-associated stimuli may contribute to aspects of compulsive drug-seeking and drug-taking behaviors. This study investigated the consequences of repeated nicotine treatment on the acquisition and performance of Pavlovian discriminative approach behavior, a measure of reward-related learning, in male rats. Water-restricted rats were trained to associate a compound conditioned stimulus (tone+light) with the availability of water (the unconditioned stimulus) in 15 consecutive daily sessions. In separate experiments, rats were repeatedly treated with nicotine (0.35 mg/kg, s.c.) either (1) prior to the onset of training, (2) after each daily training session was completed (ie postsession injections), or (3) received nicotine both before the onset of training as well as after each daily training session. In this study, all nicotine treatment schedules increased Pavlovian discriminative approach behavior and, thus, prior repeated exposure to nicotine, repeated postsession nicotine injections, or both, facilitated reward-related learning.
-
An explicit statistical model of learning lexical segmentation using multiple cues
Çöltekin, Ça ̆grı; Nerbonne, John; Lenci, Alessandro; Padró, Muntsa; Poibeau, Thierry; Villavicencio, Aline
2014-01-01
This paper presents an unsupervised and incremental model of learning segmentation that combines multiple cues whose use by children and adults were attested by experimental studies. The cues we exploit in this study are predictability statistics, phonotactics, lexical stress and partial lexical
-
Schultz, Wolfram
2004-04-01
Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.
-
Directory of Open Access Journals (Sweden)
Ulrich eKirk
2015-02-01
Full Text Available Reinforcement learning models have demonstrated that phasic activity of dopamine neurons during reward expectation encodes information about the predictability of rewards and cues that predict reward. Evidence indicates that mindfulness-based approaches reduce reward anticipation signal in the striatum to negative and positive incentives suggesting the hypothesis that such training influence basic reward processing. Using a passive conditioning task and fMRI in a group of experienced mindfulness meditators and age-matched controls, we tested the hypothesis that mindfulness meditation influence reward and reward prediction error signals. We found diminished positive and negative prediction error-related blood-oxygen level-dependent (BOLD responses in the putamen in meditators compared with controls. In the meditators, this decrease in striatal BOLD responses to reward prediction was paralleled by increased activity in posterior insula, a primary interoceptive region. Critically, responses in the putamen during early trials of the conditioning procedure (run 1 were elevated in both meditators and controls. These results provide evidence that experienced mindfulness meditators show attenuated reward prediction signals to valenced stimuli, which may be related to interoceptive processes encoded in the posterior insula.
-
Monetary reward modulates task-irrelevant perceptual learning for invisible stimuli.
Directory of Open Access Journals (Sweden)
David Pascucci
Full Text Available Task Irrelevant Perceptual Learning (TIPL shows that the brain's discriminative capacity can improve also for invisible and unattended visual stimuli. It has been hypothesized that this form of "unconscious" neural plasticity is mediated by an endogenous reward mechanism triggered by the correct task performance. Although this result has challenged the mandatory role of attention in perceptual learning, no direct evidence exists of the hypothesized link between target recognition, reward and TIPL. Here, we manipulated the reward value associated with a target to demonstrate the involvement of reinforcement mechanisms in sensory plasticity for invisible inputs. Participants were trained in a central task associated with either high or low monetary incentives, provided only at the end of the experiment, while subliminal stimuli were presented peripherally. Our results showed that high incentive-value targets induced a greater degree of perceptual improvement for the subliminal stimuli, supporting the role of reinforcement mechanisms in TIPL.
-
Reward components of feeding behavior are preserved during mouse aging
Directory of Open Access Journals (Sweden)
Mazen R. Harb
2014-09-01
Full Text Available Eating behavior depends on associations between the sensory and energetic properties of foods. Healthful balance of these factors is a challenge for industrialized societies that have an abundance of food, food choices and food-related cues. Here, we were interested in whether appetitive conditioning changes as a function of age. Operant and pavlovian conditioning experiments (rewarding stimulus was a palatable food in male mice (aged 3, 6 and 15 months showed that implicit (non-declarative memory remains intact during aging. Two other essential components of eating behavior, motivation and hedonic preference for rewarding foods, were also found not to be altered in aging mice. Specifically, hedonic responding by satiated mice to isocaloric foods of differing sensory properties (sucrose, milk was similar in all age groups; importantly, however, this paradigm disclosed that older animals adjust their energy intake according to energetic need. Based on the assumption that the mechanisms that control feeding are conserved across species, it would appear that overeating and obesity in humans reflects a mismatch between ancient physiological mechanisms and today’s cue-laden environment. The implication of the present results showing that aging does not impair the ability to learn stimulus-food associations is that the risk of overeating in response to food cues is maintained through to old age.
-
Food and drug cues activate similar brain regions: a meta-analysis of functional MRI studies.
Tang, D W; Fellows, L K; Small, D M; Dagher, A
2012-06-06
In healthy individuals, food cues can trigger hunger and feeding behavior. Likewise, smoking cues can trigger craving and relapse in smokers. Brain imaging studies report that structures involved in appetitive behaviors and reward, notably the insula, striatum, amygdala and orbital frontal cortex, tend to be activated by both visual food and smoking cues. Here, by carrying out a meta-analysis of human neuro-imaging studies, we investigate the neural network activated by: 1) food versus neutral cues (14 studies, 142 foci) 2) smoking versus neutral cues (15 studies, 176 foci) 3) smoking versus neutral cues when correlated with craving scores (7 studies, 108 foci). PubMed was used to identify cue-reactivity imaging studies that compared brain response to visual food or smoking cues to neutral cues. Fourteen articles were identified for the food meta-analysis and fifteen articles were identified for the smoking meta-analysis. Six articles were identified for the smoking cue correlated with craving analysis. Meta-analyses were carried out using activation likelihood estimation. Food cues were associated with increased blood oxygen level dependent (BOLD) response in the left amygdala, bilateral insula, bilateral orbital frontal cortex, and striatum. Smoking cues were associated with increased BOLD signal in the same areas, with the exception of the insula. However, the smoking meta-analysis of brain maps correlating cue-reactivity with subjective craving did identify the insula, suggesting that insula activation is only found when craving levels are high. The brain areas identified here are involved in learning, memory and motivation, and their cue-induced activity is an index of the incentive salience of the cues. Using meta-analytic techniques to combine a series of studies, we found that food and smoking cues activate comparable brain networks. There is significant overlap in brain regions responding to conditioned cues associated with natural and drug rewards
-
Kidd, Chloe; Loxton, Natalie J
2018-05-01
The current study aimed to identify how underlying individual differences increases vulnerability to television food advertising. In particular, this study examined how reward sensitivity, a biologically-based predisposition to approach rewards (such as appetitive foods) in the environment, influenced participants' vulnerability to television food advertising and subsequent food consumption. Ninety-eight participants were randomly assigned to a cue condition (food cues versus non-food cues) and then viewed a 30 min documentary interrupted by advertising featuring a mix of food and neutral advertising (food cue condition) or only neutral advertising (non-food cue condition). Participants' reward sensitivity, approach motivation measured as urge to eat, and food consumption were recorded. Moderated mediation regression analyses revealed the positive association between reward sensitivity and food consumption was mediated by an increase in urge to eat, but only when participants were exposed to food advertising. These findings suggest heightened reward sensitivity, exposure to appetitive food cues, and approach motivation are key interacting mechanisms that may lead to maladaptive eating behaviours. Copyright © 2018 Elsevier Inc. All rights reserved.
-
Vollstädt-Klein, Sabine; Loeber, Sabine; Kirsch, Martina; Bach, Patrick; Richter, Anne; Bühler, Mira; von der Goltz, Christoph; Hermann, Derik; Mann, Karl; Kiefer, Falk
2011-06-01
In alcohol-dependent patients, alcohol-associated cues elicit brain activation in mesocorticolimbic networks involved in relapse mechanisms. Cue-exposure based extinction training (CET) has been shown to be efficacious in the treatment of alcoholism; however, it has remained unexplored whether CET mediates its therapeutic effects via changes of activity in mesolimbic networks in response to alcohol cues. In this study, we assessed CET treatment effects on cue-induced responses using functional magnetic resonance imaging (fMRI). In a randomized controlled trial, abstinent alcohol-dependent patients were randomly assigned to a CET group (n = 15) or a control group (n = 15). All patients underwent an extended detoxification treatment comprising medically supervised detoxification, health education, and supportive therapy. The CET patients additionally received nine CET sessions over 3 weeks, exposing the patient to his/her preferred alcoholic beverage. Cue-induced fMRI activation to alcohol cues was measured at pretreatment and posttreatment. Compared with pretreatment, fMRI cue-reactivity reduction was greater in the CET relative to the control group, especially in the anterior cingulate gyrus and the insula, as well as limbic and frontal regions. Before treatment, increased cue-induced fMRI activation was found in limbic and reward-related brain regions and in visual areas. After treatment, the CET group showed less activation than the control group in the left ventral striatum. The study provides first evidence that an exposure-based psychotherapeutic intervention in the treatment of alcoholism impacts on brain areas relevant for addiction memory and attentional focus to alcohol-associated cues and affects mesocorticolimbic reward pathways suggested to be pathophysiologically involved in addiction. Copyright © 2011 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.
-
Impaired reward learning and intact motivation after serotonin depletion in rats.
Izquierdo, Alicia; Carlos, Kathleen; Ostrander, Serena; Rodriguez, Danilo; McCall-Craddolph, Aaron; Yagnik, Gargey; Zhou, Feimeng
2012-08-01
Aside from the well-known influence of serotonin (5-hydroxytryptamine, 5-HT) on emotional regulation, more recent investigations have revealed the importance of this monoamine in modulating cognition. Parachlorophenylalanine (PCPA) depletes 5-HT by inhibiting tryptophan hydroxylase, the enzyme required for 5-HT synthesis and, if administered at sufficiently high doses, can result in a depletion of at least 90% of the brain's 5-HT levels. The present study assessed the long-lasting effects of widespread 5-HT depletions on two tasks of cognitive flexibility in Long Evans rats: effort discounting and reversal learning. We assessed performance on these tasks after administration of either 250 or 500 mg/kg PCPA or saline (SAL) on two consecutive days. Consistent with a previous report investigating the role of 5-HT on effort discounting, pretreatment with either dose of PCPA resulted in normal effortful choice: All rats continued to climb tall barriers to obtain large rewards and were not work-averse. Additionally, rats receiving the lower dose of PCPA displayed normal reversal learning. However, despite intact motivation to work for food rewards, rats receiving the largest dose of PCPA were unexpectedly impaired relative to SAL rats on the pretraining stages leading up to reversal learning, ultimately failing to approach and respond to the stimuli associated with reward. High performance liquid chromatography (HPLC) with electrochemical detection confirmed 5-HT, and not dopamine, levels in the ventromedial frontal cortex were correlated with this measure of associative reward learning. Copyright © 2012 Elsevier B.V. All rights reserved.
-
Tosoni, Annalisa; Shulman, Gordon L; Pope, Anna L W; McAvoy, Mark P; Corbetta, Maurizio
2013-06-01
Success in a dynamically changing world requires both rapid shifts of attention to the location of important objects and the detection of changes in motivational contingencies that may alter future behavior. Here we addressed the relationship between these two processes by measuring the blood-oxygenation-level-dependent (BOLD) signal during a visual search task in which the location and the color of a salient cue respectively indicated where a rewarded target would appear and the monetary gain (large or small) associated with its detection. While cues that either shifted or maintained attention were presented every 4 to 8 sec, the reward magnitude indicated by the cue changed roughly every 30 sec, allowing us to distinguish a change in expected reward magnitude from a maintained state of expected reward magnitude. Posterior cingulate cortex was modulated by cues signaling an increase in expected reward magnitude, but not by cues for shifting versus maintaining spatial attention. Dorsal fronto-parietal regions in precuneus and frontal eye field (FEF) also showed increased BOLD activity for changes in expected reward magnitude from low to high, but in addition showed large independent modulations for shifting versus maintaining attention. In particular, the differential activation for shifting versus maintaining attention was not affected by expected reward magnitude. These results indicate that BOLD activations for shifts of attention and increases in expected reward magnitude are largely separate. Finally, visual cortex showed sustained spatially selective signals that were significantly enhanced when greater reward magnitude was expected, but this reward-related modulation was not observed in spatially selective regions of dorsal fronto-parietal cortex. Copyright © 2012 Elsevier Ltd. All rights reserved.
-
James, Alex S; Pennington, Zachary T; Tran, Phu; Jentsch, James David
2015-01-01
Two theories regarding the role for dopamine neurons in learning include the concepts that their activity serves as a (1) mechanism that confers incentive salience onto rewards and associated cues and/or (2) contingency teaching signal reflecting reward prediction error. While both theories are provocative, the causal role for dopamine cell activity in either mechanism remains controversial. In this study mice that either fully or partially lacked NMDARs in dopamine neurons exclusively, as well as appropriate controls, were evaluated for reward-related learning; this experimental design allowed for a test of the premise that NMDA/glutamate receptor (NMDAR)-mediated mechanisms in dopamine neurons, including NMDA-dependent regulation of phasic discharge activity of these cells, modulate either the instrumental learning processes or the likelihood of pavlovian cues to become highly motivating incentive stimuli that directly attract behavior. Loss of NMDARs in dopamine neurons did not significantly affect baseline dopamine utilization in the striatum, novelty evoked locomotor behavior, or consumption of a freely available, palatable food solution. On the other hand, animals lacking NMDARs in dopamine cells exhibited a selective reduction in reinforced lever responses that emerged over the course of instrumental learning. Loss of receptor expression did not, however, influence the likelihood of an animal acquiring a pavlovian conditional response associated with attribution of incentive salience to reward-paired cues (sign tracking). These data support the view that reductions in NMDAR signaling in dopamine neurons affect instrumental reward-related learning but do not lend support to hypotheses that suggest that the behavioral significance of this signaling includes incentive salience attribution.
-
Marsh, Brandi T; Tarigoppula, Venkata S Aditya; Chen, Chen; Francis, Joseph T
2015-05-13
For decades, neurophysiologists have worked on elucidating the function of the cortical sensorimotor control system from the standpoint of kinematics or dynamics. Recently, computational neuroscientists have developed models that can emulate changes seen in the primary motor cortex during learning. However, these simulations rely on the existence of a reward-like signal in the primary sensorimotor cortex. Reward modulation of the primary sensorimotor cortex has yet to be characterized at the level of neural units. Here we demonstrate that single units/multiunits and local field potentials in the primary motor (M1) cortex of nonhuman primates (Macaca radiata) are modulated by reward expectation during reaching movements and that this modulation is present even while subjects passively view cursor motions that are predictive of either reward or nonreward. After establishing this reward modulation, we set out to determine whether we could correctly classify rewarding versus nonrewarding trials, on a moment-to-moment basis. This reward information could then be used in collaboration with reinforcement learning principles toward an autonomous brain-machine interface. The autonomous brain-machine interface would use M1 for both decoding movement intention and extraction of reward expectation information as evaluative feedback, which would then update the decoding algorithm as necessary. In the work presented here, we show that this, in theory, is possible. Copyright © 2015 the authors 0270-6474/15/357374-14$15.00/0.
-
Teodorescu, Kinneret; Erev, Ido
2014-10-01
Exposure to uncontrollable outcomes has been found to trigger learned helplessness, a state in which the agent, because of lack of exploration, fails to take advantage of regained control. Although the implications of this phenomenon have been widely studied, its underlying cause remains undetermined. One can learn not to explore because the environment is uncontrollable, because the average reinforcement for exploring is low, or because rewards for exploring are rare. In the current research, we tested a simple experimental paradigm that contrasts the predictions of these three contributors and offers a unified psychological mechanism that underlies the observed phenomena. Our results demonstrate that learned helplessness is not correlated with either the perceived controllability of one's environment or the average reward, which suggests that reward prevalence is a better predictor of exploratory behavior than the other two factors. A simple computational model in which exploration decisions were based on small samples of past experiences captured the empirical phenomena while also providing a cognitive basis for feelings of uncontrollability. © The Author(s) 2014.
-
The attention habit: how reward learning shapes attentional selection.
Anderson, Brian A
2016-04-01
There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications. © 2015 New York Academy of Sciences.
-
Reward-prospect interacts with trial-by-trial preparation for potential distraction.
Marini, Francesco; van den Berg, Berry; Woldorff, Marty G
2015-02-01
When attending for impending visual stimuli, cognitive systems prepare to identify relevant information while ignoring irrelevant, potentially distracting input. Recent work (Marini et al., 2013) showed that a supramodal distracter-filtering mechanism is invoked in blocked designs involving expectation of possible distracter stimuli, although this entails a cost ( distraction-filtering cost ) on speeded performance when distracters are expected but not presented. Here we used an arrow-flanker task to study whether an analogous cost, potentially reflecting the recruitment of a specific distraction-filtering mechanism, occurs dynamically when potential distraction is cued trial-to-trial ( cued distracter-expectation cost ). In order to promote the maximal utilization of cue information by participants, in some experimental conditions the cue also signaled the possibility of earning a monetary reward for fast and accurate performance. This design also allowed us to investigate the interplay between anticipation for distracters and anticipation of reward, which is known to engender attentional preparation. Only in reward contexts did participants show a cued distracter-expectation cost, which was larger with higher reward prospect and when anticipation for both distracters and reward were manipulated trial-to-trial. Thus, these results indicate that reward prospect interacts with the distracter expectation during trial-by-trial preparatory processes for potential distraction. These findings highlight how reward guides cue-driven attentional preparation.
-
Long-term memory of relative reward values.
Soldati, Francesca; Burman, Oliver H P; John, Elizabeth A; Pike, Thomas W; Wilkinson, Anna
2017-02-01
Long-term memory can be adaptive as it allows animals to retain information that is crucial for survival, such as the appearance and location of key resources. This is generally examined by comparing choices of stimuli that have value to the animal with those that do not; however, in nature choices are rarely so clear cut. Animals are able to assess the relative value of a resource via direct comparison, but it remains unclear whether they are able to retain this information for a biologically meaningful amount of time. To test this, captive red-footed tortoises (Chelonoidis carbonaria) were first trained to associate visual cues with specific qualities and quantities of food, and their preferences for the different reward values determined. They were then retested after an interval of 18 months. We found that the tortoises were able to retain the information they had learned about the cues as indicators of relative reward values over this interval, demonstrating a memory for the relative quantity and quality of food over an extended period of time. This is likely to impact directly on an animal's foraging decisions, such as the exploitation of seasonally varying resources, with obvious fitness implications for the individual; however, the implications may also extend to the ecological interactions in which the animal is involved, affecting processes such as herbivory and seed dispersal. © 2017 The Author(s).
-
Impaired implicit learning and feedback processing after stroke.
Lam, J M; Globas, C; Hosp, J A; Karnath, H-O; Wächter, T; Luft, A R
2016-02-09
The ability to learn is assumed to support successful recovery and rehabilitation therapy after stroke. Hence, learning impairments may reduce the recovery potential. Here, the hypothesis is tested that stroke survivors have deficits in feedback-driven implicit learning. Stroke survivors (n=30) and healthy age-matched control subjects (n=21) learned a probabilistic classification task with brain activation measured using functional magnetic resonance imaging in a subset of these individuals (17 stroke and 10 controls). Stroke subjects learned slower than controls to classify cues. After being rewarded with a smiley face, they were less likely to give the same response when the cue was repeated. Stroke subjects showed reduced brain activation in putamen, pallidum, thalamus, frontal and prefrontal cortices and cerebellum when compared with controls. Lesion analysis identified those stroke survivors as learning-impaired who had lesions in frontal areas, putamen, thalamus, caudate and insula. Lesion laterality had no effect on learning efficacy or brain activation. These findings suggest that stroke survivors have deficits in reinforcement learning that may be related to dysfunctional processing of feedback-based decision-making, reward signals and working memory. Copyright © 2015 IBRO. Published by Elsevier Ltd. All rights reserved.
-
Allocentric spatial learning and memory deficits in Down syndrome.
Directory of Open Access Journals (Sweden)
Pamela A Banta Lavenex
2015-02-01
Full Text Available Studies have shown that persons with Down Syndrome (DS exhibit relatively poor language capacities, and impaired verbal and visuoperceptual memory, whereas their visuospatial memory capacities appear comparatively spared. Individuals with DS recall better where an object was previously seen than what object was previously seen. However, most of the evidence concerning preserved visuospatial memory comes from tabletop or computerized experiments which are biased towards testing egocentric (viewpoint-dependent spatial representations. Accordingly, allocentric (viewpoint-independent spatial learning and memory capacities may not be necessary to perform these tasks. Thus, in order to more fully characterize the spatial capacities of individuals with DS, allocentric processes underlying real-world navigation must also be investigated. We tested 20 participants with DS and 16 mental age-matched, typically developing (TD children in a real-world, allocentric spatial memory task. During local cue (LC trials, participants had to locate three rewards marked by local color cues, among 12 locations distributed in a 4 m X 4 m arena. During allocentric spatial (AS trials, participants had to locate the same three rewards, in absence of local cues, based on their relations to distal environmental cues. All TD participants chose rewarded locations in LC and AS trials at above chance level. In contrast, although all but one of the participants with DS exhibited a preference for the rewarded locations in LC trials, only 50% of participants with DS chose the rewarded locations at above chance level in AS trials. As a group, participants with DS performed worse than TD children on all measures of task performance. These findings demonstrate that individuals with DS are impaired at using an allocentric spatial representation to learn and remember discrete locations in a controlled environment, suggesting persistent and pervasive deficits in hippocampus
-
A balance of activity in brain control and reward systems predicts self-regulatory outcomes
Lopez, Richard B.; Chen, Pin-Hao A.; Huckins, Jeremy F.; Hofmann, Wilhelm; Kelley, William M.; Heatherton, Todd F.
2017-01-01
Abstract Previous neuroimaging work has shown that increased reward-related activity following exposure to food cues is predictive of self-control failure. The balance model suggests that self-regulation failures result from an imbalance in reward and executive control mechanisms. However, an open question is whether the relative balance of activity in brain systems associated with executive control (vs reward) supports self-regulatory outcomes when people encounter tempting cues in daily lif...
-
A balance of activity in brain control and reward systems predicts self-regulatory outcomes.
Lopez, Richard B; Chen, Pin-Hao A; Huckins, Jeremy F; Hofmann, Wilhelm; Kelley, William M; Heatherton, Todd F
2017-05-01
Previous neuroimaging work has shown that increased reward-related activity following exposure to food cues is predictive of self-control failure. The balance model suggests that self-regulation failures result from an imbalance in reward and executive control mechanisms. However, an open question is whether the relative balance of activity in brain systems associated with executive control (vs reward) supports self-regulatory outcomes when people encounter tempting cues in daily life. Sixty-nine chronic dieters, a population known for frequent lapses in self-control, completed a food cue-reactivity task during an fMRI scanning session, followed by a weeklong sampling of daily eating behaviors via ecological momentary assessment. We related participants' food cue activity in brain systems associated with executive control and reward to real-world eating patterns. Specifically, a balance score representing the amount of activity in brain regions associated with self-regulatory control, relative to automatic reward-related activity, predicted dieters' control over their eating behavior during the following week. This balance measure may reflect individual self-control capacity and be useful for examining self-regulation success in other domains and populations. © The Author (2017). Published by Oxford University Press.
-
Learning from sensory and reward prediction errors during motor adaptation.
Izawa, Jun; Shadmehr, Reza
2011-03-01
Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual's relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.
-
Individual differences in regulatory focus predict neural response to reward.
Scult, Matthew A; Knodt, Annchen R; Hanson, Jamie L; Ryoo, Minyoung; Adcock, R Alison; Hariri, Ahmad R; Strauman, Timothy J
2017-08-01
Although goal pursuit is related to both functioning of the brain's reward circuits and psychological factors, the literatures surrounding these concepts have often been separate. Here, we use the psychological construct of regulatory focus to investigate individual differences in neural response to reward. Regulatory focus theory proposes two motivational orientations for personal goal pursuit: (1) promotion, associated with sensitivity to potential gain, and (2) prevention, associated with sensitivity to potential loss. The monetary incentive delay task was used to manipulate reward circuit function, along with instructional framing corresponding to promotion and prevention in a within-subject design. We observed that the more promotion oriented an individual was, the lower their ventral striatum response to gain cues. Follow-up analyses revealed that greater promotion orientation was associated with decreased ventral striatum response even to no-value cues, suggesting that promotion orientation may be associated with relatively hypoactive reward system function. The findings are also likely to represent an interaction between the cognitive and motivational characteristics of the promotion system with the task demands. Prevention orientation did not correlate with ventral striatum response to gain cues, supporting the discriminant validity of regulatory focus theory. The results highlight a dynamic association between individual differences in self-regulation and reward system function.
-
Intranasal oxytocin enhances socially-reinforced learning in rhesus monkeys
Directory of Open Access Journals (Sweden)
Lisa A Parr
2014-09-01
Full Text Available There are currently no drugs approved for the treatment of social deficits associated with autism spectrum disorders (ASD. One hypothesis for these deficits is that individuals with ASD lack the motivation to attend to social cues because those cues are not implicitly rewarding. Therefore, any drug that could enhance the rewarding quality of social stimuli could have a profound impact on the treatment of ASD, and other social disorders. Oxytocin (OT is a neuropeptide that has been effective in enhancing social cognition and social reward in humans. The present study examined the ability of OT to selectively enhance learning after social compared to nonsocial reward in rhesus monkeys, an important species for modeling the neurobiology of social behavior in humans. Monkeys were required to learn an implicit visual matching task after receiving either intranasal (IN OT or Placebo (saline. Correct trials were rewarded with the presentation of positive and negative social (play faces/threat faces or nonsocial (banana/cage locks stimuli, plus food. Incorrect trials were not rewarded. Results demonstrated a strong effect of socially-reinforced learning, monkeys’ performed significantly better when reinforced with social versus nonsocial stimuli. Additionally, socially-reinforced learning was significantly better and occurred faster after IN-OT compared to placebo treatment. Performance in the IN-OT, but not Placebo, condition was also significantly better when the reinforcement stimuli were emotionally positive compared to negative facial expressions. These data support the hypothesis that OT may function to enhance prosocial behavior in primates by increasing the rewarding quality of emotionally positive, social compared to emotionally negative or nonsocial images. These data also support the use of the rhesus monkey as a model for exploring the neurobiological basis of social behavior and its impairment.
-
Brain Circuits Encoding Reward from Pain Relief.
Navratilova, Edita; Atcherley, Christopher W; Porreca, Frank
2015-11-01
Relief from pain in humans is rewarding and pleasurable. Primary rewards, or reward-predictive cues, are encoded in brain reward/motivational circuits. While considerable advances have been made in our understanding of reward circuits underlying positive reinforcement, less is known about the circuits underlying the hedonic and reinforcing actions of pain relief. We review findings from electrophysiological, neuroimaging, and behavioral studies supporting the concept that the rewarding effect of pain relief requires opioid signaling in the anterior cingulate cortex (ACC), activation of midbrain dopamine neurons, and the release of dopamine in the nucleus accumbens (NAc). Understanding of circuits that govern the reward of pain relief may allow the discovery of more effective and satisfying therapies for patients with acute or chronic pain.
-
The role of BDNF, leptin, and catecholamines in reward learning in bulimia nervosa.
Homan, Philipp; Grob, Simona; Milos, Gabriella; Schnyder, Ulrich; Eckert, Anne; Lang, Undine; Hasler, Gregor
2014-12-07
A relationship between bulimia nervosa and reward-related behavior is supported by several lines of evidence. The dopaminergic dysfunctions in the processing of reward-related stimuli have been shown to be modulated by the neurotrophin brain derived neurotrophic factor (BDNF) and the hormone leptin. Using a randomized, double-blind, placebo-controlled, crossover design, a reward learning task was applied to study the behavior of 20 female subjects with remitted bulimia nervosa and 27 female healthy controls under placebo and catecholamine depletion with alpha-methyl-para-tyrosine (AMPT). The plasma levels of BDNF and leptin were measured twice during the placebo and the AMPT condition, immediately before and 1 hour after a standardized breakfast. AMPT-induced differences in plasma BDNF levels were positively correlated with the AMPT-induced differences in reward learning in the whole sample (P=.05). Across conditions, plasma brain derived neurotrophic factor levels were higher in remitted bulimia nervosa subjects compared with controls (diagnosis effect; P=.001). Plasma BDNF and leptin levels were higher in the morning before compared with after a standardized breakfast across groups and conditions (time effect; Pbulimia nervosa and controls. A role of leptin in reward learning is not supported by this study. However, leptin levels were sensitive to a depletion of catecholamine stores in both remitted bulimia nervosa and controls. © The Author 2015. Published by Oxford University Press on behalf of CINP.
-
Dopamine prediction errors in reward learning and addiction: from theory to neural circuitry
Keiflin, Ronald; Janak, Patricia H.
2015-01-01
Summary Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error-signaling and addiction can be formulated and tested. PMID:26494275
-
Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry.
Keiflin, Ronald; Janak, Patricia H
2015-10-21
Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error signaling and addiction can be formulated and tested. Copyright © 2015 Elsevier Inc. All rights reserved.
-
The Computational Development of Reinforcement Learning during Adolescence.
Directory of Open Access Journals (Sweden)
Stefano Palminteri
2016-06-01
Full Text Available Adolescence is a period of life characterised by changes in learning and decision-making. Learning and decision-making do not rely on a unitary system, but instead require the coordination of different cognitive processes that can be mathematically formalised as dissociable computational modules. Here, we aimed to trace the developmental time-course of the computational modules responsible for learning from reward or punishment, and learning from counterfactual feedback. Adolescents and adults carried out a novel reinforcement learning paradigm in which participants learned the association between cues and probabilistic outcomes, where the outcomes differed in valence (reward versus punishment and feedback was either partial or complete (either the outcome of the chosen option only, or the outcomes of both the chosen and unchosen option, were displayed. Computational strategies changed during development: whereas adolescents' behaviour was better explained by a basic reinforcement learning algorithm, adults' behaviour integrated increasingly complex computational features, namely a counterfactual learning module (enabling enhanced performance in the presence of complete feedback and a value contextualisation module (enabling symmetrical reward and punishment learning. Unlike adults, adolescent performance did not benefit from counterfactual (complete feedback. In addition, while adults learned symmetrically from both reward and punishment, adolescents learned from reward but were less likely to learn from punishment. This tendency to rely on rewards and not to consider alternative consequences of actions might contribute to our understanding of decision-making in adolescence.
-
Ross, M; Lanyon, L J; Viswanathan, J; Manoach, D S; Barton, J J S
2011-11-24
Monkey studies report greater activity in the lateral intraparietal area and more efficient saccades when targets coincide with the location of prior reward cues, even when cue location does not indicate which responses will be rewarded. This suggests that reward can modulate spatial attention and visual selection independent of the "action value" of the motor response. Our goal was first to determine whether reward modulated visual selection similarly in humans, and next, to discover whether reward and penalty differed in effect, if cue effects were greater for cognitively demanding antisaccades, and if financial consequences that were contingent on stimulus location had spatially selective effects. We found that motivational cues reduced all latencies, more for reward than penalty. There was an "inhibition-of-return"-like effect at the location of the cue, but unlike the results in monkeys, cue valence did not modify this effect in prosaccades, and the inhibition-of-return effect was slightly increased rather than decreased in antisaccades. When financial consequences were contingent on target location, locations without reward or penalty consequences lost the benefits seen in noncontingent trials, whereas locations with consequences maintained their gains. We conclude that unlike monkeys, humans show reward effects not on visual selection but on the value of actions. The human saccadic system has both the capacity to enhance responses to multiple locations simultaneously, and the flexibility to focus motivational enhancement only on locations with financial consequences. Reward is more effective than penalty, and both interact with the additional attentional demands of the antisaccade task. Copyright © 2011 IBRO. Published by Elsevier Ltd. All rights reserved.
-
Bakic, Jasmina; Pourtois, Gilles; Jepma, Marieke; Duprat, Romain; De Raedt, Rudi; Baeken, Chris
2017-01-01
Major depressive disorder (MDD) creates debilitating effects on a wide range of cognitive functions, including reinforcement learning (RL). In this study, we sought to assess whether reward processing as such, or alternatively the complex interplay between motivation and reward might potentially account for the abnormal reward-based learning in MDD. A total of 35 treatment resistant MDD patients and 44 age matched healthy controls (HCs) performed a standard probabilistic learning task. RL was titrated using behavioral, computational modeling and event-related brain potentials (ERPs) data. MDD patients showed comparable learning rate compared to HCs. However, they showed decreased lose-shift responses as well as blunted subjective evaluations of the reinforcers used during the task, relative to HCs. Moreover, MDD patients showed normal internal (at the level of error-related negativity, ERN) but abnormal external (at the level of feedback-related negativity, FRN) reward prediction error (RPE) signals during RL, selectively when additional efforts had to be made to establish learning. Collectively, these results lend support to the assumption that MDD does not impair reward processing per se during RL. Instead, it seems to alter the processing of the emotional value of (external) reinforcers during RL, when additional intrinsic motivational processes have to be engaged. © 2016 Wiley Periodicals, Inc.
-
One-trial spatial learning: wild hummingbirds relocate a reward after a single visit.
Flores-Abreu, I Nuri; Hurly, T Andrew; Healy, Susan D
2012-07-01
Beaconing to rewarded locations is typically achieved by visual recognition of the actual goal. Spatial recognition, on the other hand, can occur in the absence of the goal itself, relying instead on the landmarks surrounding the goal location. Although the duration or frequency of experiences that an animal needs to learn the landmarks surrounding a goal have been extensively studied with a variety of laboratory tasks, little is known about the way in which wild vertebrates use them in their natural environment. Here, we allowed hummingbirds to feed once only from a rewarding flower (goal) before it was removed. When we presented a similar flower at a different height in another location, birds frequently returned to the location the flower had previously occupied (spatial recognition) before flying to the flower itself (beaconing). After experiencing three rewarded flowers, each in a different location, they were more likely to beacon to the current visible flower than they were to return to previously rewarded locations (without a visible flower). These data show that hummingbirds can encode a rewarded location on the basis of the surrounding landmarks after a single visit. After multiple goal location manipulations, however, the birds changed their strategy to beaconing presumably because they had learned that the flower itself reliably signalled reward.
-
Directory of Open Access Journals (Sweden)
Xiaomeng Xu
Full Text Available Engaging in novel/exciting ("self-expanding" activities activates the mesolimbic dopamine pathway, a brain reward pathway also associated with the rewarding effects of nicotine. This suggests that self-expanding activities can potentially substitute for the reward from nicotine. We tested this model among nicotine-deprived smokers who, during fMRI scanning, played a series of two-player cooperative games with a relationship partner. Games were randomized in a 2 (self-expanding vs. not x 2 (cigarette cue present vs. absent design. Self-expansion conditions yielded significantly greater activation in a reward region (caudate than did non-self-expansion conditions. Moreover, when exposed to smoking cues during the self-expanding versus the non-self-expanding cooperative games, smokers showed less activation in a cigarette cue-reactivity region, a priori defined [temporo-parietal junction (TPJ] from a recent meta-analysis of cue-reactivity. In smoking cue conditions, increases in excitement associated with the self-expanding condition (versus the non-self-expanding condition were also negatively correlated with TPJ activation. These results support the idea that a self-expanding activity promoting reward activation attenuates cigarette cue-reactivity among nicotine-deprived smokers. Future research could focus on the parameters of self-expanding activities that produce this effect, as well as test the utility of self-expansion in clinical interventions for smoking cessation.
-
A reward optimization method based on action subrewards in hierarchical reinforcement learning.
Fu, Yuchen; Liu, Quan; Ling, Xionghong; Cui, Zhiming
2014-01-01
Reinforcement learning (RL) is one kind of interactive learning methods. Its main characteristics are "trial and error" and "related reward." A hierarchical reinforcement learning method based on action subrewards is proposed to solve the problem of "curse of dimensionality," which means that the states space will grow exponentially in the number of features and low convergence speed. The method can reduce state spaces greatly and choose actions with favorable purpose and efficiency so as to optimize reward function and enhance convergence speed. Apply it to the online learning in Tetris game, and the experiment result shows that the convergence speed of this algorithm can be enhanced evidently based on the new method which combines hierarchical reinforcement learning algorithm and action subrewards. The "curse of dimensionality" problem is also solved to a certain extent with hierarchical method. All the performance with different parameters is compared and analyzed as well.
-
Neural correlates of contextual cueing are modulated by explicit learning.
Westerberg, Carmen E; Miller, Brennan B; Reber, Paul J; Cohen, Neal J; Paller, Ken A
2011-10-01
Contextual cueing refers to the facilitated ability to locate a particular visual element in a scene due to prior exposure to the same scene. This facilitation is thought to reflect implicit learning, as it typically occurs without the observer's knowledge that scenes repeat. Unlike most other implicit learning effects, contextual cueing can be impaired following damage to the medial temporal lobe. Here we investigated neural correlates of contextual cueing and explicit scene memory in two participant groups. Only one group was explicitly instructed about scene repetition. Participants viewed a sequence of complex scenes that depicted a landscape with five abstract geometric objects. Superimposed on each object was a letter T or L rotated left or right by 90°. Participants responded according to the target letter (T) orientation. Responses were highly accurate for all scenes. Response speeds were faster for repeated versus novel scenes. The magnitude of this contextual cueing did not differ between the two groups. Also, in both groups repeated scenes yielded reduced hemodynamic activation compared with novel scenes in several regions involved in visual perception and attention, and reductions in some of these areas were correlated with response-time facilitation. In the group given instructions about scene repetition, recognition memory for scenes was superior and was accompanied by medial temporal and more anterior activation. Thus, strategic factors can promote explicit memorization of visual scene information, which appears to engage additional neural processing beyond what is required for implicit learning of object configurations and target locations in a scene. Copyright © 2011 Elsevier Ltd. All rights reserved.
-
The role of reward and reward uncertainty in episodic memory
Mason, Alice; Farrell, Simon; Howard-Jones, Paul; Ludwig, Casimir
2017-01-01
Declarative memory has been found to be sensitive to reward-related changes in the environment. The reward signal can be broken down into information regarding the expected value of the reward, reward uncertainty and the prediction error. Research has established that high as opposed to low reward values enhance declarative memory. Research in neuroscience suggests that high uncertainty activates the reward system, which could lead to enhanced learning and memory. Here we present the results ...
-
Bats without borders: Predators learn novel prey cues from other predatory species.
Patriquin, Krista J; Kohles, Jenna E; Page, Rachel A; Ratcliffe, John M
2018-03-01
Learning from others allows individuals to adapt rapidly to environmental change. Although conspecifics tend to be reliable models, heterospecifics with similar resource requirements may be suitable surrogates when conspecifics are few or unfamiliar with recent changes in resource availability. We tested whether Trachops cirrhosus , a gleaning bat that localizes prey using their mating calls, can learn about novel prey from conspecifics and the sympatric bat Lophostoma silvicolum. Specifically, we compared the rate for naïve T. cirrhosus to learn an unfamiliar tone from either a trained conspecific or heterospecific alone through trial and error or through social facilitation. T. cirrhosus learned this novel cue from L. silvicolum as quickly as from conspecifics. This is the first demonstration of social learning of a novel acoustic cue in bats and suggests that heterospecific learning may occur in nature. We propose that auditory-based social learning may help bats learn about unfamiliar prey and facilitate their adaptive radiation.
-
Directory of Open Access Journals (Sweden)
Mathieu eBaudonnat
2013-10-01
Full Text Available Dopamine has long been tightly associated with aspects of reinforcement learning and motivation in simple situations where there are a limited number of stimuli to guide behaviour and constrained range of outcomes. In naturalistic situations, however, there are many potential cues and foraging strategies that could be adopted, and it is critical that animals determine what might be behaviourally relevant in such complex environments. This requires not only detecting discrepancies with what they have recently experienced, but also identifying similarities with past experiences stored in memory. Here, we review what role dopamine might play in determining how and when to learn about the world, and how to develop choice policies appropriate to the situation faced. We discuss evidence that dopamine is shaped by motivation and memory and in turn shapes reward-based memory formation. In particular, we suggest that hippocampal-striatal-dopamine networks may interact to determine how surprising the world is and to either inhibit or promote actions at time of behavioural uncertainty.
-
Rare Neural Correlations Implement Robotic Conditioning with Delayed Rewards and Disturbances
Soltoggio, Andrea; Lemme, Andre; Reinhart, Felix; Steil, Jochen J.
2013-01-01
Neural conditioning associates cues and actions with following rewards. The environments in which robots operate, however, are pervaded by a variety of disturbing stimuli and uncertain timing. In particular, variable reward delays make it difficult to reconstruct which previous actions are responsible for following rewards. Such an uncertainty is handled by biological neural networks, but represents a challenge for computational models, suggesting the lack of a satisfactory theory for robotic neural conditioning. The present study demonstrates the use of rare neural correlations in making correct associations between rewards and previous cues or actions. Rare correlations are functional in selecting sparse synapses to be eligible for later weight updates if a reward occurs. The repetition of this process singles out the associating and reward-triggering pathways, and thereby copes with distal rewards. The neural network displays macro-level classical and operant conditioning, which is demonstrated in an interactive real-life human-robot interaction. The proposed mechanism models realistic conditioning in humans and animals and implements similar behaviors in neuro-robotic platforms. PMID:23565092
-
Cooch, Nisha K; Stalnaker, Thomas A; Wied, Heather M; Bali-Chaudhary, Sheena; McDannald, Michael A; Liu, Tzu-Lan; Schoenbaum, Geoffrey
2015-05-21
The ventral striatum has long been proposed as an integrator of biologically significant associative information to drive actions. Although inputs from the amygdala and hippocampus have been much studied, the role of prominent inputs from orbitofrontal cortex (OFC) are less well understood. Here, we recorded single-unit activity from ventral striatum core in rats with sham or ipsilateral neurotoxic lesions of lateral OFC, as they performed an odour-guided spatial choice task. Consistent with prior reports, we found that spiking activity recorded in sham rats during cue sampling was related to both reward magnitude and reward identity, with higher firing rates observed for cues that predicted more reward. Lesioned rats also showed differential activity to the cues, but this activity was unbiased towards larger rewards. These data support a role for OFC in shaping activity in the ventral striatum to represent the biological significance of associative information in the environment.
-
O'Sullivan, Sean S; Wu, Kit; Politis, Marios; Lawrence, Andrew D; Evans, Andrew H; Bose, Subrata K; Djamshidian, Atbin; Lees, Andrew J; Piccini, Paola
2011-04-01
Impulsive-compulsive behaviours are a significant source of morbidity for patients with Parkinson's disease receiving dopaminergic therapy. The development of these behaviours may reflect sensitization of the neural response to non-drug rewards, similar to that proposed for sensitization to drug rewards in addiction. Here, by using (11)C-raclopride positron emission tomography imaging, we investigated the effects of reward-related cues and L-dopa challenge in patients with Parkinson's disease with and without impulsive-compulsive behaviours on striatal levels of synaptic dopamine. Eighteen patients (11 with and seven without impulsive-compulsive behaviours) underwent three (11)C-raclopride positron emission tomography scans. The impulsive-compulsive behaviours included hypersexuality, binge eating, punding, compulsive use of dopamine replacement therapy, compulsive buying and pathological gambling, with eight patients exhibiting more than one impulsive-compulsive behaviour. There were no significant differences in baseline dopamine D2 receptor availability between the Parkinson's disease groups. No differences were found when comparing the percentage change of raclopride binding potential between the two Parkinson's disease groups following L-dopa challenge with neutral cues. The group with Parkinson's disease with impulsive-compulsive behaviours had a greater reduction of ventral striatum (11)C-raclopride binding potential following reward-related cue exposure, relative to neutral cue exposure, following L-dopa challenge (16.3% compared with 5.8% in Parkinson's disease controls, P = 0.016). The heightened response of striatal reward circuitry to heterogeneous reward-related visual cues among a group of patients with different impulsive-compulsive behaviours is consistent with a global sensitization to appetitive behaviours with dopaminergic therapy in vulnerable individuals. Our findings are relevant for the broader debate on the relation between impulsive
-
Anger and selective attention to reward and punishment in children.
He, Jie; Jin, Xinyi; Zhang, Meng; Huang, Xiang; Shui, Rende; Shen, Mowei
2013-07-01
Anger is a negative emotion associated with approach motivation and may influence children's attention preference. Three experiments examined the effect of anger on the attentional biases accompanying reward versus punishment cues in Chinese 5- and 6-year-olds. Experiment 1 tested children who were prone to report angry feelings in an unfair game. Experiment 2 measured children who were rated by parents and teachers for temperamental anger. Experiment 3 explored children who reported angry feelings in a frustrating attention task with rigged and noncontingent feedback after controlling for temperament anger. Results suggested that both the angry and anger-prone children were faster to engage attention toward the reward cues than toward the punishment cues in the three experiments. Furthermore, the angry children in the frustrating attention task (and those with poor attention focusing by parental report) were slower in disengaging attention away from the reward versus punishment cues (especially after negative feedback). Results support the approach motivation of anger, which can facilitate children's attention toward the appetitive approach-related information. The findings are discussed in terms of the adaptive and maladaptive function of anger. Copyright © 2013 Elsevier Inc. All rights reserved.
-
The prelimbic cortex directs attention toward predictive cues during fear learning.
Sharpe, Melissa J; Killcross, Simon
2015-06-01
The prelimbic cortex is argued to promote conditioned fear expression, at odds with appetitive research implicating this region in attentional processing. Consistent with an attentional account, we report that the effect of prelimbic lesions on fear expression depends on the degree of competition between contextual and discrete cues. Further, when competition from contextual cues is low, we found that PL inactivation resulted in animals expressing fear toward irrelevant discrete cues; an effect selective to inactivation during the learning phase and not during retrieval. These data demonstrate that the prelimbic cortex modulates attention toward cues to preferentially direct fear responding on the basis of their predictive value. © 2015 Sharpe and Killcross; Published by Cold Spring Harbor Laboratory Press.
-
Skvortsova, Vasilisa; Degos, Bertrand; Welter, Marie-Laure; Vidailhet, Marie; Pessiglione, Mathias
2017-06-21
Instrumental learning is a fundamental process through which agents optimize their choices, taking into account various dimensions of available options such as the possible reward or punishment outcomes and the costs associated with potential actions. Although the implication of dopamine in learning from choice outcomes is well established, less is known about its role in learning the action costs such as effort. Here, we tested the ability of patients with Parkinson's disease (PD) to maximize monetary rewards and minimize physical efforts in a probabilistic instrumental learning task. The implication of dopamine was assessed by comparing performance ON and OFF prodopaminergic medication. In a first sample of PD patients ( n = 15), we observed that reward learning, but not effort learning, was selectively impaired in the absence of treatment, with a significant interaction between learning condition (reward vs effort) and medication status (OFF vs ON). These results were replicated in a second, independent sample of PD patients ( n = 20) using a simplified version of the task. According to Bayesian model selection, the best account for medication effects in both studies was a specific amplification of reward magnitude in a Q-learning algorithm. These results suggest that learning to avoid physical effort is independent from dopaminergic circuits and strengthen the general idea that dopaminergic signaling amplifies the effects of reward expectation or obtainment on instrumental behavior. SIGNIFICANCE STATEMENT Theoretically, maximizing reward and minimizing effort could involve the same computations and therefore rely on the same brain circuits. Here, we tested whether dopamine, a key component of reward-related circuitry, is also implicated in effort learning. We found that patients suffering from dopamine depletion due to Parkinson's disease were selectively impaired in reward learning, but not effort learning. Moreover, anti-parkinsonian medication restored the
-
Pragmatically Framed Cross-Situational Noun Learning Using Computational Reinforcement Models.
Najnin, Shamima; Banerjee, Bonny
2018-01-01
Cross-situational learning and social pragmatic theories are prominent mechanisms for learning word meanings (i.e., word-object pairs). In this paper, the role of reinforcement is investigated for early word-learning by an artificial agent. When exposed to a group of speakers, the agent comes to understand an initial set of vocabulary items belonging to the language used by the group. Both cross-situational learning and social pragmatic theory are taken into account. As social cues, joint attention and prosodic cues in caregiver's speech are considered. During agent-caregiver interaction, the agent selects a word from the caregiver's utterance and learns the relations between that word and the objects in its visual environment. The "novel words to novel objects" language-specific constraint is assumed for computing rewards. The models are learned by maximizing the expected reward using reinforcement learning algorithms [i.e., table-based algorithms: Q-learning, SARSA, SARSA-λ, and neural network-based algorithms: Q-learning for neural network (Q-NN), neural-fitted Q-network (NFQ), and deep Q-network (DQN)]. Neural network-based reinforcement learning models are chosen over table-based models for better generalization and quicker convergence. Simulations are carried out using mother-infant interaction CHILDES dataset for learning word-object pairings. Reinforcement is modeled in two cross-situational learning cases: (1) with joint attention (Attentional models), and (2) with joint attention and prosodic cues (Attentional-prosodic models). Attentional-prosodic models manifest superior performance to Attentional ones for the task of word-learning. The Attentional-prosodic DQN outperforms existing word-learning models for the same task.
-
Probability matching in perceptrons: Effects of conditional dependence and linear nonseparability.
Directory of Open Access Journals (Sweden)
Michael R W Dawson
Full Text Available Probability matching occurs when the behavior of an agent matches the likelihood of occurrence of events in the agent's environment. For instance, when artificial neural networks match probability, the activity in their output unit equals the past probability of reward in the presence of a stimulus. Our previous research demonstrated that simple artificial neural networks (perceptrons, which consist of a set of input units directly connected to a single output unit learn to match probability when presented different cues in isolation. The current paper extends this research by showing that perceptrons can match probabilities when presented simultaneous cues, with each cue signaling different reward likelihoods. In our first simulation, we presented up to four different cues simultaneously; the likelihood of reward signaled by the presence of one cue was independent of the likelihood of reward signaled by other cues. Perceptrons learned to match reward probabilities by treating each cue as an independent source of information about the likelihood of reward. In a second simulation, we violated the independence between cues by making some reward probabilities depend upon cue interactions. We did so by basing reward probabilities on a logical combination (AND or XOR of two of the four possible cues. We also varied the size of the reward associated with the logical combination. We discovered that this latter manipulation was a much better predictor of perceptron performance than was the logical structure of the interaction between cues. This indicates that when perceptrons learn to match probabilities, they do so by assuming that each signal of a reward is independent of any other; the best predictor of perceptron performance is a quantitative measure of the independence of these input signals, and not the logical structure of the problem being learned.
-
Probability matching in perceptrons: Effects of conditional dependence and linear nonseparability
2017-01-01
Probability matching occurs when the behavior of an agent matches the likelihood of occurrence of events in the agent’s environment. For instance, when artificial neural networks match probability, the activity in their output unit equals the past probability of reward in the presence of a stimulus. Our previous research demonstrated that simple artificial neural networks (perceptrons, which consist of a set of input units directly connected to a single output unit) learn to match probability when presented different cues in isolation. The current paper extends this research by showing that perceptrons can match probabilities when presented simultaneous cues, with each cue signaling different reward likelihoods. In our first simulation, we presented up to four different cues simultaneously; the likelihood of reward signaled by the presence of one cue was independent of the likelihood of reward signaled by other cues. Perceptrons learned to match reward probabilities by treating each cue as an independent source of information about the likelihood of reward. In a second simulation, we violated the independence between cues by making some reward probabilities depend upon cue interactions. We did so by basing reward probabilities on a logical combination (AND or XOR) of two of the four possible cues. We also varied the size of the reward associated with the logical combination. We discovered that this latter manipulation was a much better predictor of perceptron performance than was the logical structure of the interaction between cues. This indicates that when perceptrons learn to match probabilities, they do so by assuming that each signal of a reward is independent of any other; the best predictor of perceptron performance is a quantitative measure of the independence of these input signals, and not the logical structure of the problem being learned. PMID:28212422
-
Probability matching in perceptrons: Effects of conditional dependence and linear nonseparability.
Dawson, Michael R W; Gupta, Maya
2017-01-01
Probability matching occurs when the behavior of an agent matches the likelihood of occurrence of events in the agent's environment. For instance, when artificial neural networks match probability, the activity in their output unit equals the past probability of reward in the presence of a stimulus. Our previous research demonstrated that simple artificial neural networks (perceptrons, which consist of a set of input units directly connected to a single output unit) learn to match probability when presented different cues in isolation. The current paper extends this research by showing that perceptrons can match probabilities when presented simultaneous cues, with each cue signaling different reward likelihoods. In our first simulation, we presented up to four different cues simultaneously; the likelihood of reward signaled by the presence of one cue was independent of the likelihood of reward signaled by other cues. Perceptrons learned to match reward probabilities by treating each cue as an independent source of information about the likelihood of reward. In a second simulation, we violated the independence between cues by making some reward probabilities depend upon cue interactions. We did so by basing reward probabilities on a logical combination (AND or XOR) of two of the four possible cues. We also varied the size of the reward associated with the logical combination. We discovered that this latter manipulation was a much better predictor of perceptron performance than was the logical structure of the interaction between cues. This indicates that when perceptrons learn to match probabilities, they do so by assuming that each signal of a reward is independent of any other; the best predictor of perceptron performance is a quantitative measure of the independence of these input signals, and not the logical structure of the problem being learned.
-
Amita, Hidetoshi; Matsushima, Toshiya
2014-07-15
To investigate the role of social contexts in controlling the neuronal representation of food reward, we recorded single neuron activity in the medial striatum/nucleus accumbens of domestic chicks and examined whether activities differed between two blocks with different contexts. Chicks were trained in an operant task to associate light-emitting diode color cues with three trial types that differed in the type of food reward: no reward (S-), a small reward/short-delay option (SS), and a large reward/long-delay alternative (LL). Amount and duration of reward were set such that both of SS and LL were chosen roughly equally. Neurons showing distinct cue-period activity in rewarding trials (SS and LL) were identified during an isolation block, and activity patterns were compared with those recorded from the same neuron during a subsequent pseudo-competition block in which another chick was allowed to forage in the same area, but was separated by a transparent window. In some neurons, cue-period activity was lower in the pseudo-competition block, and the difference was not ascribed to the number of repeated trials. Comparison at neuronal population level revealed statistically significant suppression in the pseudo-competition block in both SS and LL trials, suggesting that perceived competition generally suppressed the representation of cue-associated food reward. The delay- and reward-period activities, however, did not significantly different between blocks. These results demonstrate that visual perception of a competitive forager per se weakens the neuronal representation of predicted food reward. Possible functional links to impulse control are discussed. Copyright © 2014 Elsevier B.V. All rights reserved.
-
The Responses of Young Domestic Horses to Human-Given Cues
Proops, Leanne; Rayner, Jenny; Taylor, Anna M.; McComb, Karen
2013-01-01
It has been suggested that the process of domestication, at least in some species, has led to an innate predisposition to be skilled at reading human communicative and attentional cues. Adult domestic horses (Equus caballus) are highly sensitive to subtle bodily cues when determining if a person is attending to them but they are less adept at using human communicative cues in object choice tasks. Here we provide the first study into the ontogeny of such skills in order to gain insights into the mechanisms underlying these abilities. Compared with adult horses, youngsters under the age of three could use body orientation but not more subtle cues such as head movement and open/closed eyes to correctly choose an attentive person to approach for food. Across two object choice experiments, the performance of young horses was comparable to that of adult horses – subjects were able to correctly choose a rewarded bucket using marker placement, pointing and touching cues but could not use body orientation, gaze, elbow pointing or tapping cues. Taken together these results do not support the theory that horses possess an innate predisposition to be particularly skilled at using human cues. Horses' ability to determine whether humans are attending to them using subtle body cues appears to require significant experience to fully develop and their perhaps less remarkable use of limited cues in object choice tasks, although present at a much earlier age, is likely to reflect a more general learning ability related to stimulus enhancement rather than a specific ‘human-reading’ skill. PMID:23840572
-
'You see?' Teaching and learning how to interpret visual cues during surgery.
Cope, Alexandra C; Bezemer, Jeff; Kneebone, Roger; Lingard, Lorelei
2015-11-01
The ability to interpret visual cues is important in many medical specialties, including surgery, in which poor outcomes are largely attributable to errors of perception rather than poor motor skills. However, we know little about how trainee surgeons learn to make judgements in the visual domain. We explored how trainees learn visual cue interpretation in the operating room. A multiple case study design was used. Participants were postgraduate surgical trainees and their trainers. Data included observer field notes, and integrated video- and audio-recordings from 12 cases representing more than 11 hours of observation. A constant comparative methodology was used to identify dominant themes. Visual cue interpretation was a recurrent feature of trainer-trainee interactions and was achieved largely through the pedagogic mechanism of co-construction. Co-construction was a dialogic sequence between trainer and trainee in which they explored what they were looking at together to identify and name structures or pathology. Co-construction took two forms: 'guided co-construction', in which the trainer steered the trainee to see what the trainer was seeing, and 'authentic co-construction', in which neither trainer nor trainee appeared certain of what they were seeing and pieced together the information collaboratively. Whether the co-construction activity was guided or authentic appeared to be influenced by case difficulty and trainee seniority. Co-construction was shown to occur verbally, through discussion, and also through non-verbal exchanges in which gestures made with laparoscopic instruments contributed to the co-construction discourse. In the training setting, learning visual cue interpretation occurs in part through co-construction. Co-construction is a pedagogic phenomenon that is well recognised in the context of learning to interpret verbal information. In articulating the features of co-construction in the visual domain, this work enables the development of
-
Wild rufous hummingbirds use local landmarks to return to rewarded locations.
Pritchard, David J; Scott, Renee D; Healy, Susan D; Hurly, Andrew T
2016-01-01
Animals may remember an important location with reference to one or more visual landmarks. In the laboratory, birds and mammals often preferentially use landmarks near a goal ("local landmarks") to return to that location at a later date. Although we know very little about how animals in the wild use landmarks to remember locations, mammals in the wild appear to prefer to use distant landmarks to return to rewarded locations. To examine what cues wild birds use when returning to a goal, we trained free-living hummingbirds to search for a reward at a location that was specified by three nearby visual landmarks. Following training we expanded the landmark array to test the extent that the birds relied on the local landmarks to return to the reward. During the test the hummingbirds' search was best explained by the birds having used the experimental landmarks to remember the reward location. How the birds used the landmarks was not clear and seemed to change over the course of each test. These wild hummingbirds, then, can learn locations in reference to nearby visual landmarks. Copyright © 2015 Elsevier B.V. All rights reserved.
-
Directory of Open Access Journals (Sweden)
Lara Maria Rangel
2015-07-01
Full Text Available The hippocampus is an important structure for learning and memory processes, and has strong rhythmic activity. Although a large amount of research has been dedicated towards understanding the rhythmic activity in the hippocampus during exploratory behaviors, specifically in the theta (5-10 Hz frequency range, few studies have examined the temporal interplay of theta and other frequencies during the presentation of meaningful cues. We obtained in vivo electrophysiological recordings of local field potentials (LFP in the dentate gyrus (DG of the hippocampus as rats performed three different associative learning tasks. In each task, cue presentations elicited pronounced decrements in theta amplitude in conjunction with increases in beta (15-30Hz amplitude. These changes were often transient but were sustained from the onset of cue encounters until the occurrence of a reward outcome. This oscillatory profile shifted in time to precede cue encounters over the course of the session, and was not present during similar behavior in the absence of task relevant stimuli. The observed decreases in theta amplitude and increases in beta amplitude in the dentate gyrus may thus reflect a shift in processing state that occurs when encountering meaningful cues.
-
Blum, Kenneth; Gardner, Eliot; Oscar-Berman, Marlene; Gold, Mark
2012-01-01
In an attempt to resolve controversy regarding the causal contributions of mesolimbic dopamine (DA) systems to reward, we evaluate the three main competing explanatory categories: “liking,” “learning,” and “wanting” [1]. That is, DA may mediate (a) the hedonic impact of reward (liking), (b) learned predictions about rewarding effects (learning), or (c) the pursuit of rewards by attributing incentive salience to reward-related stimuli (wanting). We evaluate these hypotheses, especially as they...
-
BAS-drive trait modulates dorsomedial striatum activity during reward response-outcome associations.
Costumero, Víctor; Barrós-Loscertales, Alfonso; Fuentes, Paola; Rosell-Negre, Patricia; Bustamante, Juan Carlos; Ávila, César
2016-09-01
According to the Reinforcement Sensitivity Theory, behavioral studies have found that individuals with stronger reward sensitivity easily detect cues of reward and establish faster associations between instrumental responses and reward. Neuroimaging studies have shown that processing anticipatory cues of reward is accompanied by stronger ventral striatum activity in individuals with stronger reward sensitivity. Even though establishing response-outcome contingencies has been consistently associated with dorsal striatum, individual differences in this process are poorly understood. Here, we aimed to study the relation between reward sensitivity and brain activity while processing response-reward contingencies. Forty-five participants completed the BIS/BAS questionnaire and performed a gambling task paradigm in which they received monetary rewards or punishments. Overall, our task replicated previous results that have related processing high reward outcomes with activation of striatum and medial frontal areas, whereas processing high punishment outcomes was associated with stronger activity in insula and middle cingulate. As expected, the individual differences in the activity of dorsomedial striatum correlated positively with BAS-Drive. Our results agree with previous studies that have related the dorsomedial striatum with instrumental performance, and suggest that the individual differences in this area may form part of the neural substrate responsible for modulating instrumental conditioning by reward sensitivity.
-
Saez, Rebecca A; Saez, Alexandre; Paton, Joseph J; Lau, Brian; Salzman, C Daniel
2017-07-05
The same reward can possess different motivational meaning depending upon its magnitude relative to other rewards. To study the neurophysiological mechanisms mediating assignment of motivational meaning, we recorded the activity of neurons in the amygdala and orbitofrontal cortex (OFC) of monkeys during a Pavlovian task in which the relative amount of liquid reward associated with one conditioned stimulus (CS) was manipulated by changing the reward amount associated with a second CS. Anticipatory licking tracked relative reward magnitude, implying that monkeys integrated information about recent rewards to adjust the motivational meaning of a CS. Upon changes in relative reward magnitude, neural responses to reward-predictive cues updated more rapidly in OFC than amygdala, and activity in OFC but not the amygdala was modulated by recent reward history. These results highlight a distinction between the amygdala and OFC in assessing reward history to support the flexible assignment of motivational meaning to sensory cues. Copyright © 2017 Elsevier Inc. All rights reserved.
-
Wood, Lara A; Whiten, Andrew
2017-11-01
Animal social learning is typically studied experimentally by the presentation of artificial foraging tasks. Although productive, results are often variable even for the same species. We present and test the hypothesis that one cause of variation is that spatial distance between rewards and the means of reward release causes conflicts for participants' attentional focus. We investigated whether spatial contiguity between a visible reward and the means of release would affect behavioral responses that evidence social learning, testing 21 brown capuchins ( Sapajus apella ), a much-studied species with variant evidence for social learning, and one hundred eighty 2- to 4-year-old human children ( Homo sapiens ), a benchmark species known for a strong social learning disposition. Participants were presented with a novel transparent apparatus where a reward was either proximal or distal to a demonstrated means of releasing it. A distal reward location decreased attention toward the location of the demonstration and impaired subsequent success in gaining rewards. Generally, the capuchins produced the alternative method to that demonstrated, whereas children copied the method demonstrated, although a distal reward location reduced copying in younger children. We conclude that some design features in common social learning tasks may significantly degrade the evidence for social learning. We have demonstrated this for 2 different primates but suggest that it is a significant factor to control for in social learning research across all taxa. (PsycINFO Database Record (c) 2017 APA, all rights reserved).
-
Toward a common theory for learning from reward, affect, and motivation: the SIMON framework.
Madan, Christopher R
2013-10-07
While the effects of reward, affect, and motivation on learning have each developed into their own fields of research, they largely have been investigated in isolation. As all three of these constructs are highly related, and use similar experimental procedures, an important advance in research would be to consider the interplay between these constructs. Here we first define each of the three constructs, and then discuss how they may influence each other within a common framework. Finally, we delineate several sources of evidence supporting the framework. By considering the constructs of reward, affect, and motivation within a single framework, we can develop a better understanding of the processes involved in learning and how they interplay, and work toward a comprehensive theory that encompasses reward, affect, and motivation.
-
Female hummingbirds do not relocate rewards using colour cues
Tello Ramos, Maria Cristina; Hurly, T. Andrew; Healy, Susan D.
2014-01-01
This research was supported by CONACYT (The Mexican National Council for Science and Technology) grant number: 310717, the University of Lethbridge and the Natural Sciences and Engineering Research Council of Canada (grant number: RGPIN 121496-2003) and the University of St Andrew's Russell Trust Award. Males generally outperform females in spatial tasks. This difference in spatial performance may reflect differences in cue preference because males often use both spatial cues 9distance and...
-
van den Akker, Karolien; Havermans, Remco C; Bouton, Mark E; Jansen, Anita
2014-10-01
Animals and humans can easily learn to associate an initially neutral cue with food intake through classical conditioning, but extinction of learned appetitive responses can be more difficult. Intermittent or partial reinforcement of food cues causes especially persistent behaviour in animals: after exposure to such learning schedules, the decline in responding that occurs during extinction is slow. After extinction, increases in responding with renewed reinforcement of food cues (reacquisition) might be less rapid after acquisition with partial reinforcement. In humans, it may be that the eating behaviour of some individuals resembles partial reinforcement schedules to a greater extent, possibly affecting dieting success by interacting with extinction and reacquisition. Furthermore, impulsivity has been associated with less successful dieting, and this association might be explained by impulsivity affecting the learning and extinction of appetitive responses. In the present two studies, the effects of different reinforcement schedules and impulsivity on the acquisition, extinction, and reacquisition of appetitive responses were investigated in a conditioning paradigm involving food rewards in healthy humans. Overall, the results indicate both partial reinforcement schedules and, possibly, impulsivity to be associated with worse extinction performance. A new model of dieting success is proposed: learning histories and, perhaps, certain personality traits (impulsivity) can interfere with the extinction and reacquisition of appetitive responses to food cues and they may be causally related to unsuccessful dieting. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Theta-band phase locking of orbitofrontal neurons during reward expectancy
van Wingerden, M.; Vinck, M.; Lankelma, J.; Pennartz, C.M.A.
2010-01-01
The expectancy of a rewarding outcome following actions and cues is coded by a network of brain structures including the orbitofrontal cortex. Thus far, predicted reward was considered to be coded by time-averaged spike rates of neurons. However, besides firing rate, the precise timing of action
-
Tactile learning and the individual evaluation of the reward in honey bees (Apis mellifera L.).
Scheiner, R; Erber, J; Page, R E
1999-07-01
Using the proboscis extension response we conditioned pollen and nectar foragers of the honey bee (Apis mellifera L.) to tactile patterns under laboratory conditions. Pollen foragers demonstrated better acquisition, extinction, and reversal learning than nectar foragers. We tested whether the known differences in response thresholds to sucrose between pollen and nectar foragers could explain the observed differences in learning and found that nectar foragers with low response thresholds performed better during acquisition and extinction than ones with higher thresholds. Conditioning pollen and nectar foragers with similar response thresholds did not yield differences in their learning performance. These results suggest that differences in the learning performance of pollen and nectar foragers are a consequence of differences in their perception of sucrose. Furthermore, we analysed the effect which the perception of sucrose reward has on associative learning. Nectar foragers with uniform low response thresholds were conditioned using varying concentrations of sucrose. We found significant positive correlations between the concentrations of the sucrose rewards and the performance during acquisition and extinction. The results are summarised in a model which describes the relationships between learning performance, response threshold to sucrose, concentration of sucrose and the number of rewards.
-
Directory of Open Access Journals (Sweden)
Xiaomeng Xu
Full Text Available Self-expanding experiences like falling in love or engaging in novel, exciting and interesting activities activate the same brain reward mechanism (mesolimbic dopamine pathway that reinforces drug use and abuse, including tobacco smoking. This suggests the possibility that reward from smoking is substitutable by self-expansion (through competition with the same neural system, potentially aiding cessation efforts. Using a model of self-expansion in the context of romantic love, the present fMRI experiment examined whether, among nicotine-deprived smokers, relationship self-expansion is associated with deactivation of cigarette cue-reactivity regions. Results indicated that among participants who were experiencing moderate levels of craving, cigarette cue-reactivity regions (e.g., cuneus and posterior cingulate cortex showed significantly less activation during self-expansion conditions compared with control conditions. These results provide evidence that rewards from one domain (self-expansion can act as a substitute for reward from another domain (nicotine to attenuate cigarette cue reactivity.
-
Xu, Xiaomeng; Wang, Jin; Aron, Arthur; Lei, Wei; Westmaas, J Lee; Weng, Xuchu
2012-01-01
Self-expanding experiences like falling in love or engaging in novel, exciting and interesting activities activate the same brain reward mechanism (mesolimbic dopamine pathway) that reinforces drug use and abuse, including tobacco smoking. This suggests the possibility that reward from smoking is substitutable by self-expansion (through competition with the same neural system), potentially aiding cessation efforts. Using a model of self-expansion in the context of romantic love, the present fMRI experiment examined whether, among nicotine-deprived smokers, relationship self-expansion is associated with deactivation of cigarette cue-reactivity regions. Results indicated that among participants who were experiencing moderate levels of craving, cigarette cue-reactivity regions (e.g., cuneus and posterior cingulate cortex) showed significantly less activation during self-expansion conditions compared with control conditions. These results provide evidence that rewards from one domain (self-expansion) can act as a substitute for reward from another domain (nicotine) to attenuate cigarette cue reactivity.
-
Soeter, Marieke; Kindt, Merel
2015-01-01
Disrupting the process of memory reconsolidation may point to a novel therapeutic strategy for the permanent reduction of fear in patients suffering from anxiety disorders. However both in animal and human studies the retrieval cue typically involves a re-exposure to the original fear-conditioned stimulus (CS). A relevant question is whether abstract cues not directly associated with the threat event also trigger reconsolidation, given that anxiety disorders often result from vicarious or unobtrusive learning for which no explicit memory exists. Insofar as the fear memory involves a flexible representation of the original learning experience, we hypothesized that the process of memory reconsolidation may also be triggered by abstract cues. We addressed this hypothesis by using a differential human fear-conditioning procedure in two distinct fear-learning groups. We predicted that if fear learning involves discrimination on basis of perceptual cues within one semantic category (i.e., the perceptual-learning group, n = 15), the subsequent ambiguity of the abstract retrieval cue would not trigger memory reconsolidation. In contrast, if fear learning involves discriminating between two semantic categories (i.e., categorical-learning group, n = 15), an abstract retrieval cue would unequivocally reactivate the fear memory and might subsequently trigger memory reconsolidation. Here we show that memory reconsolidation may indeed be triggered by another cue than the one that was present during the original learning occasion, but this effect depends on the learning history. Evidence for fear memory reconsolidation was inferred from the fear-erasing effect of one pill of propranolol (40 mg) systemically administered upon exposure to the abstract retrieval cue. Our finding that reconsolidation of a specific fear association does not require exposure to the original retrieval cue supports the feasibility of reconsolidation-based interventions for emotional disorders.
-
Directory of Open Access Journals (Sweden)
Marieke eSoeter
2015-05-01
Full Text Available Disrupting the process of memory reconsolidation may point to a novel therapeutic strategy for the permanent reduction of fear in patients suffering from anxiety disorders. However both in animal and human studies the retrieval cue typically involves a re-exposure to the original fear-conditioned stimulus. A relevant question is whether abstract cues not directly associated with the threat event also trigger reconsolidation, given that anxiety disorders often result from vicarious or unobtrusive learning for which no explicit memory exists. Insofar as the fear memory involves a flexible representation of the original learning experience, we hypothesized that the process of memory reconsolidation may also be triggered by abstract cues. We addressed this hypothesis by using a differential human fear-conditioning procedure in two distinct fear-learning groups. We predicted that if fear learning involves discrimination on basis of perceptual cues within one semantic category (i.e., the perceptual-learning group, n = 15, the subsequent ambiguity of the abstract retrieval cue would not trigger memory reconsolidation. In contrast, if fear learning involves discriminating between two semantic categories (i.e., categorical-learning group, n = 15, an abstract retrieval cue would unequivocally reactivate the fear memory and might subsequently trigger memory reconsolidation. Here we show that memory reconsolidation may indeed be triggered by another cue than the one that was present during the original learning occasion, but this effect depends on the learning history. Evidence for fear memory reconsolidation was inferred from the fear-erasing effect of one pill of propranolol (40 mg systemically administered upon exposure to the abstract retrieval cue. Our finding that reconsolidation of a specific fear association does not require exposure to the original retrieval cue supports the feasibility of reconsolidation-based interventions for emotional disorders.
-
Fear Conditioning Effects on Sensitivity to Drug Reward
2010-06-01
motivational responses and self-administration behaviors (Robbins et al., 2008). Pavlovian conditioning mechanisms link unconditioned drug responses...model. Induction of fear conditioning is followed by measurement of sensitivity to drug reward using a conditioned place preference (CPP) model to...morphine. Conditioned drug reward is a relevant model in addiction because environmental cues (e.g. a barroom) induce craving and persistent
-
DEFF Research Database (Denmark)
Dasgupta, Sakyasingha; Wörgötter, Florentin; Manoonpong, Poramate
2014-01-01
Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational...... and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role...... in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We...
-
Hellberg, Samantha N; Levit, Jeremy D; Robinson, Mike J F
2018-01-30
Gambling disorder (GD) frequently co-occurs with alcohol use and anxiety disorders, suggesting possible shared mechanisms. Recent research suggests reward uncertainty may powerfully enhance attraction towards reward cues. Here, we examined the effects of adolescent ethanol exposure, anxiety, and reward uncertainty on cue-triggered motivation. Male and female adolescent rats were given free access to ethanol or control jello for 20days. Following withdrawal, rats underwent autoshaping on a certain (100%-1) or uncertain (50%-1-2-3) reward contingency, followed by single-session conditioned reinforcement and progressive ratio tasks, and 7days of omission training, during which lever pressing resulted in omission of reward. Finally, anxiety levels were quantified on the elevated plus maze. Here, we found that uncertainty narrowed cue attraction by significantly increasing the ratio of sign-tracking to goal-tracking, particularly amongst control jello and high anxiety animals, but not in animals exposed to ethanol during adolescence. In addition, attentional bias towards the lever cue was more persistent under uncertain conditions following omission training. We also found that females consumed more ethanol, and that uncertainty mitigated the anxiolytic effects of ethanol exposure observed in high ethanol intake animals under certainty conditions. Our results further support that reward uncertainty biases attraction towards reward cues, suggesting also that heightened anxiety may enhance vulnerability to the effects of reward uncertainty. Chronic, elevated alcohol consumption may contribute to heightened anxiety levels, while high anxiety may promote the over-attribution of incentive value to reward cues, highlighting possible mechanisms that may drive concurrent anxiety, heavy drinking, and problematic gambling. Copyright © 2017 Elsevier B.V. All rights reserved.
-
Pattern of access determines influence of junk food diet on cue sensitivity and palatability.
Kosheleff, Alisa R; Araki, Jingwen; Hsueh, Jennifer; Le, Andrew; Quizon, Kevin; Ostlund, Sean B; Maidment, Nigel T; Murphy, Niall P
2018-04-01
Like drug addiction, cues associated with palatable foods can trigger food-seeking, even when sated. However, whether susceptibility to the motivating influence of food-related cues is a predisposing factor in overeating or a consequence of poor diet is difficult to determine in humans. Using a rodent model, we explored whether a highly palatable 'junk food' diet impacts responses to reward-paired cues in a Pavlovian-to-instrumental transfer test, using sweetened condensed milk (SCM) as the reward. The hedonic impact of SCM consumption was also assessed by analyzing licking microstructure. To probe the effects of pattern and duration of junk food exposure, we provided rats with either regular chow ad libitum (controls) or chow plus access to junk food for either 2 or 24 h per day for 1, 3, or 6 weeks. We also examined how individual susceptibility to weight gain related to these measures. Rats provided 24 h access to the junk food diet were insensitive to the motivational effects of a SCM-paired cue when tested sated even though their hedonic experience upon reward consumption was similar to controls. In contrast, rats provided restricted, 2 h access to junk food exhibited a cue generalization phenotype under sated conditions, lever-pressing with increased vigor in response to both a SCM-paired cue, and a cue not previously paired with reward. Hedonic response was also significantly higher in these animals relative to controls. These data demonstrate that the pattern of junk food exposure differentially alters the hedonic impact of palatable foods and susceptibility to the motivating influence of cues in the environment to promote food-seeking actions when sated, which may be consequential for understanding overeating and obesity. Copyright © 2017 Elsevier Ltd. All rights reserved.
-
Effects of reward on the accuracy and dynamics of smooth pursuit eye movements.
Brielmann, Aenne A; Spering, Miriam
2015-08-01
Reward modulates behavioral choices and biases goal-oriented behavior, such as eye or hand movements, toward locations or stimuli associated with higher rewards. We investigated reward effects on the accuracy and timing of smooth pursuit eye movements in 4 experiments. Eye movements were recorded in participants tracking a moving visual target on a computer monitor. Before target motion onset, a monetary reward cue indicated whether participants could earn money by tracking accurately, or whether the trial was unrewarded (Experiments 1 and 2, n = 11 each). Reward significantly improved eye-movement accuracy across different levels of task difficulty. Improvements were seen even in the earliest phase of the eye movement, within 70 ms of tracking onset, indicating that reward impacts visual-motor processing at an early level. We obtained similar findings when reward was not precued but explicitly associated with the pursuit target (Experiment 3, n = 16); critically, these results were not driven by stimulus prevalence or other factors such as preparation or motivation. Numerical cues (Experiment 4, n = 9) were not effective. (c) 2015 APA, all rights reserved).
-
Learning and generalization from reward and punishment in opioid addiction.
Myers, Catherine E; Rego, Janice; Haber, Paul; Morley, Kirsten; Beck, Kevin D; Hogarth, Lee; Moustafa, Ahmed A
2017-01-15
This study adapts a widely-used acquired equivalence paradigm to investigate how opioid-addicted individuals learn from positive and negative feedback, and how they generalize this learning. The opioid-addicted group consisted of 33 participants with a history of heroin dependency currently in a methadone maintenance program; the control group consisted of 32 healthy participants without a history of drug addiction. All participants performed a novel variant of the acquired equivalence task, where they learned to map some stimuli to correct outcomes in order to obtain reward, and to map other stimuli to correct outcomes in order to avoid punishment; some stimuli were implicitly "equivalent" in the sense of being paired with the same outcome. On the initial training phase, both groups performed similarly on learning to obtain reward, but as memory load grew, the control group outperformed the addicted group on learning to avoid punishment. On a subsequent testing phase, the addicted and control groups performed similarly on retention trials involving previously-trained stimulus-outcome pairs, as well as on generalization trials to assess acquired equivalence. Since prior work with acquired equivalence tasks has associated stimulus-outcome learning with the nigrostriatal dopamine system, and generalization with the hippocampal region, the current results are consistent with basal ganglia dysfunction in the opioid-addicted patients. Further, a selective deficit in learning from punishment could contribute to processes by which addicted individuals continue to pursue drug use even at the cost of negative consequences such as loss of income and the opportunity to engage in other life activities. Published by Elsevier B.V.
-
Conflict Adaptation and Cue Competition during Learning in an Eriksen Flanker Task
Ghinescu, Rodica; Ramsey, Ashley K.; Gratton, Gabriele; Fabiani, Monica
2016-01-01
Two experiments investigated competition between cues that predicted the correct target response to a target stimulus in a response conflict procedure using a flanker task. Subjects received trials with five-character arrays with a central target character and distractor flanker characters that matched (compatible) or did not match (incompatible) the central target. Subjects’ expectancies for compatible and incompatible trials were manipulated by presenting pre-trial cues that signaled the occurrence of compatible or incompatible trials. On some trials, a single cue predicted the target stimulus and the required target response. On other trials, a second redundant, predictive cue was also present on such trials. The results showed an effect of competition between cues for control over strategic responding to the target stimuli, a finding that is predicted by associative learning theories. The finding of competition between pre-trial cues that predict incompatible trials, but not cues that predict compatible trials, suggests that different strategic processes may occur during adaptation to conflict when different kinds of trials are expected. PMID:27941977
-
Functional requirements for reward-modulated spike-timing-dependent plasticity.
Frémaux, Nicolas; Sprekeler, Henning; Gerstner, Wulfram
2010-10-06
Recent experiments have shown that spike-timing-dependent plasticity is influenced by neuromodulation. We derive theoretical conditions for successful learning of reward-related behavior for a large class of learning rules where Hebbian synaptic plasticity is conditioned on a global modulatory factor signaling reward. We show that all learning rules in this class can be separated into a term that captures the covariance of neuronal firing and reward and a second term that presents the influence of unsupervised learning. The unsupervised term, which is, in general, detrimental for reward-based learning, can be suppressed if the neuromodulatory signal encodes the difference between the reward and the expected reward-but only if the expected reward is calculated for each task and stimulus separately. If several tasks are to be learned simultaneously, the nervous system needs an internal critic that is able to predict the expected reward for arbitrary stimuli. We show that, with a critic, reward-modulated spike-timing-dependent plasticity is capable of learning motor trajectories with a temporal resolution of tens of milliseconds. The relation to temporal difference learning, the relevance of block-based learning paradigms, and the limitations of learning with a critic are discussed.
-
Learning stochastic reward distributions in a speeded pointing task.
Seydell, Anna; McCann, Brian C; Trommershäuser, Julia; Knill, David C
2008-04-23
Recent studies have shown that humans effectively take into account task variance caused by intrinsic motor noise when planning fast hand movements. However, previous evidence suggests that humans have greater difficulty accounting for arbitrary forms of stochasticity in their environment, both in economic decision making and sensorimotor tasks. We hypothesized that humans can learn to optimize movement strategies when environmental randomness can be experienced and thus implicitly learned over several trials, especially if it mimics the kinds of randomness for which subjects might have generative models. We tested the hypothesis using a task in which subjects had to rapidly point at a target region partly covered by three stochastic penalty regions introduced as "defenders." At movement completion, each defender jumped to a new position drawn randomly from fixed probability distributions. Subjects earned points when they hit the target, unblocked by a defender, and lost points otherwise. Results indicate that after approximately 600 trials, subjects approached optimal behavior. We further tested whether subjects simply learned a set of stimulus-contingent motor plans or the statistics of defenders' movements by training subjects with one penalty distribution and then testing them on a new penalty distribution. Subjects immediately changed their strategy to achieve the same average reward as subjects who had trained with the second penalty distribution. These results indicate that subjects learned the parameters of the defenders' jump distributions and used this knowledge to optimally plan their hand movements under conditions involving stochastic rewards and penalties.
-
Rule Learning in Autism: The Role of Reward Type and Social Context
Jones, E. J. H.; Webb, S. J.; Estes, A.; Dawson, G.
2013-01-01
Learning abstract rules is central to social and cognitive development. Across two experiments, we used Delayed Non-Matching to Sample tasks to characterize the longitudinal development and nature of rule-learning impairments in children with Autism Spectrum Disorder (ASD). Results showed that children with ASD consistently experienced more difficulty learning an abstract rule from a discrete physical reward than children with DD. Rule learning was facilitated by the provision of more concret...
-
Directory of Open Access Journals (Sweden)
Tammy Chung
2015-12-01
Full Text Available This preliminary study examined the extent to which regional brain activation during a reward cue antisaccade (AS task was associated with 6-month treatment outcome in adolescent substance users. Antisaccade performance provides a sensitive measure of executive function and cognitive control, and generally improves with reward cues. We hypothesized that when preparing to execute an AS, greater activation in regions associated with cognitive and oculomotor control supporting AS, particularly during reward cue trials, would be associated with lower substance use severity at 6-month follow-up. Adolescents (n = 14, ages 14–18 recruited from community-based outpatient treatment completed an fMRI reward cue AS task (reward and neutral conditions, and provided follow-up data. Results indicated that AS errors decreased in reward, compared to neutral, trials. AS behavioral performance, however, was not associated with treatment outcome. As hypothesized, activation in regions of interest (ROIs associated with cognitive (e.g., ventrolateral prefrontal cortex and oculomotor control (e.g., supplementary eye field during reward trials were inversely correlated with marijuana problem severity at 6-months. ROI activation during neutral trials was not associated with outcomes. Results support the role of motivational (reward cue factors to enhance cognitive control processes, and suggest a potential brain-based correlate of youth treatment outcome.
-
Towards a common theory for learning from reward, affect, and motivation: The SIMON framework
Directory of Open Access Journals (Sweden)
Christopher R Madan
2013-10-01
Full Text Available While the effects of reward, affect, and motivation on learning have each developed into their own fields of research, they largely have been investigated in isolation. As all three of these constructs are highly related, and use similar experimental procedures, an important advance in research would be to consider the interplay between these constructs. Here we first define each of the three constructs, and then discuss how they may influence each other within a common framework. Finally, we delineate several sources of evidence supporting the framework. By considering the constructs of reward, affect, and motivation within a single framework, we can develop a better understanding of the processes involved in learning and how they interplay, and work towards a comprehensive theory that encompasses reward, affect, and motivation.
-
Meermeier, Annegret; Gremmler, Svenja; Richert, Kerstin; Eckermann, Til; Lappe, Markus
2017-10-01
Saccadic adaptation is an oculomotor learning process that maintains the accuracy of eye movements to ensure effective perception of the environment. Although saccadic adaptation is commonly considered an automatic and low-level motor calibration in the cerebellum, we recently found that strength of adaptation is influenced by the visual content of the target: pictures of humans produced stronger adaptation than noise stimuli. This suggests that meaningful images may be considered rewarding or valuable in oculomotor learning. Here we report three experiments that establish the boundaries of this effect. In the first, we tested whether stimuli that were associated with high and low value following long term self-administered reinforcement learning produce stronger adaptation. Twenty-eight expert gamers participated in two sessions of adaptation to game-related high- and low-reward stimuli, but revealed no difference in saccadic adaptation (Bayes Factor01 = 5.49). In the second experiment, we tested whether cognitive (literate) meaning could induce stronger adaptation by comparing targets consisting of words and nonwords. The results of twenty subjects revealed no difference in adaptation strength (Bayes Factor01 = 3.21). The third experiment compared images of human figures to noise patterns for reactive saccades. Twenty-two subjects adapted significantly more toward images of human figures in comparison to noise (p vs. noise), but not secondary, reinforcement affects saccadic adaptation (words vs. nonwords, high- vs. low-value video game images).
-
Reward action in the initiation of smooth pursuit eye movements
Joshua, Mati; Lisberger, Stephen G.
2012-01-01
Reward has a powerful influence on motor behavior. To probe how and where reward systems alter motor behavior, we studied smooth pursuit eye movements in monkeys trained to associate the color of a visual cue with the size of the reward to be issued at the end of the target motion. When the tracking task presented two different colored targets that moved orthogonally, monkeys biased the initiation of pursuit towards the direction of motion of the target that led to larger reward. The bias was...
-
The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study.
Mensen, Armand; Poryazova, Rositsa; Huegli, Gordana; Baumann, Christian R; Schwartz, Sophie; Khatami, Ramin
2015-01-01
The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson's disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160 ms both patient groups had reduced ERP amplitude compared to controls. Later at 250 ms, both patient groups also showed a clear event-related potential (ERP), which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN) potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson's. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.
-
The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study.
Directory of Open Access Journals (Sweden)
Armand Mensen
Full Text Available The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson's disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160 ms both patient groups had reduced ERP amplitude compared to controls. Later at 250 ms, both patient groups also showed a clear event-related potential (ERP, which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson's. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.
-
Directory of Open Access Journals (Sweden)
Joschka eBoedecker
2013-10-01
Full Text Available A common assumption in psychology, economics, and other fields holds that higher performance will result if extrinsic rewards (such as money are offered as an incentive. While this principle seems to work well for tasks that require the execution of the same sequence of steps over and over, with little uncertainty about the process, in other cases, especially where creative problem solving is required due to the difficulty in finding the optimal sequence of actions, external rewards can actually be detrimental to task performance. Furthermore, they have the potential to undermine intrinsic motivation to do an otherwise interesting activity. In this work, we extend a computational model of the prefrontal and dorsolateral striatal reinforcement learning systems to account for the effects of extrinsic and intrinsic rewards. The model assumes that the brain employs both a goal-directed and a habitual learning system, and competition between both is based on the trade-off between the cost of the reasoning process and value of information. The goal-directed system elicits internal rewards when its models of the environment improve, while the habitual system, being model-free, does not. Our results account for the phenomena that initial extrinsic reward leads to reduced activity after extinction compared to the case without any initial extrinsic rewards, and that performance in complex task settings drops when higher external rewards are promised. We also test the hypothesis that external rewards bias the competition in favor of the computationally efficient, but cruder and less flexible habitual system, which can negatively influence intrinsic motivation and task performance in the class of tasks we consider.
-
Boedecker, Joschka; Lampe, Thomas; Riedmiller, Martin
2013-01-01
A common assumption in psychology, economics, and other fields holds that higher performance will result if extrinsic rewards (such as money) are offered as an incentive. While this principle seems to work well for tasks that require the execution of the same sequence of steps over and over, with little uncertainty about the process, in other cases, especially where creative problem solving is required due to the difficulty in finding the optimal sequence of actions, external rewards can actually be detrimental to task performance. Furthermore, they have the potential to undermine intrinsic motivation to do an otherwise interesting activity. In this work, we extend a computational model of the dorsomedial and dorsolateral striatal reinforcement learning systems to account for the effects of extrinsic and intrinsic rewards. The model assumes that the brain employs both a goal-directed and a habitual learning system, and competition between both is based on the trade-off between the cost of the reasoning process and value of information. The goal-directed system elicits internal rewards when its models of the environment improve, while the habitual system, being model-free, does not. Our results account for the phenomena that initial extrinsic reward leads to reduced activity after extinction compared to the case without any initial extrinsic rewards, and that performance in complex task settings drops when higher external rewards are promised. We also test the hypothesis that external rewards bias the competition in favor of the computationally efficient, but cruder and less flexible habitual system, which can negatively influence intrinsic motivation and task performance in the class of tasks we consider.
-
Variation in reward quality and pollinator attraction: the consumer does not always get it right.
Carr, David E; Haber, Ariela I; LeCroy, Kathryn A; Lee, De'Ashia E; Link, Rosabeth I
2015-04-09
Nearly all bees rely on pollen as the sole protein source for the development of their larvae. The central importance of pollen for the bee life cycle should exert strong selection on their ability to locate the most rewarding sources of pollen. Despite this importance, very few studies have examined the influence of intraspecific variation in pollen rewards on the foraging decisions of bees. Previous studies have demonstrated that inbreeding reduces viability and hence protein content in Mimulus guttatus (seep monkeyflower) pollen and that bees strongly discriminate against inbred in favour of outbred plants. We examined whether variation in pollen viability could explain this preference using a series of choice tests with living plants, artificial plants and olfactometer tests using the bumble bee Bombus impatiens. We found that B. impatiens preferred to visit artificial plants provisioned with fertile anthers over those provisioned with sterile anthers. They also preferred fertile anthers when provided only olfactory cues. These bumble bees were unable to discriminate among live plants from subpopulations differing dramatically in pollen viability, however. They preferred outbred plants even when those plants were from subpopulations with pollen viability as low as the inbred populations. Their preference for outbred plants was evident even when only olfactory cues were available. Our data showed that bumble bees are able to differentiate between anthers that provide higher rewards when cues are isolated from the rest of the flower. When confronted with cues from the entire flower, their choices are independent of the quality of the pollen reward, suggesting that they are responding more strongly to cues unassociated with rewards than to those correlated with rewards. If so, this suggests that a sensory bias or some level of deception may be involved with advertisement to pollinators in M. guttatus. Published by Oxford University Press on behalf of the Annals of
-
The Effect of Reward on Orienting and Reorienting in Exogenous Cueing
Bucker, B.; Theeuwes, J.
2014-01-01
It is thought that reward-induced motivation influences perceptual, attentional, and cognitive control processes to facilitate behavioral performance. In this study, we investigated the effect of reward-induced motivation on exogenous attention orienting and inhibition of return (IOR). Attention was
-
Introduction: Addiction and Brain Reward and Anti-Reward Pathways
Gardner, Eliot L.
2013-01-01
bio-psycho-social” model of etiology holds very well for addiction. Addiction appears to correlate with a hypo-dopaminergic dysfunctional state within the reward circuitry of the brain. Neuroimaging studies in humans add credence to this hypothesis. Credible evidence also implicates serotonergic, opioid, endocannabinoid, GABAergic, and glutamatergic mechanisms in addiction. Critically, drug addiction progresses from occasional recreational use to impulsive use to habitual compulsive use. This correlates with a progression from reward-driven to habit-driven drug-seeking behavior. This behavioral progression correlates with a neuroanatomical progression from ventral striatal (nucleus accumbens) to dorsal striatal control over drug-seeking behavior. The three classical sets of craving and relapse triggers are a) re-exposure to addictive drugs, b) stress, and c) re-exposure to environmental cues (“people, places, things”) previously associated with drug-taking behavior. Drug-triggered relapse involves the nucleus accumbens and the neurotransmitter dopamine. Stress-triggered relapse involves a) the central nucleus of the amygdala, the bed nucleus of the stria terminalis, and the neurotransmitter CRF; and b) the lateral tegmental noradrenergic nuclei of the brain stem and the neurotransmitter norepinephrine. Cue-triggered relapse involves the basolateral nucleus of the amygdala, the hippocampus, and the neurotransmitter glutamate. Knowledge of the neuroanatomy, neurophysiology, neurochemistry, and neuropharmacology of addictive drug action in the brain is currently producing a variety of strategies for pharmacotherapeutic treatment of drug addiction, some of which appear promising. PMID:21508625
-
Nonparametric bayesian reward segmentation for skill discovery using inverse reinforcement learning
CSIR Research Space (South Africa)
Ranchod, P
2015-10-01
Full Text Available We present a method for segmenting a set of unstructured demonstration trajectories to discover reusable skills using inverse reinforcement learning (IRL). Each skill is characterised by a latent reward function which the demonstrator is assumed...
-
Dual learning processes in interactive skill acquisition.
Fu, Wai-Tat; Anderson, John R
2008-06-01
Acquisition of interactive skills involves the use of internal and external cues. Experiment 1 showed that when actions were interdependent, learning was effective with and without external cues in the single-task condition but was effective only with the presence of external cues in the dual-task condition. In the dual-task condition, actions closer to the feedback were learned faster than actions farther away but this difference was reversed in the single-task condition. Experiment 2 tested how knowledge acquired in single and dual-task conditions would transfer to a new reward structure. Results confirmed the two forms of learning mediated by the secondary task: A declarative memory encoding process that simultaneously assigned credits to actions and a reinforcement-learning process that slowly propagated credits backward from the feedback. The results showed that both forms of learning were engaged during training, but only at the response selection stage, one form of knowledge may dominate over the other depending on the availability of attentional resources. (c) 2008 APA, all rights reserved
-
Cannabinoid modulation of drug reward and the implications of marijuana legalization.
Covey, Dan P; Wenzel, Jennifer M; Cheer, Joseph F
2015-12-02
Marijuana is the most popular illegal drug worldwide. Recent trends indicate that this may soon change; not due to decreased marijuana use, but to an amendment in marijuana's illegal status. The cannabinoid type 1 (CB1) receptor mediates marijuana's psychoactive and reinforcing properties. CB1 receptors are also part of the brain endocannabinoid (eCB) system and support numerous forms of learning and memory, including the conditioned reinforcing properties of cues predicting reward or punishment. This is accomplished via eCB-dependent alterations in mesolimbic dopamine function, which plays an obligatory role in reward learning and motivation. Presynaptic CB1 receptors control midbrain dopamine neuron activity and thereby shape phasic dopamine release in target regions, particularly the nucleus accumbens (NAc). By also regulating synaptic input to the NAc, CB1 receptors modulate NAc output onto downstream neurons of the basal ganglia motor circuit, and thereby support goal-directed behaviors. Abused drugs promote short- and long-term adaptations in eCB-regulation of mesolimbic dopamine function, and thereby hijack neural systems related to the pursuit of rewards to promote drug abuse. By pharmacologically targeting the CB1 receptors, marijuana has preferential access to this neuronal system and can potently alter eCB-dependent processing of reward-related stimuli. As marijuana legalization progresses, greater access to this drug should increase the utility of marijuana as a research tool to better understand the eCB system, which has the potential to advance cannabinoid-based treatments for drug addiction. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Albasser, Mathieu M; Dumont, Julie R; Amin, Eman; Holmes, Joshua D; Horne, Murray R; Pearce, John M; Aggleton, John P
2013-12-01
Three cohorts of rats with extensive hippocampal lesions received multiple tests to examine the relationships between particular forms of associative learning and an influential account of hippocampal function (the cognitive map hypothesis). Hippocampal lesions spared both the ability to discriminate two different digging media and to discriminate two different room locations in a go/no-go task when each location was approached from a single direction. Hippocampal lesions had, however, differential effects on a more complex task (biconditional discrimination) where the correct response was signaled by the presence or absence of specific cues. For all biconditional tasks, digging in one medium (A) was rewarded in the presence of cue C, while digging in medium B was rewarded in the presences of cue D. Such biconditional tasks are "configural" as no individual cue or element predicts the solution (AC+, AD-, BD+, and BC-). When proximal context cues signaled the correct digging choice, biconditional learning was seemingly unaffected by hippocampal lesions. Severe deficits occurred, however, when the correct digging choice was signaled by distal room cues. Also, impaired was the ability to discriminate two locations when each location was approached from two directions. A task demand that predicted those tasks impaired by hippocampal damage was the need to combine specific cues with their relative spatial positions ("structural learning"). This ability makes it possible to distinguish the same cues set in different spatial arrays. Thus, the hippocampus appears necessary for configural discriminations involving structure, discriminations that potentially underlie the creation of cognitive maps. Copyright © 2013 The Authors. Hippocampus Published by Wiley Periodicals, Inc.
-
Directory of Open Access Journals (Sweden)
Carmen N. Lukie
2014-07-01
Full Text Available Children and adolescents learn to regulate their behavior by utilizing feedback from the environment but exactly how this ability develops remains unclear. To investigate this question, we recorded the event-related brain potential (ERP from children (8–13 years, adolescents (14–17 years and young adults (18–23 years while they navigated a “virtual maze” in pursuit of monetary rewards. The amplitude of the reward positivity, an ERP component elicited by feedback stimuli, was evaluated for each age group. A current theory suggests the reward positivity is produced by the impact of reinforcement learning signals carried by the midbrain dopamine system on anterior cingulate cortex, which utilizes the signals to learn and execute extended behaviors. We found that the three groups produced a reward positivity of comparable size despite relatively longer ERP component latencies for the children, suggesting that the reward processing system reaches maturity early in development. We propose that early development of the midbrain dopamine system facilitates the development of extended goal-directed behaviors in anterior cingulate cortex.
-
Vicarious Reinforcement In Rhesus Macaques (Macaca mulatta
Directory of Open Access Journals (Sweden)
Steve W. C. Chang
2011-03-01
Full Text Available What happens to others profoundly influences our own behavior. Such other-regarding outcomes can drive observational learning, as well as motivate cooperation, charity, empathy, and even spite. Vicarious reinforcement may serve as one of the critical mechanisms mediating the influence of other-regarding outcomes on behavior and decision-making in groups. Here we show that rhesus macaques spontaneously derive vicarious reinforcement from observing rewards given to another monkey, and that this reinforcement can motivate them to subsequently deliver or withhold rewards from the other animal. We exploited Pavlovian and instrumental conditioning to associate rewards to self (M1 and/or rewards to another monkey (M2 with visual cues. M1s made more errors in the instrumental trials when cues predicted reward to M2 compared to when cues predicted reward to M1, but made even more errors when cues predicted reward to no one. In subsequent preference tests between pairs of conditioned cues, M1s preferred cues paired with reward to M2 over cues paired with reward to no one. By contrast, M1s preferred cues paired with reward to self over cues paired with reward to both monkeys simultaneously. Rates of attention to M2 strongly predicted the strength and valence of vicarious reinforcement. These patterns of behavior, which were absent in nonsocial control trials, are consistent with vicarious reinforcement based upon sensitivity to observed, or counterfactual, outcomes with respect to another individual. Vicarious reward may play a critical role in shaping cooperation and competition, as well as motivating observational learning and group coordination in rhesus macaques, much as it does in humans. We propose that vicarious reinforcement signals mediate these behaviors via homologous neural circuits involved in reinforcement learning and decision-making.
-
Vicarious reinforcement in rhesus macaques (macaca mulatta).
Chang, Steve W C; Winecoff, Amy A; Platt, Michael L
2011-01-01
What happens to others profoundly influences our own behavior. Such other-regarding outcomes can drive observational learning, as well as motivate cooperation, charity, empathy, and even spite. Vicarious reinforcement may serve as one of the critical mechanisms mediating the influence of other-regarding outcomes on behavior and decision-making in groups. Here we show that rhesus macaques spontaneously derive vicarious reinforcement from observing rewards given to another monkey, and that this reinforcement can motivate them to subsequently deliver or withhold rewards from the other animal. We exploited Pavlovian and instrumental conditioning to associate rewards to self (M1) and/or rewards to another monkey (M2) with visual cues. M1s made more errors in the instrumental trials when cues predicted reward to M2 compared to when cues predicted reward to M1, but made even more errors when cues predicted reward to no one. In subsequent preference tests between pairs of conditioned cues, M1s preferred cues paired with reward to M2 over cues paired with reward to no one. By contrast, M1s preferred cues paired with reward to self over cues paired with reward to both monkeys simultaneously. Rates of attention to M2 strongly predicted the strength and valence of vicarious reinforcement. These patterns of behavior, which were absent in non-social control trials, are consistent with vicarious reinforcement based upon sensitivity to observed, or counterfactual, outcomes with respect to another individual. Vicarious reward may play a critical role in shaping cooperation and competition, as well as motivating observational learning and group coordination in rhesus macaques, much as it does in humans. We propose that vicarious reinforcement signals mediate these behaviors via homologous neural circuits involved in reinforcement learning and decision-making.
-
Cueing and Anxiety in a Visual Concept Learning Task.
Turner, Philip M.
This study investigated the relationship of two anxiety measures (the State-Trait Anxiety Inventory-Trait Form and the S-R Inventory of Anxiousness-Exam Form) to performance on a visual concept-learning task with embedded criterial information. The effect on anxiety reduction of cueing criterial information was also examined, and two levels of…
-
Lardeux, Sylvie; Paleressompoulle, Dany; Pernaud, Remy; Cador, Martine; Baunez, Christelle
2013-10-01
The search for treatment of cocaine addiction raises the challenge to find a way to diminish motivation for the drug without decreasing it for natural rewards. Subthalamic nucleus (STN) inactivation decreases motivation for cocaine while increasing motivation for food, suggesting that STN can dissociate different rewards. Here, we investigated how rat STN neurons respond to cues predicting cocaine or sucrose and to reward delivery while rats are performing a discriminative stimuli task. We show that different neuronal populations of STN neurons encode cocaine and sucrose. In addition, we show that STN activity at the cue onset predicts future error. When changing the reward predicted unexpectedly, STN neurons show capacities of adaptation, suggesting a role in reward-prediction error. Furthermore, some STN neurons show a response to executive error (i.e., "oops neurons") that is specific to the missed reward. These results position the STN as a nexus where natural rewards and drugs of abuse are coded differentially and can influence the performance. Therefore, STN can be viewed as a structure where action could be taken for the treatment of cocaine addiction.
-
Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain.
Niv, Yael; Edlund, Jeffrey A; Dayan, Peter; O'Doherty, John P
2012-01-11
Humans and animals are exquisitely, though idiosyncratically, sensitive to risk or variance in the outcomes of their actions. Economic, psychological, and neural aspects of this are well studied when information about risk is provided explicitly. However, we must normally learn about outcomes from experience, through trial and error. Traditional models of such reinforcement learning focus on learning about the mean reward value of cues and ignore higher order moments such as variance. We used fMRI to test whether the neural correlates of human reinforcement learning are sensitive to experienced risk. Our analysis focused on anatomically delineated regions of a priori interest in the nucleus accumbens, where blood oxygenation level-dependent (BOLD) signals have been suggested as correlating with quantities derived from reinforcement learning. We first provide unbiased evidence that the raw BOLD signal in these regions corresponds closely to a reward prediction error. We then derive from this signal the learned values of cues that predict rewards of equal mean but different variance and show that these values are indeed modulated by experienced risk. Moreover, a close neurometric-psychometric coupling exists between the fluctuations of the experience-based evaluations of risky options that we measured neurally and the fluctuations in behavioral risk aversion. This suggests that risk sensitivity is integral to human learning, illuminating economic models of choice, neuroscientific models of affective learning, and the workings of the underlying neural mechanisms.
-
He, Ying-Ying; Xue, Yan-Xue; Wang, Ji-Shi; Fang, Qin; Liu, Jian-Feng; Xue, Li-Fen; Lu, Lin
2011-09-01
The intense associative memories that develop between drug-paired contextual cues and rewarding stimuli or the drug withdrawal-associated aversive feeling have been suggested to contribute to the high rate of relapse. Various studies have elucidated the mechanisms underlying the formation and expression of drug-related cue memories, but how this mechanism is maintained is unknown. Protein kinase M ζ (PKMζ) was recently shown to be necessary and sufficient for long-term potentiation maintenance and memory storage. In the present study, we used conditioned place preference (CPP) and aversion (CPA) to examine whether PKMζ maintains both morphine-associated reward memory and morphine withdrawal-associated aversive memory in the basolateral amygdala (BLA). We also investigate the role of PKMζ in the infralimbic cortex in the extinction memory of morphine reward-related cues and morphine withdrawal-related aversive cues. We found that intra-BLA but not central nucleus of the amygdala injection of the selective PKMζ inhibitor ZIP 1 day after CPP and CPA training impaired the expression of CPP and CPA 1 day later, and the effect of ZIP on memory lasted at least 2 weeks. Inhibiting PKMζ activity in the infralimbic cortex, but not prelimbic cortex, disrupted the expression of the extinction memory of CPP and CPA. These results indicate that PKMζ in the BLA is required for the maintenance of associative morphine reward memory and morphine withdrawal-associated aversion memory, and PKMζ in the infralimbic cortex is required for the maintenance of extinction memory of morphine reward-related cues and morphine withdrawal-related aversive cues.
-
Ethanol-induced conditioned taste avoidance: reward or aversion?
Liu, Chuang; Showalter, John; Grigson, Patricia Sue
2009-03-01
Rats avoid intake of a palatable taste cue when paired with all drugs of abuse tested. Evidence suggests that, at least for morphine and cocaine, rats avoid the taste cue because they are anticipating the rewarding properties of the drug. Thus, the suppressive effects of a rewarding sucrose solution and cocaine, but not those of the putatively aversive agent, lithium chloride (LiCl), are exaggerated in drug-sensitive Lewis rats. Likewise, the suppressive effects of sucrose and morphine, but not those of LiCl, are eliminated by bilateral lesions of the gustatory thalamus. Unlike morphine and cocaine, it is less clear whether rewarding or aversive drug properties are responsible for ethanol-induced suppression of intake of a taste cue. The present set of studies tests whether, like cocaine, ethanol-induced suppression of intake of a taste cue also is greater in the drug-sensitive Lewis rats and whether the suppressive effects of the drug are prevented by bilateral lesions of the taste thalamus. In Experiment 1, fluid-deprived Lewis and Fischer rats were given 5-minute access to 0.15% saccharin and then injected with saline or a range of doses of ethanol (0.5, 0.75, 1.0, or 1.5 g/kg). There was a total of 6 such pairings. In Experiments 2 and 3, Sprague-Dawley rats received bilateral electrophysiologically guided lesions of the gustatory thalamus. After recovery, suppression of intake of the saccharin cue was evaluated following repeated daily pairings with either a high (1.5 g/kg) or a low (0.75 g/kg) dose of ethanol. Ethanol-induced suppression of intake of the saccharin conditioned stimulus (CS) did not differ between the drug-sensitive Lewis rats relative to the less-sensitive Fischer rats. Lesions of the taste thalamus, however, prevented the suppressive effect of the 0.75 g/kg dose of the drug, but had no impact on the suppressive effect of the 1.5 g/kg dose of ethanol. The results suggest that the suppressive effects of ethanol on CS intake are mediated by both
-
Barker, Jacqueline M; Lench, Daniel H; Chandler, L Judson
2016-01-01
Alcohol use disorders are associated with deficits in adaptive behavior. While some behavioral impairments that are associated with alcohol use disorders may predate exposure to drugs of abuse, others may result directly from exposure to drugs of abuse, including alcohol. Identifying a causal role for how alcohol exposure leads to these impairments will enable further investigation of the neurobiological mechanisms by which it acts to dysregulate adaptive behavior. In the present study, we examined the effects of chronic intermittent ethanol exposure (CIE) on the use of reward-paired cues to guide consummatory behaviors in a mouse model, and further, how manipulations of mGluR2/3 signaling-known to be dysregulated after chronic alcohol exposure-may alter the expression of this behavior. Adult male C57B/6J mice were trained to self-administer 10 % ethanol and exposed to CIE via vapor inhalation. After CIE exposure, mice were trained in a Pavlovian task wherein a cue (tone) was paired with the delivery of a 10 % sucrose unconditioned stimulus. The use of the reward-paired cue to guide licking behavior was determined across training. The effect of systemic mGluR2/3 manipulation on discrimination between cue-on and cue-off intervals was assessed by administration of the mGluR2/3 agonist LY379268 or the antagonist LY341495 prior to a testing session. Exposure to CIE resulted in reductions in discrimination between cue-on and cue-off intervals, with CIE-exposed mice exhibiting significantly lower consummatory behavior during reward-paired cues than air controls. In addition, systemic administration of an mGluR2/3 agonist restored the use of reward-paired cues in CIE-exposed animals without impacting behavior in air controls. Conversely, administration of an mGluR2/3 antagonist mimicked the effects of CIE on cue-guided licking behavior, indicating that mGluR2/3 signaling can bidirectionally regulate the ability to use reward-paired cues to guide behavior. Together
-
What Works Clearinghouse, 2010
2010-01-01
"Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning" examined the effects of different reward systems used in group learning situations on the math skills of African-American and white students. The study analyzed data on 75 African-American and 57 white fourth- and fifth-grade students from urban…
-
Fiorenza, Amanda M; Shnitko, Tatiana A; Sullivan, Kaitlin M; Vemuru, Sudheer R; Gomez-A, Alexander; Esaki, Julie Y; Boettiger, Charlotte A; Da Cunha, Claudio; Robinson, Donita L
2018-06-01
Conditioned stimuli (CS) that predict reward delivery acquire the ability to induce phasic dopamine release in the nucleus accumbens (NAc). This dopamine release may facilitate conditioned approach behavior, which often manifests as approach to the site of reward delivery (called "goal-tracking") or to the CS itself (called "sign-tracking"). Previous research has linked sign-tracking in particular to impulsivity and drug self-administration, and addictive drugs may promote the expression of sign-tracking. Ethanol (EtOH) acutely promotes phasic release of dopamine in the accumbens, but it is unknown whether an alcoholic reward alters dopamine release to a CS. We hypothesized that Pavlovian conditioning with an alcoholic reward would increase dopamine release triggered by the CS and subsequent sign-tracking behavior. Moreover, we predicted that chronic intermittent EtOH (CIE) exposure would promote sign-tracking while acute administration of naltrexone (NTX) would reduce it. Rats received 14 doses of EtOH (3 to 5 g/kg, intragastric) or water followed by 6 days of Pavlovian conditioning training. Rewards were a chocolate solution with or without 10% (w/v) alcohol. We used fast-scan cyclic voltammetry to measure phasic dopamine release in the NAc core in response to the CS and the rewards. We also determined the effect of NTX (1 mg/kg, subcutaneous) on conditioned approach. Both CIE and alcoholic reward, individually but not together, associated with greater dopamine to the CS than control conditions. However, this increase in dopamine release was not linked to greater sign-tracking, as both CIE and alcoholic reward shifted conditioned approach from sign-tracking behavior to goal-tracking behavior. However, they both also increased sensitivity to NTX, which reduced goal-tracking behavior. While a history of EtOH exposure or alcoholic reward enhanced dopamine release to a CS, they did not promote sign-tracking under the current conditions. These findings are
-
Memory Consolidation and Neural Substrate of Reward
Directory of Open Access Journals (Sweden)
Redolar-Ripoll, Diego
2012-08-01
Full Text Available The aim of this report is to analyze the relationships between reward and learning and memory processes. Different studies have described how information about rewards influences behavior and how the brain uses this reward information to control learning and memory processes. Reward nature seems to be processed in different ways by neurons in different brain structures, ranging from the detection and perception of rewards to the use of information about predicted rewards for the control of goal-directed behavior. The neural substrate underling this processing of reward information is a reliable way of improving learning and memory processes. Evidence from several studies indicates that this neural system can facilitate memory consolidation in a wide variety of learning tasks. From a molecular perspective, certain cardinal features of reward have been described as forms of memory. Studies of human addicts and studies in animal models of addiction show that chronic drug exposure produces stable changes in the brain at the cellular and molecular levels that underlie the long-lasting behavioral plasticity associated with addiction. These molecular and cellular adaptations involved in addiction are also implicated in learning and memory processes. Dopamine seems to be a critical common signal to activate different genetic mechanisms that ultimately remodel synapses and circuits. Despite memory is an active and complex process mediated by different brain areas, the neural substrate of reward is able to improve memory consolidation in a several paradigms. We believe that there are many equivalent traits between reward and learning and memory processes.
-
Premotor and Motor Cortices Encode Reward.
Directory of Open Access Journals (Sweden)
Pavan Ramkumar
Full Text Available Rewards associated with actions are critical for motivation and learning about the consequences of one's actions on the world. The motor cortices are involved in planning and executing movements, but it is unclear whether they encode reward over and above limb kinematics and dynamics. Here, we report a categorical reward signal in dorsal premotor (PMd and primary motor (M1 neurons that corresponds to an increase in firing rates when a trial was not rewarded regardless of whether or not a reward was expected. We show that this signal is unrelated to error magnitude, reward prediction error, or other task confounds such as reward consumption, return reach plan, or kinematic differences across rewarded and unrewarded trials. The availability of reward information in motor cortex is crucial for theories of reward-based learning and motivational influences on actions.
-
Floral reward, advertisement and attractiveness to honey bees in dioecious Salix caprea.
Dötterl, Stefan; Glück, Ulrike; Jürgens, Andreas; Woodring, Joseph; Aas, Gregor
2014-01-01
In dioecious, zoophilous plants potential pollinators have to be attracted to both sexes and switch between individuals of both sexes for pollination to occur. It often has been suggested that males and females require different numbers of visits for maximum reproductive success because male fertility is more likely limited by access to mates, whereas female fertility is rather limited by resource availability. According to sexual selection theory, males therefore should invest more in pollinator attraction (advertisement, reward) than females. However, our knowledge on the sex specific investment in floral rewards and advertisement, and its effects on pollinator behaviour is limited. Here, we use an approach that includes chemical, spectrophotometric, and behavioural studies i) to elucidate differences in floral nectar reward and advertisement (visual, olfactory cues) in dioecious sallow, Salix caprea, ii) to determine the relative importance of visual and olfactory floral cues in attracting honey bee pollinators, and iii) to test for differential attractiveness of female and male inflorescence cues to honey bees. Nectar amount and sugar concentration are comparable, but sugar composition varies between the sexes. Olfactory sallow cues are more attractive to honey bees than visual cues; however, a combination of both cues elicits the strongest behavioural responses in bees. Male flowers are due to the yellow pollen more colourful and emit a higher amount of scent than females. Honey bees prefer the visual but not the olfactory display of males over those of females. In all, the data of our multifaceted study are consistent with the sexual selection theory and provide novel insights on how the model organism honey bee uses visual and olfactory floral cues for locating host plants.
-
Floral reward, advertisement and attractiveness to honey bees in dioecious Salix caprea.
Directory of Open Access Journals (Sweden)
Stefan Dötterl
Full Text Available In dioecious, zoophilous plants potential pollinators have to be attracted to both sexes and switch between individuals of both sexes for pollination to occur. It often has been suggested that males and females require different numbers of visits for maximum reproductive success because male fertility is more likely limited by access to mates, whereas female fertility is rather limited by resource availability. According to sexual selection theory, males therefore should invest more in pollinator attraction (advertisement, reward than females. However, our knowledge on the sex specific investment in floral rewards and advertisement, and its effects on pollinator behaviour is limited. Here, we use an approach that includes chemical, spectrophotometric, and behavioural studies i to elucidate differences in floral nectar reward and advertisement (visual, olfactory cues in dioecious sallow, Salix caprea, ii to determine the relative importance of visual and olfactory floral cues in attracting honey bee pollinators, and iii to test for differential attractiveness of female and male inflorescence cues to honey bees. Nectar amount and sugar concentration are comparable, but sugar composition varies between the sexes. Olfactory sallow cues are more attractive to honey bees than visual cues; however, a combination of both cues elicits the strongest behavioural responses in bees. Male flowers are due to the yellow pollen more colourful and emit a higher amount of scent than females. Honey bees prefer the visual but not the olfactory display of males over those of females. In all, the data of our multifaceted study are consistent with the sexual selection theory and provide novel insights on how the model organism honey bee uses visual and olfactory floral cues for locating host plants.
-
Cue competition in evaluative conditioning as a function of the learning process.
Kattner, Florian; Green, C Shawn
2015-11-01
Evaluative conditioning (EC) is the change in the valence of a stimulus resulting from pairings with an affective (unconditioned) stimulus (US). With some exceptions, previous work has indicated that this form of conditioning might be insensitive to cue competition effects such as blocking and overshadowing. Here we assessed whether the extent of cue competition in EC depends upon the type of contingency learning during conditioning. Specifically, we contrasted a learning task that biased participants toward cognitive/inferential learning (i.e., predicting the US) with a learning task that prevented prolonged introspection (i.e., a rapid response made to the US). In all cases, standard EC effects were observed, with the subjective liking of stimuli changed in the direction of the valence of the US. More importantly, when inferential learning was likely, larger EC effects occurred for isolated stimuli than for compounds (indicating overshadowing). No blocking effects on explicit evaluations were observed for either learning task. Contingency judgments and implicit evaluations, however, were sensitive to blocking, indicating that the absence of a blocking effect on explicit evaluations might be due to inferences that occur during testing.
-
Influence of cue word perceptual information on metamemory accuracy in judgement of learning.
Hu, Xiao; Liu, Zhaomin; Li, Tongtong; Luo, Liang
2016-01-01
Previous studies have suggested that perceptual information regarding to-be-remembered words in the study phase affects the accuracy of judgement of learning (JOL). However, few have investigated whether the perceptual information in the JOL phase influences JOL accuracy. This study examined the influence of cue word perceptual information in the JOL phase on immediate and delayed JOL accuracy through changes in cue word font size. In Experiment 1, large-cue word pairs had significantly higher mean JOL magnitude than small-cue word pairs in immediate JOLs and higher relative accuracy than small-cue pairs in delayed JOLs, but font size had no influence on recall performance. Experiment 2 increased the JOL time, and mean JOL magnitude did not reliably differ for large-cue compared with small-cue pairs in immediate JOLs. However, the influence on relative accuracy still existed in delayed JOLs. Experiment 3 increased the familiarity of small-cue words in the delayed JOL phase by adding a lexical decision task. The results indicated that cue word font size no longer affected relative accuracy in delayed JOLs. The three experiments in our study indicated that the perceptual information regarding cue words in the JOL phase affects immediate and delayed JOLs in different ways.
-
Imbalance in the sensitivity to different types of rewards in pathological gambling.
Sescousse, Guillaume; Barbalat, Guillaume; Domenech, Philippe; Dreher, Jean-Claude
2013-08-01
Pathological gambling is an addictive disorder characterized by a persistent and compulsive desire to engage in gambling activities. This maladaptive behaviour has been suggested to result from a decreased sensitivity to experienced rewards, regardless of reward type. Alternatively, pathological gambling might reflect an imbalance in the sensitivity to monetary versus non-monetary incentives. To directly test these two hypotheses, we examined how the brain reward circuit of pathological gamblers responds to different types of rewards. Using functional magnetic resonance imaging, we compared the brain responses of 18 pathological gamblers and 20 healthy control subjects while they engaged in a simple incentive task manipulating both monetary and visual erotic rewards. During reward anticipation, the ventral striatum of pathological gamblers showed a differential response to monetary versus erotic cues, essentially driven by a blunted reactivity to cues predicting erotic stimuli. This differential response correlated with the severity of gambling symptoms and was paralleled by a reduced behavioural motivation for erotic rewards. During reward outcome, a posterior orbitofrontal cortex region, responding to erotic rewards in both groups, was further recruited by monetary gains in pathological gamblers but not in control subjects. Moreover, while ventral striatal activity correlated with subjective ratings assigned to monetary and erotic rewards in control subjects, it only correlated with erotic ratings in gamblers. Our results point to a differential sensitivity to monetary versus non-monetary rewards in pathological gambling, both at the motivational and hedonic levels. Such an imbalance might create a bias towards monetary rewards, potentially promoting addictive gambling behaviour.
-
Dopamine reward prediction error coding
Schultz, Wolfram
2016-01-01
Reward prediction errors consist of the differences between received and predicted rewards. They are crucial for basic forms of learning about rewards and make us strive for more rewards?an evolutionary beneficial trait. Most dopamine neurons in the midbrain of humans, monkeys, and rodents signal a reward prediction error; they are activated by more reward than predicted (positive prediction error), remain at baseline activity for fully predicted rewards, and show depressed activity with less...
-
Attentional bias for food cues in advertising among overweight and hungry children
Folkvord, F.; Anschutz, D.J.; Buijzen, M.A.
2015-01-01
Attentional bias theory suggests that an increased motivation to receive or avoid a rewarding substance elevates automatic selective attention toward cues that are related to that specific substance. Until now, no study has examined attentional bias toward food cues in food advertisements, even
-
Butterfly Learning and the Diversification of Plant Leaf Shape
Directory of Open Access Journals (Sweden)
Denise Dalbosco Dell'aglio
2016-07-01
Full Text Available Visual cues are important for insects to find flowers and host plants. It has been proposed that the diversity of leaf shape in Passiflora vines could be a result of negative frequency dependent selection driven by visual searching behavior among their butterfly herbivores. Here we tested the hypothesis that Heliconius butterflies use leaf shape as a cue to initiate approach towards a host plant. We first tested for the ability to recognize shapes using a food reward conditioning experiment. Butterflies showed an innate preference for flowers with three and five petals. However, they could be trained to increase the frequency of visits to a non-preferred flower with two petals, indicating an ability to learn to associate shape with a reward. Next we investigated shape learning specifically in the context of oviposition by conditioning females to lay eggs on two shoots associated with different artificial leaf shapes: their own host plant, Passiflora biflora, and a lanceolate non-biflora leaf shape. The conditioning treatment had a significant effect on the approach of butterflies to the two leaf shapes, consistent with a role for shape learning in oviposition behavior. This study is the first to show that Heliconius butterflies use shape as a cue for feeding and oviposition, and can learn shape preference for both flowers and leaves. This demonstrates the potential for Heliconius to drive negative frequency dependent selection on the leaf shape of their Passiflora host plants.
-
Sensitivity for cues predicting reward and punishment in young women with eating disorders
Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline
Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely
-
Sensitivity for cues predicting reward and punishment in young women with eating disorders
Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline
2017-01-01
Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely
-
[Sucrose reward promotes rats' motivation for cocaine].
Li, Yan-Qing; LE, Qiu-Min; Yu, Xiang-Chen; Ma, Lan; Wang, Fei-Fei
2016-06-25
Caloric diet, such as fat and sugar intake, has rewarding effects, and has been indicated to affect the responses to addictive substances in animal experiments. However, the possible association between sucrose reward and the motivation for addictive drugs remains to be elucidated. Thus, we carried out behavioral tests after sucrose self-administration training to determine the effects of sucrose experience on rats' motivation for cocaine, locomotor sensitivity to cocaine, basal locomotor activity, anxiety level, and associative learning ability. The sucrose-experienced (sucrose) group exhibited higher lever press, cocaine infusion and break point, as well as upshift of cocaine dose-response curve in cocaine self-administration test, as compared with the control (chow) group. Additionally, despite similar locomotor activity in open field test and comparable score in cocaine-induced conditioned place preference, the sucrose group showed higher cocaine-induced locomotor sensitivity as compared with the chow group. The anxiety level and the performance in vocal-cue induced fear memory were similar between these two groups in elevated plus maze and fear conditioning tests, respectively. Taken together, our work indicates that sucrose experience promotes the rats' motivation for cocaine.
-
Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning.
Hart, Andrew S; Clark, Jeremy J; Phillips, Paul E M
2015-01-01
Cue- and reward-evoked phasic dopamine activity during Pavlovian and operant conditioning paradigms is well correlated with reward-prediction errors from formal reinforcement learning models, which feature teaching signals in the form of discrepancies between actual and expected reward outcomes. Additionally, in learning tasks where conditioned cues probabilistically predict rewards, dopamine neurons show sustained cue-evoked responses that are correlated with the variance of reward and are maximal to cues predicting rewards with a probability of 0.5. Therefore, it has been suggested that sustained dopamine activity after cue presentation encodes the uncertainty of impending reward delivery. In the current study we examined the acquisition and maintenance of these neural correlates using fast-scan cyclic voltammetry in rats implanted with carbon fiber electrodes in the nucleus accumbens core during probabilistic Pavlovian conditioning. The advantage of this technique is that we can sample from the same animal and recording location throughout learning with single trial resolution. We report that dopamine release in the nucleus accumbens core contains correlates of both expected value and variance. A quantitative analysis of these signals throughout learning, and during the ongoing updating process after learning in probabilistic conditions, demonstrates that these correlates are dynamically encoded during these phases. Peak CS-evoked responses are correlated with expected value and predominate during early learning while a variance-correlated sustained CS signal develops during the post-asymptotic updating phase. Copyright © 2014 Elsevier Inc. All rights reserved.
-
Interaction between scene-based and array-based contextual cueing.
Rosenbaum, Gail M; Jiang, Yuhong V
2013-07-01
Contextual cueing refers to the cueing of spatial attention by repeated spatial context. Previous studies have demonstrated distinctive properties of contextual cueing by background scenes and by an array of search items. Whereas scene-based contextual cueing reflects explicit learning of the scene-target association, array-based contextual cueing is supported primarily by implicit learning. In this study, we investigated the interaction between scene-based and array-based contextual cueing. Participants searched for a target that was predicted by both the background scene and the locations of distractor items. We tested three possible patterns of interaction: (1) The scene and the array could be learned independently, in which case cueing should be expressed even when only one cue was preserved; (2) the scene and array could be learned jointly, in which case cueing should occur only when both cues were preserved; (3) overshadowing might occur, in which case learning of the stronger cue should preclude learning of the weaker cue. In several experiments, we manipulated the nature of the contextual cues present during training and testing. We also tested explicit awareness of scenes, scene-target associations, and arrays. The results supported the overshadowing account: Specifically, scene-based contextual cueing precluded array-based contextual cueing when both were predictive of the location of a search target. We suggest that explicit, endogenous cues dominate over implicit cues in guiding spatial attention.
-
A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex.
Huertas, Marco A; Hussain Shuler, Marshall G; Shouval, Harel Z
2015-09-16
Many actions performed by animals and humans depend on an ability to learn, estimate, and produce temporal intervals of behavioral relevance. Exemplifying such learning of cued expectancies is the observation of reward-timing activity in the primary visual cortex (V1) of rodents, wherein neural responses to visual cues come to predict the time of future reward as behaviorally experienced in the past. These reward-timing responses exhibit significant heterogeneity in at least three qualitatively distinct classes: sustained increase or sustained decrease in firing rate until the time of expected reward, and a class of cells that reach a peak in firing at the expected delay. We elaborate upon our existing model by including inhibitory and excitatory units while imposing simple connectivity rules to demonstrate what role these inhibitory elements and the simple architectures play in sculpting the response dynamics of the network. We find that simply adding inhibition is not sufficient for obtaining the different distinct response classes, and that a broad distribution of inhibitory projections is necessary for obtaining peak-type responses. Furthermore, although changes in connection strength that modulate the effects of inhibition onto excitatory units have a strong impact on the firing rate profile of these peaked responses, the network exhibits robustness in its overall ability to predict the expected time of reward. Finally, we demonstrate how the magnitude of expected reward can be encoded at the expected delay in the network and how peaked responses express this reward expectancy. Heterogeneity in single-neuron responses is a common feature of neuronal systems, although sometimes, in theoretical approaches, it is treated as a nuisance and seldom considered as conveying a different aspect of a signal. In this study, we focus on the heterogeneous responses in the primary visual cortex of rodents trained with a predictable delayed reward time. We describe under what
-
Directory of Open Access Journals (Sweden)
Simon eFriedmann
2013-09-01
Full Text Available In this study, we propose and analyze in simulations a new, highly flexible method of imple-menting synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. Thestudy focuses on globally modulated STDP, as a special use-case of this method. Flexibility isachieved by embedding a general-purpose processor dedicated to plasticity into the wafer. Toevaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spiketrain learning task. A single layer of neurons is trained to fire at specific points in time withonly the reward as feedback. This model is simulated to measure its performance, i.e. the in-crease in received reward after learning. Using this performance as baseline, we then simulatethe model with various constraints imposed by the proposed implementation and compare theperformance. The simulated constraints include discretized synaptic weights, a restricted inter-face between analog synapses and embedded processor, and mismatch of analog circuits. Wefind that probabilistic updates can increase the performance of low-resolution weights, a simpleinterface between analog synapses and processor is sufficient for learning, and performance isinsensitive to mismatch. Further, we consider communication latency between wafer and theconventional control computer system that is simulating the environment. This latency increasesthe delay, with which the reward is sent to the embedded processor. Because of the time continu-ous operation of the analog synapses, delay can cause a deviation of the updates as compared tothe not delayed situation. We find that for highly accelerated systems latency has to be kept to aminimum. This study demonstrates the suitability of the proposed implementation to emulatethe selected reward modulated STDP learning rule. It is therefore an ideal candidate for imple-mentation in an upgraded version of the wafer-scale system developed within the BrainScaleSproject.
-
Friedmann, Simon; Frémaux, Nicolas; Schemmel, Johannes; Gerstner, Wulfram; Meier, Karlheinz
2013-01-01
In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e., the increase in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted interface between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continuous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for implementation in an upgraded version of the wafer-scale system developed within the BrainScaleS project.
-
Guitart-Masip, Marc; Economides, Marcos; Huys, Quentin J M; Frank, Michael J; Chowdhury, Rumana; Duzel, Emrah; Dayan, Peter; Dolan, Raymond J
2014-03-01
Decision-making involves two fundamental axes of control namely valence, spanning reward and punishment, and action, spanning invigoration and inhibition. We recently exploited a go/no-go task whose contingencies explicitly decouple valence and action to show that these axes are inextricably coupled during learning. This results in a disadvantage in learning to go to avoid punishment and in learning to no-go to obtain a reward. The neuromodulators dopamine and serotonin are likely to play a role in these asymmetries: Dopamine signals anticipation of future rewards and is also involved in an invigoration of motor responses leading to reward, but it also arbitrates between different forms of control. Conversely, serotonin is implicated in motor inhibition and punishment processing. To investigate the role of dopamine and serotonin in the interaction between action and valence during learning.Methods We combined computational modeling with pharmacological manipulation in 90 healthy human volunteers, using levodopa and citalopram to affect dopamine and serotonin, respectively. We found that, after administration of levodopa,action learning was less affected by outcome valence when compared with the placebo and citalopram groups. This highlights in this context a predominant effect of levodopa in controlling the balance between different forms of control.Citalopram had distinct effects, increasing participants'tendency to perform active responses independent of outcome valence, consistent with a role in decreasing motor inhibition. Our findings highlight the rich complexities of the roles played by dopamine and serotonin during instrumental learning.
-
The issues of goal setting, interest, and reward in self-regulated learning
Okazaki, Makiko; 岡崎, 万紀子
2011-01-01
Self-regulated learning (SRL) is the theory developed in the field of educational psychology. SRL is based on the idea that behaviours are regulated by the self-concept. This self-concept is the basis of the social cognitive theory which does not depend on students' innate motivation (intrinsic motivation) at the initial stage of learning. This study examines three motivation-related factors in the concept of SRL: goals, interest, and rewards followed by the suggestion of a students' learning...
-
Biesdorf, C; Wang, A-L; Topic, B; Petri, D; Milani, H; Huston, J P; de Souza Silva, M A
2015-09-01
Microdialysis studies in rat have generally shown that appetitive stimuli release dopamine (DA) in the nucleus accumbens (NAc) shell and core. Here we examined the release of DA in the NAc during delivery of reward (food) and during extinction of food reward in the freely moving animal by use of in vivo microdialysis and HPLC. Fifty-two male Wistar rats were trained to receive food reward associated with appearance of cue-lights in a Skinner-box during in vivo microdialysis. Different behavioral protocols were used to assess the effects of extinction on DA and its metabolites. Results Exp. 1: (a) During a 20-min period of cued reward delivery, DA increased significantly in the NAc core, but not shell subregion; (b) for the next 60min period half of the rats underwent immediate extinction (with the CS light presented during non-reward) and the other half did not undergo extinction to the cue lights (CS was not presented during non-reward). DA remained significantly increased in both groups, providing no evidence for a decrease in DA during extinction in either NAc core or shell regions. (c) In half of the animals of the group that was not subjected to extinction, the cue lights were turned on for 30min, thus, initiating extinction to cue CS at a 1h delay from the period of reward. In this group DA in the NAc core, but not shell, significantly decreased. Behavioral analysis showed that while grooming is an indicator of extinction-induced behavior, glances toward the cue-lights (sign tracking) are an index of resistance to extinction. Results Exp. 2: (a) As in Exp. 1, during a 30-min period of cued reward delivery, DA levels again increased significantly in the NAc core but not in the NAc shell. (b) When extinction (the absence of reward with the cue lights presented) was administered 24h after the last reward session, DA again significantly decreased in the NAc core, but not in the NAc shell. (a) These results confirm the importance of DA release in the NAc for
-
Overt and covert attention to location-based reward.
McCoy, Brónagh; Theeuwes, Jan
2018-01-01
Recent research on the impact of location-based reward on attentional orienting has indicated that reward factors play an influential role in spatial priority maps. The current study investigated whether and how reward associations based on spatial location translate from overt eye movements to covert attention. If reward associations can be tied to locations in space, and if overt and covert attention rely on similar overlapping neuronal populations, then both overt and covert attentional measures should display similar spatial-based reward learning. Our results suggest that location- and reward-based changes in one attentional domain do not lead to similar changes in the other. Specifically, although we found similar improvements at differentially rewarded locations during overt attentional learning, this translated to the least improvement at a highly rewarded location during covert attention. We interpret this as the result of an increased motivational link between the high reward location and the trained eye movement response acquired during learning, leading to a relative slowing during covert attention when the eyes remained fixated and the saccade response was suppressed. In a second experiment participants were not required to keep fixated during the covert attention task and we no longer observed relative slowing at the high reward location. Furthermore, the second experiment revealed no covert spatial priority of rewarded locations. We conclude that the transfer of location-based reward associations is intimately linked with the reward-modulated motor response employed during learning, and alternative attentional and task contexts may interfere with learned spatial priorities. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.
-
A Computer-Assisted Learning Model Based on the Digital Game Exponential Reward System
Moon, Man-Ki; Jahng, Surng-Gahb; Kim, Tae-Yong
2011-01-01
The aim of this research was to construct a motivational model which would stimulate voluntary and proactive learning using digital game methods offering players more freedom and control. The theoretical framework of this research lays the foundation for a pedagogical learning model based on digital games. We analyzed the game reward system, which…
-
Ghrelin enhances cue-induced bar pressing for high fat food.
St-Onge, Veronique; Watts, Alexander; Abizaid, Alfonso
2016-02-01
Ghrelin is an orexigenic hormone produced by the stomach that acts on growth hormone secretagogue receptors (GHSRs) both peripherally and centrally. The presence of GHSRs in the ventral tegmental area (VTA) suggests that ghrelin signaling at this level may increase the incentive value of palatable foods as well as other natural and artificial rewards. The present investigation sought to determine if ghrelin plays a role in relapse to such foods following a period of abstinence. To achieve this, thirty-six male Long Evans rats were trained to press a lever to obtain a high fat chocolate food reward on a fixed ratio schedule of 1. Following an extinction period during which lever presses were not reinforced, rats were implanted with a cannula connected to a minipump that continuously delivered ghrelin, a GHSR antagonist ([d-Lys-3]-GHRP-6), or saline in the VTA for 14days. One week later, food reward-associated cues, food reward priming, and an overnight fast were used to induce reinstatement of the lever pressing response. Our results indicate that intra-VTA ghrelin enhances cue-induced reinstatement of responses for palatable food pellets. To the extent that the reinstatement paradigm is considered a valid model of relapse in humans, this suggests that ghrelin signaling facilitates relapse to preferred foods in response to food cues through GHSR signaling in the VTA. Copyright © 2015 Elsevier Inc. All rights reserved.
-
Dasgupta, Sakyasingha; Wörgötter, Florentin; Manoonpong, Poramate
2014-01-01
Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.
-
Personalizing a Service Robot by Learning Human Habits from Behavioral Footprints
Directory of Open Access Journals (Sweden)
Kun Li
2015-03-01
Full Text Available For a domestic personal robot, personalized services are as important as predesigned tasks, because the robot needs to adjust the home state based on the operator's habits. An operator's habits are composed of cues, behaviors, and rewards. This article introduces behavioral footprints to describe the operator's behaviors in a house, and applies the inverse reinforcement learning technique to extract the operator's habits, represented by a reward function. We implemented the proposed approach with a mobile robot on indoor temperature adjustment, and compared this approach with a baseline method that recorded all the cues and behaviors of the operator. The result shows that the proposed approach allows the robot to reveal the operator's habits accurately and adjust the environment state accordingly.
-
Morita, Kenji; Morishima, Mieko; Sakai, Katsuyuki; Kawaguchi, Yasuo
2013-05-15
Humans and animals take actions quickly when they expect that the actions lead to reward, reflecting their motivation. Injection of dopamine receptor antagonists into the striatum has been shown to slow such reward-seeking behavior, suggesting that dopamine is involved in the control of motivational processes. Meanwhile, neurophysiological studies have revealed that phasic response of dopamine neurons appears to represent reward prediction error, indicating that dopamine plays central roles in reinforcement learning. However, previous attempts to elucidate the mechanisms of these dopaminergic controls have not fully explained how the motivational and learning aspects are related and whether they can be understood by the way the activity of dopamine neurons itself is controlled by their upstream circuitries. To address this issue, we constructed a closed-circuit model of the corticobasal ganglia system based on recent findings regarding intracortical and corticostriatal circuit architectures. Simulations show that the model could reproduce the observed distinct motivational effects of D1- and D2-type dopamine receptor antagonists. Simultaneously, our model successfully explains the dopaminergic representation of reward prediction error as observed in behaving animals during learning tasks and could also explain distinct choice biases induced by optogenetic stimulation of the D1 and D2 receptor-expressing striatal neurons. These results indicate that the suggested roles of dopamine in motivational control and reinforcement learning can be understood in a unified manner through a notion that the indirect pathway of the basal ganglia represents the value of states/actions at a previous time point, an empirically driven key assumption of our model.
-
Koenig, Stephan; Uengoer, Metin; Lachnit, Harald
2017-01-01
We conducted a human fear conditioning experiment in which three different color cues were followed by an aversive electric shock on 0, 50, and 100% of the trials, and thus induced low (L), partial (P), and high (H) shock expectancy, respectively. The cues differed with respect to the strength of their shock association (L H). During conditioning we measured pupil dilation and ocular fixations to index differences in the attentional processing of the cues. After conditioning, the shock-associated colors were introduced as irrelevant distracters during visual search for a shape target while shocks were no longer administered and we analyzed the cues’ potential to capture and hold overt attention automatically. Our findings suggest that fear conditioning creates an automatic attention bias for the conditioned cues that depends on their correlation with the aversive outcome. This bias was exclusively linked to the strength of the cues’ shock association for the early attentional processing of cues in the visual periphery, but additionally was influenced by the uncertainty of the shock prediction after participants fixated on the cues. These findings are in accord with attentional learning theories that formalize how associative learning shapes automatic attention. PMID:28588466
-
Aberg, Kristoffer Carl; Doell, Kimberly C; Schwartz, Sophie
2015-10-28
Some individuals are better at learning about rewarding situations, whereas others are inclined to avoid punishments (i.e., enhanced approach or avoidance learning, respectively). In reinforcement learning, action values are increased when outcomes are better than predicted (positive prediction errors [PEs]) and decreased for worse than predicted outcomes (negative PEs). Because actions with high and low values are approached and avoided, respectively, individual differences in the neural encoding of PEs may influence the balance between approach-avoidance learning. Recent correlational approaches also indicate that biases in approach-avoidance learning involve hemispheric asymmetries in dopamine function. However, the computational and neural mechanisms underpinning such learning biases remain unknown. Here we assessed hemispheric reward asymmetry in striatal activity in 34 human participants who performed a task involving rewards and punishments. We show that the relative difference in reward response between hemispheres relates to individual biases in approach-avoidance learning. Moreover, using a computational modeling approach, we demonstrate that better encoding of positive (vs negative) PEs in dopaminergic midbrain regions is associated with better approach (vs avoidance) learning, specifically in participants with larger reward responses in the left (vs right) ventral striatum. Thus, individual dispositions or traits may be determined by neural processes acting to constrain learning about specific aspects of the world. Copyright © 2015 the authors 0270-6474/15/3514491-10$15.00/0.
-
Psychogenic and neural visual-cue response in PD dopamine dysregulation syndrome.
Loane, Clare; Wu, Kit; O'Sullivan, Sean S; Lawrence, Andrew D; Woodhead, Zoe; Lees, Andrew J; Piccini, Paola; Politis, Marios
2015-11-01
Dopamine dysregulation syndrome (DDS) in Parkinson's disease (PD) patients refers to the compulsive use of dopaminergic replacement therapy and has serious psycho-social consequences. Mechanisms underlying DDS are not clear although has been linked to dysfunctional brain reward networks. With fMRI, we investigate behavioral and neural response to drug-cues in six PD DDS patients and 12 PD control patients in both the ON and OFF medication state. Behavioral measures of liking, wanting and subjectively 'feeling ON medication' were also collected. Behaviorally, PD DDS patients feel less ON and want their drugs more at baseline compared to PD controls. Following drug-cue exposure, PD DDS patients feel significantly more ON medication, which correlates with significant increases in reward related regions. The results demonstrate that exposure to drug-cues increases the subjective feeling of being 'ON' medication which corresponds to dysfunctional activation in reward related regions in PD DDS patients. These findings should be extended in future studies. Visual stimuli being sufficient to elicit behavioral response through neuroadaptations could have direct implications to the management of addictive behavior. Copyright © 2015 Elsevier Ltd. All rights reserved.
-
Morie, Kristen P; De Sanctis, Pierfilippo; Garavan, Hugh; Foxe, John J
2016-03-01
We investigated anticipatory and consummatory reward processing in cocaine addiction. In addition, we set out to assess whether task-monitoring systems were appropriately recalibrated in light of variable reward schedules. We also examined neural measures of task-monitoring and reward processing as a function of hedonic tone, since anhedonia is a vulnerability marker for addiction that is obviously germane in the context of reward processing. High-density event-related potentials were recorded while participants performed a speeded response task that systematically varied anticipated probabilities of reward receipt. The paradigm dissociated feedback regarding task success (or failure) from feedback regarding the value of reward (or loss), so that task-monitoring and reward processing could be examined in partial isolation. Twenty-three active cocaine abusers and 23 age-matched healthy controls participated. Cocaine abusers showed amplified anticipatory responses to reward predictive cues, but crucially, these responses were not as strongly modulated by reward probability as in controls. Cocaine users also showed blunted responses to feedback about task success or failure and did not use this information to update predictions about reward. In turn, they showed clearly blunted responses to reward feedback. In controls and users, measures of anhedonia were associated with reward motivation. In cocaine users, anhedonia was also associated with diminished monitoring and reward feedback responses. Findings imply that reward anticipation and monitoring deficiencies in addiction are associated with increased responsiveness to reward cues but impaired ability to predict reward in light of task contingencies, compounded by deficits in responding to actual reward outcomes.
-
Altvater-Mackensen, Nicole; Grossmann, Tobias
2015-01-01
Infants' language exposure largely involves face-to-face interactions providing acoustic and visual speech cues but also social cues that might foster language learning. Yet, both audiovisual speech information and social information have so far received little attention in research on infants' early language development. Using a preferential…
-
Mattfeld, Aaron T.; Gluck, Mark A.; Stark, Craig E. L.
2011-01-01
The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what…
-
Baseline frontostriatal-limbic connectivity predicts reward-based memory formation.
Hamann, Janne M; Dayan, Eran; Hummel, Friedhelm C; Cohen, Leonardo G
2014-12-01
Reward mediates the acquisition and long-term retention of procedural skills in humans. Yet, learning under rewarded conditions is highly variable across individuals and the mechanisms that determine interindividual variability in rewarded learning are not known. We postulated that baseline functional connectivity in a large-scale frontostriatal-limbic network could predict subsequent interindividual variability in rewarded learning. Resting-state functional MRI was acquired in two groups of subjects (n = 30) who then trained on a visuomotor procedural learning task with or without reward feedback. We then tested whether baseline functional connectivity within the frontostriatal-limbic network predicted memory strength measured immediately, 24 h and 1 month after training in both groups. We found that connectivity in the frontostriatal-limbic network predicted interindividual variability in the rewarded but not in the unrewarded learning group. Prediction was strongest for long-term memory. Similar links between connectivity and reward-based memory were absent in two control networks, a fronto-parieto-temporal language network and the dorsal attention network. The results indicate that baseline functional connectivity within the frontostriatal-limbic network successfully predicts long-term retention of rewarded learning. © 2014 Wiley Periodicals, Inc.
-
Blum, Kenneth; Gardner, Eliot; Oscar-Berman, Marlene; Gold, Mark
2012-01-01
In an attempt to resolve controversy regarding the causal contributions of mesolimbic dopamine (DA) systems to reward, we evaluate the three main competing explanatory categories: "liking,"learning," and "wanting" [1]. That is, DA may mediate (a) the hedonic impact of reward (liking), (b) learned predictions about rewarding effects (learning), or (c) the pursuit of rewards by attributing incentive salience to reward-related stimuli (wanting). We evaluate these hypotheses, especially as they relate to the Reward Deficiency Syndrome (RDS), and we find that the incentive salience or "wanting" hypothesis of DA function is supported by a majority of the evidence. Neuroimaging studies have shown that drugs of abuse, palatable foods, and anticipated behaviors such as sex and gaming affect brain regions involving reward circuitry, and may not be unidirectional. Drugs of abuse enhance DA signaling and sensitize mesolimbic mechanisms that evolved to attribute incentive salience to rewards. Addictive drugs have in common that they are voluntarily selfadministered, they enhance (directly or indirectly) dopaminergic synaptic function in the nucleus accumbens (NAC), and they stimulate the functioning of brain reward circuitry (producing the "high" that drug users seek). Although originally believed simply to encode the set point of hedonic tone, these circuits now are believed to be functionally more complex, also encoding attention, reward expectancy, disconfirmation of reward expectancy, and incentive motivation. Elevated stress levels, together with polymorphisms of dopaminergic genes and other neurotransmitter genetic variants, may have a cumulative effect on vulnerability to addiction. The RDS model of etiology holds very well for a variety of chemical and behavioral addictions.
-
DiFeliceantonio, Alexandra G.; Berridge, Kent C.
2012-01-01
Pavlovian cues that have been paired with reward can gain incentive salience. Drug addicts find drug cues motivationally attractive and binge eaters are attracted by food cues. But the level of incentive salience elicited by a cue re-encounter still varies across time and brain states. In an animal model, cues become attractive and ‘wanted’ in an ‘autoshaping’ paradigm, where different targets of incentive salience emerge for different individuals. Some individuals (sign-trackers) find a predictive discrete cue attractive while others find a reward contiguous and goal cue more attractive (location where reward arrives: goal-trackers). Here we assessed whether central amygdala mu opioid receptor stimulation enhances the phasic incentive salience of the goal-cue for goal-trackers during moments of predictive cue presence (expressed in both approach and consummatory behaviors to goal cue), just as it enhances the attractiveness of the predictive cue target for sign-trackers. Using detailed video analysis we measured the approaches, nibbles, sniffs, and bites directed at their preferred target for both sign-trackers and goal-trackers. We report that DAMGO microinjections in central amygdala made goal-trackers, like sign-trackers, show phasic increases in appetitive nibbles and sniffs directed at the goal-cue expressed selectively whenever the predictive cue was present. This indicates enhancement of incentive salience attributed by both goal trackers and sign-trackers, but attributed in different directions: each to their own target cue. For both phenotypes, amygdala opioid stimulation makes the individual’s prepotent cue into a stronger motivational magnet at phasic moments triggered by a CS that predicts the reward UCS. PMID:22391118
-
Reward-dependent learning in neuronal networks for planning and decision making.
Dehaene, S; Changeux, J P
2000-01-01
Neuronal network models have been proposed for the organization of evaluation and decision processes in prefrontal circuitry and their putative neuronal and molecular bases. The models all include an implementation and simulation of an elementary reward mechanism. Their central hypothesis is that tentative rules of behavior, which are coded by clusters of active neurons in prefrontal cortex, are selected or rejected based on an evaluation by this reward signal, which may be conveyed, for instance, by the mesencephalic dopaminergic neurons with which the prefrontal cortex is densely interconnected. At the molecular level, the reward signal is postulated to be a neurotransmitter such as dopamine, which exerts a global modulatory action on prefrontal synaptic efficacies, either via volume transmission or via targeted synaptic triads. Negative reinforcement has the effect of destabilizing the currently active rule-coding clusters; subsequently, spontaneous activity varies again from one cluster to another, giving the organism the chance to discover and learn a new rule. Thus, reward signals function as effective selection signals that either maintain or suppress currently active prefrontal representations as a function of their current adequacy. Simulations of this variation-selection have successfully accounted for the main features of several major tasks that depend on prefrontal cortex integrity, such as the delayed-response test, the Wisconsin card sorting test, the Tower of London test and the Stroop test. For the more complex tasks, we have found it necessary to supplement the external reward input with a second mechanism that supplies an internal reward; it consists of an auto-evaluation loop which short-circuits the reward input from the exterior. This allows for an internal evaluation of covert motor intentions without actualizing them as behaviors, by simply testing them covertly by comparison with memorized former experiences. This element of architecture
-
How does reward compete with goal-directed and stimulus-driven shifts of attention?
Bourgeois, Alexia; Neveu, Rémi; Bayle, Dimitri J; Vuilleumier, Patrik
2017-01-01
In order to behave adaptively, attention can be directed in space either voluntarily (i.e. endogenously) according to strategic goals, or involuntarily (i.e. exogenously) through reflexive capture by salient or novel events. The emotional or motivational values of stimuli can also influence attentional orienting. However, little is known about how reward-related effects compete or interact with endogenous and exogenous attention mechanisms. Here we designed a visual search paradigm in which goal-driven and stimulus-driven shifts of attention were manipulated by classic spatial cueing procedures, while an irrelevant, but previously rewarded stimulus also appeared as a distractor and hence competed with both types of spatial attention during search. Our results demonstrated that stimuli previously associated with a high monetary reward received higher attentional priority in the subsequent visual search task, even though these stimuli and reward were no longer task-relevant, mitigating the attentional orienting induced by both endogenous and exogenous cues.
-
Chung, Yu Sun; Barch, Deanna M
2016-04-01
Schizophrenia is characterized by deficits of context processing, thought to be related to dorsolateral prefrontal cortex (DLPFC) impairment. Despite emerging evidence suggesting a crucial role of the DLPFC in integrating reward and goal information, we do not know whether individuals with schizophrenia can represent and integrate reward-related context information to modulate cognitive control. To address this question, 36 individuals with schizophrenia (n = 29) or schizoaffective disorder (n = 7) and 27 healthy controls performed a variant of a response conflict task (Padmala & Pessoa, 2011) during fMRI scanning, in both baseline and reward conditions, with monetary incentives on some reward trials. We used a mixed state-item design that allowed us to examine both sustained and transient reward effects on cognitive control. Different from predictions about impaired DLPFC function in schizophrenia, we found an intact pattern of increased sustained DLPFC activity during reward versus baseline blocks in individuals with schizophrenia at a group level but blunted sustained activations in the putamen. Contrary to our predictions, individuals with schizophrenia showed blunted cue-related activations in several regions of the basal ganglia responding to reward-predicting cues. Importantly, as predicted, individual differences in anhedonia/amotivation symptoms severity were significantly associated with reduced sustained DLPFC activation in the same region that showed overall increased activity as a function of reward. These results suggest that individual differences in motivational impairments in schizophrenia may be related to dysfunction of the DLPFC and striatum in motivationally salient situations. (c) 2016 APA, all rights reserved).
-
What Works Clearinghouse, 2010
2010-01-01
This paper presents an updated WWC (What Works Clearinghouse) Review of the Article "Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning". The study examined the effects of different reward systems used in group learning situations on the math skills of African-American and White students. The…
-
Rule learning in autism: the role of reward type and social context.
Jones, E J H; Webb, S J; Estes, A; Dawson, G
2013-01-01
Learning abstract rules is central to social and cognitive development. Across two experiments, we used Delayed Non-Matching to Sample tasks to characterize the longitudinal development and nature of rule-learning impairments in children with Autism Spectrum Disorder (ASD). Results showed that children with ASD consistently experienced more difficulty learning an abstract rule from a discrete physical reward than children with DD. Rule learning was facilitated by the provision of more concrete reinforcement, suggesting an underlying difficulty in forming conceptual connections. Learning abstract rules about social stimuli remained challenging through late childhood, indicating the importance of testing executive functions in both social and non-social contexts.
-
Novelty enhances visual salience independently of reward in the parietal lobe.
Foley, Nicholas C; Jangraw, David C; Peck, Christopher; Gottlieb, Jacqueline
2014-06-04
Novelty modulates sensory and reward processes, but it remains unknown how these effects interact, i.e., how the visual effects of novelty are related to its motivational effects. A widespread hypothesis, based on findings that novelty activates reward-related structures, is that all the effects of novelty are explained in terms of reward. According to this idea, a novel stimulus is by default assigned high reward value and hence high salience, but this salience rapidly decreases if the stimulus signals a negative outcome. Here we show that, contrary to this idea, novelty affects visual salience in the monkey lateral intraparietal area (LIP) in ways that are independent of expected reward. Monkeys viewed peripheral visual cues that were novel or familiar (received few or many exposures) and predicted whether the trial will have a positive or a negative outcome--i.e., end in a reward or a lack of reward. We used a saccade-based assay to detect whether the cues automatically attracted or repelled attention from their visual field location. We show that salience--measured in saccades and LIP responses--was enhanced by both novelty and positive reward associations, but these factors were dissociable and habituated on different timescales. The monkeys rapidly recognized that a novel stimulus signaled a negative outcome (and withheld anticipatory licking within the first few presentations), but the salience of that stimulus remained high for multiple subsequent presentations. Therefore, novelty can provide an intrinsic bonus for attention that extends beyond the first presentation and is independent of physical rewards. Copyright © 2014 the authors 0270-6474/14/347947-11$15.00/0.
-
Reward System Activation in Response to Alcohol Advertisements Predicts College Drinking.
Courtney, Andrea L; Rapuano, Kristina M; Sargent, James D; Heatherton, Todd F; Kelley, William M
2018-01-01
In this study, we assess whether activation of the brain's reward system in response to alcohol advertisements is associated with college drinking. Previous research has established a relationship between exposure to alcohol marketing and underage drinking. Within other appetitive domains, the relationship between cue exposure and behavioral enactment is known to rely on activation of the brain's reward system. However, the relationship between neural activation to alcohol advertisements and alcohol consumption has not been studied in a nondisordered population. In this cross-sectional study, 53 college students (32 women) completed a functional magnetic resonance imaging scan while viewing alcohol, food, and control (car and technology) advertisements. Afterward, they completed a survey about their alcohol consumption (including frequency of drinking, typical number of drinks consumed, and frequency of binge drinking) over the previous month. In 43 participants (24 women) meeting inclusion criteria, viewing alcohol advertisements elicited activation in the left orbitofrontal cortex and bilateral ventral striatum-regions of the reward system that typically activate to other appetitive rewards and relate to consumption behaviors. Moreover, the level of self-reported drinking correlated with the magnitude of activation in the left orbitofrontal cortex. Results suggest that alcohol cues are processed within the reward system in a way that may motivate drinking behavior.
-
The CB1 Receptor as an Important Mediator of Hedonic Reward Processing
Friemel, Chris M; Zimmer, Andreas; Schneider, Miriam
2014-01-01
The endocannabinoid (ECB) system has emerged recently as a key mediator for reward processing. It is well known that cannabinoids affect appetitive learning processes and can induce reinforcing and rewarding effects. However, the involvement of the ECB system in hedonic aspects of reward-related behavior is not completely understood. With the present study, we investigated the modulatory role of the ECB system on hedonic perception, measured by the pleasure attenuated startle (PAS) paradigm for a palatable food reward. Here, a conditioned odor is thought to induce a pleasant affective state that attenuates an aversive reflex—the acoustic startle response. Modulatory effects of the CB1 receptor antagonist/inverse agonist SR1411716 and the cannabinoid agonist WIN 55 212-2 on PAS were examined in rats. PAS was also measured in CB1 receptor knockout (KO) and wild-type (WT) mice. Pharmacological inhibition as well as the absence of CB1 receptors was found to reduce PAS, whereas WIN 55 212-2 administration increased PAS. Finally, presentation of a conditioned reward cue was found to induce striatal FosB/ΔFosB expression in WT mice, but not in KO mice, indicating a reduced stimulation of reward-related brain regions in conditioned KO mice by odor presentation. We here show that in addition to our previous studies in rats, PAS may also serve as a valuable and suitable measure to assess hedonic processing in mice. Our data further indicate that the ECB system, and in particular CB1 receptor signaling, appears to be highly important for the mediation of hedonic aspects of reward processing. PMID:24718372
-
Age-related changes in contextual associative learning.
Luu, Trinh T; Pirogovsky, Eva; Gilbert, Paul E
2008-01-01
The hippocampus plays a critical role in processing contextual information. Although age-related changes in the hippocampus are well documented in humans, nonhuman primates, and rodents, few studies have examined contextual learning deficits in old rats. The present study investigated age-related differences in contextual associative learning in young (6 mo) and old (24 mo) rats using olfactory stimuli. Stimuli consisted of common odors mixed in sand and placed in clear plastic cups. Testing was conducted in two boxes that represented two different contexts (Context 1 and Context 2). The contexts varied based on environmental features of the box such as color (black vs. white), visual cues on the walls of the box, and flooring texture. Each rat was simultaneously presented with two cups, one filled with Odor A and one filled with Odor B in each context. In Context 1, the rat received a food reward for digging in the cup containing Odor A, but did not receive a food reward for digging in the cup containing Odor B. In Context 2, the rat was rewarded for digging in the cup containing Odor B, but did receive a reward for digging in the cup containing Odor A. Therefore, the rat learned to associate Context 1 with Odor A and Context 2 with Odor B. The rat was tested for eight days using the same odor problem throughout all days of testing. The results showed no significant difference between young and old rats on the first two days of testing; however, young rats significantly outperformed old rats on Day 3. Young rats continued to maintain superior performance compared to old rats on Days 4-8. The results suggest that aging results in functional impairments in brain regions that support memory for associations between specific cues and their respective context.
-
Minamimoto, Takafumi; La Camera, Giancarlo; Richmond, Barry J
2009-01-01
Motivation is usually inferred from the likelihood or the intensity with which behavior is carried out. It is sensitive to external factors (e.g., the identity, amount, and timing of a rewarding outcome) and internal factors (e.g., hunger or thirst). We trained macaque monkeys to perform a nonchoice instrumental task (a sequential red-green color discrimination) while manipulating two external factors: reward size and delay-to-reward. We also inferred the state of one internal factor, level of satiation, by monitoring the accumulated reward. A visual cue indicated the forthcoming reward size and delay-to-reward in each trial. The fraction of trials completed correctly by the monkeys increased linearly with reward size and was hyperbolically discounted by delay-to-reward duration, relations that are similar to those found in free operant and choice tasks. The fraction of correct trials also decreased progressively as a function of the satiation level. Similar (albeit noiser) relations were obtained for reaction times. The combined effect of reward size, delay-to-reward, and satiation level on the proportion of correct trials is well described as a multiplication of the effects of the single factors when each factor is examined alone. These results provide a quantitative account of the interaction of external and internal factors on instrumental behavior, and allow us to extend the concept of subjective value of a rewarding outcome, usually confined to external factors, to account also for slow changes in the internal drive of the subject.
-
Learning to maximize reward rate: a model based on semi-Markov decision processes.
Khodadadi, Arash; Fakhari, Pegah; Busemeyer, Jerome R
2014-01-01
WHEN ANIMALS HAVE TO MAKE A NUMBER OF DECISIONS DURING A LIMITED TIME INTERVAL, THEY FACE A FUNDAMENTAL PROBLEM: how much time they should spend on each decision in order to achieve the maximum possible total outcome. Deliberating more on one decision usually leads to more outcome but less time will remain for other decisions. In the framework of sequential sampling models, the question is how animals learn to set their decision threshold such that the total expected outcome achieved during a limited time is maximized. The aim of this paper is to provide a theoretical framework for answering this question. To this end, we consider an experimental design in which each trial can come from one of the several possible "conditions." A condition specifies the difficulty of the trial, the reward, the penalty and so on. We show that to maximize the expected reward during a limited time, the subject should set a separate value of decision threshold for each condition. We propose a model of learning the optimal value of decision thresholds based on the theory of semi-Markov decision processes (SMDP). In our model, the experimental environment is modeled as an SMDP with each "condition" being a "state" and the value of decision thresholds being the "actions" taken in those states. The problem of finding the optimal decision thresholds then is cast as the stochastic optimal control problem of taking actions in each state in the corresponding SMDP such that the average reward rate is maximized. Our model utilizes a biologically plausible learning algorithm to solve this problem. The simulation results show that at the beginning of learning the model choses high values of decision threshold which lead to sub-optimal performance. With experience, however, the model learns to lower the value of decision thresholds till finally it finds the optimal values.
-
Schönberg, Tom; Daw, Nathaniel D; Joel, Daphna; O'Doherty, John P
2007-11-21
The computational framework of reinforcement learning has been used to forward our understanding of the neural mechanisms underlying reward learning and decision-making behavior. It is known that humans vary widely in their performance in decision-making tasks. Here, we used a simple four-armed bandit task in which subjects are almost evenly split into two groups on the basis of their performance: those who do learn to favor choice of the optimal action and those who do not. Using models of reinforcement learning we sought to determine the neural basis of these intrinsic differences in performance by scanning both groups with functional magnetic resonance imaging. We scanned 29 subjects while they performed the reward-based decision-making task. Our results suggest that these two groups differ markedly in the degree to which reinforcement learning signals in the striatum are engaged during task performance. While the learners showed robust prediction error signals in both the ventral and dorsal striatum during learning, the nonlearner group showed a marked absence of such signals. Moreover, the magnitude of prediction error signals in a region of dorsal striatum correlated significantly with a measure of behavioral performance across all subjects. These findings support a crucial role of prediction error signals, likely originating from dopaminergic midbrain neurons, in enabling learning of action selection preferences on the basis of obtained rewards. Thus, spontaneously observed individual differences in decision making performance demonstrate the suggested dependence of this type of learning on the functional integrity of the dopaminergic striatal system in humans.
-
Directory of Open Access Journals (Sweden)
Stephan Koenig
2017-05-01
Full Text Available We conducted a human fear conditioning experiment in which three different color cues were followed by an aversive electric shock on 0, 50, and 100% of the trials, and thus induced low (L, partial (P, and high (H shock expectancy, respectively. The cues differed with respect to the strength of their shock association (L < P < H and the uncertainty of their prediction (L < P > H. During conditioning we measured pupil dilation and ocular fixations to index differences in the attentional processing of the cues. After conditioning, the shock-associated colors were introduced as irrelevant distracters during visual search for a shape target while shocks were no longer administered and we analyzed the cues’ potential to capture and hold overt attention automatically. Our findings suggest that fear conditioning creates an automatic attention bias for the conditioned cues that depends on their correlation with the aversive outcome. This bias was exclusively linked to the strength of the cues’ shock association for the early attentional processing of cues in the visual periphery, but additionally was influenced by the uncertainty of the shock prediction after participants fixated on the cues. These findings are in accord with attentional learning theories that formalize how associative learning shapes automatic attention.
-
Motivational orientation modulates the neural response to reward.
Linke, Julia; Kirsch, Peter; King, Andrea V; Gass, Achim; Hennerici, Michael G; Bongers, André; Wessa, Michèle
2010-02-01
Motivational orientation defines the source of motivation for an individual to perform a particular action and can either originate from internal desires (e.g., interest) or external compensation (e.g., money). To this end, motivational orientation should influence the way positive or negative feedback is processed during learning situations and this might in turn have an impact on the learning process. In the present study, we thus investigated whether motivational orientation, i.e., extrinsic and intrinsic motivation modulates the neural response to reward and punishment as well as learning from reward and punishment in 33 healthy individuals. To assess neural responses to reward, punishment and learning of reward contingencies we employed a probabilistic reversal learning task during functional magnetic resonance imaging. Extrinsic and intrinsic motivation were assessed with a self-report questionnaire. Rewarding trials fostered activation in the medial orbitofrontal cortex and anterior cingulate gyrus (ACC) as well as the amygdala and nucleus accumbens, whereas for punishment an increased neural response was observed in the medial and inferior prefrontal cortex, the superior parietal cortex and the insula. High extrinsic motivation was positively correlated to increased neural responses to reward in the ACC, amygdala and putamen, whereas a negative relationship between intrinsic motivation and brain activation in these brain regions was observed. These findings show that motivational orientation indeed modulates the responsiveness to reward delivery in major components of the human reward system and therefore extends previous results showing a significant influence of individual differences in reward-related personality traits on the neural processing of reward. Copyright (c) 2009 Elsevier Inc. All rights reserved.
-
The role of high-frequency oscillatory activity in reward processing and learning.
Marco-Pallarés, Josep; Münte, Thomas F; Rodríguez-Fornells, Antoni
2015-02-01
Oscillatory activity has been proposed as a key mechanism in the integration of brain activity of distant structures. Particularly, high frequency brain oscillatory activity in the beta and gamma range has received increasing interest in the domains of attention and memory. In addition, a number of recent studies have revealed an increase of beta-gamma activity (20-35 Hz) after unexpected or relevant positive reward outcomes. In the present manuscript we review the literature on this phenomenon and we propose that this activity is a brain signature elicited by unexpected positive outcomes in order to transmit a fast motivational value signal to the reward network. In addition, we hypothesize that beta-gamma oscillatory activity indexes the interaction between attentional and emotional systems, and that it directly reflects the appearance of unexpected positive rewards in learning-related contexts. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
RM-SORN: a reward-modulated self-organizing recurrent neural network.
Aswolinskiy, Witali; Pipa, Gordon
2015-01-01
Neural plasticity plays an important role in learning and memory. Reward-modulation of plasticity offers an explanation for the ability of the brain to adapt its neural activity to achieve a rewarded goal. Here, we define a neural network model that learns through the interaction of Intrinsic Plasticity (IP) and reward-modulated Spike-Timing-Dependent Plasticity (STDP). IP enables the network to explore possible output sequences and STDP, modulated by reward, reinforces the creation of the rewarded output sequences. The model is tested on tasks for prediction, recall, non-linear computation, pattern recognition, and sequence generation. It achieves performance comparable to networks trained with supervised learning, while using simple, biologically motivated plasticity rules, and rewarding strategies. The results confirm the importance of investigating the interaction of several plasticity rules in the context of reward-modulated learning and whether reward-modulated self-organization can explain the amazing capabilities of the brain.
-
Veilleux, Jennifer C; Skinner, Kayla D
2016-09-01
In the current study, we aimed to extend smoking cue-reactivity research by evaluating delay discounting as an outcome of cigarette cue exposure. We also separated introspection in response to cues (e.g., self-reporting craving and affect) from cue exposure alone, to determine if introspection changes behavioral responses to cigarette cues. Finally, we included measures of quit motivation and resistance to smoking to assess motivational influences on cue exposure. Smokers were invited to participate in an online cue-reactivity study. Participants were randomly assigned to view smoking images or neutral images, and were randomized to respond to cues with either craving and affect questions (e.g., introspection) or filler questions. Following cue exposure, participants completed a delay discounting task and then reported state affect, craving, and resistance to smoking, as well as an assessment of quit motivation. We found that after controlling for trait impulsivity, participants who introspected on craving and affect showed higher delay discounting, irrespective of cue type, but we found no effect of response condition on subsequent craving (e.g., craving reactivity). We also found that motivation to quit interacted with experimental conditions to predict state craving and state resistance to smoking. Although asking about craving during cue exposure did not increase later craving, it resulted in greater delaying of discounted rewards. Overall, our findings suggest the need to further assess the implications of introspection and motivation on behavioral outcomes of cue exposure. Copyright © 2016 Elsevier Ltd. All rights reserved.
-
Reward positivity: Reward prediction error or salience prediction error?
Heydari, Sepideh; Holroyd, Clay B
2016-08-01
The reward positivity is a component of the human ERP elicited by feedback stimuli in trial-and-error learning and guessing tasks. A prominent theory holds that the reward positivity reflects a reward prediction error signal that is sensitive to outcome valence, being larger for unexpected positive events relative to unexpected negative events (Holroyd & Coles, 2002). Although the theory has found substantial empirical support, most of these studies have utilized either monetary or performance feedback to test the hypothesis. However, in apparent contradiction to the theory, a recent study found that unexpected physical punishments also elicit the reward positivity (Talmi, Atkinson, & El-Deredy, 2013). The authors of this report argued that the reward positivity reflects a salience prediction error rather than a reward prediction error. To investigate this finding further, in the present study participants navigated a virtual T maze and received feedback on each trial under two conditions. In a reward condition, the feedback indicated that they would either receive a monetary reward or not and in a punishment condition the feedback indicated that they would receive a small shock or not. We found that the feedback stimuli elicited a typical reward positivity in the reward condition and an apparently delayed reward positivity in the punishment condition. Importantly, this signal was more positive to the stimuli that predicted the omission of a possible punishment relative to stimuli that predicted a forthcoming punishment, which is inconsistent with the salience hypothesis. © 2016 Society for Psychophysiological Research.
-
Examining the role of social cues in early word learning.
Briganti, Alicia M; Cohen, Leslie B
2011-02-01
Infants watched a video of an adult pointing towards two different objects while hearing novel labels. Analyses indicated that 14- and 18-month-olds looked longer at the target object, but only 18-month-olds showed word learning. The results suggest that different types of social cues are available at different ages. Copyright © 2010 Elsevier Inc. All rights reserved.
-
The Effects of Experimental Manipulation of Sleep Duration on Neural Response to Food Cues.
Demos, Kathryn E; Sweet, Lawrence H; Hart, Chantelle N; McCaffery, Jeanne M; Williams, Samantha E; Mailloux, Kimberly A; Trautvetter, Jennifer; Owens, Max M; Wing, Rena R
2017-11-01
Despite growing literature on neural food cue responsivity in obesity, little is known about how the brain processes food cues following partial sleep deprivation and whether short sleep leads to changes similar to those observed in obesity. We used functional magnetic resonance imaging (fMRI) to test the hypothesis that short sleep leads to increased reward-related and decreased inhibitory control-related processing of food cues.In a within-subject design, 30 participants (22 female, mean age = 36.7 standard deviation = 10.8 years, body mass index range 20.4-40.7) completed four nights of 6 hours/night time-in-bed (TIB; short sleep) and four nights of 9 hours/night TIB (long sleep) in random counterbalanced order in their home environments. Following each sleep condition, participants completed an fMRI scan while viewing food and nonfood images.A priori region of interest analyses revealed increased activity to food in short versus long sleep in regions of reward processing (eg, nucleus accumbens/putamen) and sensory/motor signaling (ie, right paracentral lobule, an effect that was most pronounced in obese individuals). Contrary to the hypothesis, whole brain analyses indicated greater food cue responsivity during short sleep in an inhibitory control region (right inferior frontal gyrus) and ventral medial prefrontal cortex, which has been implicated in reward coding and decision-making (false discovery rate corrected q = 0.05).These findings suggest that sleep restriction leads to both greater reward and control processing in response to food cues. Future research is needed to understand the dynamic functional connectivity between these regions during short sleep and whether the interplay between these neural processes determines if one succumbs to food temptation. © Sleep Research Society 2017. Published by Oxford University Press on behalf of the Sleep Research Society. All rights reserved. For permissions, please e-mail journals.permissions@oup.com.
-
Blue colour preference in honeybees distracts visual attention for learning closed shapes.
Morawetz, Linde; Svoboda, Alexander; Spaethe, Johannes; Dyer, Adrian G
2013-10-01
Spatial vision is an important cue for how honeybees (Apis mellifera) find flowers, and previous work has suggested that spatial learning in free-flying bees is exclusively mediated by achromatic input to the green photoreceptor channel. However, some data suggested that bees may be able to use alternative channels for shape processing, and recent work shows conditioning type and training length can significantly influence bee learning and cue use. We thus tested the honeybees' ability to discriminate between two closed shapes considering either absolute or differential conditioning, and using eight stimuli differing in their spectral characteristics. Consistent with previous work, green contrast enabled reliable shape learning for both types of conditioning, but surprisingly, we found that bees trained with appetitive-aversive differential conditioning could additionally use colour and/or UV contrast to enable shape discrimination. Interestingly, we found that a high blue contrast initially interferes with bee shape learning, probably due to the bees innate preference for blue colours, but with increasing experience bees can learn a variety of spectral and/or colour cues to facilitate spatial learning. Thus, the relationship between bee pollinators and the spatial and spectral cues that they use to find rewarding flowers appears to be a more rich visual environment than previously thought.
-
Ortega-Castro, Nerea; Vadillo Nistal, Miguel
2013-01-01
Some researchers have attempted to determine whether situations in which a single cue is paired with several outcomes (A-B, A-C interference or interference between outcomes) involve the same learning and retrieval mechanisms as situations in which several cues are paired with a single outcome (A-B, C-B interference or interference between cues). Interestingly, current research on a related effect, which is known as retrieval-induced forgetting, can illuminate this debate. Most retrieval-indu...
-
Directory of Open Access Journals (Sweden)
Xiaoyi Zhang
2017-06-01
Full Text Available Several recent studies have suggested that cues that predict outcomes elicit a feedback-related-like negativity (FRN-like negativity reflecting initial appraisals of whether desired outcomes are probable. Some other studies, however, have found that the cues that predict outcomes elicited event-related potentials (ERPs that reflect the expectation to outcomes (e.g., outcome expectations. Given these mixed findings, this study aimed to examine whether the brain activity elicited by predictive cues in a gambling task reflected the initial evaluations of the outcomes, the outcome expectations, or both. We used a gambling task in which the participants were told to guess which of two doors hid a reward. At the beginning of each trial, a cue was presented to inform the participants of how many doors hid a reward. We found that these predictive cues elicited a FRN-like negativity at the frontal sites within around 200–300 ms. However, this negativity did not significantly differ between the cues that fully predicted gains and the cues that fully predicted losses. Furthermore, predictive cues elicited an expectation-related slow wave, and cues that predicted gains with a 50% probability elicited a larger expectation-related slow wave than cues that fully predicted gains or losses. Our results suggest that cues predicting outcomes reflect outcome expectations rather than initial evaluations of the forthcoming outcomes.
-
Probabilistically-Cued Patterns Trump Perfect Cues in Statistical Language Learning.
Lany, Jill; Gómez, Rebecca L
2013-01-01
Probabilistically-cued co-occurrence relationships between word categories are common in natural languages but difficult to acquire. For example, in English, determiner-noun and auxiliary-verb dependencies both involve co-occurrence relationships, but determiner-noun relationships are more reliably marked by correlated distributional and phonological cues, and appear to be learned more readily. We tested whether experience with co-occurrence relationships that are more reliable promotes learning those that are less reliable using an artificial language paradigm. Prior experience with deterministically-cued contingencies did not promote learning of less reliably-cued structure, nor did prior experience with relationships instantiated in the same vocabulary. In contrast, prior experience with probabilistically-cued co-occurrence relationships instantiated in different vocabulary did enhance learning. Thus, experience with co-occurrence relationships sharing underlying structure but not vocabulary may be an important factor in learning grammatical patterns. Furthermore, experience with probabilistically-cued co-occurrence relationships, despite their difficultly for naïve learners, lays an important foundation for learning novel probabilistic structure.
-
Directory of Open Access Journals (Sweden)
Steven B. Lowen
2015-01-01
Full Text Available Adolescents are highly vulnerable to addiction and are four times more likely to become addicted at first exposure than at any other age. The dopamine D1 receptor, which is typically overexpressed in the normal adolescent prefrontal cortex, is involved in drug cue responses and is associated with relapse in animal models. In human drug addicts, imaging methods have detected increased activation in response to drug cues in reward- and habit-associated brain regions. These same methods can be applied more quantitatively to rodent models. Here, changes in neuronal activation in response to cocaine-conditioned cues were observed using functional magnetic resonance imaging in juvenile rats that were made to over-express either D1 receptors or green fluorescent protein by viral-mediated transduction. Reduced activation was observed in the amygdala and dopamine cell body regions in the low cue-preferring/control juvenile rats in response to cocaine cues. In contrast, increased activation was observed in the dorsal striatum, nucleus accumbens, prefrontal cortex, and dopamine cell bodies in high cue-preferring/D1 juveniles. The increase in cue salience that is mediated by increased D1 receptor density, rather than excessive cocaine experience, appears to underlie the transition from aversion to reward in cue-induced neural response and may form the basis for habit-forming vulnerability.
-
Lowen, Steven B; Rohan, Michael L; Gillis, Timothy E; Thompson, Britta S; Wellons, Clara B W; Andersen, Susan L
2015-01-01
Adolescents are highly vulnerable to addiction and are four times more likely to become addicted at first exposure than at any other age. The dopamine D1 receptor, which is typically overexpressed in the normal adolescent prefrontal cortex, is involved in drug cue responses and is associated with relapse in animal models. In human drug addicts, imaging methods have detected increased activation in response to drug cues in reward- and habit-associated brain regions. These same methods can be applied more quantitatively to rodent models. Here, changes in neuronal activation in response to cocaine-conditioned cues were observed using functional magnetic resonance imaging in juvenile rats that were made to over-express either D1 receptors or green fluorescent protein by viral-mediated transduction. Reduced activation was observed in the amygdala and dopamine cell body regions in the low cue-preferring/control juvenile rats in response to cocaine cues. In contrast, increased activation was observed in the dorsal striatum, nucleus accumbens, prefrontal cortex, and dopamine cell bodies in high cue-preferring/D1 juveniles. The increase in cue salience that is mediated by increased D1 receptor density, rather than excessive cocaine experience, appears to underlie the transition from aversion to reward in cue-induced neural response and may form the basis for habit-forming vulnerability.
-
Dopamine reward prediction error coding.
Schultz, Wolfram
2016-03-01
Reward prediction errors consist of the differences between received and predicted rewards. They are crucial for basic forms of learning about rewards and make us strive for more rewards-an evolutionary beneficial trait. Most dopamine neurons in the midbrain of humans, monkeys, and rodents signal a reward prediction error; they are activated by more reward than predicted (positive prediction error), remain at baseline activity for fully predicted rewards, and show depressed activity with less reward than predicted (negative prediction error). The dopamine signal increases nonlinearly with reward value and codes formal economic utility. Drugs of addiction generate, hijack, and amplify the dopamine reward signal and induce exaggerated, uncontrolled dopamine effects on neuronal plasticity. The striatum, amygdala, and frontal cortex also show reward prediction error coding, but only in subpopulations of neurons. Thus, the important concept of reward prediction errors is implemented in neuronal hardware.
-
Oei, Nicole Y L; Both, Stephanie; van Heemst, Diana; van der Grond, Jeroen
2014-01-01
Stress is thought to alter motivational processes by increasing dopamine (DA) secretion in the brain's "reward system", and its key region, the nucleus accumbens (NAcc). However, stress studies using functional magnetic resonance imaging (fMRI), mainly found evidence for stress-induced decreases in NAcc responsiveness toward reward cues. Results from both animal and human PET studies indicate that the stress hormone cortisol may be crucial in the interaction between stress and dopaminergic actions. In the present study we therefore investigated whether cortisol mediated the effect of stress on DA-related responses to -subliminal-presentation of reward cues using the Trier Social Stress Test (TSST), which is known to reliably enhance cortisol levels. Young healthy males (n = 37) were randomly assigned to the TSST or control condition. After stress induction, brain activation was assessed using fMRI during a backward-masking paradigm in which potentially rewarding (sexual), emotionally negative and neutral stimuli were presented subliminally, masked by pictures of inanimate objects. A region of interest analysis showed that stress decreased activation in the NAcc in response to masked sexual cues (voxel-corrected, pcortisol levels were related to stronger NAcc activation, showing that cortisol acted as a suppressor variable in the negative relation between stress and NAcc activation. The present findings indicate that cortisol is crucially involved in the relation between stress and the responsiveness of the reward system. Although generally stress decreases activation in the NAcc in response to rewarding stimuli, high stress-induced cortisol levels suppress this relation, and are associated with stronger NAcc activation. Individuals with a high cortisol response to stress might on one hand be protected against reductions in reward sensitivity, which has been linked to anhedonia and depression, but they may ultimately be more vulnerable to increased reward
-
Reward-seeking behavior and addiction: cause or cog?
Arias-Carrión, Oscar; Salama, Mohamed
2012-09-01
Although dopaminergic system represents the cornerstone in rewarding, other neurotransmitters can modulate both the reward system and the psychomotor effects of addictive drugs. Many hypotheses have been proposed for a better understanding of the reward system and its role in drug addiction. However, after many years of investigation, no single theory can completely explain the neural basis of drug addiction. Recent reports introduce novel neurotransmitters into the game e.g. dynorphins, orexins, histamine, gheralin and galanin. The interacting functions of these neurotransmitters have shown that the reward system and its role in drug dependence, is far more complicated than was thought before. Individual variations exist regarding response to drug exposure, vulnerability for addiction and the effects of different cues on reward systems. Consequently, genetic variations of neurotransmission are thought to influence reward processing that in turn may affect distinctive social behavior and susceptibility to addiction. However, the individual variations can not be based mainly on genetics; environmental factors seem to play a role too. Here we discuss the current knowledge about the orquestic regulation of different neurotransmitters on reward-seeking behavior and their potential effect on drug addiction.
-
Inglis, W L; Olmstead, M C; Robbins, T W
2000-04-01
The role of the pedunculopontine tegmental nucleus (PPTg) in stimulus-reward learning was assessed by testing the effects of PPTg lesions on performance in visual autoshaping and conditioned reinforcement (CRf) paradigms. Rats with PPTg lesions were unable to learn an association between a conditioned stimulus (CS) and a primary reward in either paradigm. In the autoshaping experiment, PPTg-lesioned rats approached the CS+ and CS- with equal frequency, and the latencies to respond to the two stimuli did not differ. PPTg lesions also disrupted discriminated approaches to an appetitive CS in the CRf paradigm and completely abolished the acquisition of responding with CRf. These data are discussed in the context of a possible cognitive function of the PPTg, particularly in terms of lesion-induced disruptions of attentional processes that are mediated by the thalamus.
-
Lansink, Carien S; Meijer, Guido T; Lankelma, Jan V; Vinck, Martin A; Jackson, Jadin C; Pennartz, Cyriel M A
2016-10-12
The use of information from the hippocampal memory system in motivated behavior depends on its communication with the ventral striatum. When an animal encounters cues that signal subsequent reward, its reward expectancy is raised. It is unknown, however, how this process affects hippocampal dynamics and their influence on target structures, such as ventral striatum. We show that, in rats, reward-predictive cues result in enhanced hippocampal theta and beta band rhythmic activity during subsequent action, compared with uncued goal-directed navigation. The beta band component, also labeled theta's harmonic, involves selective hippocampal CA1 cell groups showing frequency doubling of firing periodicity relative to theta rhythmicity and it partitions the theta cycle into segments showing clear versus poor spike timing organization. We found that theta phase precession occurred over a wider range than previously reported. This was apparent from spikes emitted near the peak of the theta cycle exhibiting large "phase precessing jumps" relative to spikes in foregoing cycles. Neither this phenomenon nor the regular manifestation of theta phase precession was affected by reward expectancy. Ventral striatal neuronal firing phase-locked not only to hippocampal theta, but also to beta band activity. Both hippocampus and ventral striatum showed increased synchronization between neuronal firing and local field potential activity during cued compared with uncued goal approaches. These results suggest that cue-triggered reward expectancy intensifies hippocampal output to target structures, such as the ventral striatum, by which the hippocampus may gain prioritized access to systems modulating motivated behaviors. Here we show that temporally discrete cues raising reward expectancy enhance both theta and beta band activity in the hippocampus once goal-directed navigation has been initiated. These rhythmic activities are associated with increased synchronization of neuronal firing
-
Health interest modulates brain reward responses to a perceived low-caloric beverage in females.
van Rijn, Inge; Wegman, Joost; Aarts, Esther; de Graaf, Cees; Smeets, Paul A M
2017-01-01
Health labels are omnipresent in the supermarket. Such labels give rise to expectations about the product experience and may change flavor perception and perceived reward value. Consumers vary in their degree of health interest and may be differentially affected by such labels. However, how health interest influences neural reward responses to anticipation and receipt of heath-labeled foods is not known. This study assessed to what extent brain responses induced by anticipation and receipt of a beverage with different levels of perceived caloric content are associated with health interest. Twenty-five females completed an fMRI motivational taste-task in which they were presented with a low-caloric cue or a high-caloric cue and subsequently worked for sips of lemonade by moving a joystick. If they responded correctly and in time, they received the lemonade as a reward. Because of the 2 cue types, participants believed they were receiving 2 different lemonades, a high-caloric (HC-receipt) and a low-caloric (LC-receipt) one. Health interest was assessed with the General health interest subscale of the Health and Taste Attitude Scales. Health interest scores correlated significantly (r = .65) with LC-versus HC-receipt activation in the dorsal striatum (putamen), a region involved in encoding food reward. These findings suggest that the reward value of a healthy product compared to its unhealthy counterpart increases with health interest. This provides more insight into the working mechanism of government campaigns that focus on increasing health interest to encourage the formation of healthy eating habits. (PsycINFO Database Record (c) 2017 APA, all rights reserved).
-
Murawski, Carsten; Harris, Philip G; Bode, Stefan; Domínguez D, Juan F; Egan, Gary F
2012-01-01
Human decision-making is driven by subjective values assigned to alternative choice options. These valuations are based on reward cues. It is unknown, however, whether complex reward cues, such as brand logos, may bias the neural encoding of subjective value in unrelated decisions. In this functional magnetic resonance imaging (fMRI) study, we subliminally presented brand logos preceding intertemporal choices. We demonstrated that priming biased participants' preferences towards more immediate rewards in the subsequent temporal discounting task. This was associated with modulations of the neural encoding of subjective values of choice options in a network of brain regions, including but not restricted to medial prefrontal cortex. Our findings demonstrate the general susceptibility of the human decision making system to apparently incidental contextual information. We conclude that the brain incorporates seemingly unrelated value information that modifies decision making outside the decision-maker's awareness.
-
Prisciandaro, James J; Myrick, Hugh; Henderson, Scott; McRae-Clark, Aimee L; Santa Ana, Elizabeth J; Saladin, Michael E; Brady, Kathleen T
2013-09-01
The development of addiction is marked by a pathological associative learning process that imbues incentive salience to stimuli associated with drug use. Recent efforts to treat addiction have targeted this learning process using cue exposure therapy augmented with d-cycloserine (DCS), a glutamatergic agent hypothesized to enhance extinction learning. To better understand the impact of DCS-facilitated extinction on neural reactivity to drug cues, the present study reports fMRI findings from a randomized, double-blind, placebo-controlled trial of DCS-facilitated cue exposure for cocaine dependence. Twenty-five participants completed two MRI sessions (before and after intervention), with a cocaine-cue reactivity fMRI task. The intervention consisted of 50mg of DCS or placebo, combined with two sessions of cocaine cue exposure and skills training. Participants demonstrated cocaine cue activation in a variety of brain regions at baseline. From the pre- to post-study scan, participants experienced decreased activation to cues in a number of regions (e.g., accumbens, caudate, frontal poles). Unexpectedly, placebo participants experienced decreases in activation to cues in the left angular and middle temporal gyri and the lateral occipital cortex, while DCS participants did not. Three trials of DCS-facilitated cue exposure therapy for cocaine dependence have found that DCS either increases or does not significantly impact response to cocaine cues. The present study adds to this literature by demonstrating that DCS may prevent extinction to cocaine cues in temporal and occipital brain regions. Although consistent with past research, results from the present study should be considered preliminary until replicated in larger samples. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.
-
Fuel not fun: Reinterpreting attenuated brain responses to reward in obesity.
Kroemer, Nils B; Small, Dana M
2016-08-01
There is a well-established literature linking obesity to altered dopamine signaling and brain response to food-related stimuli. Neuroimaging studies frequently report enhanced responses in dopaminergic regions during food anticipation and decreased responses during reward receipt. This has been interpreted as reflecting anticipatory "reward surfeit", and consummatory "reward deficiency". In particular, attenuated response in the dorsal striatum to primary food rewards is proposed to reflect anhedonia, which leads to overeating in an attempt to compensate for the reward deficit. In this paper, we propose an alternative view. We consider brain response to food-related stimuli in a reinforcement-learning framework, which can be employed to separate the contributions of reward sensitivity and reward-related learning that are typically entangled in the brain response to reward. Consequently, we posit that decreased striatal responses to milkshake receipt reflect reduced reward-related learning rather than reward deficiency or anhedonia because reduced reward sensitivity would translate uniformly into reduced anticipatory and consummatory responses to reward. By re-conceptualizing reward deficiency as a shift in learning about subjective value of rewards, we attempt to reconcile neuroimaging findings with the putative role of dopamine in effort, energy expenditure and exploration and suggest that attenuated brain responses to energy dense foods reflect the "fuel", not the fun entailed by the reward. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Reward sensitivity, attentional bias, and executive control in early adolescent alcohol use.
van Hemel-Ruiter, Madelon E; de Jong, Peter J; Ostafin, Brian D; Wiers, Reinout W
2015-01-01
This study examined whether attentional bias for alcohol stimuli was associated with alcohol use in young adolescents, and whether the frequently demonstrated relationship between reward sensitivity and adolescent alcohol use would be partly mediated by attentional bias for alcohol cues. In addition, this study investigated the potential moderating role of executive control (EC), and tested whether the relationship between alcohol-related attentional bias and alcohol use was especially present in young adolescents with weak EC. Participants were 86 adolescents (mean age=14.86), who completed a Visual Probe Task (VPT) as an index of attentional bias, a flanker-task based Attention Network Task (ANT) as an index of EC, the sensitivity of punishment and sensitivity of reward questionnaire (SPSRQ) as an index of reward sensitivity, and an alcohol use questionnaire. High reward sensitivity, high alcohol-related attentional bias, and weak EC were all related to alcohol use. The relationship between reward sensitivity and alcohol use was not mediated by alcohol-related attentional bias. As hypothesized, attentional bias was only associated with alcohol use in participants with weak EC. Together, the present findings are consistent with the view that high reward sensitivity and low EC may be considered as risk factors for adolescent alcohol use. The independent contribution of reward sensitivity and attentional bias might suggest that adolescents who are highly reward sensitive and display an attentional bias for alcohol cues are at even higher risk for excessive alcohol use and developing alcohol abuse problems. Future research using a longitudinal approach would allow an examination of these risk factors on subsequent alcohol use. Treatment implications are discussed, including the importance of strengthening EC and reducing the rewarding value of alcohol use. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Influence of reward motivation on human declarative memory.
Miendlarzewska, Ewa A; Bavelier, Daphne; Schwartz, Sophie
2016-02-01
Motivational relevance can prioritize information for memory encoding and consolidation based on reward value. In this review, we pinpoint the possible psychological and neural mechanisms by which reward promotes learning, from guiding attention to enhancing memory consolidation. We then discuss how reward value can spill-over from one conditioned stimulus to a non-conditioned stimulus. Such generalization can occur across perceptually similar items or through more complex relations, such as associative or logical inferences. Existing evidence suggests that the neurotransmitter dopamine boosts the formation of declarative memory for rewarded information and may also control the generalization of reward values. In particular, temporally-correlated activity in the hippocampus and in regions of the dopaminergic circuit may mediate value-based decisions and facilitate cross-item integration. Given the importance of generalization in learning, our review points to the need to study not only how reward affects later memory but how learned reward values may generalize to related representations and ultimately alter memory structure. Copyright © 2015 Elsevier Ltd. All rights reserved.
-
Ventral striatal activity links adversity and reward processing in children
Directory of Open Access Journals (Sweden)
Niki H. Kamkar
2017-08-01
Full Text Available Adversity impacts many aspects of psychological and physical development including reward-based learning and decision-making. Mechanisms relating adversity and reward processing in children, however, remain unclear. Here, we show that adversity is associated with potentiated learning from positive outcomes and impulsive decision-making, but unrelated to learning from negative outcomes. We then show via functional magnetic resonance imaging that the link between adversity and reward processing is partially mediated by differences in ventral striatal response to rewards. The findings suggest that early-life adversity is associated with alterations in the brain’s sensitivity to rewards accounting, in part, for the link between adversity and altered reward processing in children.
-
Directory of Open Access Journals (Sweden)
Léa Lansade
Full Text Available Pavlovian to instrumental transfer (PIT is a central factor in how cues influence animal behavior. PIT refers to the capacity of a Pavlovian cue that predicts a reward to elicit or increase a response intended to obtain the same reward. In the present study, using an equine model, we assessed whether PIT occurs in hoofed domestic animals and whether its efficacy can be modulated by temperamental dimensions. To study PIT, horses were submitted to Pavlovian conditioning whereby an auditory-visual stimulus was repeatedly followed by food delivery. Then, horses were submitted to instrumental conditioning during which they learned to touch with their noses an object signaled by the experimenter in order to obtain the same reward. During the PIT test, the Pavlovian conditioned stimulus was presented to the animal in the absence of reward. At the end of the experiment, a battery of behavioral tests was performed on all animals to assess five temperamental dimensions and investigate their relationships with instrumental performance. The results indicate that PIT can be observed in horses and that its efficacy is greatly modulated by individual temperament. Indeed, individuals with a specific pattern of temperamental dimensions (i.e., higher levels of gregariousness, fearfulness, and sensory sensitivity exhibited the strongest PIT. The demonstration of the existence of PIT in domesticated animals (i.e., horses is important for the optimization of its use by humans and the improvement of training methods. Moreover, because PIT may be implicated in psychological phenomena, including addictive behaviors, the observation of relationships between specific temperamental dimensions and PIT efficacy may aid in identifying predisposing temperamental attributes.
-
Li, Diandian; Meng, Liang; Ma, Qingguo
2017-01-01
Trust and trustworthiness contribute to reciprocal behavior and social relationship development. To make better decisions, people need to evaluate others' trustworthiness. They often assess this kind of reputation by learning through repeated social interactions. The present event-related potential (ERP) study explored the reputation learning process in a repeated trust game where subjects made multi-round decisions of investment to different partners. We found that subjects gradually learned to discriminate trustworthy partners from untrustworthy ones based on how often their partners reciprocated the investment, which was indicated by their own investment decisions. Besides, electrophysiological data showed that the faces of the untrustworthy partners induced larger feedback negativity (FN) amplitude than those of the trustworthy partners, but only in the late phase of the game. The ERP results corresponded with the behavioral pattern and revealed that the learned trustworthiness differentiation was coded by the cue-elicited FN component. Consistent with previous research, our findings suggest that the anterior cue-elicited FN reflects the reputation appraisal and tracks the reputation learning process in social interactions.
-
Directory of Open Access Journals (Sweden)
Diandian Li
2017-06-01
Full Text Available Trust and trustworthiness contribute to reciprocal behavior and social relationship development. To make better decisions, people need to evaluate others’ trustworthiness. They often assess this kind of reputation by learning through repeated social interactions. The present event-related potential (ERP study explored the reputation learning process in a repeated trust game where subjects made multi-round decisions of investment to different partners. We found that subjects gradually learned to discriminate trustworthy partners from untrustworthy ones based on how often their partners reciprocated the investment, which was indicated by their own investment decisions. Besides, electrophysiological data showed that the faces of the untrustworthy partners induced larger feedback negativity (FN amplitude than those of the trustworthy partners, but only in the late phase of the game. The ERP results corresponded with the behavioral pattern and revealed that the learned trustworthiness differentiation was coded by the cue-elicited FN component. Consistent with previous research, our findings suggest that the anterior cue-elicited FN reflects the reputation appraisal and tracks the reputation learning process in social interactions.
-
Goschy, Harriet; Bakos, Sarolta; Müller, Hermann J; Zehetleitner, Michael
2014-01-01
Targets in a visual search task are detected faster if they appear in a probable target region as compared to a less probable target region, an effect which has been termed "probability cueing." The present study investigated whether probability cueing cannot only speed up target detection, but also minimize distraction by distractors in probable distractor regions as compared to distractors in less probable distractor regions. To this end, three visual search experiments with a salient, but task-irrelevant, distractor ("additional singleton") were conducted. Experiment 1 demonstrated that observers can utilize uneven spatial distractor distributions to selectively reduce interference by distractors in frequent distractor regions as compared to distractors in rare distractor regions. Experiments 2 and 3 showed that intertrial facilitation, i.e., distractor position repetitions, and statistical learning (independent of distractor position repetitions) both contribute to the probability cueing effect for distractor locations. Taken together, the present results demonstrate that probability cueing of distractor locations has the potential to serve as a strong attentional cue for the shielding of likely distractor locations.
-
Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking.
Stuber, Garret D; Sparta, Dennis R; Stamatakis, Alice M; van Leeuwen, Wieke A; Hardjoprajitno, Juanita E; Cho, Saemi; Tye, Kay M; Kempadoo, Kimberly A; Zhang, Feng; Deisseroth, Karl; Bonci, Antonello
2011-06-29
The basolateral amygdala (BLA) has a crucial role in emotional learning irrespective of valence. The BLA projection to the nucleus accumbens (NAc) is thought to modulate cue-triggered motivated behaviours, but our understanding of the interaction between these two brain regions has been limited by the inability to manipulate neural-circuit elements of this pathway selectively during behaviour. To circumvent this limitation, we used in vivo optogenetic stimulation or inhibition of glutamatergic fibres from the BLA to the NAc, coupled with intracranial pharmacology and ex vivo electrophysiology. Here we show that optical stimulation of the pathway from the BLA to the NAc in mice reinforces behavioural responding to earn additional optical stimulation of these synaptic inputs. Optical stimulation of these glutamatergic fibres required intra-NAc dopamine D1-type receptor signalling, but not D2-type receptor signalling. Brief optical inhibition of fibres from the BLA to the NAc reduced cue-evoked intake of sucrose, demonstrating an important role of this specific pathway in controlling naturally occurring reward-related behaviour. Moreover, although optical stimulation of glutamatergic fibres from the medial prefrontal cortex to the NAc also elicited reliable excitatory synaptic responses, optical self-stimulation behaviour was not observed by activation of this pathway. These data indicate that whereas the BLA is important for processing both positive and negative affect, the glutamatergic pathway from the BLA to the NAc, in conjunction with dopamine signalling in the NAc, promotes motivated behavioural responding. Thus, optogenetic manipulation of anatomically distinct synaptic inputs to the NAc reveals functionally distinct properties of these inputs in controlling reward-seeking behaviours.
-
Ventral striatal activity links adversity and reward processing in children.
Kamkar, Niki H; Lewis, Daniel J; van den Bos, Wouter; Morton, J Bruce
2017-08-01
Adversity impacts many aspects of psychological and physical development including reward-based learning and decision-making. Mechanisms relating adversity and reward processing in children, however, remain unclear. Here, we show that adversity is associated with potentiated learning from positive outcomes and impulsive decision-making, but unrelated to learning from negative outcomes. We then show via functional magnetic resonance imaging that the link between adversity and reward processing is partially mediated by differences in ventral striatal response to rewards. The findings suggest that early-life adversity is associated with alterations in the brain's sensitivity to rewards accounting, in part, for the link between adversity and altered reward processing in children. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.
-
Baker, Phillip M.; Ragozzino, Michael E.
2014-01-01
Switches in reward outcomes or reward-predictive cues are two fundamental ways in which information is used to flexibly shift response patterns. The rat prelimbic cortex and dorsomedial striatum support behavioral flexibility based on a change in outcomes. The present experiments investigated whether these two brain regions are necessary for…
-
Directory of Open Access Journals (Sweden)
Carsten Murawski
Full Text Available Human decision-making is driven by subjective values assigned to alternative choice options. These valuations are based on reward cues. It is unknown, however, whether complex reward cues, such as brand logos, may bias the neural encoding of subjective value in unrelated decisions. In this functional magnetic resonance imaging (fMRI study, we subliminally presented brand logos preceding intertemporal choices. We demonstrated that priming biased participants' preferences towards more immediate rewards in the subsequent temporal discounting task. This was associated with modulations of the neural encoding of subjective values of choice options in a network of brain regions, including but not restricted to medial prefrontal cortex. Our findings demonstrate the general susceptibility of the human decision making system to apparently incidental contextual information. We conclude that the brain incorporates seemingly unrelated value information that modifies decision making outside the decision-maker's awareness.
-
Dopamine mediates testosterone-induced social reward in male Syrian hamsters.
Bell, Margaret R; Sisk, Cheryl L
2013-03-01
Adolescent maturation of responses to social stimuli is essential for adult-typical sociosexual behavior. Naturally occurring developmental changes in male Syrian hamster responses to a salient social cue, female hamster vaginal secretions (VS), provide a good model system for investigating neuroendocrine mechanisms of adolescent change in social reward. Sexually naïve adult, but not juvenile, males show a conditioned place preference (CPP) to VS, indicating that VS is not rewarding before puberty. In this series of experiments, the authors examined the roles of testosterone and dopamine receptor activation in mediating the adolescent gain in positive valence of VS. Experiment 1 showed that testosterone replacement is necessary for gonadectomized adult hamsters to form a CPP to VS. Experiment 2 showed that testosterone treatment is sufficient for juvenile hamsters to form a CPP to VS, and that the dopamine receptor antagonist haloperidol blocks formation of a CPP to VS in these animals. Experiments 3 and 4 demonstrated that the disruption of VS CPP with low doses of haloperidol is the result of a reduction in the attractive properties of VS and not attributable to aversive properties of haloperidol. Together, these studies demonstrate that the unconditioned rewarding properties of a social cue necessary for successful adult sociosexual interactions come about as the result of the pubertal increase in circulating testosterone in male hamsters. Furthermore, this social reward can be prevented by dopamine receptor antagonism, indicating that hypothalamic and/or mesocorticolimbic dopaminergic circuits are targets for hormonal activation of social reward.
-
Directory of Open Access Journals (Sweden)
Jacqueline Scholl
2017-02-01
Full Text Available To make good decisions, humans need to learn about and integrate different sources of appetitive and aversive information. While serotonin has been linked to value-based decision-making, its role in learning is less clear, with acute manipulations often producing inconsistent results. Here, we show that when the effects of a selective serotonin reuptake inhibitor (SSRI, citalopram are studied over longer timescales, learning is robustly improved. We measured brain activity with functional magnetic resonance imaging (fMRI in volunteers as they performed a concurrent appetitive (money and aversive (effort learning task. We found that 2 weeks of citalopram enhanced reward and effort learning signals in a widespread network of brain regions, including ventromedial prefrontal and anterior cingulate cortex. At a behavioral level, this was accompanied by more robust reward learning. This suggests that serotonin can modulate the ability to learn via a mechanism that is independent of stimulus valence. Such effects may partly underlie SSRIs' impact in treating psychological illnesses. Our results highlight both a specific function in learning for serotonin and the importance of studying its role across longer timescales.
-
Fostering participation in learning networks by using reward systems and face-to-face meetings
Hummel, Hans; Tattersall, Colin; Burgos, Daniel; Brouns, Francis; Koper, Rob
2006-01-01
Hummel, H. G. K., Tattersall, C., Burgos, D., Brouns, F. M. R., & Koper, E. J. R. (Submitted). Fostering participation in learning networks by using reward systems and face-to-face meetings. In Proceedings of ICALT 2006 Conference. July 5-7, Kerkrade, The Netherlands
-
Altered motivation masks appetitive learning potential of obese mice
Directory of Open Access Journals (Sweden)
Mazen R. Harb
2014-10-01
Full Text Available Eating depends strongly on learning processes which, in turn, depend on motivation. Conditioned learning, where individuals associate environmental cues with receipt of a reward, forms an important part of hedonic mechanisms; the latter contribute to the development of human overweight and obesity by driving excessive eating in what may become a vicious cycle. Although mice are commonly used to explore the regulation of human appetite, it is not known whether their conditioned learning of food rewards varies as a function of body mass. To address this, groups of adult male mice of differing body weights were tested two appetitive conditioning paradigms (pavlovian and operant as well as in food retrieval and hedonic preference tests in an attempt to dissect the respective roles of learning/motivation and energy state in the regulation of feeding behavior. We found that i the rate of pavlovian conditioning to an appetitive reward develops as an inverse function of body weight; ii higher body weight associates with increased latency to collect food reward; and iii mice with lower body weights are more motivated to work for a food reward, as compared to animals with higher body weights. Interestingly, as compared to controls, overweight and obese mice consumed smaller amounts of palatable foods (isocaloric milk or sucrose, in either the presence or absence of their respective maintenance diets: standard, low fat-high carbohydrate or high fat-high carbohydrate. Notably, however, all groups adjusted their consumption of the different food types, such that their body weight-corrected daily intake of calories remained constant. Thus, overeating in mice does not reflect a reward deficiency syndrome and, in contrast to humans, mice regulate their caloric intake according to metabolic status rather than to the hedonic properties of a particular food. Together, these observations demonstrate that excess weight masks the capacity for appetitive learning in
-
Intersection of reward and memory in monkey rhinal cortex.
Clark, Andrew M; Bouret, Sebastien; Young, Adrienne M; Richmond, Barry J
2012-05-16
In humans and other animals, the vigor with which a reward is pursued depends on its desirability, that is, on the reward's predicted value. Predicted value is generally context-dependent, varying according to the value of rewards obtained in the recent and distant past. Signals related to reward prediction and valuation are believed to be encoded in a circuit centered around midbrain dopamine neurons and their targets in the prefrontal cortex and basal ganglia. Notably absent from this hypothesized reward pathway are dopaminergic targets in the medial temporal lobe. Here we show that a key part of the medial temporal lobe memory system previously reported to be important for sensory mnemonic and perceptual processing, the rhinal cortex (Rh), is required for using memories of previous reward values to predict the value of forthcoming rewards. We tested monkeys with bilateral Rh lesions on a task in which reward size varied across blocks of uncued trials. In this experiment, the only cues for predicting current reward value are the sizes of rewards delivered in previous blocks. Unexpectedly, monkeys with Rh ablations, but not intact controls, were insensitive to differences in predicted reward, responding as if they expected all rewards to be of equal magnitude. Thus, it appears that Rh is critical for using memory of previous rewards to predict the value of forthcoming rewards. These results are in agreement with accumulating evidence that Rh is critical for establishing the relationships between temporally interleaved events, which is a key element of episodic memory.
-
Sigmund, K; Hauert, C; Nowak, M A
2001-09-11
Minigames capturing the essence of Public Goods experiments show that even in the absence of rationality assumptions, both punishment and reward will fail to bring about prosocial behavior. This result holds in particular for the well-known Ultimatum Game, which emerges as a special case. But reputation can induce fairness and cooperation in populations adapting through learning or imitation. Indeed, the inclusion of reputation effects in the corresponding dynamical models leads to the evolution of economically productive behavior, with agents contributing to the public good and either punishing those who do not or rewarding those who do. Reward and punishment correspond to two types of bifurcation with intriguing complementarity. The analysis suggests that reputation is essential for fostering social behavior among selfish agents, and that it is considerably more effective with punishment than with reward.
-
Rosenthal, M Zachary; Kutlu, Munir G
2014-09-01
Despite experimental findings and some treatment research supporting the use of cues as a means to induce and extinguish cravings, interventions using cue exposure have not been well integrated into contemporary substance abuse treatments. A primary problem with exposure-based interventions for addiction is that after learning not to use substances in the presence of addiction cues inside the clinic (i.e., extinction), stimuli in the naturalistic setting outside the clinic may continue to elicit craving, drug use, or other maladaptive conditioned responses. For exposure-based substance use interventions to be efficacious, new approaches are needed that can prevent relapse by directly generalizing learning from the therapeutic setting into naturalistic settings associated with a high risk for relapse. Basic research suggests that extinction reminders (ERs) can be paired with the context of learning new and more adaptive conditioned responses to substance abuse cues in exposure therapies for addiction. Using mobile phones and automated dialing and data collection software, ERs can be delivered in everyday high-risk settings to inhibit conditioned responses to substance-use-related stimuli. In this review, we describe how associative learning mechanisms (e.g., conditioned inhibition) can inform how ERs are conceptualized, learned, and implemented to prevent substance use when delivered via mobile phones. This approach, exposure with portable reminders of extinction, is introduced as an adjunctive intervention that uses brief automated ERs between clinic visits when individuals are in high-risk settings for drug use.
-
Attention to irrelevant cues is related to positive symptoms in schizophrenia.
Morris, Richard; Griffiths, Oren; Le Pelley, Michael E; Weickert, Thomas W
2013-05-01
Many modern learning theories assume that the amount of attention to a cue depends on how well that cue predicted important events in the past. Schizophrenia is associated with deficits in attention and recent theories of psychosis have argued that positive symptoms such as delusions and hallucinations are related to a failure of selective attention. However, evidence demonstrating that attention to irrelevant cues is related to positive symptoms in schizophrenia is lacking. We used a novel method of measuring attention to nonpredictive (and thus irrelevant) cues in a causal learning test (Le Pelley ME, McLaren IP. Learned associability and associative change in human causal learning. Q J Exp Psychol B. 2003;56:68-79) to assess whether healthy adults and people with schizophrenia discriminate previously predictive and nonpredictive cues. In a series of experiments with independent samples, we demonstrated: (1) when people with schizophrenia who had severe positive symptoms successfully distinguished between predictive and nonpredictive cues during training, they failed to discriminate between predictive and nonpredictive cues relative to healthy adults during subsequent testing and (2) learning about nonpredictive cues was correlated with more severe positive symptoms scores in schizophrenia. These results suggest that positive symptoms of schizophrenia are related to increased attention to nonpredictive cues during causal learning. This deficit in selective attention results in learning irrelevant causal associations and may be the basis of positive symptoms in schizophrenia.
-
Sensitivity for Cues Predicting Reward and Punishment in Young Women with Eating Disorders.
Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline
2017-11-01
Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely ignored. This pilot study used a spatial orientation task to examine transdiagnostic and interdiagnostic differences in SR/SP. Participants (14-29 years) were patients with anorexia nervosa of restricting type (AN-R, n = 20), binge/purge ED group [AN of binge/purge type and bulimia nervosa (n = 16)] and non-symptomatic individuals (n = 23). Results revealed stronger difficulties to redirect attention away from signals of rewards in AN-R compared with binge/purge EDs, and binge/purge EDs showed stronger difficulties to direct attention away from signals of punishment compared with AN-R. Findings demonstrate interdiagnostic differences and show that the spatial orientation task is sensitive for individual differences in SP/SR within the context of EDs, thereby sustaining its usefulness as behavioural measure of reinforcement sensitivity. Copyright © 2017 John Wiley & Sons, Ltd and Eating Disorders Association. Copyright © 2017 John Wiley & Sons, Ltd and Eating Disorders Association.
-
Ivanov, Iliyan; Liu, Xun; Clerkin, Suzanne; Schulz, Kurt; Fan, Jin; Friston, Karl; London, Edythe D; Schwartz, Jeffrey; Newcorn, Jeffrey H
2014-06-01
Psychostimulants, such as methylphenidate, are thought to improve information processing in motivation-reward and attention-activation networks by enhancing the effects of more relevant signals and suppressing those of less relevant ones; however the nature of such reciprocal influences remains poorly understood. To explore this question, we tested the effect of methylphenidate on performance and associated brain activity in the Anticipation, Conflict, Reward (ACR) task. Sixteen healthy adult volunteers, ages 21-45, were scanned twice using functional magnetic resonance imaging (fMRI) as they performed the ACR task under placebo and methylphenidate conditions. A three-way repeated measures analysis of variance, with cue (reward vs. non-reward), target (congruent vs. incongruent) and medication condition (methylphenidate vs. placebo) as the factors, was used to analyze behaviors on the task. Blood oxygen level dependent (BOLD) signals, reflecting task-related neural activity, were evaluated using linear contrasts. Participants exhibited significantly greater accuracy in the methylphenidate condition than the placebo condition. Compared with placebo, the methylphenidate condition also was associated with lesser task-related activity in components of attention-activation systems irrespective of the reward cue, and less task-related activity in components of the reward-motivation system, particularly the insula, during reward trials irrespective of target difficulty. These results suggest that methylphenidate enhances task performance by improving efficiency of information processing in both reward-motivation and in attention-activation systems. Published by Elsevier B.V.
-
Pervasive competition between threat and reward in the brain.
Choi, Jong Moon; Padmala, Srikanth; Spechler, Philip; Pessoa, Luiz
2014-06-01
In the current functional MRI study, we investigated interactions between reward and threat processing. Visual cues at the start of each trial informed participants about the chance of winning monetary reward and/or receiving a mild aversive shock. We tested two competing hypothesis: according to the 'salience hypothesis', in the condition involving both reward and threat, enhanced activation would be observed because of increased salience; according to the 'competition hypothesis', the processing of reward and threat would trade-off against each other, leading to reduced activation. Analysis of skin conductance data during a delay phase revealed an interaction between reward and threat processing, such that the effect of reward was reduced during threat and the effect of threat was reduced during reward. Analysis of imaging data during the same task phase revealed interactions between reward and threat processing in several regions, including the midbrain/ventral tegmental area, caudate, putamen, bed nucleus of the stria terminalis, anterior insula, middle frontal gyrus and dorsal anterior cingulate cortex. Taken together, our findings reveal conditions during which reward and threat trade-off against each other across multiple sites. Such interactions are suggestive of competitive processes and may reflect the organization of opponent systems in the brain. © The Author (2013). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Floral odor learning within the hive affects honeybees' foraging decisions
Arenas, Andrés; Fernández, Vanesa M.; Farina, Walter M.
2007-03-01
Honeybees learn odor cues quickly and efficiently when visiting rewarding flowers. Memorization of these cues facilitates the localization and recognition of food sources during foraging flights. Bees can also use information gained inside the hive during social interactions with successful foragers. An important information cue that can be learned during these interactions is food odor. However, little is known about how floral odors learned in the hive affect later decisions of foragers in the field. We studied the effect of food scent on foraging preferences when this learning is acquired directly inside the hive. By using in-hive feeders that were removed 24 h before the test, we showed that foragers use the odor information acquired during a 3-day stimulation period with a scented solution during a food-choice situation outside the nest. This bias in food preference is maintained even 24 h after the replacement of all the hive combs. Thus, without being previously collected outside by foragers, food odors learned within the hive can be used during short-range foraging flights. Moreover, correct landings at a dual-choice device after replacing the storing combs suggests that long-term memories formed within the colony can be retrieved while bees search for food in the field.
-
Ebrahimi, Claudia; Koch, Stefan P; Friedel, Eva; Crespo, Ilsoray; Fydrich, Thomas; Ströhle, Andreas; Heinz, Andreas; Schlagenhauf, Florian
2017-07-01
Appetitive Pavlovian conditioning plays a crucial role in the pathogenesis of drug addiction and conditioned reward cues can trigger craving and relapse even after long phases of abstinence. Promising preclinical work showed that the NMDA-receptor partial agonist D-cycloserine (DCS) facilitates Pavlovian extinction learning of fear and drug cues. Furthermore, DCS-augmented exposure therapy seems to be beneficial in various anxiety disorders, while the supposed working mechanism of DCS during human appetitive or aversive extinction learning is still not confirmed. To test the hypothesis that DCS administration before extinction training improves extinction learning, healthy adults (n=32) underwent conditioning, extinction, and extinction recall on three successive days in a randomized, double-blind, placebo-controlled fMRI design. Monetary wins and losses served as unconditioned stimuli during conditioning to probe appetitive and aversive learning. An oral dose of 50mg of DCS or placebo was administered 1h before extinction training and DCS effects during extinction recall were evaluated on a behavioral and neuronal level. We found attenuated amygdala activation in the DCS compared to the placebo group during recall of the extinguished appetitive cue, along with evidence for enhanced functional amygdala-vmPFC coupling in the DCS group. While the absence of additional physiological measures of conditioned responses during recall in this study prevent the evaluation of a behavioral DCS effect, our neuronal findings are in accordance with recent theories linking successful extinction recall in humans to modulatory top-down influences from the vmPFC that inhibit amygdala activation. Our results should encourage further translational studies concerning the usefulness of DCS to target maladaptive Pavlovian reward associations. Copyright © 2017 Elsevier Inc. All rights reserved.
-
Learning to Produce Syllabic Speech Sounds via Reward-Modulated Neural Plasticity
Warlaumont, Anne S.; Finnegan, Megan K.
2016-01-01
At around 7 months of age, human infants begin to reliably produce well-formed syllables containing both consonants and vowels, a behavior called canonical babbling. Over subsequent months, the frequency of canonical babbling continues to increase. How the infant’s nervous system supports the acquisition of this ability is unknown. Here we present a computational model that combines a spiking neural network, reinforcement-modulated spike-timing-dependent plasticity, and a human-like vocal tract to simulate the acquisition of canonical babbling. Like human infants, the model’s frequency of canonical babbling gradually increases. The model is rewarded when it produces a sound that is more auditorily salient than sounds it has previously produced. This is consistent with data from human infants indicating that contingent adult responses shape infant behavior and with data from deaf and tracheostomized infants indicating that hearing, including hearing one’s own vocalizations, is critical for canonical babbling development. Reward receipt increases the level of dopamine in the neural network. The neural network contains a reservoir with recurrent connections and two motor neuron groups, one agonist and one antagonist, which control the masseter and orbicularis oris muscles, promoting or inhibiting mouth closure. The model learns to increase the number of salient, syllabic sounds it produces by adjusting the base level of muscle activation and increasing their range of activity. Our results support the possibility that through dopamine-modulated spike-timing-dependent plasticity, the motor cortex learns to harness its natural oscillations in activity in order to produce syllabic sounds. It thus suggests that learning to produce rhythmic mouth movements for speech production may be supported by general cortical learning mechanisms. The model makes several testable predictions and has implications for our understanding not only of how syllabic vocalizations develop
-
Directory of Open Access Journals (Sweden)
Nadine Matton
2013-09-01
Full Text Available Individuals differ in their ability to deal with unpredictable environments. Could impaired performances on learning an unpredictable cue-criteria relationship in a laboratory task be associated with impaired learning of complex skills in a natural setting? We focused on a multiple-cue probability learning (MCPL laboratory task and on the natural setting of pilot training. We used data from three selection sessions and from the three corresponding selected pilot student classes of a national airline pilot selection and training system. First, applicants took an MCPL task at the selection stage (N=556; N=701; N=412. Then, pilot trainees selected from the applicant pools (N=44; N=60; N=28 followed the training for 2.5 to 3 yrs. Differences in final MCPL performance were associated with pilot training difficulties. Indeed, poor MCPL performers experienced almost twice as many pilot training difficulties as better MCPL performers (44.0% and 25.0%, respectively.
-
Reward sensitivity is associated with brain activity during erotic stimulus processing.
Costumero, Victor; Barrós-Loscertales, Alfonso; Bustamante, Juan Carlos; Ventura-Campos, Noelia; Fuentes, Paola; Rosell-Negre, Patricia; Ávila, César
2013-01-01
The behavioral approach system (BAS) from Gray's reinforcement sensitivity theory is a neurobehavioral system involved in the processing of rewarding stimuli that has been related to dopaminergic brain areas. Gray's theory hypothesizes that the functioning of reward brain areas is modulated by BAS-related traits. To test this hypothesis, we performed an fMRI study where participants viewed erotic and neutral pictures, and cues that predicted their appearance. Forty-five heterosexual men completed the Sensitivity to Reward scale (from the Sensitivity to Punishment and Sensitivity to Reward Questionnaire) to measure BAS-related traits. Results showed that Sensitivity to Reward scores correlated positively with brain activity during reactivity to erotic pictures in the left orbitofrontal cortex, left insula, and right ventral striatum. These results demonstrated a relationship between the BAS and reward sensitivity during the processing of erotic stimuli, filling the gap of previous reports that identified the dopaminergic system as a neural substrate for the BAS during the processing of other rewarding stimuli such as money and food.
-
Reward sensitivity is associated with brain activity during erotic stimulus processing.
Directory of Open Access Journals (Sweden)
Victor Costumero
Full Text Available The behavioral approach system (BAS from Gray's reinforcement sensitivity theory is a neurobehavioral system involved in the processing of rewarding stimuli that has been related to dopaminergic brain areas. Gray's theory hypothesizes that the functioning of reward brain areas is modulated by BAS-related traits. To test this hypothesis, we performed an fMRI study where participants viewed erotic and neutral pictures, and cues that predicted their appearance. Forty-five heterosexual men completed the Sensitivity to Reward scale (from the Sensitivity to Punishment and Sensitivity to Reward Questionnaire to measure BAS-related traits. Results showed that Sensitivity to Reward scores correlated positively with brain activity during reactivity to erotic pictures in the left orbitofrontal cortex, left insula, and right ventral striatum. These results demonstrated a relationship between the BAS and reward sensitivity during the processing of erotic stimuli, filling the gap of previous reports that identified the dopaminergic system as a neural substrate for the BAS during the processing of other rewarding stimuli such as money and food.
-
Reward inference by primate prefrontal and striatal neurons.
Pan, Xiaochuan; Fan, Hongwei; Sawa, Kosuke; Tsuda, Ichiro; Tsukada, Minoru; Sakagami, Masamichi
2014-01-22
The brain contains multiple yet distinct systems involved in reward prediction. To understand the nature of these processes, we recorded single-unit activity from the lateral prefrontal cortex (LPFC) and the striatum in monkeys performing a reward inference task using an asymmetric reward schedule. We found that neurons both in the LPFC and in the striatum predicted reward values for stimuli that had been previously well experienced with set reward quantities in the asymmetric reward task. Importantly, these LPFC neurons could predict the reward value of a stimulus using transitive inference even when the monkeys had not yet learned the stimulus-reward association directly; whereas these striatal neurons did not show such an ability. Nevertheless, because there were two set amounts of reward (large and small), the selected striatal neurons were able to exclusively infer the reward value (e.g., large) of one novel stimulus from a pair after directly experiencing the alternative stimulus with the other reward value (e.g., small). Our results suggest that although neurons that predict reward value for old stimuli in the LPFC could also do so for new stimuli via transitive inference, those in the striatum could only predict reward for new stimuli via exclusive inference. Moreover, the striatum showed more complex functions than was surmised previously for model-free learning.
-
Adapting to the destitute situations: poverty cues lead to short-term choice.
Liu, Lei; Feng, Tingyong; Suo, Tao; Lee, Kang; Li, Hong
2012-01-01
Why do some people live for the present, whereas others save for the future? The evolutionary framework of life history theory predicts that preference for delay of gratification should be influenced by social economic status (SES). However, here we propose that the decision to choose alternatives in immediate and delayed gratification in poverty environments may have a psychological dimension. Specifically, the perception of environmental poverty cues may induce people alike to favor choices with short-term, likely smaller benefit than choices with long-term, greater benefit. The present study was conducted to explore how poverty and affluence cues affected individuals' intertemporal choices. In our first two experiments, individuals exposed explicitly (Experiment 1) and implicitly (Experiment 2) to poverty pictures (the poverty cue) were induced to prefer immediate gratification compared with those exposed to affluence pictures (the affluence cue). Furthermore, by the manipulation of temporary perceptions of poverty and affluence status using a lucky draw game; individuals in the poverty state were more impulsive in a manner, which made them pursue immediate gratification in intertemporal choices (Experiment 3). Thus, poverty cues can lead to short-term choices. Decision makers chose more frequently the sooner-smaller reward over the later-larger reward as they were exposed to the poverty cue. This indicates that it is that just the feeling of poverty influences intertemporal choice - the actual reality of poverty (restricted resources, etc.) is not necessary to get the effect. Furthermore, our findings emphasize that it is a change of the poverty-affluence status, not a trait change, can influence individual preference in intertemporal choice.
-
Fat and lean tissue accretion in relation to reward motivation in children.
De Decker, Annelies; De Clercq, Bart; Verbeken, Sandra; Wells, Jonathan C K; Braet, Caroline; Michels, Nathalie; De Henauw, Stefaan; Sioen, Isabelle
2017-01-01
'Reward sensitivity' explains inter-individual differences in the motivation to obtain rewards when reward cues are perceived. This psychobiological trait has been linked to higher consumption of palatable food when exposed to palatable food cues. The current study aims to examine if reward sensitivity explains differences in patterns of fat and lean tissue accretion over time in children. A longitudinal observational study with measurement waves in 2011 (baseline), 2012, 2013, and 2015 was conducted. The sample was a population-based Flemish cohort of children (n = 446, 50% boys and 5.5-12 years at baseline; 38.8% of the baseline sample also participated in 2015). Baseline reward sensitivity of the children was assessed by parent ratings on the Drive subscale of the Behavioral Inhibition System/Behavioral Approach System scales. Age- and sex-independent Fat and Lean Mass Index z-scores (zFMI and zLMI respectively) were computed for each study wave based on air-displacement plethysmography. In girls, but not boys, reward sensitivity was positively associated with the baseline zFMI and zLMI (95% confidence intervals of unstandardized estimates: 0.01 to 0.11 and 0.01 to 0.10 respectively, P values 0.01 and 0.02 respectively). Further, reward sensitivity explained 14.8% and 11.6% of the change in girls' zFMI and zLMI respectively over four years: the zFMI and zLMI increased and decreased respectively in high reward sensitive girls (95% confidence intervals of unstandardized estimates: 0.01 to 0.11 and -0.12 to -0.01 respectively, P values 0.01 and 0.02 respectively). Hence, girls high in reward sensitivity had significantly higher adiposity gain over four years parallel with lower increase in lean mass than was expected on the basis of their age and height. These results may help to identify appropriate targets for interventions for obesity prevention. Copyright © 2016 Elsevier Ltd. All rights reserved.
-
Xie, Heping; Wang, Fuxing; Hao, Yanbin; Chen, Jiaxue; An, Jing; Wang, Yuxin; Liu, Huashan
2017-01-01
Cueing facilitates retention and transfer of multimedia learning. From the perspective of cognitive load theory (CLT), cueing has a positive effect on learning outcomes because of the reduction in total cognitive load and avoidance of cognitive overload. However, this has not been systematically evaluated. Moreover, what remains ambiguous is the direct relationship between the cue-related cognitive load and learning outcomes. A meta-analysis and two subsequent meta-regression analyses were conducted to explore these issues. Subjective total cognitive load (SCL) and scores on a retention test and transfer test were selected as dependent variables. Through a systematic literature search, 32 eligible articles encompassing 3,597 participants were included in the SCL-related meta-analysis. Among them, 25 articles containing 2,910 participants were included in the retention-related meta-analysis and the following retention-related meta-regression, while there were 29 articles containing 3,204 participants included in the transfer-related meta-analysis and the transfer-related meta-regression. The meta-analysis revealed a statistically significant cueing effect on subjective ratings of cognitive load (d = -0.11, 95% CI = [-0.19, -0.02], p < 0.05), retention performance (d = 0.27, 95% CI = [0.08, 0.46], p < 0.01), and transfer performance (d = 0.34, 95% CI = [0.12, 0.56], p < 0.01). The subsequent meta-regression analyses showed that dSCL for cueing significantly predicted dretention for cueing (β = -0.70, 95% CI = [-1.02, -0.38], p < 0.001), as well as dtransfer for cueing (β = -0.60, 95% CI = [-0.92, -0.28], p < 0.001). Thus in line with CLT, adding cues in multimedia materials can indeed reduce SCL and promote learning outcomes, and the more SCL is reduced by cues, the better retention and transfer of multimedia learning.
-
Seid-Fatemi, Azade; Tobler, Philippe N
2015-05-01
When we are learning to associate novel cues with outcomes, learning is more efficient if we take advantage of previously learned associations and thereby avoid redundant learning. The blocking effect represents this sort of efficiency mechanism and refers to the phenomenon in which a novel stimulus is blocked from learning when it is associated with a fully predicted outcome. Although there is sufficient evidence that this effect manifests itself when individuals learn about their own rewards, it remains unclear whether it also does when they learn about others' rewards. We employed behavioral and neuroimaging methods to address this question. We demonstrate that blocking does indeed occur in the social domain and it does so to a similar degree as observed in the individual domain. On the neural level, activations in the medial prefrontal cortex (mPFC) show a specific contribution to blocking and learning-related prediction errors in the social domain. These findings suggest that the efficiency principle that applies to reward learning in the individual domain also applies to that in the social domain, with the mPFC playing a central role in implementing it. © The Author (2014). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.
-
Cannabinoid CB1 receptor antagonist rimonabant disrupts nicotine reward-associated memory in rats.
Fang, Qin; Li, Fang-Qiong; Li, Yan-Qin; Xue, Yan-Xue; He, Ying-Ying; Liu, Jian-Feng; Lu, Lin; Wang, Ji-Shi
2011-10-01
Exposure to cues previously associated with drug intake leads to relapse by activating previously acquired memories. Based on previous findings, in which cannabinoid CB(1) receptors were found to be critically involved in specific aspects of learning and memory, we investigated the role of CB(1) receptors in nicotine reward memory using a rat conditioned place preference (CPP) model. In Experiment 1, rats were trained for CPP with alternating injections of nicotine (0.5mg/kg, s.c.) and saline to acquire the nicotine-conditioned memory. To examine the effects of rimonabant on the reconsolidation of nicotine reward memory, rats were administered rimonabant (0, 0.3, and 3.0mg/kg, i.p.) immediately after reexposure to the drug-paired context. In Experiment 2, rats were trained for CPP similarly to Experiment 1. To examine the effects of rimonabant on the reinstatement of nicotine reward memory, rimonabant (0, 0.3, and 3.0mg/kg, i.p.) was administered before the test of nicotine-induced CPP reinstatement. In Experiment 3, to evaluate whether rimonabant itself produces a reward memory, rats were trained for CPP with alternating injections of different doses of rimonabant (0, 0.3, and 3.0mg/kg) and saline. Rimonabant at a dose of 3.0mg/kg significantly disrupted the reconsolidation of nicotine memory and significantly blocked the reinstatement of nicotine-induced CPP. However, rimonabant itself did not produce CPP. These findings provide clear evidence that CB(1) receptors play a role in nicotine reward memory, suggesting that CB(1) receptor antagonists may be a potential target for managing nicotine addiction. Copyright © 2011 Elsevier Inc. All rights reserved.
-
Slave to habit? Obesity is associated with decreased behavioural sensitivity to reward devaluation.
Horstmann, Annette; Dietrich, Anja; Mathar, David; Pössel, Maria; Villringer, Arno; Neumann, Jane
2015-04-01
The motivational value of food is lower during satiety compared to fasting. Dynamic changes in motivational value promote food seeking or meal cessation. In obesity this mechanism might be compromised since obese subjects ingest energy beyond homeostatic needs. Thus, lower adaptation of eating behaviour with respect to changes in motivational value might cause food overconsumption in obesity. To test this hypothesis, we implemented a selective satiation procedure to investigate the relationship between obesity and the size of the behavioural devaluation effect in humans. Lean to obese men (mean age 25.9, range 19-30 years; mean BMI 29.1, range 19.2-45.1 kg/m(2)) were trained on a free operant paradigm and learned to associate cues with the possibility to win different food rewards by pressing a button. After the initial training phase, one of the rewards was devalued by consumption. Response rates for and wanting of the different rewards were measured pre and post devaluation. Behavioural sensitivity to reward devaluation, measured as the magnitude of difference between pre and post responses, was regressed against BMI. Results indicate that (1) higher BMI compared to lower BMI in men led to an attenuated behavioural adjustment to reward devaluation, and (2) the decrease in motivational value was associated with the decrease in response rate between pre and post. Change in explicitly reported motivational value, however, was not affected by BMI. Thus, we conclude that high BMI in men is associated with lower behavioural adaptation with respect to changes in motivational value of food, possibly resulting in automatic overeating patterns that are hard to control in daily life. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Serotonergic neurons signal reward and punishment on multiple timescales
Cohen, Jeremiah Y; Amoroso, Mackenzie W; Uchida, Naoshige
2015-01-01
Serotonin's function in the brain is unclear. One challenge in testing the numerous hypotheses about serotonin's function has been observing the activity of identified serotonergic neurons in animals engaged in behavioral tasks. We recorded the activity of dorsal raphe neurons while mice experienced a task in which rewards and punishments varied across blocks of trials. We ‘tagged’ serotonergic neurons with the light-sensitive protein channelrhodopsin-2 and identified them based on their responses to light. We found three main features of serotonergic neuron activity: (1) a large fraction of serotonergic neurons modulated their tonic firing rates over the course of minutes during reward vs punishment blocks; (2) most were phasically excited by punishments; and (3) a subset was phasically excited by reward-predicting cues. By contrast, dopaminergic neurons did not show firing rate changes across blocks of trials. These results suggest that serotonergic neurons signal information about reward and punishment on multiple timescales. DOI: http://dx.doi.org/10.7554/eLife.06346.001 PMID:25714923
-
Reward, Context, and Human Behaviour
Directory of Open Access Journals (Sweden)
Clare L. Blaukopf
2007-01-01
Full Text Available Animal models of reward processing have revealed an extensive network of brain areas that process different aspects of reward, from expectation and prediction to calculation of relative value. These results have been confirmed and extended in human neuroimaging to encompass secondary rewards more unique to humans, such as money. The majority of the extant literature covers the brain areas associated with rewards whilst neglecting analysis of the actual behaviours that these rewards generate. This review strives to redress this imbalance by illustrating the importance of looking at the behavioural outcome of rewards and the context in which they are produced. Following a brief review of the literature of reward-related activity in the brain, we examine the effect of reward context on actions. These studies reveal how the presence of reward vs. reward and punishment, or being conscious vs. unconscious of reward-related actions, differentially influence behaviour. The latter finding is of particular importance given the extent to which animal models are used in understanding the reward systems of the human mind. It is clear that further studies are needed to learn about the human reaction to reward in its entirety, including any distinctions between conscious and unconscious behaviours. We propose that studies of reward entail a measure of the animal's (human or nonhuman knowledge of the reward and knowledge of its own behavioural outcome to achieve that reward.
-
Eating beyond metabolic need: how environmental cues influence feeding behavior.
Johnson, Alexander W
2013-02-01
Animals use current, past, and projected future states of the organism and the world in a finely tuned system to control ingestion. They must not only deal effectively with current nutrient deficiencies, but also manage energy resources to meet future needs, all within the constraints of the mechanisms of metabolism. Many recent approaches to understanding the control of ingestive behavior distinguish between homeostatic mechanisms concerned with energy balance, and hedonic and incentive processes based on palatability and reward characteristics of food. In this review, I consider how learning about environmental cues influences homeostatic and hedonic brain signals, which may lead to increases in the affective taste properties of food and desire to over consume. Understanding these mechanisms may be critical for elucidating the etiology of the obesity epidemic. Copyright © 2013 Elsevier Ltd. All rights reserved.
-
van den Akker, Karolien; Schyns, Ghislaine; Jansen, Anita
2016-07-01
The prevalence of overweight and obesity has increased substantially over the last decades. Weight loss attempts in overweight individuals are common, though they seldom result in successful long-term weight loss. One very promising treatment is food cue exposure therapy, during which overweight individuals are repeatedly exposed to food-associated cues (e.g., the sight, smell and taste of high-calorie foods, overeating environments) without eating in order to extinguish cue-elicited appetitive responses to food cues. However, only few studies have tested the effectiveness of cue exposure, especially with regards to weight loss. For exposure treatment of anxiety disorders, it has been proposed that inhibitory learning is critical for exposure to be effective. In this RCT, we translated techniques proposed by Craske et al. (2014) to the appetitive domain and developed a novel cue exposure therapy for overeating aimed at maximizing inhibitory learning. The current RCT tested the effectiveness of this 8-session cue exposure intervention relative to a control intervention in 45 overweight adult (aged 18-60) females at post-treatment and 3-month follow-up, of which 39 participants completed the study. Weight loss, eating psychopathology, food cue reactivity, and snacking behaviour were studied as main treatment outcomes, and mediators and moderators of treatment effects were studied. The presented study design represents an innovative effort to provide valuable clinical recommendations for the treatment of overeating and obesity. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Circadian timed episodic-like memory - a bee knows what to do when, and also where.
Pahl, Mario; Zhu, Hong; Pix, Waltraud; Tautz, Juergen; Zhang, Shaowu
2007-10-01
This study investigates how the colour, shape and location of patterns could be memorized within a time frame. Bees were trained to visit two Y-mazes, one of which presented yellow vertical (rewarded) versus horizontal (non-rewarded) gratings at one site in the morning, while another presented blue horizontal (rewarded) versus vertical (non-rewarded) gratings at another site in the afternoon. The bees could perform well in the learning tests and various transfer tests, in which (i) all contextual cues from the learning test were present; (ii) the colour cues of the visual patterns were removed, but the location cue, the orientation of the visual patterns and the temporal cue still existed; (iii) the location cue was removed, but other contextual cues, i.e. the colour and orientation of the visual patterns and the temporal cue still existed; (iv) the location cue and the orientation cue of the visual patterns were removed, but the colour cue and temporal cue still existed; (v) the location cue, and the colour cue of the visual patterns were removed, but the orientation cue and the temporal cue still existed. The results reveal that the honeybee can recall the memory of the correct visual patterns by using spatial and/or temporal information. The relative importance of different contextual cues is compared and discussed. The bees' ability to integrate elements of circadian time, place and visual stimuli is akin to episodic-like memory; we have therefore named this kind of memory circadian timed episodic-like memory.
-
Bedecarrats, Alexis; Cornet, Charles; Simmers, John; Nargeot, Romuald
2013-01-01
Feeding in "Aplysia" provides an amenable model system for analyzing the neuronal substrates of motivated behavior and its adaptability by associative reward learning and neuromodulation. Among such learning processes, appetitive operant conditioning that leads to a compulsive-like expression of feeding actions is known to be associated…
-
Reward Motivation Enhances Task Coding in Frontoparietal Cortex.
Etzel, Joset A; Cole, Michael W; Zacks, Jeffrey M; Kay, Kendrick N; Braver, Todd S
2016-04-01
Reward motivation often enhances task performance, but the neural mechanisms underlying such cognitive enhancement remain unclear. Here, we used a multivariate pattern analysis (MVPA) approach to test the hypothesis that motivation-related enhancement of cognitive control results from improved encoding and representation of task set information. Participants underwent two fMRI sessions of cued task switching, the first under baseline conditions, and the second with randomly intermixed reward incentive and no-incentive trials. Information about the upcoming task could be successfully decoded from cue-related activation patterns in a set of frontoparietal regions typically associated with task control. More critically, MVPA classifiers trained on the baseline session had significantly higher decoding accuracy on incentive than non-incentive trials, with decoding improvement mediating reward-related enhancement of behavioral performance. These results strongly support the hypothesis that reward motivation enhances cognitive control, by improving the discriminability of task-relevant information coded and maintained in frontoparietal brain regions. © The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
-
Cognitive Neurostimulation: Learning to Volitionally Sustain Ventral Tegmental Area Activation.
MacInnes, Jeff J; Dickerson, Kathryn C; Chen, Nan-Kuei; Adcock, R Alison
2016-03-16
Activation of the ventral tegmental area (VTA) and mesolimbic networks is essential to motivation, performance, and learning. Humans routinely attempt to motivate themselves, with unclear efficacy or impact on VTA networks. Using fMRI, we found untrained participants' motivational strategies failed to consistently activate VTA. After real-time VTA neurofeedback training, however, participants volitionally induced VTA activation without external aids, relative to baseline, Pre-test, and control groups. VTA self-activation was accompanied by increased mesolimbic network connectivity. Among two comparison groups (no neurofeedback, false neurofeedback) and an alternate neurofeedback group (nucleus accumbens), none sustained activation in target regions of interest nor increased VTA functional connectivity. The results comprise two novel demonstrations: learning and generalization after VTA neurofeedback training and the ability to sustain VTA activation without external reward or reward cues. These findings suggest theoretical alignment of ideas about motivation and midbrain physiology and the potential for generalizable interventions to improve performance and learning. Copyright © 2016 Elsevier Inc. All rights reserved.
-
Vanes, L.D.; Holst, R.J. van; Jansen, J.M.; Brink, W. van den; Oosterlaan, J.; Goudriaan, A.E.
2014-01-01
BACKGROUND: Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex
-
Vanes, L.D.; Holst, R.; Jansen, J.D.; van den Brink, W.A.; Oosterlaan, J.; Goudriaan, A.E.
2014-01-01
Background: Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex
-
Maximize Producer Rewards in Distributed Windmill Environments: A Q-Learning Approach
Directory of Open Access Journals (Sweden)
Bei Li
2015-03-01
Full Text Available In Smart Grid environments, homes equipped with windmills are encouraged to generate energy and sell it back to utilities. Time of Use pricing and the introduction of storage devices would greatly influence a user in deciding when to sell back energy and how much to sell. Therefore, a study of sequential decision making algorithms that can optimize the total pay off for the user is necessary. In this paper, reinforcement learning is used to tackle this optimization problem. The problem of determining when to sell back energy is formulated as a Markov decision process and the model is learned adaptively using Q-learning. Experiments are done with varying sizes of storage capacities and under periodic energy generation rates of different levels of fluctuations. The results show a notable increase in discounted total rewards from selling back energy with the proposed approach.
-
van Wouwe, N.C.; Ridderinkhof, K.R.; van den Wildenberg, W.P.M.; Band, G.P.H.; Abisogun, A.; Elias, W.J.; Frysinger, R.; Wylie, S.A.
2011-01-01
Recently, the subthalamic nucleus (STN) has been shown to be critically involved in decision-making, action selection, and motor control. Here we investigate the effect of deep brain stimulation (DBS) of the STN on reward-based decision-learning in patients diagnosed with Parkinson's disease (PD).
-
Vanes, Lucy D.; van Holst, Ruth J.; Jansen, Jochem M.; van den Brink, Wim; Oosterlaan, Jaap; Goudriaan, Anna E.
2014-01-01
Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex mix of
-
Increasing the efficacy of cue exposure treatment in preventing relapse of addictive behavior.
Havermans, Remco C; Jansen, Anita T M
2003-07-01
Theoretically, cue exposure treatment should be able to prevent relapse by extinguishing conditioned drug responding (e.g. cue-elicited craving). According to contemporary learning theory, though, extinction does not eliminate conditioned responding. Analogous cue exposure with response prevention (CERP) as a treatment of addictive behavior might not eliminate the learned relation between drug-related cues and drug use. This does not necessarily mean that cue exposure cannot successfully prevent relapse. Various suggestions for increasing the efficacy of cue exposure treatment are being discussed from a contemporary learning theory perspective. It is suggested that cue exposure treatment incorporating retrieval cues can be a beneficial treatment in preventing relapse of addictive behavior.
-
Smoking-Cue Induced Brain Activation In Adolescent Light Smokers
Rubinstein, Mark L.; Luks, Tracy L.; Moscicki, Anna-Barbara; Dryden, Wendy; Rait, Michelle A.; Simpson, Gregory V.
2010-01-01
Purpose Using fMRI, we examined whether or not adolescents with low levels of nicotine exposure (light smokers) display neural activation in areas shown to be involved with addiction in response to smoking-related stimuli. Design/Setting/Participants Twelve adolescent light smokers (aged 13 to17, smoked 1 to 5 cigarettes per day) and 12 non-smokers (ages 13 to 17, never smoked a cigarette) from the San Francisco Bay Area underwent fMRI scanning. During scanning they viewed blocks of photographic smoking and control cues. Smoking cues consisted of pictures of people smoking cigarettes and smoking-related objects such as lighters and ashtrays. Neutral cues consisted of everyday objects and people engaged in everyday activities. Findings For smokers, smoking cues elicited greater activation than neutral cues in the mesolimbic reward circuit (left anterior cingulate (T=7.88, pbrain regions seen in adult and heavy teen smokers suggests that even at low levels of smoking, adolescents exhibit heightened reactivity to smoking cues. This paper adds to the existing literature suggesting that nicotine dependence may begin with exposure to low levels of nicotine, underscoring the need for early intervention among adolescent smokers. PMID:21185518
-
Optogenetic activation of dorsal raphe serotonin neurons enhances patience for future rewards.
Miyazaki, Kayoko W; Miyazaki, Katsuhiko; Tanaka, Kenji F; Yamanaka, Akihiro; Takahashi, Aki; Tabuchi, Sawako; Doya, Kenji
2014-09-08
Serotonin is a neuromodulator that is involved extensively in behavioral, affective, and cognitive functions in the brain. Previous recording studies of the midbrain dorsal raphe nucleus (DRN) revealed that the activation of putative serotonin neurons correlates with the levels of behavioral arousal [1], rhythmic motor outputs [2], salient sensory stimuli [3-6], reward, and conditioned cues [5-8]. The classic theory on serotonin states that it opposes dopamine and inhibits behaviors when aversive events are predicted [9-14]. However, the therapeutic effects of serotonin signal-enhancing medications have been difficult to reconcile with this theory [15, 16]. In contrast, a more recent theory states that serotonin facilitates long-term optimal behaviors and suppresses impulsive behaviors [17-21]. To test these theories, we developed optogenetic mice that selectively express channelrhodopsin in serotonin neurons and tested how the activation of serotonergic neurons in the DRN affects animal behavior during a delayed reward task. The activation of serotonin neurons reduced the premature cessation of waiting for conditioned cues and food rewards. In reward omission trials, serotonin neuron stimulation prolonged the time animals spent waiting. This effect was observed specifically when the animal was engaged in deciding whether to keep waiting and was not due to motor inhibition. Control experiments showed that the prolonged waiting times observed with optogenetic stimulation were not due to behavioral inhibition or the reinforcing effects of serotonergic activation. These results show, for the first time, that the timed activation of serotonin neurons during waiting promotes animals' patience to wait for a delayed reward. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Olausson, Peter; Kiraly, Drew D; Gourley, Shannon L; Taylor, Jane R
2013-02-01
Repeated or prolonged exposure to stress has profound effects on a wide spectrum of behavioral and neurobiological processes and has been associated with the pathophysiology of depression. The multifaceted nature of this disorder includes despair, anhedonia, diminished motivation, and disrupted cognition, and it has been proposed that depression is also associated with reduced reward-motivated learning. We have previously reported that prior chronic corticosterone exposure to mice produces a lasting depressive-like state that can be reversed by chronic antidepressant treatment. In the present study, we tested the effects of prior chronic exposure to corticosterone (50 μg/ml) administered to rats or to mice in drinking water for 14 days followed by dose-tapering over 9 days. The exposure to corticosterone produced lasting deficits in the acquisition of reward-related learning tested on a food-motivated instrumental task conducted 10-20 days after the last day of full dose corticosterone exposure. Rats exposed to corticosterone also displayed reduced responding on a progressive ratio schedule of reinforcement when tested on day 21 after exposure. Amitriptyline (200 mg/ml in drinking water) exposure for 14 days to mice produced the opposite effect, enhancing food-motivated instrumental acquisition and performance. Repeated treatment with amitriptyline (5 mg/kg, intraperitoneally; bid) subsequent to corticosterone exposure also prevented the corticosterone-induced deficits in rats. These results are consistent with aberrant reward-related learning and motivational processes in depressive states and provide new evidence that stress-induced neuroadaptive alterations in cortico-limbic-striatal brain circuits involved in learning and motivation may play a critical role in aspects of mood disorders.
-
Henry, Alastair
2017-01-01
In order to reinstate interest and motivation for learning foreign languages (FLs) other than English, the Swedish government has recently reformed the system for admission to higher education. Upper secondary students who continue with the FL learnt in secondary school are rewarded with extra credits that considerably enhance their grade point…
-
Achievement-Based Rewards and Intrinsic Motivation: A Test of Cognitive Mediators
Cameron, Judy; Pierce, W. David; Banko, Katherine M.; Gear, Amber
2005-01-01
This study assessed how rewards impacted intrinsic motivation when students were rewarded for achievement while learning an activity, for performing at a specific level on a test, or for both. Undergraduate university students engaged in a problem-solving activity. The design was a 2 * 2 factorial with 2 levels of reward in a learning phase…
-
Reward and punishment enhance motor adaptation in stroke.
Quattrocchi, Graziella; Greenwood, Richard; Rothwell, John C; Galea, Joseph M; Bestmann, Sven
2017-09-01
The effects of motor learning, such as motor adaptation, in stroke rehabilitation are often transient, thus mandating approaches that enhance the amount of learning and retention. Previously, we showed in young individuals that reward and punishment feedback have dissociable effects on motor adaptation, with punishment improving adaptation and reward enhancing retention. If these findings were able to generalise to patients with stroke, they would provide a way to optimise motor learning in these patients. Therefore, we tested this in 45 patients with chronic stroke allocated in three groups. Patients performed reaching movements with their paretic arm with a robotic manipulandum. After training (day 1), day 2 involved adaptation to a novel force field. During the adaptation phase, patients received performance-based feedback according to the group they were allocated: reward, punishment or no feedback (neutral). On day 3, patients readapted to the force field but all groups now received neutral feedback. All patients adapted, with reward and punishment groups displaying greater adaptation and readaptation than the neutral group, irrespective of demographic, cognitive or functional differences. Remarkably, the reward and punishment groups adapted to similar degree as healthy controls. Finally, the reward group showed greater retention. This study provides, for the first time, evidence that reward and punishment can enhance motor adaptation in patients with stroke. Further research on reinforcement-based motor learning regimes is warranted to translate these promising results into clinical practice and improve motor rehabilitation outcomes in patients with stroke. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.
-
Thomas, Cyril; Didierjean, André; Maquestiaux, François; Goujon, Annabelle
2018-04-12
Since the seminal study by Chun and Jiang (Cognitive Psychology, 36, 28-71, 1998), a large body of research based on the contextual-cueing paradigm has shown that the cognitive system is capable of extracting statistical contingencies from visual environments. Most of these studies have focused on how individuals learn regularities found within an intratrial temporal window: A context predicts the target position within a given trial. However, Ono, Jiang, and Kawahara (Journal of Experimental Psychology, 31, 703-712, 2005) provided evidence of an intertrial implicit-learning effect when a distractor configuration in preceding trials N - 1 predicted the target location in trials N. The aim of the present study was to gain further insight into this effect by examining whether it occurs when predictive relationships are impeded by interfering task-relevant noise (Experiments 2 and 3) or by a long delay (Experiments 1, 4, and 5). Our results replicated the intertrial contextual-cueing effect, which occurred in the condition of temporally close contingencies. However, there was no evidence of integration across long-range spatiotemporal contingencies, suggesting a temporal limitation of statistical learning.
-
Zack, Elizabeth; Gerhardstein, Peter; Meltzoff, Andrew N.; Barr, Rachel
2012-01-01
Infants have difficulty transferring information between 2D and 3D sources. The current study extends Zack et al.’s (2009) touch screen imitation task to examine whether the addition of specific language cues significantly facilitates 15-month-olds’ transfer of learning between touch screens and real-world 3D objects. The addition of two kinds of linguistic cues (object label plus verb or nonsense name) did not elevate action imitation significantly above levels observed when such language cues were not used. Language cues hindered infants’ performance in the 3D→2D direction of transfer, but only for the object label plus verb condition. The lack of a facilitative effect of language is discussed in terms of competing cognitive loads imposed by conjointly transferring information across dimensions and processing linguistic cues in an action imitation task at this age. PMID:23121508
-
Applying extinction research and theory to cue-exposure addiction treatments.
Conklin, Cynthia A; Tiffany, Stephen T
2002-02-01
To evaluate the efficacy of cue-exposure addiction treatment and review modern animal learning research to generate recommendations for substantially enhancing the effectiveness of this treatment. Meta-analysis of cue-exposure addiction treatment outcome studies (N=9), review of animal extinction research and theory, and evaluation of whether major principles from this literature are addressed adequately in cue-exposure treatments. The meta-analytical review showed that there is no consistent evidence for the efficacy of cue-exposure treatment as currently implemented. Moreover, procedures derived from the animal learning literature that should maximize the potential of extinction training are rarely used in cue-exposure treatments. Given what is known from animal extinction theory and research about extinguishing learned behavior, it is not surprising that cue-exposure treatments so often fail. This paper reviews current animal research regarding the most salient threats to the development and maintenance of extinction, and suggests several major procedures for increasing the efficacy of cue-exposure addiction treatment.
-
Ventral striatal activity links adversity and reward processing in children
Kamkar, N.H.; Lewis, D.J.; van den Bos, W.; Morton, J.B.
2017-01-01
Adversity impacts many aspects of psychological and physical development including reward-based learning and decision-making. Mechanisms relating adversity and reward processing in children, however, remain unclear. Here, we show that adversity is associated with potentiated learning from positive
-
Doremus-Fitzwater, Tamara L; Spear, Linda P
2016-11-01
Adolescence is an evolutionarily conserved developmental period, with neural circuits and behaviors contributing to the detection, procurement, and receipt of rewards bearing similarity across species. Studies with laboratory animals suggest that adolescence is typified by a "reward-centric" phenotype-an increased sensitivity to rewards relative to adults. In contrast, adolescent rodents are reportedly less sensitive to the aversive properties of many drugs and naturally aversive stimuli. Alterations within the mesocorticolimbic dopamine and endocannabinoid systems likely contribute to an adolescent reward-sensitive, yet aversion-resistant, phenotype. Although early hypotheses postulated that developmental changes in dopaminergic circuitry would result in a "reward deficiency" syndrome, evidence now suggests the opposite: that adolescents are uniquely poised to seek out hedonic stimuli, experience greater "pleasure" from rewards, and consume rewarding stimuli in excess. Future studies that more clearly define the role of specific brain regions and neurotransmitter systems in the expression of behaviors toward reward- and aversive-related cues and stimuli are necessary to more fully understand an adolescent-proclivity for and vulnerability to rewards and drugs of potential abuse. Copyright © 2016 Elsevier Ltd. All rights reserved.
-
The dorsomedial striatum mediates Pavlovian appetitive conditioning and food consumption.
Cole, Sindy; Stone, Andrew D; Petrovich, Gorica D
2017-12-01
The dorsomedial striatum (DMS) is an important sensorimotor region mediating the acquisition of goal-directed instrumental reward learning and behavioral flexibility. However, whether the DMS also regulates Pavlovian cue-food learning is less clear. The current study used excitotoxic lesions to determine whether the DMS is critical in Pavlovian appetitive learning and behavior, using discriminative conditioning and reversal paradigms. The results showed that DMS lesions transiently retarded cue-food learning and subsequent reversal of this learning. Rats with DMS lesions selectively attenuated responding to a food cue but not a control cue, early in training, suggesting the DMS is involved when initial associations are formed. Similarly, initial reversal learning was attenuated in rats with DMS lesions, which suggests impaired flexibility to adjust behavior when the cue meaning is reversed. We also examined the effect of DMS lesions on food intake during tests with access to a highly palatable food along with standard chow diet. Rats with DMS lesions showed an altered pattern of intake, with an initial reduction in high-fat diet followed by an increase in chow consumption. These results demonstrate that the DMS has a role in mediating cue-food learning and its subsequent reversal, as well as changes in food intake when a choice is provided. Together, these results demonstrate the DMS is involved in reward associative learning and reward consumption, when behavioral flexibility is needed to adjust responding or consumption to match the current value. (PsycINFO Database Record (c) 2017 APA, all rights reserved).
-
Low putamen activity associated with poor reward sensitivity in childhood chronic fatigue syndrome
Directory of Open Access Journals (Sweden)
Kei Mizuno, Ph.D.
2016-01-01
Full Text Available Motivational signals influence a wide variety of cognitive processes and components of behavioral performance. Cognitive dysfunction in patients with childhood chronic fatigue syndrome (CCFS may be closely associated with a low motivation to learn induced by impaired neural reward processing. However, the extent to which reward processing is impaired in CCFS patients is unclear. The aim of the present functional magnetic resonance imaging (fMRI study was to determine whether brain activity in regions related to reward sensitivity is impaired in CCFS patients. fMRI data were collected from 13 CCFS patients (mean age, 13.6 ± 1.0 years and 13 healthy children and adolescents (HCA (mean age, 13.7 ± 1.3 years performing a monetary reward task. Neural activity in high- and low-monetary-reward conditions was compared between CCFS and HCA groups. Severity of fatigue and the reward obtained from learning in daily life were evaluated by questionnaires. Activity of the putamen was lower in the CCFS group than in the HCA group in the low-reward condition, but not in the high-reward condition. Activity of the putamen in the low-reward condition in CCFS patients was negatively and positively correlated with severity of fatigue and the reward from learning in daily life, respectively. We previously revealed that motivation to learn was correlated with striatal activity, particularly the neural activity in the putamen. This suggests that in CCFS patients low putamen activity, associated with altered dopaminergic function, decreases reward sensitivity and lowers motivation to learn.
-
Murty, Vishnu P.; Adcock, R. Alison
2013-01-01
Learning how to obtain rewards requires learning about their contexts and likely causes. How do long-term memory mechanisms balance the need to represent potential determinants of reward outcomes with the computational burden of an over-inclusive memory? One solution would be to enhance memory for salient events that occur during reward anticipation, because all such events are potential determinants of reward. We tested whether reward motivation enhances encoding of salient events like expec...
-
Exploiting risk-reward structures in decision making under uncertainty.
Leuker, Christina; Pachur, Thorsten; Hertwig, Ralph; Pleskac, Timothy J
2018-06-01
People often have to make decisions under uncertainty-that is, in situations where the probabilities of obtaining a payoff are unknown or at least difficult to ascertain. One solution to this problem is to infer the probability from the magnitude of the potential payoff and thus exploit the inverse relationship between payoffs and probabilities that occurs in many domains in the environment. Here, we investigated how the mind may implement such a solution: (1) Do people learn about risk-reward relationships from the environment-and if so, how? (2) How do learned risk-reward relationships impact preferences in decision-making under uncertainty? Across three experiments (N = 352), we found that participants can learn risk-reward relationships from being exposed to choice environments with a negative, positive, or uncorrelated risk-reward relationship. They were able to learn the associations both from gambles with explicitly stated payoffs and probabilities (Experiments 1 & 2) and from gambles about epistemic events (Experiment 3). In subsequent decisions under uncertainty, participants often exploited the learned association by inferring probabilities from the magnitudes of the payoffs. This inference systematically influenced their preferences under uncertainty: Participants who had been exposed to a negative risk-reward relationship tended to prefer the uncertain option over a smaller sure option for low payoffs, but not for high payoffs. This pattern reversed in the positive condition and disappeared in the uncorrelated condition. This adaptive change in preferences is consistent with the use of the risk-reward heuristic. Copyright © 2018 Elsevier B.V. All rights reserved.
-
Directory of Open Access Journals (Sweden)
Ioan eOpris
2011-05-01
Full Text Available Although reward probability is an important factor that shapes animal behavior, it is not well understood however, how the primate brain translates reward expectation into the vigor of movement (reaction time and speed. To address this question, we trained two monkeys in a reaction time task that required wrist movements in response to vibrotactile and visual stimuli, with a variable reward schedule. Correct performance was rewarded in 75 % of the trials. Monkeys were certain that they would be rewarded only in the trials immediately following withheld rewards. In these trials, the animals responded sooner and moved faster. Single-unit recordings from the dorsal striatum revealed that modulations in striatal neurons reflected such modulations of movement vigor. First, in the trials with certain rewards, striatal neurons modulated their firing rates earlier. Second, magnitudes of changes in neuronal firing rates depended on whether or not monkeys were certain about the reward. Third, these modulations depended on the sensory modality of the cue (visual vs. vibratory and/or movement direction (flexions vs. extensions. We conclude that dorsal striatum may be a part of the mechanism responsible for the modulation of movement vigor in response to changes of reward predictability.
-
Heterogeneity of reward mechanisms.
Lajtha, A; Sershen, H
2010-06-01
The finding that many drugs that have abuse potential and other natural stimuli such as food or sexual activity cause similar chemical changes in the brain, an increase in extracellular dopamine (DA) in the shell of the nucleus accumbens (NAccS), indicated some time ago that the reward mechanism is at least very similar for all stimuli and that the mechanism is relatively simple. The presently available information shows that the mechanisms involved are more complex and have multiple elements. Multiple brain regions, multiple receptors, multiple distinct neurons, multiple transmitters, multiple transporters, circuits, peptides, proteins, metabolism of transmitters, and phosphorylation, all participate in reward mechanisms. The system is variable, is changed during development, is sex-dependent, and is influenced by genetic differences. Not all of the elements participate in the reward of all stimuli. Different set of mechanisms are involved in the reward of different drugs of abuse, yet different mechanisms in the reward of natural stimuli such as food or sexual activity; thus there are different systems that distinguish different stimuli. Separate functions of the reward system such as anticipation, evaluation, consummation and identification; all contain function-specific elements. The level of the stimulus also influences the participation of the elements of the reward system, there are possible reactions to even below threshold stimuli, and excessive stimuli can change reward to aversion involving parts of the system. Learning and memory of past reward is an important integral element of reward and addictive behavior. Many of the reward elements are altered by repeated or chronic stimuli, and chronic exposure to one drug is likely to alter the response to another stimulus. To evaluate and identify the reward stimulus thus requires heterogeneity of the reward components in the brain.
-
Directory of Open Access Journals (Sweden)
Ulla K Seidel
Full Text Available Besides motor function, the basal ganglia have been implicated in feedback learning. In patients with chronic basal ganglia infarcts, deficits in reward-based reversal learning have previously been described.We re-examined the acquisition and reversal of stimulus-stimulus-reward associations and acquired equivalence in eleven patients with acute basal ganglia stroke (8 men, 3 women; 57.8±13.3 years, whose performance was compared eleven healthy subjects of comparable age, sex distribution and education, who were recruited outside the hospital. Eleven hospitalized patients with a similar vascular risk profile as the stroke patients but without stroke history served as clinical control group.In a neuropsychological assessment 7±3 days post-stroke, verbal and spatial short-term and working memory and inhibition control did not differ between groups. Compared with healthy subjects, control patients with vascular risk factors exhibited significantly reduced performance in the reversal phase (F[2,30] = 3.47; p = 0.044; post-hoc comparison between risk factor controls and healthy controls: p = 0.030, but not the acquisition phase (F[2,30] = 1.01; p = 0.376 and the acquired equivalence (F[2,30] = 1.04; p = 0.367 tasks. In all tasks, the performance of vascular risk factor patients closely resembled that of basal ganglia stroke patients. Correlation studies revealed a significant association of the number of vascular risk factors with reversal learning (r = -0.33, p = 0.012, but not acquisition learning (r = -0.20, p = 0.121 or acquired equivalence (r = -0.22, p = 0.096.The previously reported impairment of reward-based learning may be attributed to vascular risk factors and associated diseases, which are enriched in stroke patients. This study emphasizes the necessity of appropriate control subjects in cognition studies.
-
de Premorel, Géraud; Giurfa, Martin; Andraud, Christine; Gomez, Doris
2017-10-25
Iridescence-change of colour with changes in the angle of view or of illumination-is widespread in the living world, but its functions remain poorly understood. The presence of iridescence has been suggested in flowers where diffraction gratings generate iridescent colours. Such colours have been suggested to serve plant-pollinator communication. Here we tested whether a higher iridescence relative to corolla pigmentation would facilitate discrimination, learning and retention of iridescent visual targets. We conditioned bumblebees ( Bombus terrestris ) to discriminate iridescent from non-iridescent artificial flowers and we varied iridescence detectability by varying target iridescent relative to pigment optical effect. We show that bees rewarded on targets with higher iridescent relative to pigment effect required fewer choices to complete learning, showed faster generalization to novel targets exhibiting the same iridescence-to-pigment level and had better long-term memory retention. Along with optical measurements, behavioural results thus demonstrate that bees can learn iridescence-related cues as bona fide signals for flower reward. They also suggest that floral advertising may be shaped by competition between iridescence and corolla pigmentation, a fact that has important evolutionary implications for pollinators. Optical measurements narrow down the type of cues that bees may have used for learning. Beyond pollinator-plant communication, our experiments help understanding how receivers influence the evolution of iridescence signals generated by gratings. © 2017 The Author(s).
-
Reward eliminates retrieval-induced forgetting.
Imai, Hisato; Kim, Dongho; Sasaki, Yuka; Watanabe, Takeo
2014-12-02
Although it is well known that reward enhances learning and memory, how extensively such enhancement occurs remains unclear. To address this question, we examined how reward influences retrieval-induced forgetting (RIF) in which the retrieval of a nonpracticed item under the same category as a practiced item is worse than the retrieval of a nonpracticed item outside the category. Subjects were asked to try to encode category-exemplar pairs (e.g., FISH-salmon). Then, they were presented with a category name and a two-letter word stem (e.g., FISH-sa) and were asked to complete an encoded word (retrieval practice). For a correct response, apple juice was given as a reward in the reward condition and a beeping sound was presented in the no-reward condition. Finally, subjects were asked to report whether each exemplar had been presented in the first phase. RIF was replicated in the no-reward condition. However, in the reward condition, RIF was eliminated. These results suggest that reward enhances processing of retrieval of unpracticed members by mechanisms such as spreading activation within the same category, irrespective of whether items were practiced or not.
-
Boucheix, Jean-Michel; Lowe, Richard K.
2010-01-01
Two experiments used eye tracking to investigate a novel cueing approach for directing learner attention to low salience, high relevance aspects of a complex animation. In the first experiment, comprehension of a piano mechanism animation containing spreading-colour cues was compared with comprehension obtained with arrow cues or no cues. Eye…
-
Ortega-Castro, Nerea; Vadillo, Miguel A
2013-03-01
Some researchers have attempted to determine whether situations in which a single cue is paired with several outcomes (A-B, A-C interference or interference between outcomes) involve the same learning and retrieval mechanisms as situations in which several cues are paired with a single outcome (A-B, C-B interference or interference between cues). Interestingly, current research on a related effect, which is known as retrieval-induced forgetting, can illuminate this debate. Most retrieval-induced forgetting experiments are based on an experimental design that closely resembles the A-B, A-C interference paradigm. In the present experiment, we found that a similar effect may be observed when items are rearranged such that the general structure of the task more closely resembles the A-B, C-B interference paradigm. This result suggests that, as claimed by other researchers in the area of contingency learning, the two types of interference, namely A-B, A-C and A-B, C-B interference, may share some basic mechanisms. Moreover, the type of inhibitory processes assumed to underlie retrieval-induced forgetting may also play a role in these phenomena. Copyright © 2012 Elsevier B.V. All rights reserved.
-
Connor, David A; Kutlu, Munir G; Gould, Thomas J
2017-07-01
Learned safety, a learning process in which a cue becomes associated with the absence of threat, is disrupted in individuals with post-traumatic stress disorder (PTSD). A bi-directional relationship exists between smoking and PTSD and one potential explanation is that nicotine-associated changes in cognition facilitate PTSD emotional dysregulation by disrupting safety associations. Therefore, we investigated whether nicotine would disrupt learned safety by enhancing fear associated with a safety cue. In the present study, C57BL/6 mice were administered acute or chronic nicotine and trained over three days in a differential backward trace conditioning paradigm consisting of five trials of a forward conditioned stimulus (CS)+ (Light) co-terminating with a footshock unconditioned stimulus followed by a backward CS- (Tone) presented 20 s after cessation of the unconditioned stimulus. Summation testing found that acute nicotine disrupted learned safety, but chronic nicotine had no effect. Another group of animals administered acute nicotine showed fear when presented with the backward CS (Light) alone, indicating the formation of a maladaptive fear association with the backward CS. Finally, we investigated the brain regions involved by administering nicotine directly into the dorsal hippocampus, ventral hippocampus, and prelimbic cortex. Infusion of nicotine into the dorsal hippocampus disrupted safety learning.
-
Directory of Open Access Journals (Sweden)
Lauren Lederle
2011-01-01
Full Text Available The mouse has emerged as a uniquely valuable species for studying the molecular and genetic basis of complex behaviors and modeling neuropsychiatric disease states. While valid and reliable preclinical assays for reward-related behaviors are critical to understanding addiction-related processes, and various behavioral procedures have been developed and characterized in rats and primates, there have been relatively few studies using operant-based addiction-relevant behavioral paradigms in the mouse. Here we describe the performance of the C57BL/6J inbred mouse strain on three major reward-related paradigms, and replicate the same procedures in two other commonly used inbred strains (DBA/2J, BALB/cJ. We examined Pavlovian-instrumental transfer (PIT by measuring the ability of an auditory cue associated with food reward to promote an instrumental (lever press response. In a separate experiment, we assessed the acquisition and extinction of a simple stimulus-reward instrumental behavior on a touch screen based task. Reinstatement of this behavior was then examined following either continuous exposure to cues (conditioned reinforcers, CRs associated with reward, brief reward and CR exposure, or brief reward exposure followed by continuous CR exposure. The third paradigm examined sensitivity of an instrumental (lever press response to devaluation of food reward (a probe for outcome insensitive, habitual behavior by repeated pairing with malaise. Results showed that C57BL/6J mice displayed robust PIT, as well as clear extinction and reinstatement, but were insensitive to reinforcer devaluation. DBA/2J mice showed good PIT and (rewarded reinstatement, but were slow to extinguish and did not show reinforcer devaluation or significant CR-reinstatement. BALB/cJ mice also displayed good PIT, extinction and reinstatement, and retained instrumental responding following devaluation, but, unlike the other strains, demonstrated reduced Pavlovian approach
-
Lederle, Lauren; Weber, Susanna; Wright, Tara; Feyder, Michael; Brigman, Jonathan L; Crombag, Hans S; Saksida, Lisa M; Bussey, Timothy J; Holmes, Andrew
2011-01-10
The mouse has emerged as a uniquely valuable species for studying the molecular and genetic basis of complex behaviors and modeling neuropsychiatric disease states. While valid and reliable preclinical assays for reward-related behaviors are critical to understanding addiction-related processes, and various behavioral procedures have been developed and characterized in rats and primates, there have been relatively few studies using operant-based addiction-relevant behavioral paradigms in the mouse. Here we describe the performance of the C57BL/6J inbred mouse strain on three major reward-related paradigms, and replicate the same procedures in two other commonly used inbred strains (DBA/2J, BALB/cJ). We examined Pavlovian-instrumental transfer (PIT) by measuring the ability of an auditory cue associated with food reward to promote an instrumental (lever press) response. In a separate experiment, we assessed the acquisition and extinction of a simple stimulus-reward instrumental behavior on a touch screen based task. Reinstatement of this behavior was then examined following either continuous exposure to cues (conditioned reinforcers, CRs) associated with reward, brief reward and CR exposure, or brief reward exposure followed by continuous CR exposure. The third paradigm examined sensitivity of an instrumental (lever press) response to devaluation of food reward (a probe for outcome insensitive, habitual behavior) by repeated pairing with malaise. Results showed that C57BL/6J mice displayed robust PIT, as well as clear extinction and reinstatement, but were insensitive to reinforcer devaluation. DBA/2J mice showed good PIT and (rewarded) reinstatement, but were slow to extinguish and did not show reinforcer devaluation or significant CR-reinstatement. BALB/cJ mice also displayed good PIT, extinction and reinstatement, and retained instrumental responding following devaluation, but, unlike the other strains, demonstrated reduced Pavlovian approach behavior (food
-
Addiction and brain reward and antireward pathways.
Gardner, Eliot L
2011-01-01
etiology holds very well for addiction. Addiction appears to correlate with a hypodopaminergic dysfunctional state within the reward circuitry of the brain. Neuroimaging studies in humans add credence to this hypothesis. Credible evidence also implicates serotonergic, opioid, endocannabinoid, GABAergic and glutamatergic mechanisms in addiction. Critically, drug addiction progresses from occasional recreational use to impulsive use to habitual compulsive use. This correlates with a progression from reward-driven to habit-driven drug-seeking behavior. This behavioral progression correlates with a neuroanatomical progression from ventral striatal (nucleus accumbens) to dorsal striatal control over drug-seeking behavior. The three classical sets of craving and relapse triggers are (a) reexposure to addictive drugs, (b) stress, and (c) reexposure to environmental cues (people, places, things) previously associated with drug-taking behavior. Drug-triggered relapse involves the nucleus accumbens and the neurotransmitter dopamine. Stress-triggered relapse involves (a) the central nucleus of the amygdala, the bed nucleus of the stria terminalis, and the neurotransmitter corticotrophin-releasing factor, and (b) the lateral tegmental noradrenergic nuclei of the brain stem and the neurotransmitter norepinephrine. Cue-triggered relapse involves the basolateral nucleus of the amygdala, the hippocampus and the neurotransmitter glutamate. Knowledge of the neuroanatomy, neurophysiology, neurochemistry and neuropharmacology of addictive drug action in the brain is currently producing a variety of strategies for pharmacotherapeutic treatment of drug addiction, some of which appear promising. Copyright © 2011 S. Karger AG, Basel.
-
The role of social stimuli content in neuroimaging studies investigating alcohol cue-reactivity
Groefsema, M.M.; Engels, R.C.M.E.; Luijten, M.
2016-01-01
Introduction: Cue-reactivity is thought to play a fundamental role in the maintenance of addiction. The incentive sensitization theory proposes that conditioned responses are related to increased sensitivity of the reward-related dopaminergic pathways in the brain. However, neuroimaging studies on
-
Monetary reward speeds up voluntary saccades.
Chen, Lewis L; Chen, Y Mark; Zhou, Wu; Mustain, William D
2014-01-01
Past studies have shown that reward contingency is critical for sensorimotor learning, and reward expectation speeds up saccades in animals. Whether monetary reward speeds up saccades in human remains unknown. Here we addressed this issue by employing a conditional saccade task, in which human subjects performed a series of non-reflexive, visually-guided horizontal saccades. The subjects were (or were not) financially compensated for making a saccade in response to a centrally-displayed visual congruent (or incongruent) stimulus. Reward modulation of saccadic velocities was quantified independently of the amplitude-velocity coupling. We found that reward expectation significantly sped up voluntary saccades up to 30°/s, and the reward modulation was consistent across tests. These findings suggest that monetary reward speeds up saccades in human in a fashion analogous to how juice reward sped up saccades in monkeys. We further noticed that the idiosyncratic nasal-temporal velocity asymmetry was highly consistent regardless of test order, and its magnitude was not correlated with the magnitude of reward modulation. This suggests that reward modulation and the intrinsic velocity asymmetry may be governed by separate mechanisms that regulate saccade generation.
-
Strategy selection in cue-based decision making.
Bryant, David J
2014-06-01
People can make use of a range of heuristic and rational, compensatory strategies to perform a multiple-cue judgment task. It has been proposed that people are sensitive to the amount of cognitive effort required to employ decision strategies. Experiment 1 employed a dual-task methodology to investigate whether participants' preference for heuristic versus compensatory decision strategies can be altered by increasing the cognitive demands of the task. As indicated by participants' decision times, a secondary task interfered more with the performance of a heuristic than compensatory decision strategy but did not affect the proportions of participants using either type of strategy. A stimulus set effect suggested that the conjunction of cue salience and cue validity might play a determining role in strategy selection. The results of Experiment 2 indicated that when a perceptually salient cue was also the most valid, the majority of participants preferred a single-cue heuristic strategy. Overall, the results contradict the view that heuristics are more likely to be adopted when a task is made more cognitively demanding. It is argued that people employ 2 learning processes during training, one an associative learning process in which cue-outcome associations are developed by sampling multiple cues, and another that involves the sequential examination of single cues to serve as a basis for a single-cue heuristic.
-
Bunce, Scott C; Harris, Jonathan D; Bixler, Edward O; Taylor, Megan; Muelly, Emilie; Deneke, Erin; Thompson, Kenneth W; Meyer, Roger E
2015-01-01
There is growing evidence for a neuroadaptive model underlying vulnerability to relapse in opioid dependence. The purpose of this study was to evaluate clinical measures hypothesized to mirror elements of allostatic dysregulation in patients dependent on prescription opioids at 2 time points after withdrawal, compared with healthy control participants. Recently withdrawn (n = 7) prescription opioid-dependent patients were compared with the patients in supervised residential care for 2 to 3 months (extended care; n = 7) and healthy controls (n = 7) using drug cue reactivity, affect-modulated startle response tasks, salivary cortisol, and 8 days of sleep actigraphy. Prefrontal cortex was monitored with functional near-infrared spectroscopy during the cue reactivity task. Startle response results indicated reduced hedonic response to natural rewards among patients recently withdrawn from opioids relative to extended care patients. The recently withdrawn patients showed increased activation to pill stimuli in right dorsolateral prefrontal cortex relative to extended care patients. Cortisol levels were elevated among recently withdrawn patients and intermediate for extended care relative to healthy controls. Actigraphy indicated disturbed sleep between recently withdrawn patients and extended care patients; extended care patients were similar to controls. Dorsolateral prefrontal cortex activation to drug and natural reward cues, startle responses to natural reward cues, day-time cortisol levels, time in bed, and total time spent sleeping were all correlated with the number of days since last drug use (ie, time in supervised residential treatment). These results suggest possible re-regulation of dysregulated hypothalamic-pituitary-adrenal axis and brain reward systems in prescription opioid-dependent patients over the drug-free period in residential treatment.
-
Jastreboff, Ania M; Sinha, Rajita; Lacadie, Cheryl; Small, Dana M; Sherwin, Robert S; Potenza, Marc N
2013-02-01
Obesity is associated with alterations in corticolimbic-striatal brain regions involved in food motivation and reward. Stress and the presence of food cues may each motivate eating and engage corticolimibic-striatal neurocircuitry. It is unknown how these factors interact to influence brain responses and whether these interactions are influenced by obesity, insulin levels, and insulin sensitivity. We hypothesized that obese individuals would show greater responses in corticolimbic-striatal neurocircuitry after exposure to stress and food cues and that brain activations would correlate with subjective food craving, insulin levels, and HOMA-IR. Fasting insulin levels were assessed in obese and lean subjects who were exposed to individualized stress and favorite-food cues during functional MRI. Obese, but not lean, individuals exhibited increased activation in striatal, insular, and hypothalamic regions during exposure to favorite-food and stress cues. In obese but not lean individuals, food craving, insulin, and HOMA-IR levels correlated positively with neural activity in corticolimbic-striatal brain regions during favorite-food and stress cues. The relationship between insulin resistance and food craving in obese individuals was mediated by activity in motivation-reward regions including the striatum, insula, and thalamus. These findings demonstrate that obese, but not lean, individuals exhibit increased corticolimbic-striatal activation in response to favorite-food and stress cues and that these brain responses mediate the relationship between HOMA-IR and food craving. Improving insulin sensitivity and in turn reducing corticolimbic-striatal reactivity to food cues and stress may diminish food craving and affect eating behavior in obesity.
-
Associations between sleep parameters and food reward.
McNeil, Jessica; Cadieux, Sébastien; Finlayson, Graham; Blundell, John E; Doucet, Éric
2015-06-01
This study examined the effects of acute, isocaloric aerobic and resistance exercise on different sleep parameters, and whether changes in these sleep parameters between sessions were related to next morning food reward. Fourteen men and women (age: 21.9 ± 2.7 years; body mass index: 22.7 ± 1.9 kg m(-) ²) participated in three randomized crossover sessions: aerobic exercise; resistance exercise; and sedentary control. Target exercise energy expenditure was matched at 4 kcal kg(-1) of body weight, and performed at 70% of VO2peak or 70% of 1 repetition-maximal. Sleep was measured (accelerometry) for 22 h following each session. The 'wanting' for visual food cues (validated computer task) was assessed the next morning. There were no differences in sleep parameters and food 'wanting' between conditions. Decreases in sleep duration and earlier wake-times were significantly associated with increased food 'wanting' between sessions (P = 0.001). However, these associations were no longer significant after controlling for elapsed time between wake-time and the food reward task. These findings suggest that shorter sleep durations and earlier wake-times are associated with increased food reward, but these associations are driven by elapsed time between awakening and completion of the food reward task. © 2015 European Sleep Research Society.
-
Oei, Nicole Y. L.; Both, Stephanie; van Heemst, Diana; van der Grond, Jeroen
2014-01-01
Summary Stress is thought to alter motivational processes by increasing dopamine (DA) secretion in the brain’s ‘‘reward system’’, and its key region, the nucleus accumbens (NAcc). However, stress studies using functional magnetic resonance imaging (fMRI), mainly found evidence for stress-induced decreases in NAcc responsiveness toward reward cues. Results from both animal and human PETstudies indicate that the stress hormone cortisol may be crucial in the interaction between st...
-
Zack, Elizabeth; Gerhardstein, Peter; Meltzoff, Andrew N; Barr, Rachel
2013-02-01
Infants have difficulty transferring information between 2D and 3D sources. The current study extends Zack, Barr, Gerhardstein, Dickerson & Meltzoff's (2009) touch screen imitation task to examine whether the addition of specific language cues significantly facilitates 15-month-olds' transfer of learning between touch screens and real-world 3D objects. The addition of two kinds of linguistic cues (object label plus verb or nonsense name) did not elevate action imitation significantly above levels observed when such language cues were not used. Language cues hindered infants' performance in the 3D→2D direction of transfer, but only for the object label plus verb condition. The lack of a facilitative effect of language is discussed in terms of competing cognitive loads imposed by conjointly transferring information across dimensions and processing linguistic cues in an action imitation task at this age. © 2012 The Authors. Scandinavian Journal of Psychology © 2012 The Scandinavian Psychological Associations.
-
Versaggi, Cassandra L; King, Christopher P; Meyer, Paul J
2016-08-01
Some individuals are particularly responsive to reward-associated stimuli ("cues"), including the effects of these cues on craving and relapse to drug-seeking behavior. In the cases of nicotine and alcohol, cues may acquire these abilities via the incentive-enhancing properties of the drug. To determine the interaction between cue-responsivity and nicotine reinforcement, we studied the patterns of nicotine self-administration in rats categorized based on their tendency to approach a food-predictive cue ("sign-trackers") or a reward-delivery location ("goal-trackers"). In a second experiment, we determined whether nicotine and ethanol altered the incentive value of a food cue. Rats were classified as sign- or goal-trackers during a Pavlovian conditioned approach paradigm. Rats then self-administered intravenous nicotine (0.03 mg/kg infusions) followed by extinction and cue-induced reinstatement tests. We also tested the effects of nicotine (0.4 mg/kg base s.c.) or ethanol (0.7 g/kg i.p.) on the approach to, and reinforcing efficacy of, a food cue. Sign-trackers showed greater reinstatement in response to a nicotine cue. Further, nicotine enhanced sign-tracking but not goal-tracking to a food cue and also enhanced responding for the food cue during the conditioned reinforcement test. Conversely, ethanol reduced sign-tracking and increased goal-tracking, but had no effect on conditioned reinforcement. Our studies demonstrate that the tendency to attribute incentive value to a food cue predicts enhanced cue-induced reinstatement. Additionally, the incentive value of food cues is differentially modulated by nicotine and ethanol, which may be related to the reinforcing effects of these drugs.
-
Impulsivity in binge eating disorder: food cues elicit increased reward responses and disinhibition.
Directory of Open Access Journals (Sweden)
Kathrin Schag
Full Text Available BACKGROUND: Binge eating disorder (BED represents a distinct eating disorder diagnosis. Current approaches assume increased impulsivity to be one factor leading to binge eating and weight gain. We used eye tracking to investigate both components of impulsivity, namely reward sensitivity and rash-spontaneous behaviour towards food in BED for the first time. METHODS: Overweight and obese people with BED (BED+; n = 25, without BED (BED-; n = 26 and healthy normal-weight controls (NWC; n = 25 performed a free exploration paradigm measuring reward sensitivity (experiment 1 and a modified antisaccade paradigm measuring disinhibited, rash-spontaneous behaviour (experiment 2 using food and nonfood stimuli. Additionally, trait impulsivity was assessed. RESULTS: In experiment 1, all participants located their initial fixations more often on food stimuli and BED+ participants gazed longer on food stimuli in comparison with BED- and NWC participants. In experiment 2, BED+ participants had more difficulties inhibiting saccades towards food and nonfood stimuli compared with both other groups in first saccades, and especially towards food stimuli in second saccades and concerning sequences of first and second saccades. BED- participants did not differ significantly from NWC participants in both experiments. Additionally, eye tracking performance was associated with self-reported reward responsiveness and self-control. CONCLUSIONS: According to these results, food-related reward sensitivity and rash-spontaneous behaviour, as the two components of impulsivity, are increased in BED in comparison with weight-matched and normal-weight controls. This indicates that BED represents a neurobehavioural phenotype of obesity that is characterised by increased impulsivity. Interventions for BED should target these special needs of affected patients.
-
Wouwe, N.C. van; Ridderinkhof, K.R.; Wildenberg, W.P.M. van den; Band, G.P.H.; Abisogun, A.; Elias, W.J.; Frysinger, R.; Wylie, S.A.
2011-01-01
Recently, the subthalamic nucleus (STN) has been shown to be critically involved in decision-making, action selection, and motor control. Here we investigate the effect of deep brain stimulation (DBS) of the STN on reward-based decision-learning in patients diagnosed with Parkinson’s disease (PD).
-
Calpain-GRIP Signaling in Nucleus Accumbens Core Mediates the Reconsolidation of Drug Reward Memory.
Liang, Jie; Li, Jia-Li; Han, Ying; Luo, Yi-Xiao; Xue, Yan-Xue; Zhang, Yàn; Zhang, Yán; Zhang, Li-Bo; Chen, Man-Li; Lu, Lin; Shi, Jie
2017-09-13
Exposure to drug-paired cues causes drug memories to be in a destabilized state and interfering with memory reconsolidation can inhibit relapse. Calpain, a calcium-dependent neutral cysteine protease, is involved in synaptic plasticity and the formation of long-term fear memory. However, the role of calpain in the reconsolidation of drug reward memory is still unknown. In the present study, using a conditioned place preference (CPP) model, we found that exposure to drug-paired contextual stimuli induced the activation of calpain and decreased the expression of glutamate receptor interacting protein 1 (GRIP1) in the nucleus accumbens (NAc) core, but not shell, of male rats. Infusions of calpain inhibitors in the NAc core immediately after retrieval disrupted the reconsolidation of cocaine/morphine cue memory and blocked retrieval-induced calpain activation and GRIP1 degradation. The suppressive effect of calpain inhibitors on the expression of drug-induced CPP lasted for at least 14 d. The inhibition of calpain without retrieval 6 h after retrieval or after exposure to an unpaired context had no effects on the expression of reward memory. Calpain inhibition after retrieval also decreased cocaine seeking in a self-administration model and this effect did not recover spontaneously after 28 d. Moreover, the knock-down of GRIP1 expression in the NAc core by lentivirus-mediated short-hairpin RNA blocked disruption of the reconsolidation of drug cue memories that was induced by calpain inhibitor treatment. These results suggest that calpain activity in the NAc core is crucial for the reconsolidation of drug reward memory via the regulation of GRIP1 expression. SIGNIFICANCE STATEMENT Calpain plays an important role in synaptic plasticity and long-term memory consolidation, however, its role in the reconsolidation of drug cue memory remains unknown. Using conditioned place preference and self-administration procedures, we found that exposure to drug-paired cues induced the
-
Sieling, Fred; Bédécarrats, Alexis; Simmers, John; Prinz, Astrid A; Nargeot, Romuald
2014-05-05
Rewarding stimuli in associative learning can transform the irregularly and infrequently generated motor patterns underlying motivated behaviors into output for accelerated and stereotyped repetitive action. This transition to compulsive behavioral expression is associated with modified synaptic and membrane properties of central neurons, but establishing the causal relationships between cellular plasticity and motor adaptation has remained a challenge. We found previously that changes in the intrinsic excitability and electrical synapses of identified neurons in Aplysia's central pattern-generating network for feeding are correlated with a switch to compulsive-like motor output expression induced by in vivo operant conditioning. Here, we used specific computer-simulated ionic currents in vitro to selectively replicate or suppress the membrane and synaptic plasticity resulting from this learning. In naive in vitro preparations, such experimental manipulation of neuronal membrane properties alone increased the frequency but not the regularity of feeding motor output found in preparations from operantly trained animals. On the other hand, changes in synaptic strength alone switched the regularity but not the frequency of feeding output from naive to trained states. However, simultaneously imposed changes in both membrane and synaptic properties reproduced both major aspects of the motor plasticity. Conversely, in preparations from trained animals, experimental suppression of the membrane and synaptic plasticity abolished the increase in frequency and regularity of the learned motor output expression. These data establish direct causality for the contributions of distinct synaptic and nonsynaptic adaptive processes to complementary facets of a compulsive behavior resulting from operant reward learning. Copyright © 2014 Elsevier Ltd. All rights reserved.
-
Model-free and model-based reward prediction errors in EEG.
Sambrook, Thomas D; Hardwick, Ben; Wills, Andy J; Goslin, Jeremy
2018-05-24
Learning theorists posit two reinforcement learning systems: model-free and model-based. Model-based learning incorporates knowledge about structure and contingencies in the world to assign candidate actions with an expected value. Model-free learning is ignorant of the world's structure; instead, actions hold a value based on prior reinforcement, with this value updated by expectancy violation in the form of a reward prediction error. Because they use such different learning mechanisms, it has been previously assumed that model-based and model-free learning are computationally dissociated in the brain. However, recent fMRI evidence suggests that the brain may compute reward prediction errors to both model-free and model-based estimates of value, signalling the possibility that these systems interact. Because of its poor temporal resolution, fMRI risks confounding reward prediction errors with other feedback-related neural activity. In the present study, EEG was used to show the presence of both model-based and model-free reward prediction errors and their place in a temporal sequence of events including state prediction errors and action value updates. This demonstration of model-based prediction errors questions a long-held assumption that model-free and model-based learning are dissociated in the brain. Copyright © 2018 Elsevier Inc. All rights reserved.
-
Cortisol, but not intranasal insulin, affects the central processing of visual food cues
Ferreira de Sá, D.S.; Schulz, A.; Streit, F.E.; Turner, J.D.; Oitzl, M.S.; Blumenthal, T.D.; Schächinger, H.
2014-01-01
Stress glucocorticoids and insulin are important endocrine regulators of energy homeostasis, but little is known about their central interaction on the reward-related processing of food cues. According to a balanced group design, healthy food deprived men received either 40IU intranasal insulin
-
Cassidy, Clifford M.; Brodeur, Mathieu B.; Lepage, Martin; Malla, Ashok
2014-01-01
Background Dysfunctional reward processing is present in individuals with schizophrenia-spectrum disorders (SSD) and may confer vulnerability to addiction. Our objective was to identify a deficit in patients with SSD on response to rewarding stimuli and determine whether this deficit predicts cannabis use. Methods We divided a group of patients with SSD and nonpsychotic controls into cannabis users and nonusers. Response to emotional and cannabis-associated visual stimuli was assessed using self-report, event-related potentials (using the late positive potential [LPP]), facial electromyography and skin-conductance response. Results Our sample comprised 35 patients with SSD and 35 nonpsychotic controls. Compared with controls, the patients with SSD showed blunted LPP response to pleasant stimuli (p = 0.003). Across measures, cannabis-using controls showed greater response to pleasant stimuli than to cannabis stimuli whereas cannabis-using patients showed little bias toward pleasant stimuli. Reduced LPP response to pleasant stimuli was predictive of more frequent subsequent cannabis use (β = −0.24, p = 0.034). Limitations It is not clear if the deficit associated with cannabis use is specific to rewarding stimuli or nonspecific to any kind of emotionally salient stimuli. Conclusion The LPP captures a reward-processing deficit in patients with SSD and shows potential as a biomarker for identifying patients at risk of heavy cannabis use. PMID:24913137
-
Dopamine reward prediction errors reflect hidden state inference across time
Starkweather, Clara Kwon; Babayan, Benedicte M.; Uchida, Naoshige; Gershman, Samuel J.
2017-01-01
Midbrain dopamine neurons signal reward prediction error (RPE), or actual minus expected reward. The temporal difference (TD) learning model has been a cornerstone in understanding how dopamine RPEs could drive associative learning. Classically, TD learning imparts value to features that serially track elapsed time relative to observable stimuli. In the real world, however, sensory stimuli provide ambiguous information about the hidden state of the environment, leading to the proposal that TD learning might instead compute a value signal based on an inferred distribution of hidden states (a ‘belief state’). In this work, we asked whether dopaminergic signaling supports a TD learning framework that operates over hidden states. We found that dopamine signaling exhibited a striking difference between two tasks that differed only with respect to whether reward was delivered deterministically. Our results favor an associative learning rule that combines cached values with hidden state inference. PMID:28263301
-
Cerebral interactions of pain and reward and their relevance for chronic pain.
Becker, Susanne; Gandhi, Wiebke; Schweinhardt, Petra
2012-06-29
Pain and reward are opponent, interacting processes. Such interactions are enabled by neuroanatomical and neurochemical overlaps of brain systems that process pain and reward. Cerebral processing of hedonic ('liking') and motivational ('wanting') aspects of reward can be separated: the orbitofrontal cortex and opioids play an important role for the hedonic experience, and the ventral striatum and dopamine predominantly process motivation for reward. Supported by neuroimaging studies, we present here the hypothesis that the orbitofrontal cortex and opioids are responsible for pain modulation by hedonic experience, while the ventral striatum and dopamine mediate motivational effects on pain. A rewarding stimulus that appears to be particularly important in the context of pain is pain relief. Further, reward, including pain relief, leads to operant learning, which can affect pain sensitivity. Indirect evidence points at brain mechanisms that might underlie pain relief as a reward and related operant learning but studies are scarce. Investigating the cerebral systems underlying pain-reward interactions as well as related operant learning holds the potential of better understanding mechanisms that contribute to the development and maintenance of chronic pain, as detailed in the last section of this review. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.
-
Gibson, Brett M; Wasserman, Edward A
2003-01-01
The authors taught pigeons to discriminate displays of 16 identical items from displays of 16 nonidentical items. Unlike most same-different discrimination studies--where only stimulus relations could serve a discriminative function--both the identity of the items and the relations among the items were discriminative features of the displays. The pigeons learned about both stimulus identity and stimulus relations when these 2 sources of information served as redundant, relevant cues. In tests of associative competition, identity cues exerted greater stimulus control than relational cues. These results suggest that the pigeon can respond to both specific stimuli and general relations in the environment.
-
Versaggi, Cassandra L.; King, Christopher P.; Meyer, Paul J.
2016-01-01
Rationale Some individuals are particularly responsive to reward-associated stimuli (“cues”), including the effects of these cues on craving and relapse to drug-seeking behavior. In the cases of nicotine and alcohol, cues may acquire these abilities via the incentive-enhancing properties of the drug. Objectives To determine the interaction between cue-responsivity and nicotine reinforcement, we studied the patterns of nicotine self-administration in rats categorized based on their tendency to approach a food predictive cue (“sign-trackers”) or a reward-delivery location (“goal-trackers”). In a second experiment, we determined whether nicotine and ethanol altered the incentive value of a food cue. Methods Rats were classified as sign- or goal-trackers during a Pavlovian conditioned approach paradigm. Rats then self-administered intravenous nicotine (0.03 mg/kg infusions) followed by extinction and cue induced reinstatement tests. We also tested the effects of nicotine (0.4 mg/kg base s.c.) or ethanol (0.7 g/kg i.p.) on the approach to, and reinforcing efficacy of, a food cue. Results Sign-trackers showed greater reinstatement in response to a nicotine cue. Further, nicotine enhanced sign-tracking but not goal-tracking to a food cue, and also enhanced responding for the food cue during the conditioned reinforcement test. Conversely, ethanol reduced sign-tracking and increased goal-tracking, but had no effect on conditioned reinforcement. Conclusions Our studies demonstrate that the tendency to attribute incentive value to a food cue predicts enhanced cue-induced reinstatement. Additionally, the incentive value of food cues is differentially modulated by nicotine and ethanol, which may be related to the reinforcing effects of these drugs. PMID:27282365
-
Reward and Attentional Control in Visual Search
Anderson, Brian A.; Wampler, Emma K.; Laurent, Patryk A.
2015-01-01
It has long been known that the control of attention in visual search depends both on voluntary, top-down deployment according to context-specific goals, and on involuntary, stimulus-driven capture based on the physical conspicuity of perceptual objects. Recent evidence suggests that pairing target stimuli with reward can modulate the voluntary deployment of attention, but there is little evidence that reward modulates the involuntary deployment of attention to task-irrelevant distractors. We report several experiments that investigate the role of reward learning on attentional control. Each experiment involved a training phase and a test phase. In the training phase, different colors were associated with different amounts of monetary reward. In the test phase, color was not task-relevant and participants searched for a shape singleton; in most experiments no reward was delivered in the test phase. We first show that attentional capture by physically salient distractors is magnified by a previous association with reward. In subsequent experiments we demonstrate that physically inconspicuous stimuli previously associated with reward capture attention persistently during extinction—even several days after training. Furthermore, vulnerability to attentional capture by high-value stimuli is negatively correlated across individuals with working memory capacity and positively correlated with trait impulsivity. An analysis of intertrial effects reveals that value-driven attentional capture is spatially specific. Finally, when reward is delivered at test contingent on the task-relevant shape feature, recent reward history modulates value-driven attentional capture by the irrelevant color feature. The influence of learned value on attention may provide a useful model of clinical syndromes characterized by similar failures of cognitive control, including addiction, attention-deficit/hyperactivity disorder, and obesity. PMID:23437631
-
Reward networks in the brain as captured by connectivity measures
Directory of Open Access Journals (Sweden)
Estela Camara
2009-12-01
Full Text Available An assortment of human behaviors is thought to be driven by rewards including reinforcement learning, novelty processing, learning, decision making, economic choice, incentive motivation, and addiction. In each case the ventral tegmental area / ventral striatum (Nucleus accumbens system (VTA-VS has been implicated as a key structure by functional imaging studies, mostly on the basis of standard, univariate analyses. Here we propose that standard fMRI analysis needs to be complemented by methods that take into account the differential connectivity of the VTA-VS system in the different behavioral contexts in order to describe reward based processes more appropriately. We first consider the wider network for reward processing as it emerged from animal experimentation. Subsequently, an example for a method to assess functional connectivity is given. Finally, we illustrate the usefulness of such analyses by examples regarding reward valuation, reward expectation and the role of reward in addiction.
-
Ihssen, Niklas; Sokunbi, Moses O; Lawrence, Andrew D; Lawrence, Natalia S; Linden, David E J
2017-06-01
FMRI-based neurofeedback transforms functional brain activation in real-time into sensory stimuli that participants can use to self-regulate brain responses, which can aid the modification of mental states and behavior. Emerging evidence supports the clinical utility of neurofeedback-guided up-regulation of hypoactive networks. In contrast, down-regulation of hyperactive neural circuits appears more difficult to achieve. There are conditions though, in which down-regulation would be clinically useful, including dysfunctional motivational states elicited by salient reward cues, such as food or drug craving. In this proof-of-concept study, 10 healthy females (mean age = 21.40 years, mean BMI = 23.53) who had fasted for 4 h underwent a novel 'motivational neurofeedback' training in which they learned to down-regulate brain activation during exposure to appetitive food pictures. FMRI feedback was given from individually determined target areas and through decreases/increases in food picture size, thus providing salient motivational consequences in terms of cue approach/avoidance. Our preliminary findings suggest that motivational neurofeedback is associated with functionally specific activation decreases in diverse cortical/subcortical regions, including key motivational areas. There was also preliminary evidence for a reduction of hunger after neurofeedback and an association between down-regulation success and the degree of hunger reduction. Decreasing neural cue responses by motivational neurofeedback may provide a useful extension of existing behavioral methods that aim to modulate cue reactivity. Our pilot findings indicate that reduction of neural cue reactivity is not achieved by top-down regulation but arises in a bottom-up manner, possibly through implicit operant shaping of target area activity.
-
Valence of facial cues influences sheep learning in a visual discrimination task
Bellegarde, Lucille; Erhard, Hans; Weiss, A.; Boissy, Alain; Haskell, M.J.
2017-01-01
Sheep are one of the most studied farm species in terms of their ability to process information from faces, but little is known about their face-based emotion recognition abilities. We investigated (a) whether sheep could use images of sheep faces taken in situation of varying valence as cues in a simultaneous discrimination task and (b) whether the valence of the situation affects their learning performance. To accomplish this, we photographed faces of sheep in three situations inducing emot...
-
Valence of Facial Cues Influences Sheep Learning in a Visual Discrimination Task
Lucille G. A. Bellegarde; Lucille G. A. Bellegarde; Lucille G. A. Bellegarde; Hans W. Erhard; Alexander Weiss; Alain Boissy; Marie J. Haskell
2017-01-01
Sheep are one of the most studied farm species in terms of their ability to process information from faces, but little is known about their face-based emotion recognition abilities. We investigated (a) whether sheep could use images of sheep faces taken in situation of varying valence as cues in a simultaneous discrimination task and (b) whether the valence of the situation affects their learning performance. To accomplish this, we photographed faces of sheep in three situations inducing emot...
-
Effects of orbitofrontal cortex lesions on autoshaped lever pressing and reversal learning.
Chang, Stephen E
2014-10-15
A cue associated with a rewarding event can trigger behavior towards the cue itself due to the cue acquiring incentive value through its pairing with the rewarding outcome (i.e., sign-tracking). For example, rats will approach, press, and attempt to "consume" a retractable lever conditioned stimulus (CS) that signals delivery of a food unconditioned stimulus (US). Attending to food-predictive CSs is important when seeking out food, and it is just as important to be able to modify one's behavior when the relationships between CSs and USs are changed. Using a discriminative autoshaping procedure with lever CSs, the present study investigated the effects of orbitofrontal cortex (OFC) lesions on sign-tracking and reversal learning. Insertion of one lever was followed by sucrose delivery upon retraction, and insertion of another lever was followed by nothing. After the acquisition phase, the contingencies between the levers and outcomes were reversed. Bilateral OFC lesions had no effect on the acquisition of sign-tracking. However, OFC-lesioned rats showed substantial deficits in acquiring sign-tracking compared to sham-lesioned rats once the stimulus-outcome contingencies were reversed. Over the course of reversal learning, OFC-lesioned rats were able to reach comparable levels of sign-tracking as sham-lesioned rats. These findings suggest that OFC is not necessary for the ability of a CS to acquire incentive value and provide more evidence that OFC is critical for modifying behavior appropriately following a change in stimulus-outcome contingencies. Copyright © 2014 Elsevier B.V. All rights reserved.
-
Directory of Open Access Journals (Sweden)
Takashi Ochiai
Full Text Available BACKGROUND: Coordinated movement in social animal groups via social learning facilitates foraging activity. Few studies have examined the behavioral cause-and-effect between group members that mediates this social learning. METHODOLOGY/PRINCIPAL FINDINGS: We first established a behavioral paradigm for visual food learning using medaka fish and demonstrated that a single fish can learn to associate a visual cue with a food reward. Grouped medaka fish (6 fish learn to respond to the visual cue more rapidly than a single fish, indicating that medaka fish undergo social learning. We then established a data-mining method based on Kullback-Leibler divergence (KLD to search for candidate behaviors that induce alignment and found that high-speed movement of a focal fish tended to induce alignment of the other members locally and transiently under free-swimming conditions without presentation of a visual cue. The high-speed movement of the informed and trained fish during visual cue presentation appeared to facilitate the alignment of naïve fish in response to some visual cues, thereby mediating social learning. Compared with naïve fish, the informed fish had a higher tendency to induce alignment of other naïve fish under free-swimming conditions without visual cue presentation, suggesting the involvement of individual recognition in social learning. CONCLUSIONS/SIGNIFICANCE: Behavioral cause-and-effect studies of the high-speed movement between fish group members will contribute to our understanding of the dynamics of social behaviors. The data-mining method used in the present study is a powerful method to search for candidates factors associated with inter-individual interactions using a dataset for time-series coordinate data of individuals.
-
Addiction: beyond dopamine reward circuitry.
Volkow, Nora D; Wang, Gene-Jack; Fowler, Joanna S; Tomasi, Dardo; Telang, Frank
2011-09-13
Dopamine (DA) is considered crucial for the rewarding effects of drugs of abuse, but its role in addiction is much less clear. This review focuses on studies that used PET to characterize the brain DA system in addicted subjects. These studies have corroborated in humans the relevance of drug-induced fast DA increases in striatum [including nucleus accumbens (NAc)] in their rewarding effects but have unexpectedly shown that in addicted subjects, drug-induced DA increases (as well as their subjective reinforcing effects) are markedly blunted compared with controls. In contrast, addicted subjects show significant DA increases in striatum in response to drug-conditioned cues that are associated with self-reports of drug craving and appear to be of a greater magnitude than the DA responses to the drug. We postulate that the discrepancy between the expectation for the drug effects (conditioned responses) and the blunted pharmacological effects maintains drug taking in an attempt to achieve the expected reward. Also, whether tested during early or protracted withdrawal, addicted subjects show lower levels of D2 receptors in striatum (including NAc), which are associated with decreases in baseline activity in frontal brain regions implicated in salience attribution (orbitofrontal cortex) and inhibitory control (anterior cingulate gyrus), whose disruption results in compulsivity and impulsivity. These results point to an imbalance between dopaminergic circuits that underlie reward and conditioning and those that underlie executive function (emotional control and decision making), which we postulate contributes to the compulsive drug use and loss of control in addiction.
-
Addiction: Beyond dopamine reward circuitry
International Nuclear Information System (INIS)
Volkow, N.D.; Wang, G.-J.; Fowler, J.S.; Tomasi, D.; Telang, F.
2011-01-01
Dopamine (DA) is considered crucial for the rewarding effects of drugs of abuse, but its role in addiction is much less clear. This review focuses on studies that used PET to characterize the brain DA system in addicted subjects. These studies have corroborated in humans the relevance of drug-induced fast DA increases in striatum [including nucleus accumbens (NAc)] in their rewarding effects but have unexpectedly shown that in addicted subjects, drug-induced DA increases (as well as their subjective reinforcing effects) are markedly blunted compared with controls. In contrast, addicted subjects show significant DA increases in striatum in response to drug-conditioned cues that are associated with self-reports of drug craving and appear to be of a greater magnitude than the DA responses to the drug. We postulate that the discrepancy between the expectation for the drug effects (conditioned responses) and the blunted pharmacological effects maintains drug taking in an attempt to achieve the expected reward. Also, whether tested during early or protracted withdrawal, addicted subjects show lower levels of D2 receptors in striatum (including NAc), which are associated with decreases in baseline activity in frontal brain regions implicated in salience attribution (orbitofrontal cortex) and inhibitory control (anterior cingulate gyrus), whose disruption results in compulsivity and impulsivity. These results point to an imbalance between dopaminergic circuits that underlie reward and conditioning and those that underlie executive function (emotional control and decision making), which we postulate contributes to the compulsive drug use and loss of control in addiction.
-
Addiction: Beyond dopamine reward circuitry
Energy Technology Data Exchange (ETDEWEB)
Volkow, N.D.; Wang, G.; Volkow, N.D.; Wang, G.-J.; Fowler, J.S.; Tomasi, D.; Telang, F.
2011-09-13
Dopamine (DA) is considered crucial for the rewarding effects of drugs of abuse, but its role in addiction is much less clear. This review focuses on studies that used PET to characterize the brain DA system in addicted subjects. These studies have corroborated in humans the relevance of drug-induced fast DA increases in striatum [including nucleus accumbens (NAc)] in their rewarding effects but have unexpectedly shown that in addicted subjects, drug-induced DA increases (as well as their subjective reinforcing effects) are markedly blunted compared with controls. In contrast, addicted subjects show significant DA increases in striatum in response to drug-conditioned cues that are associated with self-reports of drug craving and appear to be of a greater magnitude than the DA responses to the drug. We postulate that the discrepancy between the expectation for the drug effects (conditioned responses) and the blunted pharmacological effects maintains drug taking in an attempt to achieve the expected reward. Also, whether tested during early or protracted withdrawal, addicted subjects show lower levels of D2 receptors in striatum (including NAc), which are associated with decreases in baseline activity in frontal brain regions implicated in salience attribution (orbitofrontal cortex) and inhibitory control (anterior cingulate gyrus), whose disruption results in compulsivity and impulsivity. These results point to an imbalance between dopaminergic circuits that underlie reward and conditioning and those that underlie executive function (emotional control and decision making), which we postulate contributes to the compulsive drug use and loss of control in addiction.
-
Probability differently modulating the effects of reward and punishment on visuomotor adaptation.
Song, Yanlong; Smiley-Oyen, Ann L
2017-12-01
Recent human motor learning studies revealed that punishment seemingly accelerated motor learning but reward enhanced consolidation of motor memory. It is not evident how intrinsic properties of reward and punishment modulate the potentially dissociable effects of reward and punishment on motor learning and motor memory. It is also not clear what causes the dissociation of the effects of reward and punishment. By manipulating probability of distribution, a critical property of reward and punishment, the present study demonstrated that probability had distinct modulation on the effects of reward and punishment in adapting to a sudden visual rotation and consolidation of the adaptation memory. Specifically, two probabilities of monetary reward and punishment distribution, 50 and 100%, were applied during young adult participants adapting to a sudden visual rotation. Punishment and reward showed distinct effects on motor adaptation and motor memory. The group that received punishments in 100% of the adaptation trials adapted significantly faster than the other three groups, but the group that received rewards in 100% of the adaptation trials showed marked savings in re-adapting to the same rotation. In addition, the group that received punishments in 50% of the adaptation trials that were randomly selected also had savings in re-adapting to the same rotation. Sensitivity to sensory prediction error or difference in explicit process induced by reward and punishment may likely contribute to the distinct effects of reward and punishment.
-
Nawroth, Christian; Prentice, Pamela M; McElligott, Alan G
2017-01-01
Variation in common personality traits, such as boldness or exploration, is often associated with risk-reward trade-offs and behavioural flexibility. To date, only a few studies have examined the effects of consistent behavioural traits on both learning and cognition. We investigated whether certain personality traits ('exploration' and 'sociability') of individuals were related to cognitive performance, learning flexibility and learning style in a social ungulate species, the goat (Capra hircus). We also investigated whether a preference for feature cues rather than impaired learning abilities can explain performance variation in a visual discrimination task. We found that personality scores were consistent across time and context. Less explorative goats performed better in a non-associative cognitive task, in which subjects had to follow the trajectory of a hidden object (i.e. testing their ability for object permanence). We also found that less sociable subjects performed better compared to more sociable goats in a visual discrimination task. Good visual learning performance was associated with a preference for feature cues, indicating personality-dependent learning strategies in goats. Our results suggest that personality traits predict the outcome in visual discrimination and non-associative cognitive tasks in goats and that impaired performance in a visual discrimination tasks does not necessarily imply impaired learning capacities, but rather can be explained by a varying preference for feature cues. Copyright © 2016 Elsevier B.V. All rights reserved.