WorldWideScience

Sample records for learning reward cues

  1. Morphine Reward Promotes Cue-Sensitive Learning: Implication of Dorsal Striatal CREB Activity

    Directory of Open Access Journals (Sweden)

    Mathieu Baudonnat

    2017-05-01

    Full Text Available Different parallel neural circuits interact and may even compete to process and store information: whereas stimulus–response (S–R learning critically depends on the dorsal striatum (DS, spatial memory relies on the hippocampus (HPC. Strikingly, despite its potential importance for our understanding of addictive behaviors, the impact of drug rewards on memory systems dynamics has not been extensively studied. Here, we assessed long-term effects of drug- vs food reinforcement on the subsequent use of S–R vs spatial learning strategies and their neural substrates. Mice were trained in a Y-maze cue-guided task, during which either food or morphine injections into the ventral tegmental area (VTA were used as rewards. Although drug- and food-reinforced mice learned the Y-maze task equally well, drug-reinforced mice exhibited a preferential use of an S–R learning strategy when tested in a water-maze competition task designed to dissociate cue-based and spatial learning. This cognitive bias was associated with a persistent increase in the phosphorylated form of cAMP response element-binding protein phosphorylation (pCREB within the DS, and a decrease of pCREB expression in the HPC. Pharmacological inhibition of striatal PKA pathway in drug-rewarded mice limited the morphine-induced increase in levels of pCREB in DS and restored a balanced use of spatial vs cue-based learning. Our findings suggest that drug (opiate reward biases the engagement of separate memory systems toward a predominant use of the cue-dependent system via an increase in learning-related striatal pCREB activity. Persistent functional imbalance between striatal and hippocampal activity could contribute to the persistence of addictive behaviors, or counteract the efficiency of pharmacological or psychotherapeutic treatments.

  2. 'Proactive' use of cue-context congruence for building reinforcement learning's reward function.

    Science.gov (United States)

    Zsuga, Judit; Biro, Klara; Tajti, Gabor; Szilasi, Magdolna Emma; Papp, Csaba; Juhasz, Bela; Gesztelyi, Rudolf

    2016-10-28

    Reinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent's knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent's control either using, or not using a model. In the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively. Based on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.

  3. Stingless bees (Melipona scutellaris) learn to associate footprint cues at food sources with a specific reward context.

    Science.gov (United States)

    Roselino, Ana Carolina; Rodrigues, André Vieira; Hrncir, Michael

    2016-10-01

    Foraging insects leave chemical footprints on flowers that subsequent foragers may use as indicators for recent flower visits and, thus, potential resource depletion. Accordingly, foragers should reject food sources presenting these chemical cues. Contrasting this assumption, experimental studies in stingless bees (Apidae, Meliponini), so far, demonstrated an attractive effect of footprints. These findings lead to doubts about the meaning of these chemical cues in natural foraging contexts. Here, we asked whether foragers of stingless bees (Melipona scutellaris) use footprints according to the previously experienced reward level of visited food sources. Bees were trained to artificial flower patches, at which the reward of a flower either decreased or, alternatively, increased after a visit by a forager. Individuals were allowed a total of nine foraging bouts to the patch, after which their preference for visited or unvisited flowers was tested. In the choice tests, bees trained under the decreasing reward context preferred unvisited flowers, whereas individuals trained under the increasing reward context preferred visited flowers. Foragers without experience chose randomly between visited and unvisited flowers. These results demonstrate that M. scutellaris learns to associate unspecific footprint cues at food sources with differential, specific reward contexts, and uses these chemical cues accordingly for their foraging decisions.

  4. Reward processing in the value-driven attention network: reward signals tracking cue identity and location.

    Science.gov (United States)

    Anderson, Brian A

    2017-03-01

    Through associative reward learning, arbitrary cues acquire the ability to automatically capture visual attention. Previous studies have examined the neural correlates of value-driven attentional orienting, revealing elevated activity within a network of brain regions encompassing the visual corticostriatal loop [caudate tail, lateral occipital complex (LOC) and early visual cortex] and intraparietal sulcus (IPS). Such attentional priority signals raise a broader question concerning how visual signals are combined with reward signals during learning to create a representation that is sensitive to the confluence of the two. This study examines reward signals during the cued reward training phase commonly used to generate value-driven attentional biases. High, compared with low, reward feedback preferentially activated the value-driven attention network, in addition to regions typically implicated in reward processing. Further examination of these reward signals within the visual system revealed information about the identity of the preceding cue in the caudate tail and LOC, and information about the location of the preceding cue in IPS, while early visual cortex represented both location and identity. The results reveal teaching signals within the value-driven attention network during associative reward learning, and further suggest functional specialization within different regions of this network during the acquisition of an integrated representation of stimulus value. © The Author (2016). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

  5. Extinction and renewal of cue-elicited reward-seeking.

    Science.gov (United States)

    Bezzina, Louise; Lee, Jessica C; Lovibond, Peter F; Colagiuri, Ben

    2016-12-01

    Reward cues can contribute to overconsumption of food and drugs and can relapse. The failure of exposure therapies to reduce overconsumption and relapse is generally attributed to the context-specificity of extinction. However, no previous study has examined whether cue-elicited reward-seeking (as opposed to cue-reactivity) is sensitive to context renewal. We tested this possibility in 160 healthy volunteers using a Pavlovian-instrumental transfer (PIT) design involving voluntary responding for a high value natural reward (chocolate). One reward cue underwent Pavlovian extinction in the same (Group AAA) or different context (Group ABA) to all other phases. This cue was compared with a second non-extinguished reward cue and an unpaired control cue. There was a significant overall PIT effect with both reward cues eliciting reward-seeking on test relative to the unpaired cue. Pavlovian extinction substantially reduced this effect, with the extinguished reward cue eliciting less reward-seeking than the non-extinguished reward cue. Most interestingly, extinction of cue-elicited reward-seeking was sensitive to renewal, with extinction less effective for reducing PIT when conducted in a different context. These findings have important implications for extinction-based interventions for reducing maladaptive reward-seeking in practice. Copyright © 2016 Elsevier Ltd. All rights reserved.

  6. Facilitation of voluntary goal-directed action by reward cues.

    Science.gov (United States)

    Lovibond, Peter F; Colagiuri, Ben

    2013-10-01

    Reward-associated cues are known to influence motivation to approach both natural and man-made rewards, such as food and drugs. However, the mechanisms underlying these effects are not well understood. To model these processes in the laboratory with humans, we developed an appetitive Pavlovian-instrumental transfer procedure with a chocolate reward. We used a single unconstrained response that led to an actual rather than symbolic reward to assess the strength of reward motivation. Presentation of a chocolate-paired cue, but not an unpaired cue, markedly enhanced instrumental responding over a 30-s period. The same pattern was observed with 10-s and 30-s cues, showing that close cue-reward contiguity is not necessary for facilitation of reward-directed action. The results confirm that reward-related cues can instigate voluntary action to obtain that reward. The effectiveness of long-duration cues suggests that in clinical settings, attention should be directed to both proximal and distal cues for reward.

  7. Multiple reward-cue contingencies favor expectancy over uncertainty in shaping the reward-cue attentional salience.

    Science.gov (United States)

    De Tommaso, Matteo; Mastropasqua, Tommaso; Turatto, Massimo

    2018-01-25

    Reward-predicting cues attract attention because of their motivational value. A debated question regards the conditions under which the cue's attentional salience is governed more by reward expectancy rather than by reward uncertainty. To help shedding light on this relevant issue, here, we manipulated expectancy and uncertainty using three levels of reward-cue contingency, so that, for example, a high level of reward expectancy (p = .8) was compared with the highest level of reward uncertainty (p = .5). In Experiment 1, the best reward-cue during conditioning was preferentially attended in a subsequent visual search task. This result was replicated in Experiment 2, in which the cues were matched in terms of response history. In Experiment 3, we implemented a hybrid procedure consisting of two phases: an omission contingency procedure during conditioning, followed by a visual search task as in the previous experiments. Crucially, during both phases, the reward-cues were never task relevant. Results confirmed that, when multiple reward-cue contingencies are explored by a human observer, expectancy is the major factor controlling both the attentional and the oculomotor salience of the reward-cue.

  8. Global cue inconsistency diminishes learning of cue validity

    Directory of Open Access Journals (Sweden)

    Tony Wang

    2016-11-01

    Full Text Available We present a novel two-stage probabilistic learning task that examines the participants’ ability to learn and utilize valid cues across several levels of probabilistic feedback. In the first stage, participants sample from one of three cues that gives predictive information about the outcome of the second stage. Participants are rewarded for correct prediction of the outcome in stage two. Only one of the three cues gives valid predictive information and thus participants can maximise their reward by learning to sample from the valid cue. The validity of this predictive information, however, is reinforced across several levels of probabilistic feedback. A second manipulation involved changing the consistency of the predictive information in stage one and the outcome in stage two. The results show that participants, with higher probabilistic feedback, learned to utilise the valid cue. In inconsistent task conditions, however, participants were significantly less successful in utilising higher validity cues. We interpret this result as implying that learning in probabilistic categorization is based on developing a representation of the task that allows for goal-directed action.

  9. Incidental rewarding cues influence economic decisions in people with obesity

    Science.gov (United States)

    Simmank, Jakob; Murawski, Carsten; Bode, Stefan; Horstmann, Annette

    2015-01-01

    Recent research suggests that obesity is linked to prominent alterations in learning and decision-making. This general difference may also underlie the preference for immediately consumable, highly palatable but unhealthy and high-calorie foods. Such poor food-related inter-temporal decision-making can explain weight gain; however, it is not yet clear whether this deficit can be generalized to other domains of inter-temporal decision-making, for example financial decisions. Further, little is known about the stability of decision-making behavior in obesity, especially in the presence of rewarding cues. To answer these questions, obese and lean participants (n = 52) completed two sessions of a novel priming paradigm including a computerized monetary delay discounting task. In the first session, general differences between groups in financial delay discounting were measured. In the second session, we tested the general stability of discount rates. Additionally, participants were primed by affective visual cues of different contextual categories before making financial decisions. We found that the obese group showed stronger discounting of future monetary rewards than the lean group, but groups did not differ in their general stability between sessions nor in their sensitivity toward changes in reward magnitude. In the obese group, a fast decrease of subjective value over time was directly related to a higher tendency for opportunistic eating. Obese in contrast to lean people were primed by the affective cues, showing a sex-specific pattern of priming direction. Our findings demonstrate that environments rich of cues, aiming at inducing unhealthy consumer decisions, can be highly detrimental for obese people. It also underscores that obesity is not merely a medical condition but has a strong cognitive component, meaning that current dietary and medical treatment strategies may fall too short. PMID:26528158

  10. Incidental rewarding cues influence economic decision-making in obesity

    Directory of Open Access Journals (Sweden)

    Jakob eSimmank

    2015-10-01

    Full Text Available Recent research suggests that obesity is linked to prominent alterations in learning and decision-making. This general difference may also underlie the preference for immediately consumable, highly palatable but unhealthy and high-calorie foods. Such poor food-related inter-temporal decision-making can explain weight gain; however, it is not yet clear whether this deficit can be generalized to other domains of inter-temporal decision-making, for example financial decisions. Further, little is known about the stability of decision-making behavior in obesity, especially in the presence of rewarding cues. To answer these questions, obese and lean participants (n=52 completed two sessions of a novel priming paradigm including a computerized monetary delay discounting task. In the first session, general differences between groups in financial delay discounting were measured. In the second session, we tested the general stability of discounting rates. Additionally, participants were primed by affective visual cues of different contextual categories before the financial decision. We found that the obese group showed stronger discounting of future monetary rewards than the lean group, but groups did not differ in their general stability between sessions nor in their sensitivity towards changes in reward magnitude. In the obese group, a fast decrease of subjective value over time was directly related to a higher tendency for opportunistic eating. Obese in contrast to lean people were primed by the affective cues, showing a sex-specific pattern of priming direction. Our findings demonstrate that environments rich of cues, aiming at inducing unhealthy consumer decisions, can be highly detrimental for obese people. It also underscores that obesity is not merely a medical condition but has a strong cognitive component, meaning that current dietary and medical treatment strategies may fall too short.

  11. Dorsolateral neostriatum contribution to incentive salience: Opioid or dopamine stimulation makes one reward cue more motivationally attractive than another

    OpenAIRE

    DiFeliceantonio, Alexandra G.; Berridge, Kent C.

    2016-01-01

    Pavlovian cues for rewards can become attractive incentives: approached and ‘wanted’ as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here we report that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individu...

  12. The habenula governs the attribution of incentive salience to reward predictive cues

    Science.gov (United States)

    Danna, Carey L.; Shepard, Paul D.; Elmer, Greg I.

    2013-01-01

    The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr), the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms. PMID:24368898

  13. The habenula governs the attribution of incentive salience to reward predictive cues.

    Directory of Open Access Journals (Sweden)

    Carey L. Danna

    2013-12-01

    Full Text Available The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb, a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr, the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms.

  14. On the motivational properties of reward cues: Individual differences.

    Science.gov (United States)

    Robinson, Terry E; Yager, Lindsay M; Cogan, Elizabeth S; Saunders, Benjamin T

    2014-01-01

    Cues associated with rewards, such as food or drugs of abuse, can themselves acquire motivational properties. Acting as incentive stimuli, such cues can exert powerful control over motivated behavior, and in the case of cues associated with drugs, they can goad continued drug-seeking behavior and relapse. However, recent studies reviewed here suggest that there are large individual differences in the extent to which food and drug cues are attributed with incentive salience. Rats prone to approach reward cues (sign-trackers) attribute greater motivational value to discrete localizable cues and interoceptive cues than do rats less prone to approach reward cues (goal-trackers). In contrast, contextual cues appear to exert greater control over motivated behavior in goal-trackers than sign-trackers. It is possible to predict, therefore, before any experience with drugs, in which animals specific classes of drug cues will most likely reinstate drug-seeking behavior. The finding that different individuals may be sensitive to different triggers capable of motivating behavior and producing relapse suggests there may be different pathways to addiction, and has implications for thinking about individualized treatment. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'. Copyright © 2013 Elsevier Ltd. All rights reserved.

  15. Consolidation power of extrinsic rewards: reward cues enhance long-term memory for irrelevant past events.

    Science.gov (United States)

    Murayama, Kou; Kitagami, Shinji

    2014-02-01

    Recent research suggests that extrinsic rewards promote memory consolidation through dopaminergic modulation processes. However, no conclusive behavioral evidence exists given that the influence of extrinsic reward on attention and motivation during encoding and consolidation processes are inherently confounded. The present study provides behavioral evidence that extrinsic rewards (i.e., monetary incentives) enhance human memory consolidation independently of attention and motivation. Participants saw neutral pictures, followed by a reward or control cue in an unrelated context. Our results (and a direct replication study) demonstrated that the reward cue predicted a retrograde enhancement of memory for the preceding neutral pictures. This retrograde effect was observed only after a delay, not immediately upon testing. An additional experiment showed that emotional arousal or unconscious resource mobilization cannot explain the retrograde enhancement effect. These results provide support for the notion that the dopaminergic memory consolidation effect can result from extrinsic reward.

  16. Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction.

    Science.gov (United States)

    Flagel, Shelly B; Akil, Huda; Robinson, Terry E

    2009-01-01

    Drugs of abuse acquire different degrees of control over thoughts and actions based not only on the effects of drugs themselves, but also on predispositions of the individual. Those individuals who become addicted are unable to shift their thoughts and actions away from drugs and drug-associated stimuli. Thus in addicts, exposure to places or things (cues) that has been previously associated with drug-taking often instigates renewed drug-taking. We and others have postulated that drug-associated cues acquire the ability to maintain and instigate drug-taking behavior in part because they acquire incentive motivational properties through Pavlovian (stimulus-stimulus) learning. In the case of compulsive behavioral disorders, including addiction, such cues may be attributed with pathological incentive value ("incentive salience"). For this reason, we have recently begun to explore individual differences in the tendency to attribute incentive salience to cues that predict rewards. When discrete cues are associated with the non-contingent delivery of food or drug rewards some animals come to quickly approach and engage the cue even if it is located at a distance from where the reward will be delivered. In these animals the reward-predictive cue itself becomes attractive, eliciting approach towards it, presumably because it is attributed with incentive salience. Animals that develop this type of conditional response are called "sign-trackers". Other animals, "goal-trackers", do not approach the reward-predictive cue, but upon cue presentation they immediately go to the location where food will be delivered (the "goal"). For goal-trackers the reward-predictive cue is not attractive, presumably because it is not attributed with incentive salience. We review here preliminary data suggesting that these individual differences in the tendency to attribute incentive salience to cues predictive of reward may confer vulnerability or resistance to compulsive behavioral disorders

  17. Individual differences in anticipatory activity to food rewards predict cue-induced appetitive 50-kHz calls in rats.

    Science.gov (United States)

    Brenes, Juan C; Schwarting, Rainer K W

    2015-10-01

    Reward-related stimuli come to acquire incentive salience through Pavlovian learning and become capable of controlling reward-oriented behaviors. Here, we examined individual differences in anticipatory activity elicited by reward-related cues as indicative of how animals attribute incentive salience to otherwise neutral stimuli. Since adult rats can signal incentive motivation states through ultrasonic vocalizations (USVs) at around 50-kHz, such calls were recorded in food-deprived rats trained to associate cues with food rewards, which were subsequently devalued by satiation.We found that the extent to which animals developed conditioned anticipatory activity to food cues while food deprived determined the level of cue-induced appetitive USVs while sated. Re-exposure to reward cues after a free-testing period reinstated USVs, invigorated reward seeking and consumption, and again, increases in calling occurred only in animals with high levels of cue-induced anticipatory activity. Reward-experienced rats systemically challenged with the catecholamine agonist amphetamine or with the dopamine receptor antagonist flupenthixol showed attenuated responses to these drugs, especially for USVs and in subjects with high levels of cue-induced anticipatory activity. Our results suggest that individuals prone to attribute incentive salience to reward cues showed heightened reward-induced USVs which were reliably expressed over time and persisted despite physiological needs being fulfilled. Also, prone subjects seemed to undergo particular adaptations in their dopaminergic system related with incentive learning. Our findings may have translational relevance in preclinical research modeling compulsive disorders, which may be due to excessive attribution of incentive salience to reward cues, such as overeating, pathological gambling, and drug addiction.

  18. Extinction Can Reduce the Impact of Reward Cues on Reward-Seeking Behavior.

    Science.gov (United States)

    Lovibond, Peter F; Satkunarajah, Michelle; Colagiuri, Ben

    2015-07-01

    Reward-associated cues are thought to promote relapse after treatment of appetitive disorders such as drug-taking, binge eating, and gambling. This process has been modelled in the laboratory using a Pavlovian-instrumental transfer (PIT) design in which Pavlovian cues facilitate instrumental reward-directed action. Attempts to reduce facilitation by cue exposure (extinction) have produced mixed results. We tested the effect of extinction in a recently developed PIT procedure using a natural reward, chocolate, in human participants. Facilitation of instrumental responding was only observed in participants who were aware of the Pavlovian contingencies. Pavlovian extinction successfully reduced, but did not completely eliminate, expectancy of reward and facilitation of instrumental responding. The results indicate that exposure can reduce the ability of cues to promote reward-directed behavior in the laboratory. However, the residual potency of extinguished cues means that additional active strategies may be needed in clinical practice to train patients to resist the impact of these cues in their environment. Copyright © 2015. Published by Elsevier Ltd.

  19. An Animal Model of Genetic Vulnerability to Behavioral Disinhibition and Responsiveness to Reward-Related Cues: Implications for Addiction

    OpenAIRE

    Flagel, Shelly B; Robinson, Terry E; Clark, Jeremy J; Clinton, Sarah M; Watson, Stanley J; Seeman, Phillip; Phillips, Paul E M; Akil, Huda

    2009-01-01

    Rats selectively bred based on high or low reactivity to a novel environment were characterized for other behavioral and neurobiological traits thought to be relevant to addiction vulnerability. The two lines of animals, which differ in their propensity to self-administer drugs, also differ in the value they attribute to cues associated with reward, in impulsive behavior, and in their dopamine system. When a cue was paired with food or cocaine reward bred high-responder rats (bHRs) learned to...

  20. Impaired associative learning with food rewards in obese women.

    Science.gov (United States)

    Zhang, Zhihao; Manson, Kirk F; Schiller, Daniela; Levy, Ifat

    2014-08-04

    Obesity is a major epidemic in many parts of the world. One of the main factors contributing to obesity is overconsumption of high-fat and high-calorie food, which is driven by the rewarding properties of these types of food. Previous studies have suggested that dysfunction in reward circuits may be associated with overeating and obesity. The nature of this dysfunction, however, is still unknown. Here, we demonstrate impairment in reward-based associative learning specific to food in obese women. Normal-weight and obese participants performed an appetitive reversal learning task in which they had to learn and modify cue-reward associations. To test whether any learning deficits were specific to food reward or were more general, we used a between-subject design in which half of the participants received food reward and the other half received money reward. Our results reveal a marked difference in associative learning between normal-weight and obese women when food was used as reward. Importantly, no learning deficits were observed with money reward. Multiple regression analyses also established a robust negative association between body mass index and learning performance in the food domain in female participants. Interestingly, such impairment was not observed in obese men. These findings suggest that obesity may be linked to impaired reward-based associative learning and that this impairment may be specific to the food domain. Copyright © 2014 Elsevier Ltd. All rights reserved.

  1. Dorsolateral neostriatum contribution to incentive salience: opioid or dopamine stimulation makes one reward cue more motivationally attractive than another.

    Science.gov (United States)

    DiFeliceantonio, Alexandra G; Berridge, Kent C

    2016-05-01

    Pavlovian cues for rewards can become attractive incentives: approached and 'wanted' as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here it was reported that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individualized fashion, at the expense of a competing cue. In an autoshaping (sign-tracking vs. goal-tracking) paradigm, microinjection of the mu opioid receptor agonist (DAMGO) or dopamine indirect agonist (amphetamine) in the DLS of sign-tracker individuals selectively enhanced their sign-tracking attraction toward the reward-predictive lever cue. By contrast, DAMGO or amphetamine in the DLS of goal-trackers selectively enhanced prepotent attraction toward the reward-proximal cue of sucrose dish. Amphetamine also enhanced goal-tracking in some sign-tracker individuals (if they ever defected to the dish even once). That DLS enhancement of cue attraction was due to stronger motivation, not stronger habits, was suggested by: (i) sign-trackers flexibly followed their cue to a new location when the lever was suddenly moved after DLS DAMGO microinjection; and (ii) DAMGO in the DLS also made sign-trackers work harder on a new instrumental nose-poke response required to earn presentations of their Pavlovian lever cue (instrumental conditioned reinforcement). Altogether, the current results suggest that DLS circuitry can enhance the incentive salience of a Pavlovian reward cue, selectively making that cue a stronger motivational magnet. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.

  2. Improved memory for reward cues following acute buprenorphine administration in humans

    NARCIS (Netherlands)

    Syal, Supriya; Ipser, Jonathan; Terburg, David|info:eu-repo/dai/nl/32304087X; Solms, Mark; Panksepp, Jaak; Malcolm-Smith, Susan; Bos, Peter A.|info:eu-repo/dai/nl/337018995; Montoya, Estrella R.|info:eu-repo/dai/nl/34141347X; Stein, Dan J.; van Honk, Jack|info:eu-repo/dai/nl/188602801

    2015-01-01

    In rodents, there is abundant evidence for the involvement of the opioid system in the processing of reward cues, but this system has remained understudied in humans. In humans, the happy facial expression is a pivotal reward cue. Happy facial expressions activate the brain's reward system and are

  3. Integration of reward signalling and appetite regulating peptide systems in the control of food-cue responses.

    Science.gov (United States)

    Reichelt, A C; Westbrook, R F; Morris, M J

    2015-11-01

    Understanding the neurobiological substrates that encode learning about food-associated cues and how those signals are modulated is of great clinical importance especially in light of the worldwide obesity problem. Inappropriate or maladaptive responses to food-associated cues can promote over-consumption, leading to excessive energy intake and weight gain. Chronic exposure to foods rich in fat and sugar alters the reinforcing value of foods and weakens inhibitory neural control, triggering learned, but maladaptive, associations between environmental cues and food rewards. Thus, responses to food-associated cues can promote cravings and food-seeking by activating mesocorticolimbic dopamine neurocircuitry, and exert physiological effects including salivation. These responses may be analogous to the cravings experienced by abstaining drug addicts that can trigger relapse into drug self-administration. Preventing cue-triggered eating may therefore reduce the over-consumption seen in obesity and binge-eating disorder. In this review we discuss recent research examining how cues associated with palatable foods can promote reward-based feeding behaviours and the potential involvement of appetite-regulating peptides including leptin, ghrelin, orexin and melanin concentrating hormone. These peptide signals interface with mesolimbic dopaminergic regions including the ventral tegmental area to modulate reactivity to cues associated with palatable foods. Thus, a novel target for anti-obesity therapeutics is to reduce non-homeostatic, reward driven eating behaviour, which can be triggered by environmental cues associated with highly palatable, fat and sugar rich foods. © 2015 The British Pharmacological Society.

  4. Belief reward shaping in reinforcement learning

    CSIR Research Space (South Africa)

    Marom, O

    2018-02-01

    Full Text Available A key challenge in many reinforcement learning problems is delayed rewards, which can significantly slow down learning. Although reward shaping has previously been introduced to accelerate learning by bootstrapping an agent with additional...

  5. Operant conditioning of rat navigation using electrical stimulation for directional cues and rewards.

    Science.gov (United States)

    Lee, Maan-Gee; Jun, Gayoung; Choi, Hyo-Soon; Jang, Hwan Soo; Bae, Yong Chul; Suk, Kyoungho; Jang, Il-Sung; Choi, Byung-Ju

    2010-07-01

    Operant conditioning is often used to train a desired behavior in an animal. The contingency between a specific behavior and a reward is required for successful training. Here, we compared the effectiveness of two different mazes for training turning behaviors in response to directional cues in Sprague-Dawley rats. Forty-three rats were implanted with electrodes into the medial forebrain bundle and the left and right somatosensory cortices for reward and cues. Among them, thirteen rats discriminated between the left and right somatosensory stimulations to obtain rewards. They were trained to learn ipsilateral turning response to the stimulation of the left or right somatosensory cortex in either the T-maze (Group T) or the E| maze (Group W). Performance was measured by the navigation speed in the mazes. Performances of rats in Group T were enhanced faster than those in Group W. A significant correlation between performances during training and performance in final testing was observed in Group T starting with the fifth training session while such a correlation was not observed in Group W until the tenth training session. The training mazes did not however affect the performances in the final test. These results suggest that a simple maze is better than a complicated maze for training animals to learn directions and direct cortical stimulation can be used as a cue for direction training. Copyright (c) 2010 Elsevier B.V. All rights reserved.

  6. Which Cue to ‘Want?’ Central Amygdala Opioid Activation Enhances and Focuses Incentive Salience on a Prepotent Reward Cue

    Science.gov (United States)

    Mahler, Stephen V.; Berridge, Kent C.

    2009-01-01

    The central nucleus of the amygdala (CeA) helps translate learning into motivation, and here we show that opioid stimulation of CeA magnifies and focuses learned incentive salience onto a specific reward cue (Pavlovian conditioned stimulus, or CS). This motivation enhancement makes that cue more attractive, noticeable, and liable to elicit appetitive and consummatory behaviors. To reveal the focusing of incentive salience, we exploited individual differences in an autoshaping paradigm in which a rat prefers to approach, nibble and sniff one of two reward-associated stimuli (its prepotent stimulus). The individually-prepotent cue is either a predictive CS+ that signals reward (8sec metal lever insertion), or instead the metal cup that delivers sucrose pellets (the reward source). Results indicated that CeA opioid activation by microinjection of the μ agonist DAMGO (0.1μg) selectively and reversibly enhanced the attractiveness of whichever reward CS was that rat's prepotent cue. CeA DAMGO microinjections made rats more vigorously approach their particular prepotent CS, and to energetically sniff and nibble it in a nearly frenzied consummatory fashion. Only the prepotent cue was enhanced as an incentive target, and alternative cues were not enhanced. Conversely, inactivation of CeA by muscimol microinjection (0.25μg) suppressed approach, nibbles and sniffs of the prepotent CS. Confirming modulation of incentive salience, unconditioned food intake was similarly increased by DAMGO microinjection and decreased by muscimol in CeA. We conclude that opioid neurotransmission in central amygdala helps determine which environmental stimuli become most ‘wanted,’ and how ‘wanted’ they become. This may powerfully guide reward-seeking behavior. PMID:19458221

  7. Visual sexual stimuli – cue or reward? A key for interpreting brain imaging studies on human sexual behaviors

    Directory of Open Access Journals (Sweden)

    Mateusz Gola

    2016-08-01

    Full Text Available There is an increasing number of neuroimaging studies using visual sexual stimuli (VSS for human sexuality studies, including emerging field of research on compulsive sexual behaviors. A central question in this field is whether behaviors such as extensive pornography consumption share common brain mechanisms with widely studied substance and behavioral addictions. Depending on how VSS are conceptualized, different predictions can be formulated within the frameworks of Reinforcement Learning or Incentive Salience Theory, where a crucial distinction is made between conditioned (cue and unconditioned (reward stimuli (related to reward anticipation vs reward consumption, respectively. Surveying 40 recent human neuroimaging studies we show existing ambiguity about the conceptualization of VSS. Therefore, we feel that it is important to address the question of whether VSS should be considered as cues (conditioned stimuli or rewards (unconditioned stimuli. Here we present our own perspective, which is that in most laboratory settings VSS play a role of reward (unconditioned stimuli, as evidenced by: 1. experience of pleasure while watching VSS, possibly accompanied by genital reaction 2. reward-related brain activity correlated with these pleasurable feelings in response to VSS, 3. a willingness to exert effort to view VSS similarly as for other rewarding stimuli such as money, and/or 4. conditioning for cues (CS predictive for. We hope that this perspective paper will initiate a scientific discussion on this important and overlooked topic and increase attention for appropriate interpretations of results of human neuroimaging studies using VSS.

  8. Cortical Brain Activity Reflecting Attentional Biasing Toward Reward-Predicting Cues Covaries with Economic Decision-Making Performance.

    Science.gov (United States)

    San Martín, René; Appelbaum, Lawrence G; Huettel, Scott A; Woldorff, Marty G

    2016-01-01

    Adaptive choice behavior depends critically on identifying and learning from outcome-predicting cues. We hypothesized that attention may be preferentially directed toward certain outcome-predicting cues. We studied this possibility by analyzing event-related potential (ERP) responses in humans during a probabilistic decision-making task. Participants viewed pairs of outcome-predicting visual cues and then chose to wager either a small (i.e., loss-minimizing) or large (i.e., gain-maximizing) amount of money. The cues were bilaterally presented, which allowed us to extract the relative neural responses to each cue by using a contralateral-versus-ipsilateral ERP contrast. We found an early lateralized ERP response, whose features matched the attention-shift-related N2pc component and whose amplitude scaled with the learned reward-predicting value of the cues as predicted by an attention-for-reward model. Consistently, we found a double dissociation involving the N2pc. Across participants, gain-maximization positively correlated with the N2pc amplitude to the most reliable gain-predicting cue, suggesting an attentional bias toward such cues. Conversely, loss-minimization was negatively correlated with the N2pc amplitude to the most reliable loss-predicting cue, suggesting an attentional avoidance toward such stimuli. These results indicate that learned stimulus-reward associations can influence rapid attention allocation, and that differences in this process are associated with individual differences in economic decision-making performance. © The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

  9. Reward prediction error signal enhanced by striatum-amygdala interaction explains the acceleration of probabilistic reward learning by emotion.

    Science.gov (United States)

    Watanabe, Noriya; Sakagami, Masamichi; Haruno, Masahiko

    2013-03-06

    Learning does not only depend on rationality, because real-life learning cannot be isolated from emotion or social factors. Therefore, it is intriguing to determine how emotion changes learning, and to identify which neural substrates underlie this interaction. Here, we show that the task-independent presentation of an emotional face before a reward-predicting cue increases the speed of cue-reward association learning in human subjects compared with trials in which a neutral face is presented. This phenomenon was attributable to an increase in the learning rate, which regulates reward prediction errors. Parallel to these behavioral findings, functional magnetic resonance imaging demonstrated that presentation of an emotional face enhanced reward prediction error (RPE) signal in the ventral striatum. In addition, we also found a functional link between this enhanced RPE signal and increased activity in the amygdala following presentation of an emotional face. Thus, this study revealed an acceleration of cue-reward association learning by emotion, and underscored a role of striatum-amygdala interactions in the modulation of the reward prediction errors by emotion.

  10. Motivational state, reward value, and Pavlovian cues differentially affect skilled forelimb grasping in rats

    Science.gov (United States)

    de Clauser, Larissa; Kasper, Hansjörg; Schwab, Martin E.

    2016-01-01

    Motor skills represent high-precision movements performed at optimal speed and accuracy. Such motor skills are learned with practice over time. Besides practice, effects of motivation have also been shown to influence speed and accuracy of movements, suggesting that fast movements are performed to maximize gained reward over time as noted in previous studies. In rodents, skilled motor performance has been successfully modeled with the skilled grasping task, in which animals use their forepaw to grasp for sugar pellet rewards through a narrow window. Using sugar pellets, the skilled grasping task is inherently tied to motivation processes. In the present study, we performed three experiments modulating animals’ motivation during skilled grasping by changing the motivational state, presenting different reward value ratios, and displaying Pavlovian stimuli. We found in all three studies that motivation affected the speed of skilled grasping movements, with the strongest effects seen due to motivational state and reward value. Furthermore, accuracy of the movement, measured in success rate, showed a strong dependence on motivational state as well. Pavlovian cues had only minor effects on skilled grasping, but results indicate an inverse Pavlovian-instrumental transfer effect on movement speed. These findings have broad implications considering the increasing use of skilled grasping in studies of motor system structure, function, and recovery after injuries. PMID:27194796

  11. Quantifying individual variation in the propensity to attribute incentive salience to reward cues.

    Directory of Open Access Journals (Sweden)

    Paul J Meyer

    Full Text Available If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience" to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878 subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience, to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers" vs. others that approached the location of reward delivery ("goal-trackers". This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.

  12. Quantifying individual variation in the propensity to attribute incentive salience to reward cues.

    Science.gov (United States)

    Meyer, Paul J; Lovic, Vedran; Saunders, Benjamin T; Yager, Lindsay M; Flagel, Shelly B; Morrow, Jonathan D; Robinson, Terry E

    2012-01-01

    If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.

  13. Behavioral stress may increase the rewarding valence of cocaine-associated cues through a dynorphin/kappa-opioid receptor-mediated mechanism without affecting associative learning or memory retrieval mechanisms.

    Science.gov (United States)

    Schindler, Abigail G; Li, Shuang; Chavkin, Charles

    2010-08-01

    Stress exposure increases the risk of addictive drug use in human and animal models of drug addiction by mechanisms that are not completely understood. Mice subjected to repeated forced swim stress (FSS) before cocaine develop significantly greater conditioned place preference (CPP) for the drug-paired chamber than unstressed mice. Analysis of the dose dependency showed that FSS increased both the maximal CPP response and sensitivity to cocaine. To determine whether FSS potentiated CPP by enhancing associative learning mechanisms, mice were conditioned with cocaine in the absence of stress, then challenged after association was complete with the kappa-opioid receptor (KOR) agonist U50,488 or repeated FSS, before preference testing. Mice challenged with U50,488 60 min before CPP preference testing expressed significantly greater cocaine-CPP than saline-challenged mice. Potentiation by U50,488 was dose and time dependent and blocked by the KOR antagonist norbinaltorphimine (norBNI). Similarly, mice subjected to repeated FSS before the final preference test expressed significantly greater cocaine-CPP than unstressed controls, and FSS-induced potentiation was blocked by norBNI. Novel object recognition (NOR) performance was not affected by U50,488 given 60 min before assay, but was impaired when given 15 min before NOR assay, suggesting that KOR activation did not potentiate CPP by facilitating memory retrieval or expression. The results from this study show that the potentiation of cocaine-CPP by KOR activation does not result from an enhancement of associative learning mechanisms and that stress may instead enhance the rewarding valence of cocaine-associated cues by a dynorphin-dependent mechanism.

  14. Visual Sexual Stimuli-Cue or Reward? A Perspective for Interpreting Brain Imaging Findings on Human Sexual Behaviors.

    Science.gov (United States)

    Gola, Mateusz; Wordecha, Małgorzata; Marchewka, Artur; Sescousse, Guillaume

    2016-01-01

    There is an increasing number of neuroimaging studies using visual sexual stimuli (VSS), especially within the emerging field of research on compulsive sexual behaviors (CSB). A central question in this field is whether behaviors such as excessive pornography consumption share common brain mechanisms with widely studied substance and behavioral addictions. Depending on how VSS are conceptualized, different predictions can be formulated within the frameworks of Reinforcement Learning or Incentive Salience Theory, where a crucial distinction is made between conditioned and unconditioned stimuli (related to reward anticipation vs. reward consumption, respectively). Surveying 40 recent human neuroimaging studies we show existing ambiguity about the conceptualization of VSS. Therefore, we feel that it is important to address the question of whether VSS should be considered as conditioned stimuli (cue) or unconditioned stimuli (reward). Here we present our own perspective, which is that in most laboratory settings VSS play a role of reward, as evidenced by: (1) experience of pleasure while watching VSS, possibly accompanied by genital reaction; (2) reward-related brain activity correlated with these pleasurable feelings in response to VSS; (3) a willingness to exert effort to view VSS similarly as for other rewarding stimuli such as money; and (4) conditioning for cues predictive of VSS. We hope that this perspective article will initiate a scientific discussion on this important and overlooked topic and increase attention for appropriate interpretations of results of human neuroimaging studies using VSS.

  15. Visual Sexual Stimuli—Cue or Reward? A Perspective for Interpreting Brain Imaging Findings on Human Sexual Behaviors

    Science.gov (United States)

    Gola, Mateusz; Wordecha, Małgorzata; Marchewka, Artur; Sescousse, Guillaume

    2016-01-01

    There is an increasing number of neuroimaging studies using visual sexual stimuli (VSS), especially within the emerging field of research on compulsive sexual behaviors (CSB). A central question in this field is whether behaviors such as excessive pornography consumption share common brain mechanisms with widely studied substance and behavioral addictions. Depending on how VSS are conceptualized, different predictions can be formulated within the frameworks of Reinforcement Learning or Incentive Salience Theory, where a crucial distinction is made between conditioned and unconditioned stimuli (related to reward anticipation vs. reward consumption, respectively). Surveying 40 recent human neuroimaging studies we show existing ambiguity about the conceptualization of VSS. Therefore, we feel that it is important to address the question of whether VSS should be considered as conditioned stimuli (cue) or unconditioned stimuli (reward). Here we present our own perspective, which is that in most laboratory settings VSS play a role of reward, as evidenced by: (1) experience of pleasure while watching VSS, possibly accompanied by genital reaction; (2) reward-related brain activity correlated with these pleasurable feelings in response to VSS; (3) a willingness to exert effort to view VSS similarly as for other rewarding stimuli such as money; and (4) conditioning for cues predictive of VSS. We hope that this perspective article will initiate a scientific discussion on this important and overlooked topic and increase attention for appropriate interpretations of results of human neuroimaging studies using VSS. PMID:27574507

  16. Learning Reward Uncertainty in the Basal Ganglia.

    Directory of Open Access Journals (Sweden)

    John G Mikhael

    2016-09-01

    Full Text Available Learning the reliability of different sources of rewards is critical for making optimal choices. However, despite the existence of detailed theory describing how the expected reward is learned in the basal ganglia, it is not known how reward uncertainty is estimated in these circuits. This paper presents a class of models that encode both the mean reward and the spread of the rewards, the former in the difference between the synaptic weights of D1 and D2 neurons, and the latter in their sum. In the models, the tendency to seek (or avoid options with variable reward can be controlled by increasing (or decreasing the tonic level of dopamine. The models are consistent with the physiology of and synaptic plasticity in the basal ganglia, they explain the effects of dopaminergic manipulations on choices involving risks, and they make multiple experimental predictions.

  17. Do cattle (Bos taurus) retain an association of a visual cue with a food reward for a year?

    Science.gov (United States)

    Hirata, Masahiko; Takeno, Nozomi

    2014-06-01

    Use of visual cues to locate specific food resources from a distance is a critical ability of animals foraging in a spatially heterogeneous environment. However, relatively little is known about how long animals can retain the learned cue-reward association without reinforcement. We compared feeding behavior of experienced and naive Japanese Black cows (Bos taurus) in discovering food locations in a pasture. Experienced animals had been trained to respond to a visual cue (plastic washtub) for a preferred food (grain-based concentrate) 1 year prior to the experiment, while naive animals had no exposure to the cue. Cows were tested individually in a test arena including tubs filled with the concentrate on three successive days (Days 1-3). Experienced cows located the first tub more quickly and visited more tubs than naive cows on Day 1 (usually P visual cue with a food reward within a day and retain the association for 1 year despite a slight decay. © 2014 Japanese Society of Animal Science.

  18. Dopamine receptor blockade attenuates the general incentive motivational effects of noncontingently delivered rewards and reward-paired cues without affecting their ability to bias action selection.

    Science.gov (United States)

    Ostlund, Sean B; Maidment, Nigel T

    2012-01-01

    Environmental cues affect our behavior in a variety of ways. Despite playing an invaluable role in guiding our daily activities, such cues also appear to trigger the harmful, compulsive behaviors that characterize addiction and other disorders of behavioral control. In instrumental conditioning, rewards and reward-paired cues bias action selection and invigorate reward-seeking behaviors, and appear to do so through distinct neurobehavioral processes. Although reward-paired cues are known to invigorate performance through a dopamine-dependent incentive motivational process, it is not known if dopamine also mediates the influence of rewards and reward-paired cues over action selection. The current study contrasted the effects of systemic administration of the nonspecific dopamine receptor antagonist flupentixol on response invigoration and action bias in Pavlovian-instrumental transfer, a test of cue-elicited responding, and in instrumental reinstatement, a test of noncontingent reward-elicited responding. Hungry rats were trained on two different stimulus-outcome relationships (eg, tone-grain pellets and noise-sucrose solution) and two different action-outcome relationships (eg, left press-grain and right press-sucrose). At test, we found that flupentixol pretreatment blocked the response invigoration generated by the cues but spared their ability to bias action selection to favor the action whose outcome was signaled by the cue being presented. The response-biasing influence of noncontingent reward deliveries was also unaffected by flupentixol. Interestingly, although flupentixol had a modest effect on the immediate response invigoration produced by those rewards, it was particularly potent in countering the lingering enhancement of responding produced by multiple reward deliveries. These findings indicate that dopamine mediates the general incentive motivational effects of noncontingent rewards and reward-paired cues but does not support their ability to bias

  19. Learned reward association improves visual working memory.

    Science.gov (United States)

    Gong, Mengyuan; Li, Sheng

    2014-04-01

    Statistical regularities in the natural environment play a central role in adaptive behavior. Among other regularities, reward association is potentially the most prominent factor that influences our daily life. Recent studies have suggested that pre-established reward association yields strong influence on the spatial allocation of attention. Here we show that reward association can also improve visual working memory (VWM) performance when the reward-associated feature is task-irrelevant. We established the reward association during a visual search training session, and investigated the representation of reward-associated features in VWM by the application of a change detection task before and after the training. The results showed that the improvement in VWM was significantly greater for items in the color associated with high reward than for those in low reward-associated or nonrewarded colors. In particular, the results from control experiments demonstrate that the observed reward effect in VWM could not be sufficiently accounted for by attentional capture toward the high reward-associated item. This was further confirmed when the effect of attentional capture was minimized by presenting the items in the sample and test displays of the change detection task with the same color. The results showed significantly larger improvement in VWM performance when the items in a display were in the high reward-associated color than those in the low reward-associated or nonrewarded colors. Our findings suggest that, apart from inducing space-based attentional capture, the learned reward association could also facilitate the perceptual representation of high reward-associated items through feature-based attentional modulation.

  20. Female hummingbirds do not relocate rewards using colour cues

    OpenAIRE

    Tello Ramos, Maria Cristina; Hurly, T. Andrew; Healy, Susan D.

    2014-01-01

    This research was supported by CONACYT (The Mexican National Council for Science and Technology) grant number: 310717, the University of Lethbridge and the Natural Sciences and Engineering Research Council of Canada (grant number: RGPIN 121496-2003) and the University of St Andrew's Russell Trust Award. Males generally outperform females in spatial tasks. This difference in spatial performance may reflect differences in cue preference because males often use both spatial cues 9distance and...

  1. The habenula governs the attribution of incentive salience to reward predictive cues

    OpenAIRE

    Danna, Carey L.; Shepard, Paul D.; Elmer, Greg I.

    2013-01-01

    The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral...

  2. Dopamine, reward learning, and active inference

    Directory of Open Access Journals (Sweden)

    Thomas eFitzgerald

    2015-11-01

    Full Text Available Temporal difference learning models propose phasic dopamine signalling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behaviour. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

  3. Dopamine, reward learning, and active inference.

    Science.gov (United States)

    FitzGerald, Thomas H B; Dolan, Raymond J; Friston, Karl

    2015-01-01

    Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

  4. Reward dependence moderates smoking-cue- and stress-induced cigarette cravings.

    Science.gov (United States)

    Michalowski, Alexandra; Erblich, Joel

    2014-12-01

    Cigarette cravings following exposure to smoking cues in a smoker's environment are thought to play an important role in cessation failure. The possibility that dispositional factors may impact cue-induced cravings, though intriguing, has received little attention. According to Cloninger's Tridimensional Personality Theory, factors such as reward dependence (RD), harm avoidance (HA), and novelty seeking (NS) may figure prominently in risk for addiction, as well as relapse, in individuals attempting to abstain from drug and alcohol use. Particularly interesting in this regard is the possibility that smokers with higher levels of RD, who are especially sensitive to reward signals, will have heightened craving reactions to smoking cues. To that end, non-treatment-seeking nicotine dependent smokers (n=96, mean age=41.1, 47% African American, 17% Caucasian, 22% Hispanic, 19.3cigs/day, FTND=7.5) underwent a classic experimental cue-induction, during which they were exposed to imagery of: (1) smoking, (2) neutral, and (3) stress cues, and reported their cigarette cravings (0-100) before and after each exposure. Participants also completed the Tridimensional Personality Questionnaire. Not surprisingly, smoking and stress cues (but not neutral cues) elicited significant elevations in craving (p'scues (pcues (pcues. Furthermore, the similar effects of RD on stress-induced craving suggest that both cue-and stress-induced cravings may be influenced by a common underlying disposition. Copyright © 2014 Elsevier Ltd. All rights reserved.

  5. Prosocial reward learning in children and adolescents

    Directory of Open Access Journals (Sweden)

    Youngbin Kwak

    2016-10-01

    Full Text Available Adolescence is a period of increased sensitivity to social contexts. To evaluate how social context sensitivity changes over development – and influences reward learning – we investigated how children and adolescents perceive and integrate rewards for oneself and others during a dynamic risky-decision-making task. Children and adolescents (N=75, 8-16 yrs performed the Social Gambling Task (SGT, (Kwak et al., 2014 and completed a set of questionnaires measuring other-regarding behavior. In the SGT, participants choose amongst four card decks that have different payout structures for oneself and for a charity. We examined patterns of choices, overall decision strategies, and how reward outcomes led to trial-by-trial adjustments in behavior, as estimated using a reinforcement-learning model. Performance of children and adolescents was compared to data from a previously collected sample of adults (N=102 performing the identical task. We found that that children/adolescents were not only more sensitive to rewards directed to the charity than self but also showed greater prosocial tendencies on independent measures of other-regarding behavior. Children and adolescents also showed less use of a strategy that prioritizes rewards for self at the expense of rewards for others. These results support the conclusion that, compared to adults, children and adolescents show greater sensitivity to outcomes for others when making decisions and learning about potential rewards.

  6. Environmental manipulations alter age differences in attribution of incentive salience to reward-paired cues.

    Science.gov (United States)

    Anderson, Rachel I; Bush, Peter C; Spear, Linda P

    2013-11-15

    Cues repeatedly paired with rewards often themselves become imbued with enhanced motivational value, or incentive salience. During Pavlovian conditioned approach procedures, a cue repeatedly preceding reward delivery often elicits conditioned responses at either the reward delivery location ("goal-tracking") or the cue itself ("sign-tracking"). Sign-tracking behavior is thought to reflect the individual differences in attribution of incentive salience to reward-paired cues that may contribute to addiction vulnerability. Adolescent rats typically demonstrate less sign-tracking behavior than adult rats, a surprising finding given that adolescence is hypothesized to be a time of heightened addiction vulnerability. Given evidence that adult sign-tracking behavior can be influenced by environmental conditions, the present study compared the effects of isolate housing and food deprivation on expression of sign-tacking and goal-tracking behavior in adolescent and adult male rats across eight days of a Pavlovian conditioned approach procedure. Pair-housed adults exhibited more sign-tracking behavior than pair-housed adolescents; however, this age difference was not apparent in isolate-housed subjects. Adolescents often appeared more sensitive than adults to both food restriction- and isolate housing-induced changes in behavior, with food restriction promoting an increase in sign-tracking among isolate-housed adolescents and an increase in goal-tracking among pair-housed adolescents. For adults, food restriction resulted in a modest increase in overall expression of both sign- and goal-tracking behavior. To the extent that sign-tracking behavior reflects attribution of incentive salience to reward-paired cues, results from the present study provide evidence that reactivity to rewards during adolescence is strongly related to the nature of the surrounding environment. Copyright © 2013 Elsevier B.V. All rights reserved.

  7. Amygdala mu-opioid receptors mediate the motivating influence of cue-triggered reward expectations.

    Science.gov (United States)

    Lichtenberg, Nina T; Wassum, Kate M

    2017-02-01

    Environmental reward-predictive stimuli can retrieve from memory a specific reward expectation that allows them to motivate action and guide choice. This process requires the basolateral amygdala (BLA), but little is known about the signaling systems necessary within this structure. Here we examined the role of the neuromodulatory opioid receptor system in the BLA in such cue-directed action using the outcome-specific Pavlovian-to-instrumental transfer (PIT) test in rats. Inactivation of BLA mu-, but not delta-opioid receptors was found to dose-dependently attenuate the ability of a reward-predictive cue to selectively invigorate the performance of actions directed at the same unique predicted reward (i.e. to express outcome-specific PIT). BLA mu-opioid receptor inactivation did not affect the ability of a reward itself to similarly motivate action (outcome-specific reinstatement), suggesting a more selective role for the BLA mu-opioid receptor in the motivating influence of currently unobservable rewarding events. These data reveal a new role for BLA mu-opioid receptor activation in the cued recall of precise reward memories and the use of this information to motivate specific action plans. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.

  8. Reward-based spatial learning in unmedicated adults with obsessive-compulsive disorder.

    Science.gov (United States)

    Marsh, Rachel; Tau, Gregory Z; Wang, Zhishun; Huo, Yuankai; Liu, Ge; Hao, Xuejun; Packard, Mark G; Peterson, Bradley S; Simpson, H Blair

    2015-04-01

    The authors assessed the functioning of mesolimbic and striatal areas involved in reward-based spatial learning in unmedicated adults with obsessive-compulsive disorder (OCD). Functional MRI blood-oxygen-level-dependent response was compared in 33 unmedicated adults with OCD and 33 healthy, age-matched comparison subjects during a reward-based learning task that required learning to use extramaze cues to navigate a virtual eight-arm radial maze to find hidden rewards. The groups were compared in their patterns of brain activation associated with reward-based spatial learning versus a control condition in which rewards were unexpected because they were allotted pseudorandomly to experimentally prevent learning. Both groups learned to navigate the maze to find hidden rewards, but group differences in neural activity during navigation and reward processing were detected in mesolimbic and striatal areas. During navigation, the OCD group, unlike the healthy comparison group, exhibited activation in the left posterior hippocampus. Unlike healthy subjects, participants in the OCD group did not show activation in the left ventral putamen and amygdala when anticipating rewards or in the left hippocampus, amygdala, and ventral putamen when receiving unexpected rewards (control condition). Signal in these regions decreased relative to baseline during unexpected reward receipt among those in the OCD group, and the degree of activation was inversely associated with doubt/checking symptoms. Participants in the OCD group displayed abnormal recruitment of mesolimbic and ventral striatal circuitry during reward-based spatial learning. Whereas healthy comparison subjects exhibited activation in this circuitry in response to the violation of reward expectations, unmedicated OCD participants did not and instead over-relied on the posterior hippocampus during learning. Thus, dopaminergic innervation of reward circuitry may be altered, and future study of anterior/posterior hippocampal

  9. The Role of Extrinsic Rewards and Cue-Intention Association in Prospective Memory in Young Children

    NARCIS (Netherlands)

    Sheppard, D.P.; Kretschmer, A.; Knispel, E.; Vollert, B.; Altgassen, A.M.

    2015-01-01

    The current study examined, for the first time, the effect of cue-intention association, as well as the effects of promised extrinsic rewards, on prospective memory in young children, aged 5-years-old (n = 39) and 7-years-old (n = 40). Children were asked to name pictures for a toy mole, whilst also

  10. Improved memory for reward cues following acute buprenorphine administration in humans.

    Science.gov (United States)

    Syal, Supriya; Ipser, Jonathan; Terburg, David; Solms, Mark; Panksepp, Jaak; Malcolm-Smith, Susan; Bos, Peter A; Montoya, Estrella R; Stein, Dan J; van Honk, Jack

    2015-03-01

    In rodents, there is abundant evidence for the involvement of the opioid system in the processing of reward cues, but this system has remained understudied in humans. In humans, the happy facial expression is a pivotal reward cue. Happy facial expressions activate the brain's reward system and are disregarded by subjects scoring high on depressive mood who are low in reward drive. We investigated whether a single 0.2mg administration of the mixed mu-opioid agonist/kappa-antagonist, buprenorphine, would influence short-term memory for happy, angry or fearful expressions relative to neutral faces. Healthy human subjects (n38) participated in a randomized placebo-controlled within-subject design, and performed an emotional face relocation task after administration of buprenorphine and placebo. We show that, compared to placebo, buprenorphine administration results in a significant improvement of memory for happy faces. Our data demonstrate that acute manipulation of the opioid system by buprenorphine increases short-term memory for social reward cues. Copyright © 2015. Published by Elsevier Ltd.

  11. How Performance-Contingent Reward Prospect Modulates Cognitive Control: Increased Cue Maintenance at the Cost of Decreased Flexibility

    Science.gov (United States)

    Hefer, Carmen; Dreisbach, Gesine

    2017-01-01

    Growing evidence suggests that reward prospect promotes cognitive stability in terms of increased context or cue maintenance. In 3 Experiments, using different versions of the AX-continuous performance task, we investigated whether this reward effect comes at the cost of decreased cognitive flexibility. Experiment 1 shows that the reward induced…

  12. Signed reward prediction errors drive declarative learning

    NARCIS (Netherlands)

    De Loof, E.; Ergo, K.; Naert, L.; Janssens, C.; Talsma, D.; van Opstal, F.; Verguts, T.

    2018-01-01

    Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning–a quintessentially human form of learning–remains surprisingly absent. We

  13. Colour cues facilitate learning flower refill schedules in wild hummingbirds.

    Science.gov (United States)

    Samuels, Michael; Hurly, T Andrew; Healy, Susan D

    2014-11-01

    Free-living hummingbirds can learn the refill schedules of individual experimental flowers but little is known about what information they use to do this. Colour cues, in particular, may be important to hummingbirds when learning about rewarded flower properties. We investigated, therefore, whether colour cues facilitated the learning of flower refill schedules in wild, free-living rufous hummingbirds (Selasphorus rufus). In the Cued condition, we presented birds with an array of six flowers, three of one colour, each of which were refilled 10min after being emptied by the bird and three of a different colour, which were refilled 20min after being emptied. In the Uncued condition we presented birds with six flowers of the same colour, three of which were refilled after 10min and three of which were refilled after 20min as for the birds in the Cued condition. In the second part of the experiment, we moved the array 2m and changed the shape of the array. Across both phases, birds in the Cued condition learned to discriminate between 10 and 20-min flowers more quickly than did the birds in the Uncued condition. The Cued birds were also better at discriminating between the two distinct refill intervals. Colour cues can, therefore, facilitate learning the refill schedules of experimental flowers in these birds. This article is part of a Special Issue entitled: Cognition in the wild. Copyright © 2014 Elsevier B.V. All rights reserved.

  14. The Role of Extrinsic Rewards and Cue-Intention Association in Prospective Memory in Young Children.

    Science.gov (United States)

    Sheppard, Daniel Patrick; Kretschmer, Anett; Knispel, Elisa; Vollert, Bianka; Altgassen, Mareike

    2015-01-01

    The current study examined, for the first time, the effect of cue-intention association, as well as the effects of promised extrinsic rewards, on prospective memory in young children, aged 5-years-old (n = 39) and 7-years-old (n = 40). Children were asked to name pictures for a toy mole, whilst also having to remember to respond differently to certain target pictures (prospective memory task). The level to which the target picture was associated with the intention was manipulated across two conditions (low- or high-association) for all participants, whilst half of the participants were promised a reward for good prospective memory performance. Results showed a main effect of age, with the 7-year-olds outperforming the 5-year-olds. Furthermore, there was a main effect of reward, with those promised a reward performing better than those who were not. No effect was found for cue-association, with the participants of both age groups performing equally well in both association conditions. No significant interactions were found between any of the variables. The potentially important role of reward in young children's everyday prospective memory tasks, and possible reasons for the lack of a reflexive-associative effect, are discussed.

  15. On Principle of Rewards in English Learning

    Institute of Scientific and Technical Information of China (English)

    熊莉芸

    2004-01-01

    There is no question that learning a foreign language like English is different from learning other subjects, mainly be cause it is new to us Chinese and there is no enough enviroment. But that doesn't mean we have no way to learn it and do it well .If asked to identify the most powerful influences on learning, motivation would probably be high on most teachers' and learners' lists. It seens only sensible to assume that English learning is most likely to occur when the learners want to learn. That is, when motivation such as interest, curiosity, or a desire achieves, the learners would be engaged in learning. However, how do we teachers motivate our students to like learning and learn well? Here, rewards both extrinsic and intrinsic are of great value and play a vital role in English learning.

  16. Functional states of rat cortical circuits during the unpredictable availability of a reward-related cue.

    Science.gov (United States)

    Fernández-Lamo, Iván; Sánchez-Campusano, Raudel; Gruart, Agnès; Delgado-García, José M

    2016-11-21

    Proper performance of acquired abilities can be disturbed by the unexpected occurrence of external changes. Rats trained with an operant conditioning task (to press a lever in order to obtain a food pellet) using a fixed-ratio (1:1) schedule were subsequently placed in a Skinner box in which the lever could be removed randomly. Field postsynaptic potentials (fPSPs) were chronically evoked in perforant pathway-hippocampal CA1 (PP-CA1), CA1-subiculum (CA1-SUB), CA1-medial prefrontal cortex (CA1-mPFC), mPFC-nucleus accumbens (mPFC-NAc), and mPFC-basolateral amygdala (mPFC-BLA) synapses during lever IN and lever OUT situations. While lever presses were accompanied by a significant increase in fPSP slopes at the five synapses, the unpredictable absence of the lever were accompanied by decreased fPSP slopes in all, except PP-CA1 synapses. Spectral analysis of local field potentials (LFPs) recorded when the animal approached the corresponding area in the lever OUT situation presented lower spectral powers than during lever IN occasions for all recording sites, apart from CA1. Thus, the unpredictable availability of a reward-related cue modified the activity of cortical and subcortical areas related with the acquisition of operant learning tasks, suggesting an immediate functional reorganization of these neural circuits to address the changed situation and to modify ongoing behaviors accordingly.

  17. Learning Analytics: Readiness and Rewards

    Science.gov (United States)

    Friesen, Norm

    2013-01-01

    This position paper introduces the relatively new field of learning analytics, first by considering the relevant meanings of both "learning" and "analytics," and then by looking at two main levels at which learning analytics can be or has been implemented in educational organizations. Although integrated turnkey systems or…

  18. Response of neural reward regions to food cues in autism spectrum disorders

    Directory of Open Access Journals (Sweden)

    Cascio Carissa J

    2012-05-01

    Full Text Available Abstract Background One hypothesis for the social deficits that characterize autism spectrum disorders (ASD is diminished neural reward response to social interaction and attachment. Prior research using established monetary reward paradigms as a test of non-social reward to compare with social reward may involve confounds in the ability of individuals with ASD to utilize symbolic representation of money and the abstraction required to interpret monetary gains. Thus, a useful addition to our understanding of neural reward circuitry in ASD includes a characterization of the neural response to primary rewards. Method We asked 17 children with ASD and 18 children without ASD to abstain from eating for at least four hours before an MRI scan in which they viewed images of high-calorie foods. We assessed the neural reward network for increases in the blood oxygenation level dependent (BOLD signal in response to the food images Results We found very similar patterns of increased BOLD signal to these images in the two groups; both groups showed increased BOLD signal in the bilateral amygdala, as well as in the nucleus accumbens, orbitofrontal cortex, and insula. Direct group comparisons revealed that the ASD group showed a stronger response to food cues in bilateral insula along the anterior-posterior gradient and in the anterior cingulate cortex than the control group, whereas there were no neural reward regions that showed higher activation for controls than for ASD. Conclusion These results suggest that neural response to primary rewards is not diminished but in fact shows an aberrant enhancement in children with ASD.

  19. Nucleus accumbens corticotropin-releasing factor increases cue-triggered motivation for sucrose reward: paradoxical positive incentive effects in stress?

    Directory of Open Access Journals (Sweden)

    Schulkin Jay

    2006-04-01

    Full Text Available Abstract Background Corticotropin-releasing factor (CRF is typically considered to mediate aversive aspects of stress, fear and anxiety. However, CRF release in the brain is also elicited by natural rewards and incentive cues, raising the possibility that some CRF systems in the brain mediate an independent function of positive incentive motivation, such as amplifying incentive salience. Here we asked whether activation of a limbic CRF subsystem magnifies the increase in positive motivation for reward elicited by incentive cues previously associated with that reward, in a way that might exacerbate cue-triggered binge pursuit of food or other incentives? We assessed the impact of CRF microinjections into the medial shell of nucleus accumbens using a pure incentive version of Pavlovian-Instrumental transfer, a measure specifically sensitive to the incentive salience of reward cues (which it separates from influences of aversive stress, stress reduction, frustration and other traditional explanations for stress-increased behavior. Rats were first trained to press one of two levers to obtain sucrose pellets, and then separately conditioned to associate a Pavlovian cue with free sucrose pellets. On test days, rats received microinjections of vehicle, CRF (250 or 500 ng/0.2 μl or amphetamine (20 μg/0.2 μl. Lever pressing was assessed in the presence or absence of the Pavlovian cues during a half-hour test. Results Microinjections of the highest dose of CRF (500 ng or amphetamine (20 μg selectively enhanced the ability of Pavlovian reward cues to trigger phasic peaks of increased instrumental performance for a sucrose reward, each peak lasting a minute or so before decaying after the cue. Lever pressing was not enhanced by CRF microinjections in the baseline absence of the Pavlovian cue or during the presentation without a cue, showing that the CRF enhancement could not be explained as a result of generalized motor arousal, frustration or stress

  20. Nucleus accumbens corticotropin-releasing factor increases cue-triggered motivation for sucrose reward: paradoxical positive incentive effects in stress?

    Science.gov (United States)

    Peciña, Susana; Schulkin, Jay; Berridge, Kent C

    2006-04-13

    Corticotropin-releasing factor (CRF) is typically considered to mediate aversive aspects of stress, fear and anxiety. However, CRF release in the brain is also elicited by natural rewards and incentive cues, raising the possibility that some CRF systems in the brain mediate an independent function of positive incentive motivation, such as amplifying incentive salience. Here we asked whether activation of a limbic CRF subsystem magnifies the increase in positive motivation for reward elicited by incentive cues previously associated with that reward, in a way that might exacerbate cue-triggered binge pursuit of food or other incentives? We assessed the impact of CRF microinjections into the medial shell of nucleus accumbens using a pure incentive version of Pavlovian-Instrumental transfer, a measure specifically sensitive to the incentive salience of reward cues (which it separates from influences of aversive stress, stress reduction, frustration and other traditional explanations for stress-increased behavior). Rats were first trained to press one of two levers to obtain sucrose pellets, and then separately conditioned to associate a Pavlovian cue with free sucrose pellets. On test days, rats received microinjections of vehicle, CRF (250 or 500 ng/0.2 microl) or amphetamine (20 microg/0.2 microl). Lever pressing was assessed in the presence or absence of the Pavlovian cues during a half-hour test. Microinjections of the highest dose of CRF (500 ng) or amphetamine (20 microg) selectively enhanced the ability of Pavlovian reward cues to trigger phasic peaks of increased instrumental performance for a sucrose reward, each peak lasting a minute or so before decaying after the cue. Lever pressing was not enhanced by CRF microinjections in the baseline absence of the Pavlovian cue or during the presentation without a cue, showing that the CRF enhancement could not be explained as a result of generalized motor arousal, frustration or stress, or by persistent attempts to

  1. Signed reward prediction errors drive declarative learning.

    Directory of Open Access Journals (Sweden)

    Esther De Loof

    Full Text Available Reward prediction errors (RPEs are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning. However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.

  2. Signed reward prediction errors drive declarative learning.

    Science.gov (United States)

    De Loof, Esther; Ergo, Kate; Naert, Lien; Janssens, Clio; Talsma, Durk; Van Opstal, Filip; Verguts, Tom

    2018-01-01

    Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals) during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.

  3. Continuous, but not intermittent, antipsychotic drug delivery intensifies the pursuit of reward cues.

    Science.gov (United States)

    Bédard, Anne-Marie; Maheux, Jérôme; Lévesque, Daniel; Samaha, Anne-Noël

    2011-05-01

    Chronic exposure to antipsychotic medications can persistently change brain dopamine systems. Most studies on the functional significance of these neural changes have focused on motor behavior and few have addressed how long-term antipsychotic treatment might influence dopamine-mediated reward function. We asked, therefore, whether a clinically relevant antipsychotic treatment regimen would alter the incentive motivational properties of a reward cue. We assessed the ability of a Pavlovian-conditioned stimulus to function as a conditioned reward, as well as to elicit approach behavior in rats treated with haloperidol, either continuously (achieved via subcutaneous osmotic minipump) or intermittently (achieved via daily subcutaneous injections). Continuous, but not intermittent, treatment enhanced the ability of amphetamine to potentiate the conditioned reinforcing effects of a cue associated with water. This effect was not related to differences in the ability to attribute predictive value to a conditioned stimulus (as measured by conditioned approach behavior), but was potentially linked to the development of behavioral supersensitivity to amphetamine and to augmented amphetamine-induced immediate early-gene expression (c-fos and Nur77) in dorsal striatopallidal and striatonigral cells. By enhancing the ability of reward cues to control behavior and by intensifying dopamine-mediated striatopallidal and striatonigral cell activity, standard (ie, continuous) antipsychotic treatment regimens might exacerbate drug-seeking and drug-taking behavior in schizophrenia. Achieving regular but transiently high antipsychotic levels in the brain (as modeled in the intermittent condition) might be a viable option to prevent these changes. This possibility should be explored in the clinic.

  4. Inhibition of histone deacetylase 3 via RGFP966 facilitates cortical plasticity underlying unusually accurate auditory associative cue memory for excitatory and inhibitory cue-reward associations.

    Science.gov (United States)

    Shang, Andrea; Bylipudi, Sooraz; Bieszczad, Kasia M

    2018-05-31

    Epigenetic mechanisms are key for regulating long-term memory (LTM) and are known to exert control on memory formation in multiple systems of the adult brain, including the sensory cortex. One epigenetic mechanism is chromatin modification by histone acetylation. Blocking the action of histone de-acetylases (HDACs) that normally negatively regulate LTM by repressing transcription, has been shown to enable memory formation. Indeed, HDAC-inhibition appears to facilitate memory by altering the dynamics of gene expression events important for memory consolidation. However less understood are the ways in which molecular-level consolidation processes alter subsequent memory to enhance storage or facilitate retrieval. Here we used a sensory perspective to investigate whether the characteristics of memory formed with HDAC inhibitors are different from naturally-formed memory. One possibility is that HDAC inhibition enables memory to form with greater sensory detail than normal. Because the auditory system undergoes learning-induced remodeling that provides substrates for sound-specific LTM, we aimed to identify behavioral effects of HDAC inhibition on memory for specific sound features using a standard model of auditory associative cue-reward learning, memory, and cortical plasticity. We found that three systemic post-training treatments of an HDAC3-inhibitor (RGPF966, Abcam Inc.) in rats in the early phase of training facilitated auditory discriminative learning, changed auditory cortical tuning, and increased the specificity for acoustic frequency formed in memory of both excitatory (S+) and inhibitory (S-) associations for at least 2 weeks. The findings support that epigenetic mechanisms act on neural and behavioral sensory acuity to increase the precision of associative cue memory, which can be revealed by studying the sensory characteristics of long-term associative memory formation with HDAC inhibitors. Published by Elsevier B.V.

  5. Motivational State, Reward Value, and Pavlovian Cues Differentially Affect Skilled Forelimb Grasping in Rats

    Science.gov (United States)

    Mosberger, Alice C.; de Clauser, Larissa; Kasper, Hansjörg; Schwab, Martin E.

    2016-01-01

    Motor skills represent high-precision movements performed at optimal speed and accuracy. Such motor skills are learned with practice over time. Besides practice, effects of motivation have also been shown to influence speed and accuracy of movements, suggesting that fast movements are performed to maximize gained reward over time as noted in…

  6. Activation of dopamine D3 receptors inhibits reward-related learning induced by cocaine.

    Science.gov (United States)

    Kong, H; Kuang, W; Li, S; Xu, M

    2011-03-10

    Memories of learned associations between the rewarding properties of drugs and environmental cues contribute to craving and relapse in humans. The mesocorticolimbic dopamine (DA) system is involved in reward-related learning induced by drugs of abuse. DA D3 receptors are preferentially expressed in mesocorticolimbic DA projection areas. Genetic and pharmacological studies have shown that DA D3 receptors suppress locomotor-stimulant effects of cocaine and reinstatement of cocaine-seeking behaviors. Activation of the extracellular signal-regulated kinase (ERK) induced by acute cocaine administration is also inhibited by D3 receptors. How D3 receptors modulate cocaine-induced reward-related learning and associated changes in cell signaling in reward circuits in the brain, however, have not been fully investigated. In the present study, we show that D3 receptor mutant mice exhibit potentiated acquisition of conditioned place preference (CPP) at low doses of cocaine compared to wild-type mice. Activation of ERK and CaMKIIα, but not the c-Jun N-terminal kinase and p38, in the nucleus accumbens, amygdala and prefrontal cortex is also potentiated in D3 receptor mutant mice compared to that in wild-type mice following CPP expression. These results support a model in which D3 receptors modulate reward-related learning induced by low doses of cocaine by inhibiting activation of ERK and CaMKIIα in reward circuits in the brain. Copyright © 2011 IBRO. Published by Elsevier Ltd. All rights reserved.

  7. Individual differences in sensitivity to reward and punishment and neural activity during reward and avoidance learning.

    Science.gov (United States)

    Kim, Sang Hee; Yoon, HeungSik; Kim, Hackjin; Hamann, Stephan

    2015-09-01

    In this functional neuroimaging study, we investigated neural activations during the process of learning to gain monetary rewards and to avoid monetary loss, and how these activations are modulated by individual differences in reward and punishment sensitivity. Healthy young volunteers performed a reinforcement learning task where they chose one of two fractal stimuli associated with monetary gain (reward trials) or avoidance of monetary loss (avoidance trials). Trait sensitivity to reward and punishment was assessed using the behavioral inhibition/activation scales (BIS/BAS). Functional neuroimaging results showed activation of the striatum during the anticipation and reception periods of reward trials. During avoidance trials, activation of the dorsal striatum and prefrontal regions was found. As expected, individual differences in reward sensitivity were positively associated with activation in the left and right ventral striatum during reward reception. Individual differences in sensitivity to punishment were negatively associated with activation in the left dorsal striatum during avoidance anticipation and also with activation in the right lateral orbitofrontal cortex during receiving monetary loss. These results suggest that learning to attain reward and learning to avoid loss are dependent on separable sets of neural regions whose activity is modulated by trait sensitivity to reward or punishment. © The Author (2015). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

  8. Obese adults have visual attention bias for food cue images: evidence for altered reward system function.

    Science.gov (United States)

    Castellanos, E H; Charboneau, E; Dietrich, M S; Park, S; Bradley, B P; Mogg, K; Cowan, R L

    2009-09-01

    The major aim of this study was to investigate whether the motivational salience of food cues (as reflected by their attention-grabbing properties) differs between obese and normal-weight subjects in a manner consistent with altered reward system function in obesity. A total of 18 obese and 18 normal-weight, otherwise healthy, adult women between the ages of 18 and 35 participated in an eye-tracking paradigm in combination with a visual probe task. Eye movements and reaction time to food and non-food images were recorded during both fasted and fed conditions in a counterbalanced design. Eating behavior and hunger level were assessed by self-report measures. Obese individuals had higher scores than normal-weight individuals on self-report measures of responsiveness to external food cues and vulnerability to disruptions in control of eating behavior. Both obese and normal-weight individuals demonstrated increased gaze duration for food compared to non-food images in the fasted condition. In the fed condition, however, despite reduced hunger in both groups, obese individuals maintained the increased attention to food images, whereas normal-weight individuals had similar gaze duration for food and non-food images. Additionally, obese individuals had preferential orienting toward food images at the onset of each image. Obese and normal-weight individuals did not differ in reaction time measures in the fasted or fed condition. Food cue incentive salience is elevated equally in normal-weight and obese individuals during fasting. Obese individuals retain incentive salience for food cues despite feeding and decreased self-report of hunger. Sensitization to food cues in the environment and their dysregulation in obese individuals may play a role in the development and/or maintenance of obesity.

  9. Neural correlates of reward-based spatial learning in persons with cocaine dependence.

    Science.gov (United States)

    Tau, Gregory Z; Marsh, Rachel; Wang, Zhishun; Torres-Sanchez, Tania; Graniello, Barbara; Hao, Xuejun; Xu, Dongrong; Packard, Mark G; Duan, Yunsuo; Kangarlu, Alayar; Martinez, Diana; Peterson, Bradley S

    2014-02-01

    Dysfunctional learning systems are thought to be central to the pathogenesis of and impair recovery from addictions. The functioning of the brain circuits for episodic memory or learning that support goal-directed behavior has not been studied previously in persons with cocaine dependence (CD). Thirteen abstinent CD and 13 healthy participants underwent MRI scanning while performing a task that requires the use of spatial cues to navigate a virtual-reality environment and find monetary rewards, allowing the functional assessment of the brain systems for spatial learning, a form of episodic memory. Whereas both groups performed similarly on the reward-based spatial learning task, we identified disturbances in brain regions involved in learning and reward in CD participants. In particular, CD was associated with impaired functioning of medial temporal lobe (MTL), a brain region that is crucial for spatial learning (and episodic memory) with concomitant recruitment of striatum (which normally participates in stimulus-response, or habit, learning), and prefrontal cortex. CD was also associated with enhanced sensitivity of the ventral striatum to unexpected rewards but not to expected rewards earned during spatial learning. We provide evidence that spatial learning in CD is characterized by disturbances in functioning of an MTL-based system for episodic memory and a striatum-based system for stimulus-response learning and reward. We have found additional abnormalities in distributed cortical regions. Consistent with findings from animal studies, we provide the first evidence in humans describing the disruptive effects of cocaine on the coordinated functioning of multiple neural systems for learning and memory.

  10. Neuropsychology of Reward Learning and Negative Symptoms in Schizophrenia

    OpenAIRE

    Nestor, Paul G.; Choate, Victoria; Niznikiewicz, Margaret; Levitt, James J.; Shenton, Martha E; McCarley, Robert W.

    2014-01-01

    We used the Iowa Gambling Test (IGT) to examine the relationship of reward learning to both neuropsychological functioning and symptom formation in 65 individuals with schizophrenia. Results indicated that compared to controls, participants with schizophrenia showed significantly reduced reward learning, which in turn correlated with reduced intelligence, memory and executive function, and increased negative symptoms. The current findings suggested that a disease-related disturbance in reward...

  11. Reward-Guided Learning with and without Causal Attribution

    Science.gov (United States)

    Jocham, Gerhard; Brodersen, Kay H.; Constantinescu, Alexandra O.; Kahn, Martin C.; Ianni, Angela M.; Walton, Mark E.; Rushworth, Matthew F.S.; Behrens, Timothy E.J.

    2016-01-01

    Summary When an organism receives a reward, it is crucial to know which of many candidate actions caused this reward. However, recent work suggests that learning is possible even when this most fundamental assumption is not met. We used novel reward-guided learning paradigms in two fMRI studies to show that humans deploy separable learning mechanisms that operate in parallel. While behavior was dominated by precise contingent learning, it also revealed hallmarks of noncontingent learning strategies. These learning mechanisms were separable behaviorally and neurally. Lateral orbitofrontal cortex supported contingent learning and reflected contingencies between outcomes and their causal choices. Amygdala responses around reward times related to statistical patterns of learning. Time-based heuristic mechanisms were related to activity in sensorimotor corticostriatal circuitry. Our data point to the existence of several learning mechanisms in the human brain, of which only one relies on applying known rules about the causal structure of the task. PMID:26971947

  12. Differential Contributions of Nucleus Accumbens Subregions to Cue-Guided Risk/Reward Decision Making and Implementation of Conditional Rules.

    Science.gov (United States)

    Floresco, Stan B; Montes, David R; Tse, Maric M T; van Holstein, Mieke

    2018-02-21

    The nucleus accumbens (NAc) is a key node within corticolimbic circuitry for guiding action selection and cost/benefit decision making in situations involving reward uncertainty. Preclinical studies have typically assessed risk/reward decision making using assays where decisions are guided by internally generated representations of choice-outcome contingencies. Yet, real-life decisions are often influenced by external stimuli that inform about likelihoods of obtaining rewards. How different subregions of the NAc mediate decision making in such situations is unclear. Here, we used a novel assay colloquially termed the "Blackjack" task that models these types of situations. Male Long-Evans rats were trained to choose between one lever that always delivered a one-pellet reward and another that delivered four pellets with different probabilities [either 50% (good-odds) or 12.5% (poor-odds)], which were signaled by one of two auditory cues. Under control conditions, rats selected the large/risky option more often on good-odds versus poor-odds trials. Inactivation of the NAc core caused indiscriminate choice patterns. In contrast, NAc shell inactivation increased risky choice, more prominently on poor-odds trials. Additional experiments revealed that both subregions contribute to auditory conditional discrimination. NAc core or shell inactivation reduced Pavlovian approach elicited by an auditory CS+, yet shell inactivation also increased responding during presentation of a CS-. These data highlight distinct contributions for NAc subregions in decision making and reward seeking guided by discriminative stimuli. The core is crucial for implementation of conditional rules, whereas the shell refines reward seeking by mitigating the allure of larger, unlikely rewards and reducing expression of inappropriate or non-rewarded actions. SIGNIFICANCE STATEMENT Using external cues to guide decision making is crucial for adaptive behavior. Deficits in cue-guided behavior have been

  13. Amphetamine-induced sensitization and reward uncertainty similarly enhance incentive salience for conditioned cues

    Science.gov (United States)

    Robinson, Mike J.F.; Anselme, Patrick; Suchomel, Kristen; Berridge, Kent C.

    2015-01-01

    Amphetamine and stress can sensitize mesolimbic dopamine-related systems. In Pavlovian autoshaping, repeated exposure to uncertainty of reward prediction can enhance motivated sign-tracking or attraction to a discrete reward-predicting cue (lever CS+), as well as produce cross-sensitization to amphetamine. However, it remains unknown how amphetamine-sensitization or repeated restraint stress interact with uncertainty in controlling CS+ incentive salience attribution reflected in sign-tracking. Here rats were tested in three successive phases. First, different groups underwent either induction of amphetamine sensitization or repeated restraint stress, or else were not sensitized or stressed as control groups (either saline injections only, or no stress or injection at all). All next received Pavlovian autoshaping training under either certainty conditions (100% CS-UCS association) or uncertainty conditions (50% CS-UCS association and uncertain reward magnitude). During training, rats were assessed for sign-tracking to the lever CS+ versus goal-tracking to the sucrose dish. Finally, all groups were tested for psychomotor sensitization of locomotion revealed by an amphetamine challenge. Our results confirm that reward uncertainty enhanced sign-tracking attraction toward the predictive CS+ lever, at the expense of goal-tracking. We also report that amphetamine sensitization promoted sign-tracking even in rats trained under CS-UCS certainty conditions, raising them to sign-tracking levels equivalent to the uncertainty group. Combining amphetamine sensitization and uncertainty conditions together did not add together to elevate sign-tracking further above the relatively high levels induced by either manipulation alone. In contrast, repeated restraint stress enhanced subsequent amphetamine-elicited locomotion, but did not enhance CS+ attraction. PMID:26076340

  14. Amphetamine-induced sensitization and reward uncertainty similarly enhance incentive salience for conditioned cues.

    Science.gov (United States)

    Robinson, Mike J F; Anselme, Patrick; Suchomel, Kristen; Berridge, Kent C

    2015-08-01

    Amphetamine and stress can sensitize mesolimbic dopamine-related systems. In Pavlovian autoshaping, repeated exposure to uncertainty of reward prediction can enhance motivated sign-tracking or attraction to a discrete reward-predicting cue (lever-conditioned stimulus; CS+), as well as produce cross-sensitization to amphetamine. However, it remains unknown how amphetamine sensitization or repeated restraint stress interact with uncertainty in controlling CS+ incentive salience attribution reflected in sign-tracking. Here rats were tested in 3 successive phases. First, different groups underwent either induction of amphetamine sensitization or repeated restraint stress, or else were not sensitized or stressed as control groups (either saline injections only, or no stress or injection at all). All next received Pavlovian autoshaping training under either certainty conditions (100% CS-UCS association) or uncertainty conditions (50% CS-UCS association and uncertain reward magnitude). During training, rats were assessed for sign-tracking to the CS+ lever versus goal-tracking to the sucrose dish. Finally, all groups were tested for psychomotor sensitization of locomotion revealed by an amphetamine challenge. Our results confirm that reward uncertainty enhanced sign-tracking attraction toward the predictive CS+ lever, at the expense of goal-tracking. We also reported that amphetamine sensitization promoted sign-tracking even in rats trained under CS-UCS certainty conditions, raising them to sign-tracking levels equivalent to the uncertainty group. Combining amphetamine sensitization and uncertainty conditions did not add together to elevate sign-tracking further above the relatively high levels induced by either manipulation alone. In contrast, repeated restraint stress enhanced subsequent amphetamine-elicited locomotion, but did not enhance CS+ attraction. (c) 2015 APA, all rights reserved).

  15. Finding intrinsic rewards by embodied evolution and constrained reinforcement learning.

    Science.gov (United States)

    Uchibe, Eiji; Doya, Kenji

    2008-12-01

    Understanding the design principle of reward functions is a substantial challenge both in artificial intelligence and neuroscience. Successful acquisition of a task usually requires not only rewards for goals, but also for intermediate states to promote effective exploration. This paper proposes a method for designing 'intrinsic' rewards of autonomous agents by combining constrained policy gradient reinforcement learning and embodied evolution. To validate the method, we use Cyber Rodent robots, in which collision avoidance, recharging from battery packs, and 'mating' by software reproduction are three major 'extrinsic' rewards. We show in hardware experiments that the robots can find appropriate 'intrinsic' rewards for the vision of battery packs and other robots to promote approach behaviors.

  16. Working memory and reward association learning impairments in obesity.

    Science.gov (United States)

    Coppin, Géraldine; Nolan-Poupart, Sarah; Jones-Gotman, Marilyn; Small, Dana M

    2014-12-01

    Obesity has been associated with impaired executive functions including working memory. Less explored is the influence of obesity on learning and memory. In the current study we assessed stimulus reward association learning, explicit learning and memory and working memory in healthy weight, overweight and obese individuals. Explicit learning and memory did not differ as a function of group. In contrast, working memory was significantly and similarly impaired in both overweight and obese individuals compared to the healthy weight group. In the first reward association learning task the obese, but not healthy weight or overweight participants consistently formed paradoxical preferences for a pattern associated with a negative outcome (fewer food rewards). To determine if the deficit was specific to food reward a second experiment was conducted using money. Consistent with Experiment 1, obese individuals selected the pattern associated with a negative outcome (fewer monetary rewards) more frequently than healthy weight individuals and thus failed to develop a significant preference for the most rewarded patterns as was observed in the healthy weight group. Finally, on a probabilistic learning task, obese compared to healthy weight individuals showed deficits in negative, but not positive outcome learning. Taken together, our results demonstrate deficits in working memory and stimulus reward learning in obesity and suggest that obese individuals are impaired in learning to avoid negative outcomes. Copyright © 2014 Elsevier Ltd. All rights reserved.

  17. Observing others stay or switch - How social prediction errors are integrated into reward reversal learning.

    Science.gov (United States)

    Ihssen, Niklas; Mussweiler, Thomas; Linden, David E J

    2016-08-01

    Reward properties of stimuli can undergo sudden changes, and the detection of these 'reversals' is often made difficult by the probabilistic nature of rewards/punishments. Here we tested whether and how humans use social information (someone else's choices) to overcome uncertainty during reversal learning. We show a substantial social influence during reversal learning, which was modulated by the type of observed behavior. Participants frequently followed observed conservative choices (no switches after punishment) made by the (fictitious) other player but ignored impulsive choices (switches), even though the experiment was set up so that both types of response behavior would be similarly beneficial/detrimental (Study 1). Computational modeling showed that participants integrated the observed choices as a 'social prediction error' instead of ignoring or blindly following the other player. Modeling also confirmed higher learning rates for 'conservative' versus 'impulsive' social prediction errors. Importantly, this 'conservative bias' was boosted by interpersonal similarity, which in conjunction with the lack of effects observed in a non-social control experiment (Study 2) confirmed its social nature. A third study suggested that relative weighting of observed impulsive responses increased with increased volatility (frequency of reversals). Finally, simulations showed that in the present paradigm integrating social and reward information was not necessarily more adaptive to maximize earnings than learning from reward alone. Moreover, integrating social information increased accuracy only when conservative and impulsive choices were weighted similarly during learning. These findings suggest that to guide decisions in choice contexts that involve reward reversals humans utilize social cues conforming with their preconceptions more strongly than cues conflicting with them, especially when the other is similar. Copyright © 2016 The Authors. Published by Elsevier B

  18. Associations among smoking, anhedonia, and reward learning in depression.

    Science.gov (United States)

    Liverant, Gabrielle I; Sloan, Denise M; Pizzagalli, Diego A; Harte, Christopher B; Kamholz, Barbara W; Rosebrock, Laina E; Cohen, Andrew L; Fava, Maurizio; Kaplan, Gary B

    2014-09-01

    Depression and cigarette smoking co-occur at high rates. However, the etiological mechanisms that contribute to this relationship remain unclear. Anhedonia and associated impairments in reward learning are key features of depression, which also have been linked to the onset and maintenance of cigarette smoking. However, few studies have investigated differences in anhedonia and reward learning among depressed smokers and depressed nonsmokers. The goal of this study was to examine putative differences in anhedonia and reward learning in depressed smokers (n=36) and depressed nonsmokers (n=44). To this end, participants completed self-report measures of anhedonia and behavioral activation (BAS reward responsiveness scores) and as well as a probabilistic reward task rooted in signal detection theory, which measures reward learning (Pizzagalli, Jahn, & O'Shea, 2005). When considering self-report measures, depressed smokers reported higher trait anhedonia and reduced BAS reward responsiveness scores compared to depressed nonsmokers. In contrast to self-report measures, nicotine-satiated depressed smokers demonstrated greater acquisition of reward-based learning compared to depressed nonsmokers as indexed by the probabilistic reward task. Findings may point to a potential mechanism underlying the frequent co-occurrence of smoking and depression. These results highlight the importance of continued investigation of the role of anhedonia and reward system functioning in the co-occurrence of depression and nicotine abuse. Results also may support the use of treatments targeting reward learning (e.g., behavioral activation) to enhance smoking cessation among individuals with depression. Copyright © 2014. Published by Elsevier Ltd.

  19. Fasting for 24 hours heightens reward from food and food-related cues.

    Science.gov (United States)

    Cameron, Jameason D; Goldfield, Gary S; Finlayson, Graham; Blundell, John E; Doucet, Eric

    2014-01-01

    We examined the impact of a 24 hour complete fast (vs. fed state) on two measures of food reward: 1) 'wanting', as measured by response to food images and by the relative-reinforcing value of food (RRV), and 2) 'liking', as measured by response to food images and the hedonic evaluation of foods consumed. Utilizing a randomized crossover design, 15 subjects (9 male; 6 female) aged 28.6±4.5 yrs with body mass index 25.3±1.4 kg/m(2) were randomized and counterbalanced to normal feeding (FED) and 24-hour fast (FASTED) conditions. Trait characteristics were measured with the Three Factor Eating Questionnaire. Two computer tasks measured food reward: 1) RRV progressive ratio task, 2) explicit 'liking' and 'wanting' (Leeds Food Preference Questionnaire, LFPQ). Also measured were ad libitum energy intake (EI; buffet) and food 'liking' (visual analogue scale) of personalized stimuli. There were no significant anthropometric changes between conditions. Appetite scores, hedonic ratings of 'liking', and ad libitum EI all significantly increased under the FASTED condition (pFASTED condition there were significant increases in the RRV of snack foods; similarly, explicit 'wanting' and 'liking' significantly increased for all food categories. 'Liking' of sweet foods remained high across-meals under FASTED, but savory foods decreased in hedonic saliency. Relative to a fed state, we observed an increase in hedonic ratings of food, the rewarding value of food, and food intake after a 24 hr fast. Alliesthesia to food and food cues is suggested by heightened hedonic ratings under the FASTED condition relative to FED.

  20. Model Appreciative Learning Untuk Perancangan Reward Pada Game Pendidikan

    OpenAIRE

    Haryanto, Hanny; Kardianawati, Acun; Rosyidah, Umi

    2017-01-01

    Reward adalah elemen dari game yang sangat penting untuk membentuk pengalaman positif dan motivasi bagi pemain. Reward dalam suatu game pendidikan memegang peranan penting dalam menjaga motivasi pembelajar dan memberikan evaluasi dari apa yang dikerjakan. Namun perancangan reward seringkali masih tidak terkonsep dengan baik, acak dan bersifat subjektif. Penelitian ini menggunakan model Appreciative Learning, yang terdiri dari tahapan Discovery, Dream, Design dan Destiny, untuk mel...

  1. Impairment of probabilistic reward-based learning in schizophrenia.

    Science.gov (United States)

    Weiler, Julia A; Bellebaum, Christian; Brüne, Martin; Juckel, Georg; Daum, Irene

    2009-09-01

    Recent models assume that some symptoms of schizophrenia originate from defective reward processing mechanisms. Understanding the precise nature of reward-based learning impairments might thus make an important contribution to the understanding of schizophrenia and the development of treatment strategies. The present study investigated several features of probabilistic reward-based stimulus association learning, namely the acquisition of initial contingencies, reversal learning, generalization abilities, and the effects of reward magnitude. Compared to healthy controls, individuals with schizophrenia exhibited attenuated overall performance during acquisition, whereas learning rates across blocks were similar to the rates of controls. On the group level, persons with schizophrenia were, however, unable to learn the reversal of the initial reward contingencies. Exploratory analysis of only the subgroup of individuals with schizophrenia who showed significant learning during acquisition yielded deficits in reversal learning with low reward magnitudes only. There was further evidence of a mild generalization impairment of the persons with schizophrenia in an acquired equivalence task. In summary, although there was evidence of intact basic processing of reward magnitudes, individuals with schizophrenia were impaired at using this feedback for the adaptive guidance of behavior.

  2. Neuropsychology of reward learning and negative symptoms in schizophrenia.

    Science.gov (United States)

    Nestor, Paul G; Choate, Victoria; Niznikiewicz, Margaret; Levitt, James J; Shenton, Martha E; McCarley, Robert W

    2014-11-01

    We used the Iowa Gambling Test (IGT) to examine the relationship of reward learning to both neuropsychological functioning and symptom formation in 65 individuals with schizophrenia. Results indicated that compared to controls, participants with schizophrenia showed significantly reduced reward learning, which in turn correlated with reduced intelligence, memory and executive function, and negative symptoms. The current findings suggested that a disease-related disturbance in reward learning may underlie both cognitive and motivation deficits, as expressed by neuropsychological impairment and negative symptoms in schizophrenia. Copyright © 2014 Elsevier B.V. All rights reserved.

  3. Neural correlates of water reward in thirsty Drosophila

    OpenAIRE

    Lin, Suewei; Owald, David; Chandra, Vikram; Talbot, Clifford; Huetteroth, Wolf; Waddell, Scott

    2014-01-01

    Drinking water is innately rewarding to thirsty animals. In addition, the consumed value can be assigned to behavioral actions and predictive sensory cues by associative learning. Here we show that thirst converts water avoidance into water-seeking in naive Drosophila melanogaster. Thirst also permitted flies to learn olfactory cues paired with water reward. Water learning required water taste and

  4. Reward learning and negative emotion during rapid attentional competition

    Directory of Open Access Journals (Sweden)

    Takemasa eYokoyama

    2015-03-01

    Full Text Available Learned stimulus-reward associations influence how attention is allocated, such that stimuli rewarded in the past are favored in situations involving limited resources and competition. At the same time, task-irrelevant, high-arousal negative stimuli capture attention and divert resources away from tasks resulting in poor behavioral performance. Yet, investigations of how reward learning and negative stimuli affect perceptual and attentional processing have been conducted in a largely independent fashion. We have recently reported that performance-based monetary rewards reduce negative stimuli interference during perception. The goal of the present study was to investigate how stimuli associated with past monetary rewards compete with negative stimuli during a subsequent attentional task when, critically, no performance-based rewards were at stake. Across two experiments, we found that target stimuli that were associated with high reward reduced the interference effect of potent, negative distractors. Similar to our recent findings with performance-based rewards, our results demonstrate that reward-associated stimuli reduce the deleterious impact of negative stimuli on behavior.

  5. Reward Draws the Eye, Uncertainty Holds the Eye: Associative Learning Modulates Distractor Interference in Visual Search

    Directory of Open Access Journals (Sweden)

    Stephan Koenig

    2017-07-01

    Full Text Available Stimuli in our sensory environment differ with respect to their physical salience but moreover may acquire motivational salience by association with reward. If we repeatedly observed that reward is available in the context of a particular cue but absent in the context of another cue the former typically attracts more attention than the latter. However, we also may encounter cues uncorrelated with reward. A cue with 50% reward contingency may induce an average reward expectancy but at the same time induces high reward uncertainty. In the current experiment we examined how both values, reward expectancy and uncertainty, affected overt attention. Two different colors were established as predictive cues for low reward and high reward respectively. A third color was followed by high reward on 50% of the trials and thus induced uncertainty. Colors then were introduced as distractors during search for a shape target, and we examined the relative potential of the color distractors to capture and hold the first fixation. We observed that capture frequency corresponded to reward expectancy while capture duration corresponded to uncertainty. The results may suggest that within trial reward expectancy is represented at an earlier time window than uncertainty.

  6. Reminder cues modulate the renewal effect in human predictive learning

    Directory of Open Access Journals (Sweden)

    Javier Bustamante

    2016-12-01

    Full Text Available Associative learning refers to our ability to learn about regularities in our environment. When a stimulus is repeatedly followed by a specific outcome, we learn to expect the outcome in the presence of the stimulus. We are also able to modify established expectations in the face of disconfirming information (the stimulus is no longer followed by the outcome. Both the change of environmental regularities and the related processes of adaptation are referred to as extinction. However, extinction does not erase the initially acquired expectations. For instance, following successful extinction, the initially learned expectations can recover when there is a context change – a phenomenon called the renewal effect, which is considered as a model for relapse after exposure therapy. Renewal was found to be modulated by reminder cues of acquisition and extinction. However, the mechanisms underlying the effectiveness of reminder cues are not well understood. The aim of the present study was to investigate the impact of reminder cues on renewal in the field of human predictive learning. Experiment I demonstrated that renewal in human predictive learning is modulated by cues related to acquisition or extinction. Initially, participants received pairings of a stimulus and an outcome in one context. These stimulus-outcome pairings were preceded by presentations of a reminder cue (acquisition cue. Then, participants received extinction in a different context in which presentations of the stimulus were no longer followed by the outcome. These extinction trials were preceded by a second reminder cue (extinction cue. During a final phase conducted in a third context, participants showed stronger expectations of the outcome in the presence of the stimulus when testing was accompanied by the acquisition cue compared to the extinction cue. Experiment II tested an explanation of the reminder cue effect in terms of simple cue-outcome associations. Therefore

  7. Dopamine D2/3- and μ-opioid receptor antagonists reduce cue-induced responding and reward impulsivity in humans.

    Science.gov (United States)

    Weber, S C; Beck-Schimmer, B; Kajdi, M-E; Müller, D; Tobler, P N; Quednow, B B

    2016-07-05

    Increased responding to drug-associated stimuli (cue reactivity) and an inability to tolerate delayed gratification (reward impulsivity) have been implicated in the development and maintenance of drug addiction. Whereas data from animal studies suggest that both the dopamine and opioid system are involved in these two reward-related processes, their role in humans is less clear. Moreover, dopaminergic and opioidergic drugs have not been directly compared with regard to these functions, even though a deeper understanding of the underlying mechanisms might inform the development of specific treatments for elevated cue reactivity and reward impulsivity. In a randomized, double-blind, between-subject design we administered the selective dopamine D2/D3 receptor antagonist amisulpride (400 mg, n=41), the unspecific opioid receptor antagonist naltrexone (50 mg, n=40) or placebo (n=40) to healthy humans and measured cue-induced responding with a Pavlovian-instrumental transfer task and reward impulsivity with a delay discounting task. Mood was assessed using a visual analogue scale. Compared with placebo, amisulpride significantly suppressed cue-induced responding and reward impulsivity. The effects of naltrexone were similar, although less pronounced. Both amisulpride and naltrexone decreased average mood ratings compared with placebo. Our results demonstrate that a selective blockade of dopamine D2/D3 receptors reduces cue-induced responding and reward impulsivity in healthy humans. Antagonizing μ-opioid receptors has similar effects for cue-induced responding and to a lesser extent for reward impulsivity.

  8. Audiovisual Cues and Perceptual Learning of Spectrally Distorted Speech

    Science.gov (United States)

    Pilling, Michael; Thomas, Sharon

    2011-01-01

    Two experiments investigate the effectiveness of audiovisual (AV) speech cues (cues derived from both seeing and hearing a talker speak) in facilitating perceptual learning of spectrally distorted speech. Speech was distorted through an eight channel noise-vocoder which shifted the spectral envelope of the speech signal to simulate the properties…

  9. Initial uncertainty in Pavlovian reward prediction persistently elevates incentive salience and extends sign-tracking to normally unattractive cues.

    Science.gov (United States)

    Robinson, Mike J F; Anselme, Patrick; Fischer, Adam M; Berridge, Kent C

    2014-06-01

    Uncertainty is a component of many gambling games and may play a role in incentive motivation and cue attraction. Uncertainty can increase the attractiveness for predictors of reward in the Pavlovian procedure of autoshaping, visible as enhanced sign-tracking (or approach and nibbles) by rats of a metal lever whose sudden appearance acts as a conditioned stimulus (CS+) to predict sucrose pellets as an unconditioned stimulus (UCS). Here we examined how reward uncertainty might enhance incentive salience as sign-tracking both in intensity and by broadening the range of attractive CS+s. We also examined whether initially induced uncertainty enhancements of CS+ attraction can endure beyond uncertainty itself, and persist even when Pavlovian prediction becomes 100% certain. Our results show that uncertainty can broaden incentive salience attribution to make CS cues attractive that would otherwise not be (either because they are too distal from reward or too risky to normally attract sign-tracking). In addition, uncertainty enhancement of CS+ incentive salience, once induced by initial exposure, persisted even when Pavlovian CS-UCS correlations later rose toward 100% certainty in prediction. Persistence suggests an enduring incentive motivation enhancement potentially relevant to gambling, which in some ways resembles incentive-sensitization. Higher motivation to uncertain CS+s leads to more potent attraction to these cues when they predict the delivery of uncertain rewards. In humans, those cues might possibly include the sights and sounds associated with gambling, which contribute a major component of the play immersion experienced by problematic gamblers. Copyright © 2014 Elsevier B.V. All rights reserved.

  10. Ethanol Exposure History and Alcoholic Reward Differentially Alter Dopamine Release in the Nucleus Accumbens to a Reward-Predictive Cue.

    Science.gov (United States)

    Fiorenza, Amanda M; Shnitko, Tatiana A; Sullivan, Kaitlin M; Vemuru, Sudheer R; Gomez-A, Alexander; Esaki, Julie Y; Boettiger, Charlotte A; Da Cunha, Claudio; Robinson, Donita L

    2018-06-01

    Conditioned stimuli (CS) that predict reward delivery acquire the ability to induce phasic dopamine release in the nucleus accumbens (NAc). This dopamine release may facilitate conditioned approach behavior, which often manifests as approach to the site of reward delivery (called "goal-tracking") or to the CS itself (called "sign-tracking"). Previous research has linked sign-tracking in particular to impulsivity and drug self-administration, and addictive drugs may promote the expression of sign-tracking. Ethanol (EtOH) acutely promotes phasic release of dopamine in the accumbens, but it is unknown whether an alcoholic reward alters dopamine release to a CS. We hypothesized that Pavlovian conditioning with an alcoholic reward would increase dopamine release triggered by the CS and subsequent sign-tracking behavior. Moreover, we predicted that chronic intermittent EtOH (CIE) exposure would promote sign-tracking while acute administration of naltrexone (NTX) would reduce it. Rats received 14 doses of EtOH (3 to 5 g/kg, intragastric) or water followed by 6 days of Pavlovian conditioning training. Rewards were a chocolate solution with or without 10% (w/v) alcohol. We used fast-scan cyclic voltammetry to measure phasic dopamine release in the NAc core in response to the CS and the rewards. We also determined the effect of NTX (1 mg/kg, subcutaneous) on conditioned approach. Both CIE and alcoholic reward, individually but not together, associated with greater dopamine to the CS than control conditions. However, this increase in dopamine release was not linked to greater sign-tracking, as both CIE and alcoholic reward shifted conditioned approach from sign-tracking behavior to goal-tracking behavior. However, they both also increased sensitivity to NTX, which reduced goal-tracking behavior. While a history of EtOH exposure or alcoholic reward enhanced dopamine release to a CS, they did not promote sign-tracking under the current conditions. These findings are

  11. Conditioned Object Preference: An Alternative Approach to Measuring Reward Learning in Rats

    Science.gov (United States)

    Kennedy, Bruce C.; Kohli, Maulika; Maertens, Jamie J.; Marell, Paulina S.; Gewirtz, Jonathan C.

    2016-01-01

    Pavlovian conditioned approach behavior can be directed as much toward discrete cues as it is toward the environmental contexts in which those cues are encountered. The current experiments characterized a tendency of rats to approach object cues whose prior exposure had been paired with reward (conditioned object preference, COP). To demonstrate…

  12. Reward/Punishment reversal learning in older suicide attempters.

    Science.gov (United States)

    Dombrovski, Alexandre Y; Clark, Luke; Siegle, Greg J; Butters, Meryl A; Ichikawa, Naho; Sahakian, Barbara J; Szanto, Katalin

    2010-06-01

    Suicide rates are high in old age, and the contribution of cognitive risk factors remains poorly understood. Suicide may be viewed as an outcome of an altered decision process. The authors hypothesized that impairment in reward/punishment-based learning, a component of affective decision making, is associated with attempted suicide in late-life depression. They expected that suicide attempters would discount past reward/punishment history, focusing excessively on the most recent rewards and punishments. The authors further hypothesized that this impairment could be dissociated from executive abilities, such as forward planning. The authors assessed reward/punishment-based learning using the probabilistic reversal learning task in 65 individuals age 60 and older: suicide attempters, suicide ideators, nonsuicidal depressed elderly, and nondepressed comparison subjects. The authors used a reinforcement learning computational model to decompose reward/punishment processing over time. The Stockings of Cambridge test served as a control measure of executive function. Suicide attempters but not suicide ideators showed impaired probabilistic reversal learning compared to both nonsuicidal depressed elderly and nondepressed comparison subjects, after controlling for effects of education, global cognitive function, and substance use. Model-based analyses revealed that suicide attempters discounted previous history to a higher degree relative to comparison subjects, basing their choice largely on reward/punishment received on the last trial. Groups did not differ in their performance on the Stockings of Cambridge test. Older suicide attempters display impaired reward/punishment-based learning. The authors propose a hypothesis that older suicide attempters make overly present-focused decisions, ignoring past experiences. Modification of this "myopia for the past" may have therapeutic potential.

  13. Reward and punishment learning in daily life : A replication study

    NARCIS (Netherlands)

    Heininga, Vera E; van Roekel, G.H.; Wichers, Marieke; Oldehinkel, Albertine J

    2017-01-01

    Day-to-day experiences are accompanied by feelings of Positive Affect (PA) and Negative Affect (NA). Implicitly, without conscious processing, individuals learn about the reward and punishment value of each context and activity. These associative learning processes, in turn, affect the probability

  14. Sensitivity for cues predicting reward and punishment in young women with eating disorders

    NARCIS (Netherlands)

    Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline

    Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely

  15. Sensitivity for cues predicting reward and punishment in young women with eating disorders

    NARCIS (Netherlands)

    Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline

    2017-01-01

    Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely

  16. Boosting Vocabulary Learning by Verbal Cueing During Sleep.

    Science.gov (United States)

    Schreiner, Thomas; Rasch, Björn

    2015-11-01

    Reactivating memories during sleep by re-exposure to associated memory cues (e.g., odors or sounds) improves memory consolidation. Here, we tested for the first time whether verbal cueing during sleep can improve vocabulary learning. We cued prior learned Dutch words either during non-rapid eye movement sleep (NonREM) or during active or passive waking. Re-exposure to Dutch words during sleep improved later memory for the German translation of the cued words when compared with uncued words. Recall of uncued words was similar to an additional group receiving no verbal cues during sleep. Furthermore, verbal cueing failed to improve memory during active and passive waking. High-density electroencephalographic recordings revealed that successful verbal cueing during NonREM sleep is associated with a pronounced frontal negativity in event-related potentials, a higher frequency of frontal slow waves as well as a cueing-related increase in right frontal and left parietal oscillatory theta power. Our results indicate that verbal cues presented during NonREM sleep reactivate associated memories, and facilitate later recall of foreign vocabulary without impairing ongoing consolidation processes. Likewise, our oscillatory analysis suggests that both sleep-specific slow waves as well as theta oscillations (typically associated with successful memory encoding during wakefulness) might be involved in strengthening memories by cueing during sleep. © The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

  17. Incidental Learning of Rewarded Associations Bolsters Learning on an Associative Task

    Science.gov (United States)

    Freedberg, Michael; Schacherer, Jonathan; Hazeltine, Eliot

    2016-01-01

    Reward has been shown to change behavior as a result of incentive learning (by motivating the individual to increase their effort) and instrumental learning (by increasing the frequency of a particular behavior). However, Palminteri et al. (2011) demonstrated that reward can also improve the incidental learning of a motor skill even when…

  18. Anticipated Reward Enhances Offline Learning during Sleep

    Science.gov (United States)

    Fischer, Stefan; Born, Jan

    2009-01-01

    Sleep is known to promote the consolidation of motor memories. In everyday life, typically more than 1 isolated motor skill is acquired at a time, and this possibly gives rise to interference during consolidation. Here, it is shown that reward expectancy determines the amount of sleep-dependent memory consolidation. Subjects were trained on 2…

  19. Differential effects of fructose versus glucose on brain and appetitive responses to food cues and decisions for food rewards.

    Science.gov (United States)

    Luo, Shan; Monterosso, John R; Sarpelleh, Kayan; Page, Kathleen A

    2015-05-19

    Prior studies suggest that fructose compared with glucose may be a weaker suppressor of appetite, and neuroimaging research shows that food cues trigger greater brain reward responses in a fasted relative to a fed state. We sought to determine the effects of ingesting fructose versus glucose on brain, hormone, and appetitive responses to food cues and food-approach behavior. Twenty-four healthy volunteers underwent two functional magnetic resonance imaging (fMRI) sessions with ingestion of either fructose or glucose in a double-blinded, random-order cross-over design. fMRI was performed while participants viewed images of high-calorie foods and nonfood items using a block design. After each block, participants rated hunger and desire for food. Participants also performed a decision task in which they chose between immediate food rewards and delayed monetary bonuses. Hormones were measured at baseline and 30 and 60 min after drink ingestion. Ingestion of fructose relative to glucose resulted in smaller increases in plasma insulin levels and greater brain reactivity to food cues in the visual cortex (in whole-brain analysis) and left orbital frontal cortex (in region-of-interest analysis). Parallel to the neuroimaging findings, fructose versus glucose led to greater hunger and desire for food and a greater willingness to give up long-term monetary rewards to obtain immediate high-calorie foods. These findings suggest that ingestion of fructose relative to glucose results in greater activation of brain regions involved in attention and reward processing and may promote feeding behavior.

  20. Stimulus homogeneity enhances implicit learning: evidence from contextual cueing.

    Science.gov (United States)

    Feldmann-Wüstefeld, Tobias; Schubö, Anna

    2014-04-01

    Visual search for a target object is faster if the target is embedded in a repeatedly presented invariant configuration of distractors ('contextual cueing'). It has also been shown that the homogeneity of a context affects the efficiency of visual search: targets receive prioritized processing when presented in a homogeneous context compared to a heterogeneous context, presumably due to grouping processes at early stages of visual processing. The present study investigated in three Experiments whether context homogeneity also affects contextual cueing. In Experiment 1, context homogeneity varied on three levels of the task-relevant dimension (orientation) and contextual cueing was most pronounced for context configurations with high orientation homogeneity. When context homogeneity varied on three levels of the task-irrelevant dimension (color) and orientation homogeneity was fixed, no modulation of contextual cueing was observed: high orientation homogeneity led to large contextual cueing effects (Experiment 2) and low orientation homogeneity led to low contextual cueing effects (Experiment 3), irrespective of color homogeneity. Enhanced contextual cueing for homogeneous context configurations suggest that grouping processes do not only affect visual search but also implicit learning. We conclude that memory representation of context configurations are more easily acquired when context configurations can be processed as larger, grouped perceptual units. However, this form of implicit perceptual learning is only improved by stimulus homogeneity when stimulus homogeneity facilitates grouping processes on a dimension that is currently relevant in the task. Copyright © 2014 Elsevier B.V. All rights reserved.

  1. Reward Learning, Neurocognition, Social Cognition, and Symptomatology in Psychosis.

    Science.gov (United States)

    Lewandowski, Kathryn E; Whitton, Alexis E; Pizzagalli, Diego A; Norris, Lesley A; Ongur, Dost; Hall, Mei-Hua

    2016-01-01

    Patients with psychosis spectrum disorders exhibit deficits in social and neurocognition, as well as hallmark abnormalities in motivation and reward processing. Aspects of reward processing may overlap behaviorally and neurobiologically with some elements of cognitive functioning, and abnormalities in these processes may share partially overlapping etiologies in patients. However, whether reward processing and cognition are associated across the psychoses and linked to state and trait clinical symptomatology is unclear. The present study examined associations between cognitive functioning, reward learning, and clinical symptomatology in a cross-diagnostic sample. Patients with schizophrenia (SZ; n = 37), bipolar I disorder with psychosis (BD; n = 42), and healthy controls (n = 29) were assessed for clinical symptoms (patients only), neurocognitive functioning using the MATRICS Battery (MCCB) and reward learning using the probabilistic reward task (PRT). Groups were compared on neurocognition and PRT response bias, and associations between PRT response bias and neurocognition or clinical symptoms were examined controlling for demographic variables and PRT task difficulty (discriminability). Patients with SZ performed worse than controls on most measures of neurocognition; patients with BD exhibited deficits in some domains between the level of patients with SZ and controls. The SZ - but not BD - group exhibited deficits in social cognition compared to controls. Patients and controls did not differ on PRT response bias, but did differ on PRT discriminability. Better response bias across the sample was associated with poorer social cognition, but not neurocognition; conversely, discriminability was associated with neurocognition but not social cognition. Symptoms of psychosis, particularly negative symptoms, were associated with poorer response bias across patient groups. Reward learning was associated with symptoms of psychosis - in particular negative

  2. Adolescent cannabinoid exposure effects on natural reward seeking and learning in rats.

    Science.gov (United States)

    Schoch, H; Huerta, M Y; Ruiz, C M; Farrell, M R; Jung, K M; Huang, J J; Campbell, R R; Piomelli, D; Mahler, S V

    2018-01-01

    Adolescence is characterized by endocannabinoid (ECB)-dependent refinement of neural circuits underlying emotion, learning, and motivation. As a result, adolescent cannabinoid receptor stimulation (ACRS) with phytocannabinoids or synthetic agonists like "Spice" cause robust and persistent changes in both behavior and circuit architecture in rodents, including in reward-related regions like medial prefrontal cortex and nucleus accumbens (NAc). Here, we examine persistent effects of ACRS with the cannabinoid receptor 1/2 specific agonist WIN55-212,2 (WIN; 1.2 mg/kg/day, postnatal day (PD) 30-43), on natural reward-seeking behaviors and ECB system function in adult male Long Evans rats (PD 60+). WIN ACRS increased palatable food intake, and altered attribution of incentive salience to food cues in a sign-/goal-tracking paradigm. ACRS also blunted hunger-induced sucrose intake, and resulted in increased anandamide and oleoylethanolamide levels in NAc after acute food restriction not seen in controls. ACRS did not affect food neophobia or locomotor response to a novel environment, but did increase preference for exploring a novel environment. These results demonstrate that ACRS causes long-term increases in natural reward-seeking behaviors and ECB system function that persist into adulthood, potentially increasing liability to excessive natural reward seeking later in life.

  3. Rewards.

    Science.gov (United States)

    Gunderman, Richard B; Kamer, Aaron P

    2011-05-01

    For much of the 20th century, psychologists and economists operated on the assumption that work is devoid of intrinsic rewards, and the only way to get people to work harder is through the use of rewards and punishments. This so-called carrot-and-stick model of workplace motivation, when applied to medical practice, emphasizes the use of financial incentives and disincentives to manipulate behavior. More recently, however, it has become apparent that, particularly when applied to certain kinds of work, such approaches can be ineffective or even frankly counterproductive. Instead of focusing on extrinsic rewards such as compensation, organizations and their leaders need to devote more attention to the intrinsic rewards of work itself. This article reviews this new understanding of rewards and traces out its practical implications for radiology today. Copyright © 2011. Published by Elsevier Inc.

  4. Examining the durability of incidentally learned trust from gaze cues.

    Science.gov (United States)

    Strachan, James W A; Tipper, Steven P

    2017-10-01

    In everyday interactions we find our attention follows the eye gaze of faces around us. As this cueing is so powerful and difficult to inhibit, gaze can therefore be used to facilitate or disrupt visual processing of the environment, and when we experience this we infer information about the trustworthiness of the cueing face. However, to date no studies have investigated how long these impressions last. To explore this we used a gaze-cueing paradigm where faces consistently demonstrated either valid or invalid cueing behaviours. Previous experiments show that valid faces are subsequently rated as more trustworthy than invalid faces. We replicate this effect (Experiment 1) and then include a brief interference task in Experiment 2 between gaze cueing and trustworthiness rating, which weakens but does not completely eliminate the effect. In Experiment 3, we explore whether greater familiarity with the faces improves the durability of trust learning and find that the effect is more resilient with familiar faces. Finally, in Experiment 4, we push this further and show that evidence of trust learning can be seen up to an hour after cueing has ended. Taken together, our results suggest that incidentally learned trust can be durable, especially for faces that deceive.

  5. The Effect of Reward on Orienting and Reorienting in Exogenous Cueing

    NARCIS (Netherlands)

    Bucker, B.; Theeuwes, J.

    2014-01-01

    It is thought that reward-induced motivation influences perceptual, attentional, and cognitive control processes to facilitate behavioral performance. In this study, we investigated the effect of reward-induced motivation on exogenous attention orienting and inhibition of return (IOR). Attention was

  6. Valence of Facial Cues Influences Sheep Learning in a Visual Discrimination Task

    Directory of Open Access Journals (Sweden)

    Lucille G. A. Bellegarde

    2017-11-01

    Full Text Available Sheep are one of the most studied farm species in terms of their ability to process information from faces, but little is known about their face-based emotion recognition abilities. We investigated (a whether sheep could use images of sheep faces taken in situation of varying valence as cues in a simultaneous discrimination task and (b whether the valence of the situation affects their learning performance. To accomplish this, we photographed faces of sheep in three situations inducing emotional states of neutral (ruminating in the home pen or negative valence (social isolation or aggressive interaction. Sheep (n = 35 first had to learn a discrimination task with colored cards. Animals that reached the learning criterion (n = 16 were then presented with pairs of images of the face of a single individual taken in the neutral situation and in one of the negative situations. Finally, sheep had to generalize what they had learned to new pairs of images of faces taken in the same situation, but of a different conspecific. All sheep that learned the discrimination task with colored cards reached the learning criterion with images of faces. Sheep that had to associate a negative image with a food reward learned faster than sheep that had to associate a neutral image with a reward. With the exception of sheep from the aggression-rewarded group, sheep generalized this discrimination to images of faces of different individuals. Our results suggest that sheep can perceive the emotional valence displayed on faces of conspecifics and that this valence affects learning processes.

  7. Learning from sensory and reward prediction errors during motor adaptation.

    Science.gov (United States)

    Izawa, Jun; Shadmehr, Reza

    2011-03-01

    Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual's relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.

  8. Amygdala subsystems and control of feeding behavior by learned cues.

    Science.gov (United States)

    Petrovich, Gorica D; Gallagher, Michela

    2003-04-01

    A combination of behavioral studies and a neural systems analysis approach has proven fruitful in defining the role of the amygdala complex and associated circuits in fear conditioning. The evidence presented in this chapter suggests that this approach is also informative in the study of other adaptive functions that involve the amygdala. In this chapter we present a novel model to study learning in an appetitive context. Furthermore, we demonstrate that long-recognized connections between the amygdala and the hypothalamus play a crucial role in allowing learning to modulate feeding behavior. In the first part we describe a behavioral model for motivational learning. In this model a cue that acquires motivational properties through pairings with food delivery when an animal is hungry can override satiety and promote eating in sated rats. Next, we present evidence that a specific amygdala subsystem (basolateral area) is responsible for allowing such learned cues to control eating (override satiety and promote eating in sated rats). We also show that basolateral amygdala mediates these actions via connectivity with the lateral hypothalamus. Lastly, we present evidence that the amygdalohypothalamic system is specific for the control of eating by learned motivational cues, as it does not mediate another function that depends on intact basolateral amygdala, namely, the ability of a conditioned cue to support new learning based on its acquired value. Knowledge about neural systems through which food-associated cues specifically control feeding behavior provides a defined model for the study of learning. In addition, this model may be informative for understanding mechanisms of maladaptive aspects of learned control of eating that contribute to eating disorders and more moderate forms of overeating.

  9. Cocaine-associated odor cue re-exposure increases blood oxygenation level dependent signal in memory and reward regions of the maternal rat brain.

    Science.gov (United States)

    Caffrey, Martha K; Febo, Marcelo

    2014-01-01

    Cue triggered relapse during the postpartum period can negatively impact maternal care. Given the high reward value of pups in maternal rats, we designed an fMRI experiment to test whether offspring presence reduces the neural response to a cocaine associated olfactory cue. Cocaine conditioned place preference was carried out before pregnancy in the presence of two distinct odors that were paired with cocaine or saline (+Cue and -Cue). The BOLD response to +Cue and -Cue was measured in dams on postpartum days 2-4. Odor cues were delivered to dams in the absence and then the presence of pups. Our data indicate that several limbic and cognitive regions of the maternal rat brain show a greater BOLD signal response to a +Cue versus -Cue. These include dorsal striatum, prelimbic cortex, parietal cortex, habenula, bed nucleus of stria terminalis, lateral septum and the mediodorsal and the anterior thalamic nucleus. Of the aforementioned brain regions, only the parietal cortex of cocaine treated dams showed a significant modulatory effect of pup presence. In this area of the cortex, cocaine exposed maternal rats showed a greater BOLD activation in response to the +Cue in the presence than in the absence of pups. Specific regions of the cocaine exposed maternal rat brain are strongly reactive to drug associated cues. The regions implicated in cue reactivity have been previously reported in clinical imaging work, and previous work supports their role in various motivational and cognitive functions. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

  10. COCAINE-ASSOCIATED ODOR CUE RE-EXPOSURE INCREASES BLOOD OXYGENATION LEVEL DEPENDENT SIGNAL IN MEMORY AND REWARD REGIONS OF THE MATERNAL RAT BRAIN*

    Science.gov (United States)

    Caffrey, Martha K.; Febo, Marcelo

    2013-01-01

    BACKGROUND Cue triggered relapse during the postpartum period can negatively impact maternal care. Given the high reward value of pups in maternal rats, we designed an fMRI experiment to test whether offspring presence reduces the neural response to a cocaine associated olfactory cue. METHODS Cocaine conditioned place preference was carried out before pregnancy in the presence of two distinct odors that were paired with cocaine or saline (+Cue and −Cue). The BOLD response to +Cue and −Cue was measured in dams on postpartum days 2–4. Odor cues were delivered to dams in the absence and then the presence of pups. RESULTS Our data indicate that several limbic and cognitive regions of the maternal rat brain show a greater BOLD signal response to a +Cue versus −Cue. These include dorsal striatum, prelimbic cortex, parietal cortex, habenula, bed nucleus of stria terminalis, lateral septum and the mediodorsal and the anterior thalamic nucleus. Of the aforementioned brain regions, only the parietal cortex of cocaine treated dams showed a significant modulatory effect of pup presence. In this area of the cortex, cocaine exposed maternal rats showed a greater BOLD activation in response to the +Cue in the presence than in the absence of pups. CONCLUSIONS Specific regions of the cocaine exposed maternal rat brain are strongly reactive to drug associated cues. The regions implicated in cue reactivity have been previously reported in clinical imaging work, and previous work supports their role in various motivational and cognitive functions. PMID:24183499

  11. The attention habit: how reward learning shapes attentional selection.

    Science.gov (United States)

    Anderson, Brian A

    2016-04-01

    There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications. © 2015 New York Academy of Sciences.

  12. Examining the role of social cues in early word learning.

    Science.gov (United States)

    Briganti, Alicia M; Cohen, Leslie B

    2011-02-01

    Infants watched a video of an adult pointing towards two different objects while hearing novel labels. Analyses indicated that 14- and 18-month-olds looked longer at the target object, but only 18-month-olds showed word learning. The results suggest that different types of social cues are available at different ages. Copyright © 2010 Elsevier Inc. All rights reserved.

  13. Cueing and Anxiety in a Visual Concept Learning Task.

    Science.gov (United States)

    Turner, Philip M.

    This study investigated the relationship of two anxiety measures (the State-Trait Anxiety Inventory-Trait Form and the S-R Inventory of Anxiousness-Exam Form) to performance on a visual concept-learning task with embedded criterial information. The effect on anxiety reduction of cueing criterial information was also examined, and two levels of…

  14. Food approach conditioning and discrimination learning using sound cues in benthic sharks.

    Science.gov (United States)

    Vila Pouca, Catarina; Brown, Culum

    2018-07-01

    The marine environment is filled with biotic and abiotic sounds. Some of these sounds predict important events that influence fitness while others are unimportant. Individuals can learn specific sound cues and 'soundscapes' and use them for vital activities such as foraging, predator avoidance, communication and orientation. Most research with sounds in elasmobranchs has focused on hearing thresholds and attractiveness to sound sources, but very little is known about their abilities to learn about sounds, especially in benthic species. Here we investigated if juvenile Port Jackson sharks could learn to associate a musical stimulus with a food reward, discriminate between two distinct musical stimuli, and whether individual personality traits were linked to cognitive performance. Five out of eight sharks were successfully conditioned to associate a jazz song with a food reward delivered in a specific corner of the tank. We observed repeatable individual differences in activity and boldness in all eight sharks, but these personality traits were not linked to the learning performance assays we examined. These sharks were later trained in a discrimination task, where they had to distinguish between the same jazz and a novel classical music song, and swim to opposite corners of the tank according to the stimulus played. The sharks' performance to the jazz stimulus declined to chance levels in the discrimination task. Interestingly, some sharks developed a strong side bias to the right, which in some cases was not the correct side for the jazz stimulus.

  15. The reward of seeing: Different types of visual reward and their ability to modify oculomotor learning.

    Science.gov (United States)

    Meermeier, Annegret; Gremmler, Svenja; Richert, Kerstin; Eckermann, Til; Lappe, Markus

    2017-10-01

    Saccadic adaptation is an oculomotor learning process that maintains the accuracy of eye movements to ensure effective perception of the environment. Although saccadic adaptation is commonly considered an automatic and low-level motor calibration in the cerebellum, we recently found that strength of adaptation is influenced by the visual content of the target: pictures of humans produced stronger adaptation than noise stimuli. This suggests that meaningful images may be considered rewarding or valuable in oculomotor learning. Here we report three experiments that establish the boundaries of this effect. In the first, we tested whether stimuli that were associated with high and low value following long term self-administered reinforcement learning produce stronger adaptation. Twenty-eight expert gamers participated in two sessions of adaptation to game-related high- and low-reward stimuli, but revealed no difference in saccadic adaptation (Bayes Factor01 = 5.49). In the second experiment, we tested whether cognitive (literate) meaning could induce stronger adaptation by comparing targets consisting of words and nonwords. The results of twenty subjects revealed no difference in adaptation strength (Bayes Factor01 = 3.21). The third experiment compared images of human figures to noise patterns for reactive saccades. Twenty-two subjects adapted significantly more toward images of human figures in comparison to noise (p vs. noise), but not secondary, reinforcement affects saccadic adaptation (words vs. nonwords, high- vs. low-value video game images).

  16. Effort-Reward Imbalance for Learning Is Associated with Fatigue in School Children

    Science.gov (United States)

    Fukuda, Sanae; Yamano, Emi; Joudoi, Takako; Mizuno, Kei; Tanaka, Masaaki; Kawatani, Junko; Takano, Miyuki; Tomoda, Akemi; Imai-Matsumura, Kyoko; Miike, Teruhisa; Watanabe, Yasuyoshi

    2010-01-01

    We examined relationships among fatigue, sleep quality, and effort-reward imbalance for learning in school children. We developed an effort-reward for learning scale in school students and examined its reliability and validity. Self-administered surveys, including the effort reward for leaning scale and fatigue scale, were completed by 1,023…

  17. Learning Grammatical Categories from Distributional Cues: Flexible Frames for Language Acquisition

    Science.gov (United States)

    St. Clair, Michelle C.; Monaghan, Padraic; Christiansen, Morten H.

    2010-01-01

    Numerous distributional cues in the child's environment may potentially assist in language learning, but what cues are useful to the child and when are these cues utilised? We propose that the most useful source of distributional cue is a flexible frame surrounding the word, where the language learner integrates information from the preceding and…

  18. Sensitivity for Cues Predicting Reward and Punishment in Young Women with Eating Disorders.

    Science.gov (United States)

    Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline

    2017-11-01

    Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely ignored. This pilot study used a spatial orientation task to examine transdiagnostic and interdiagnostic differences in SR/SP. Participants (14-29 years) were patients with anorexia nervosa of restricting type (AN-R, n = 20), binge/purge ED group [AN of binge/purge type and bulimia nervosa (n = 16)] and non-symptomatic individuals (n = 23). Results revealed stronger difficulties to redirect attention away from signals of rewards in AN-R compared with binge/purge EDs, and binge/purge EDs showed stronger difficulties to direct attention away from signals of punishment compared with AN-R. Findings demonstrate interdiagnostic differences and show that the spatial orientation task is sensitive for individual differences in SP/SR within the context of EDs, thereby sustaining its usefulness as behavioural measure of reinforcement sensitivity. Copyright © 2017 John Wiley & Sons, Ltd and Eating Disorders Association. Copyright © 2017 John Wiley & Sons, Ltd and Eating Disorders Association.

  19. HIV infection results in ventral-striatal reward system hypo-activation during cue processing

    NARCIS (Netherlands)

    Plessis, Stéfan du; Vink, Matthijs; Joska, John A; Koutsilieri, Eleni; Bagadia, Asif; Stein, Dan J; Emsley, Robin

    2015-01-01

    OBJECTIVE: Functional MRI has thus far demonstrated that HIV has an impact on frontal-striatal systems involved in executive functioning. The potential impact of HIV on frontal-striatal systems involved in reward processing has yet to be examined by functional MRI. This study therefore aims to

  20. Dopamine or opioid stimulation of nucleus accumbens similarly amplify cue-triggered 'wanting' for reward: entire core and medial shell mapped as substrates for PIT enhancement.

    Science.gov (United States)

    Peciña, Susana; Berridge, Kent C

    2013-05-01

    Pavlovian cues [conditioned stimulus (CS+)] often trigger intense motivation to pursue and consume related reward [unconditioned stimulus (UCS)]. But cues do not always trigger the same intensity of motivation. Encountering a reward cue can be more tempting on some occasions than on others. What makes the same cue trigger more intense motivation to pursue reward on a particular encounter? The answer may be the level of incentive salience ('wanting') that is dynamically generated by mesocorticolimbic brain systems, influenced especially by dopamine and opioid neurotransmission in the nucleus accumbens (NAc) at that moment. We tested the ability of dopamine stimulation (by amphetamine microinjection) vs. mu opioid stimulation [by d-Ala, nMe-Phe, Glyol-enkephalin (DAMGO) microinjection] of either the core or shell of the NAc to amplify cue-triggered levels of motivation to pursue sucrose reward, measured with a Pavlovian-Instrumental Transfer (PIT) procedure, a relatively pure assay of incentive salience. Cue-triggered 'wanting' in PIT was enhanced by amphetamine or DAMGO microinjections equally, and also equally at nearly all sites throughout the entire core and medial shell (except for a small far-rostral strip of shell). NAc dopamine/opioid stimulations specifically enhanced CS+ ability to trigger phasic peaks of 'wanting' to obtain UCS, without altering baseline efforts when CS+ was absent. We conclude that dopamine/opioid stimulation throughout nearly the entire NAc can causally amplify the reactivity of mesocorticolimbic circuits, and so magnify incentive salience or phasic UCS 'wanting' peaks triggered by a CS+. Mesolimbic amplification of incentive salience may explain why a particular cue encounter can become irresistibly tempting, even when previous encounters were successfully resisted before. © 2013 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.

  1. Human protein status modulates brain reward responses to food cues1–3

    NARCIS (Netherlands)

    Griffioen-Roose, S.; Smeets, P.A.M.; Heuvel, van den E.M.; Boesveldt, S.; Finlayson, G.; Graaf, de C.

    2014-01-01

    Background: Protein is indispensable in the human diet, and its intake appears tightly regulated. The role of sensory attributes of foods in protein intake regulation is far from clear. Objective: We investigated the effect of human protein status on neural responses to different food cues with the

  2. Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology.

    Science.gov (United States)

    Schultz, Wolfram

    2004-04-01

    Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.

  3. Reward-related learning via multiple memory systems.

    Science.gov (United States)

    Delgado, Mauricio R; Dickerson, Kathryn C

    2012-07-15

    The application of a neuroeconomic approach to the study of reward-related processes has provided significant insights in our understanding of human learning and decision making. Much of this research has focused primarily on the contributions of the corticostriatal circuitry, involved in trial-and-error reward learning. As a result, less consideration has been allotted to the potential influence of different neural mechanisms such as the hippocampus or to more common ways in human society in which information is acquired and utilized to reach a decision, such as through explicit instruction rather than trial-and-error learning. This review examines the individual contributions of multiple learning and memory neural systems and their interactions during human decision making in both normal and neuropsychiatric populations. Specifically, the anatomical and functional connectivity across multiple memory systems are highlighted to suggest that probing the role of the hippocampus and its interactions with the corticostriatal circuitry via the application of model-based neuroeconomic approaches may provide novel insights into neuropsychiatric populations that suffer from damage to one of these structures and as a consequence have deficits in learning, memory, or decision making. Copyright © 2012 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

  4. Impulsivity in binge eating disorder: food cues elicit increased reward responses and disinhibition.

    Directory of Open Access Journals (Sweden)

    Kathrin Schag

    Full Text Available BACKGROUND: Binge eating disorder (BED represents a distinct eating disorder diagnosis. Current approaches assume increased impulsivity to be one factor leading to binge eating and weight gain. We used eye tracking to investigate both components of impulsivity, namely reward sensitivity and rash-spontaneous behaviour towards food in BED for the first time. METHODS: Overweight and obese people with BED (BED+; n = 25, without BED (BED-; n = 26 and healthy normal-weight controls (NWC; n = 25 performed a free exploration paradigm measuring reward sensitivity (experiment 1 and a modified antisaccade paradigm measuring disinhibited, rash-spontaneous behaviour (experiment 2 using food and nonfood stimuli. Additionally, trait impulsivity was assessed. RESULTS: In experiment 1, all participants located their initial fixations more often on food stimuli and BED+ participants gazed longer on food stimuli in comparison with BED- and NWC participants. In experiment 2, BED+ participants had more difficulties inhibiting saccades towards food and nonfood stimuli compared with both other groups in first saccades, and especially towards food stimuli in second saccades and concerning sequences of first and second saccades. BED- participants did not differ significantly from NWC participants in both experiments. Additionally, eye tracking performance was associated with self-reported reward responsiveness and self-control. CONCLUSIONS: According to these results, food-related reward sensitivity and rash-spontaneous behaviour, as the two components of impulsivity, are increased in BED in comparison with weight-matched and normal-weight controls. This indicates that BED represents a neurobehavioural phenotype of obesity that is characterised by increased impulsivity. Interventions for BED should target these special needs of affected patients.

  5. Learning stochastic reward distributions in a speeded pointing task.

    Science.gov (United States)

    Seydell, Anna; McCann, Brian C; Trommershäuser, Julia; Knill, David C

    2008-04-23

    Recent studies have shown that humans effectively take into account task variance caused by intrinsic motor noise when planning fast hand movements. However, previous evidence suggests that humans have greater difficulty accounting for arbitrary forms of stochasticity in their environment, both in economic decision making and sensorimotor tasks. We hypothesized that humans can learn to optimize movement strategies when environmental randomness can be experienced and thus implicitly learned over several trials, especially if it mimics the kinds of randomness for which subjects might have generative models. We tested the hypothesis using a task in which subjects had to rapidly point at a target region partly covered by three stochastic penalty regions introduced as "defenders." At movement completion, each defender jumped to a new position drawn randomly from fixed probability distributions. Subjects earned points when they hit the target, unblocked by a defender, and lost points otherwise. Results indicate that after approximately 600 trials, subjects approached optimal behavior. We further tested whether subjects simply learned a set of stimulus-contingent motor plans or the statistics of defenders' movements by training subjects with one penalty distribution and then testing them on a new penalty distribution. Subjects immediately changed their strategy to achieve the same average reward as subjects who had trained with the second penalty distribution. These results indicate that subjects learned the parameters of the defenders' jump distributions and used this knowledge to optimally plan their hand movements under conditions involving stochastic rewards and penalties.

  6. Learning and generalization from reward and punishment in opioid addiction.

    Science.gov (United States)

    Myers, Catherine E; Rego, Janice; Haber, Paul; Morley, Kirsten; Beck, Kevin D; Hogarth, Lee; Moustafa, Ahmed A

    2017-01-15

    This study adapts a widely-used acquired equivalence paradigm to investigate how opioid-addicted individuals learn from positive and negative feedback, and how they generalize this learning. The opioid-addicted group consisted of 33 participants with a history of heroin dependency currently in a methadone maintenance program; the control group consisted of 32 healthy participants without a history of drug addiction. All participants performed a novel variant of the acquired equivalence task, where they learned to map some stimuli to correct outcomes in order to obtain reward, and to map other stimuli to correct outcomes in order to avoid punishment; some stimuli were implicitly "equivalent" in the sense of being paired with the same outcome. On the initial training phase, both groups performed similarly on learning to obtain reward, but as memory load grew, the control group outperformed the addicted group on learning to avoid punishment. On a subsequent testing phase, the addicted and control groups performed similarly on retention trials involving previously-trained stimulus-outcome pairs, as well as on generalization trials to assess acquired equivalence. Since prior work with acquired equivalence tasks has associated stimulus-outcome learning with the nigrostriatal dopamine system, and generalization with the hippocampal region, the current results are consistent with basal ganglia dysfunction in the opioid-addicted patients. Further, a selective deficit in learning from punishment could contribute to processes by which addicted individuals continue to pursue drug use even at the cost of negative consequences such as loss of income and the opportunity to engage in other life activities. Published by Elsevier B.V.

  7. Neural correlates of contextual cueing are modulated by explicit learning.

    Science.gov (United States)

    Westerberg, Carmen E; Miller, Brennan B; Reber, Paul J; Cohen, Neal J; Paller, Ken A

    2011-10-01

    Contextual cueing refers to the facilitated ability to locate a particular visual element in a scene due to prior exposure to the same scene. This facilitation is thought to reflect implicit learning, as it typically occurs without the observer's knowledge that scenes repeat. Unlike most other implicit learning effects, contextual cueing can be impaired following damage to the medial temporal lobe. Here we investigated neural correlates of contextual cueing and explicit scene memory in two participant groups. Only one group was explicitly instructed about scene repetition. Participants viewed a sequence of complex scenes that depicted a landscape with five abstract geometric objects. Superimposed on each object was a letter T or L rotated left or right by 90°. Participants responded according to the target letter (T) orientation. Responses were highly accurate for all scenes. Response speeds were faster for repeated versus novel scenes. The magnitude of this contextual cueing did not differ between the two groups. Also, in both groups repeated scenes yielded reduced hemodynamic activation compared with novel scenes in several regions involved in visual perception and attention, and reductions in some of these areas were correlated with response-time facilitation. In the group given instructions about scene repetition, recognition memory for scenes was superior and was accompanied by medial temporal and more anterior activation. Thus, strategic factors can promote explicit memorization of visual scene information, which appears to engage additional neural processing beyond what is required for implicit learning of object configurations and target locations in a scene. Copyright © 2011 Elsevier Ltd. All rights reserved.

  8. A causal link between prediction errors, dopamine neurons and learning.

    Science.gov (United States)

    Steinberg, Elizabeth E; Keiflin, Ronald; Boivin, Josiah R; Witten, Ilana B; Deisseroth, Karl; Janak, Patricia H

    2013-07-01

    Situations in which rewards are unexpectedly obtained or withheld represent opportunities for new learning. Often, this learning includes identifying cues that predict reward availability. Unexpected rewards strongly activate midbrain dopamine neurons. This phasic signal is proposed to support learning about antecedent cues by signaling discrepancies between actual and expected outcomes, termed a reward prediction error. However, it is unknown whether dopamine neuron prediction error signaling and cue-reward learning are causally linked. To test this hypothesis, we manipulated dopamine neuron activity in rats in two behavioral procedures, associative blocking and extinction, that illustrate the essential function of prediction errors in learning. We observed that optogenetic activation of dopamine neurons concurrent with reward delivery, mimicking a prediction error, was sufficient to cause long-lasting increases in cue-elicited reward-seeking behavior. Our findings establish a causal role for temporally precise dopamine neuron signaling in cue-reward learning, bridging a critical gap between experimental evidence and influential theoretical frameworks.

  9. A Model to Explain the Emergence of Reward Expectancy neurons using Reinforcement Learning and Neural Network

    OpenAIRE

    Shinya, Ishii; Munetaka, Shidara; Katsunari, Shibata

    2006-01-01

    In an experiment of multi-trial task to obtain a reward, reward expectancy neurons,###which responded only in the non-reward trials that are necessary to advance###toward the reward, have been observed in the anterior cingulate cortex of monkeys.###In this paper, to explain the emergence of the reward expectancy neuron in###terms of reinforcement learning theory, a model that consists of a recurrent neural###network trained based on reinforcement learning is proposed. The analysis of the###hi...

  10. Drug versus sweet reward: greater attraction to and preference for sweet versus drug cues.

    Science.gov (United States)

    Madsen, Heather B; Ahmed, Serge H

    2015-05-01

    Despite the unique ability of addictive drugs to directly activate brain reward circuits, recent evidence suggests that drugs induce reinforcing and incentive effects that are comparable to, or even lower than some nondrug rewards. In particular, when rats have a choice between pressing a lever associated with intravenous cocaine or heroin delivery and another lever associated with sweet water delivery, most respond on the latter. This outcome suggests that sweet water is more reinforcing and attractive than either drug. However, this outcome may also be due to the differential ability of sweet versus drug levers to elicit Pavlovian feeding-like conditioned responses that can cause involuntary lever pressing, such as pawing and biting the lever. To test this hypothesis, rats first underwent Pavlovian conditioning to associate one lever with sweet water (0.2% saccharin) and a different lever with intravenous cocaine (0.25 mg) or heroin (0.01 mg). Choice between these two levers was then assessed under two operant choice procedures: one that permitted the expression of Pavlovian-conditioned lever press responses during choice, the other not. During conditioning, Pavlovian-conditioned lever press responses were considerably higher on the sweet lever than on either drug lever, and slightly greater on the heroin lever than on the cocaine lever. Importantly, though these differences in Pavlovian-conditioned behavior predicted subsequent preference for sweet water during choice, they were not required for its expression. Overall, this study confirms that rats prefer the sweet lever because sweet water is more reinforcing and attractive than cocaine or heroin. © 2014 Society for the Study of Addiction.

  11. Pressure to cooperate: is positive reward interdependence really needed in cooperative learning?

    Science.gov (United States)

    Buchs, Céline; Gilles, Ingrid; Dutrévis, Marion; Butera, Fabrizio

    2011-03-01

    BACKGROUND. Despite extensive research on cooperative learning, the debate regarding whether or not its effectiveness depends on positive reward interdependence has not yet found clear evidence. AIMS. We tested the hypothesis that positive reward interdependence, as compared to reward independence, enhances cooperative learning only if learners work on a 'routine task'; if the learners work on a 'true group task', positive reward interdependence induces the same level of learning as reward independence. SAMPLE. The study involved 62 psychology students during regular workshops. METHOD. Students worked on two psychology texts in cooperative dyads for three sessions. The type of task was manipulated through resource interdependence: students worked on either identical (routine task) or complementary (true group task) information. Students expected to be assessed with a Multiple Choice Test (MCT) on the two texts. The MCT assessment type was introduced according to two reward interdependence conditions, either individual (reward independence) or common (positive reward interdependence). A follow-up individual test took place 4 weeks after the third session of dyadic work to examine individual learning. RESULTS. The predicted interaction between the two types of interdependence was significant, indicating that students learned more with positive reward interdependence than with reward independence when they worked on identical information (routine task), whereas students who worked on complementary information (group task) learned the same with or without reward interdependence. CONCLUSIONS. This experiment sheds light on the conditions under which positive reward interdependence enhances cooperative learning, and suggests that creating a real group task allows to avoid the need for positive reward interdependence. © 2010 The British Psychological Society.

  12. Autistic Traits Moderate the Impact of Reward Learning on Social Behaviour.

    Science.gov (United States)

    Panasiti, Maria Serena; Puzzo, Ignazio; Chakrabarti, Bhismadev

    2016-04-01

    A deficit in empathy has been suggested to underlie social behavioural atypicalities in autism. A parallel theoretical account proposes that reduced social motivation (i.e., low responsivity to social rewards) can account for the said atypicalities. Recent evidence suggests that autistic traits modulate the link between reward and proxy metrics related to empathy. Using an evaluative conditioning paradigm to associate high and low rewards with faces, a previous study has shown that individuals high in autistic traits show reduced spontaneous facial mimicry of faces associated with high vs. low reward. This observation raises the possibility that autistic traits modulate the magnitude of evaluative conditioning. To test this, we investigated (a) if autistic traits could modulate the ability to implicitly associate a reward value to a social stimulus (reward learning/conditioning, using the Implicit Association Task, IAT); (b) if the learned association could modulate participants' prosocial behaviour (i.e., social reciprocity, measured using the cyberball task); (c) if the strength of this modulation was influenced by autistic traits. In 43 neurotypical participants, we found that autistic traits moderated the relationship of social reward learning on prosocial behaviour but not reward learning itself. This evidence suggests that while autistic traits do not directly influence social reward learning, they modulate the relationship of social rewards with prosocial behaviour. © 2015 The Authors Autism Research published by Wiley Periodicals, Inc. on behalf of International Society for Autism Research.

  13. Individual differences in personality in laying hens are related to learning a colour cue association.

    Science.gov (United States)

    de Haas, Elske N; Lee, Caroline; Hernandez, Carlos E; Naguib, Marc; Rodenburg, T Bas

    2017-01-01

    Personality can influence how animals perceive and learn cues. The behaviour and physiological responses animals show during stressful events is indicative of their personality. Acute induced stress prior to a cognitive test are known to affect the judgement of a stimulus, but personality of an individual could also affect learning of a specific cognitive paradigm. Here, we assessed if adult laying hens' behaviour and physiological responses, as indicators of their personality, were related to their cognitive performance. We assessed their behavioural responses to a tonic immobility test, an open field test, and a manual restraint test, and measured plasma corticosterone levels after manual restraint. After that, hens (n=20) were trained in a pre-set training schedule to associate a colour-cue with a reward. In a two-choice go-go test, hens needed to choose between a baited or non-baited food container displayed randomly on the left or right side of an arena. Success in learning was related to personality, with better performance of hens which showed a reactive personality type by a long latency to walk, struggle or vocalize during the tests. Only eight out of 20 hens reached the training criteria. The non-learners showed a strong side preference during all training days. Side preferences were strong in hens with high levels of plasma corticosterone and with a long duration of tonic immobility, indicating that fearful, stress-sensitive hens are more prone to develop side biases. Our results show that learning can be hindered by side biases, and fearful animals with a more proactive personality type are more sensitive to develop such biases. Copyright © 2016 Elsevier B.V. All rights reserved.

  14. Implicit Sequence Learning and Contextual Cueing Do Not Compete for Central Cognitive Resources

    Science.gov (United States)

    Jimenez, Luis; Vazquez, Gustavo A.

    2011-01-01

    Sequence learning and contextual cueing explore different forms of implicit learning, arising from practice with a structured serial task, or with a search task with informative contexts. We assess whether these two learning effects arise simultaneously when both remain implicit. Experiments 1 and 2 confirm that a cueing effect can be observed…

  15. Dose Dependent Dopaminergic Modulation of Reward-Based Learning in Parkinson's Disease

    Science.gov (United States)

    van Wouwe, N. C.; Ridderinkhof, K. R.; Band, G. P. H.; van den Wildenberg, W. P. M.; Wylie, S. A.

    2012-01-01

    Learning to select optimal behavior in new and uncertain situations is a crucial aspect of living and requires the ability to quickly associate stimuli with actions that lead to rewarding outcomes. Mathematical models of reinforcement-based learning to select rewarding actions distinguish between (1) the formation of stimulus-action-reward…

  16. Selectivity in associative learning: A cognitive stage framework for blocking and cue competition phenomena

    Directory of Open Access Journals (Sweden)

    Yannick eBoddez

    2014-11-01

    Full Text Available Blocking is the most important phenomenon in the history of associative learning theory: For over 40 years, blocking has inspired a whole generation of learning models. Blocking is part of a family of effects that are typically termed cue competition effects. Common amongst all cue competition effects is that a cue-outcome relation is poorly learned or poorly expressed because the cue is trained in the presence of an alternative predictor or cause of the outcome. We provide an overview of the cognitive processes involved in cue competition effects in humans and propose a stage framework that brings these processes together. The framework contends that the behavioral display of cue competition is cognitively construed following three stages that include (1 an encoding stage, (2 a retention stage, and (3 a performance stage. We argue that the stage framework supports a comprehensive understanding of cue competition effects.

  17. Reward-related brain response and craving correlates of marijuana cue exposure: a preliminary study in treatment-seeking marijuana-dependent subjects.

    Science.gov (United States)

    Goldman, Marina; Szucs-Reed, Regina P; Jagannathan, Kanchana; Ehrman, Ronald N; Wang, Ze; Li, Yin; Suh, Jesse J; Kampman, Kyle; O'Brien, Charles P; Childress, Anna Rose; Franklin, Teresa R

    2013-01-01

    : Determining the brain substrates underlying the motivation to abuse addictive drugs is critical for understanding and treating addictive disorders. Laboratory neuroimaging studies have demonstrated differential activation of limbic and motivational circuitry (eg, amygdala, hippocampus, ventral striatum, insula, and orbitofrontal cortex) triggered by cocaine, heroin, nicotine, and alcohol cues. The literature on neural responses to marijuana cues is sparse. Thus, the goals of this study were to characterize the brain's response to marijuana cues, a major motivator underlying drug use and relapse, and determine whether these responses are linked to self-reported craving in a clinically relevant population of treatment-seeking marijuana-dependent subjects. : Marijuana craving was assessed in 12 marijuana-dependent subjects using the Marijuana Craving Questionnaire-Short Form. Subsequently, blood oxygen level dependent functional magnetic resonance imaging data were acquired during exposure to alternating 20-second blocks of marijuana-related versus matched nondrug visual cues. : Brain activation during marijuana cue exposure was significantly greater in the bilateral amygdala and the hippocampus. Significant positive correlations between craving scores and brain activation were found in the ventral striatum and the medial and lateral orbitofrontal cortex (P cues and craving and extends the current literature on marijuana cue reactivity. Furthermore, the correlative relationship between craving and brain activity in reward-related regions was observed in a clinically relevant sample (treatment-seeking marijuana-dependent subjects). Results are consistent with prior findings in cocaine, heroin, nicotine, and alcohol cue studies, indicating that the brain substrates of cue-triggered drug motivation are shared across abused substances.

  18. Post-learning hippocampal dynamics promote preferential retention of rewarding events

    Science.gov (United States)

    Gruber, Matthias J.; Ritchey, Maureen; Wang, Shao-Fang; Doss, Manoj K.; Ranganath, Charan

    2016-01-01

    Reward motivation is known to modulate memory encoding, and this effect depends on interactions between the substantia nigra/ ventral tegmental area complex (SN/VTA) and the hippocampus. It is unknown, however, whether these interactions influence offline neural activity in the human brain that is thought to promote memory consolidation. Here, we used functional magnetic resonance imaging (fMRI) to test the effect of reward motivation on post-learning neural dynamics and subsequent memory for objects that were learned in high- or low-reward motivation contexts. We found that post-learning increases in resting-state functional connectivity between the SN/VTA and hippocampus predicted preferential retention of objects that were learned in high-reward contexts. In addition, multivariate pattern classification revealed that hippocampal representations of high-reward contexts were preferentially reactivated during post-learning rest, and the number of hippocampal reactivations was predictive of preferential retention of items learned in high-reward contexts. These findings indicate that reward motivation alters offline post-learning dynamics between the SN/VTA and hippocampus, providing novel evidence for a potential mechanism by which reward could influence memory consolidation. PMID:26875624

  19. Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans.

    Science.gov (United States)

    Fouragnan, Elsa; Queirazza, Filippo; Retzler, Chris; Mullinger, Karen J; Philiastides, Marios G

    2017-07-06

    Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.

  20. SOVEREIGN: An autonomous neural system for incrementally learning planned action sequences to navigate towards a rewarded goal.

    Science.gov (United States)

    Gnadt, William; Grossberg, Stephen

    2008-06-01

    How do reactive and planned behaviors interact in real time? How are sequences of such behaviors released at appropriate times during autonomous navigation to realize valued goals? Controllers for both animals and mobile robots, or animats, need reactive mechanisms for exploration, and learned plans to reach goal objects once an environment becomes familiar. The SOVEREIGN (Self-Organizing, Vision, Expectation, Recognition, Emotion, Intelligent, Goal-oriented Navigation) animat model embodies these capabilities, and is tested in a 3D virtual reality environment. SOVEREIGN includes several interacting subsystems which model complementary properties of cortical What and Where processing streams and which clarify similarities between mechanisms for navigation and arm movement control. As the animat explores an environment, visual inputs are processed by networks that are sensitive to visual form and motion in the What and Where streams, respectively. Position-invariant and size-invariant recognition categories are learned by real-time incremental learning in the What stream. Estimates of target position relative to the animat are computed in the Where stream, and can activate approach movements toward the target. Motion cues from animat locomotion can elicit head-orienting movements to bring a new target into view. Approach and orienting movements are alternately performed during animat navigation. Cumulative estimates of each movement are derived from interacting proprioceptive and visual cues. Movement sequences are stored within a motor working memory. Sequences of visual categories are stored in a sensory working memory. These working memories trigger learning of sensory and motor sequence categories, or plans, which together control planned movements. Predictively effective chunk combinations are selectively enhanced via reinforcement learning when the animat is rewarded. Selected planning chunks effect a gradual transition from variable reactive exploratory

  1. Monetary rewards influence retrieval orientations.

    Science.gov (United States)

    Halsband, Teresa M; Ferdinand, Nicola K; Bridger, Emma K; Mecklinger, Axel

    2012-09-01

    Reward anticipation during learning is known to support memory formation, but its role in retrieval processes is so far unclear. Retrieval orientations, as a reflection of controlled retrieval processing, are one aspect of retrieval that might be modulated by reward. These processes can be measured using the event-related potentials (ERPs) elicited by retrieval cues from tasks with different retrieval requirements, such as via changes in the class of targeted memory information. To determine whether retrieval orientations of this kind are modulated by reward during learning, we investigated the effects of high and low reward expectancy on the ERP correlates of retrieval orientation in two separate experiments. The reward manipulation at study in Experiment 1 was associated with later memory performance, whereas in Experiment 2, reward was directly linked to accuracy in the study task. In both studies, the participants encoded mixed lists of pictures and words preceded by high- or low-reward cues. After 24 h, they performed a recognition memory exclusion task, with words as the test items. In addition to a previously reported material-specific effect of retrieval orientation, a frontally distributed, reward-associated retrieval orientation effect was found in both experiments. These findings suggest that reward motivation during learning leads to the adoption of a reward-associated retrieval orientation to support the retrieval of highly motivational information. Thus, ERP retrieval orientation effects not only reflect retrieval processes related to the sought-for materials, but also relate to the reward conditions with which items were combined during encoding.

  2. Tiger salamanders' (Ambystoma tigrinum) response learning and usage of visual cues.

    Science.gov (United States)

    Kundey, Shannon M A; Millar, Roberto; McPherson, Justin; Gonzalez, Maya; Fitz, Aleyna; Allen, Chadbourne

    2016-05-01

    We explored tiger salamanders' (Ambystoma tigrinum) learning to execute a response within a maze as proximal visual cue conditions varied. In Experiment 1, salamanders learned to turn consistently in a T-maze for reinforcement before the maze was rotated. All learned the initial task and executed the trained turn during test, suggesting that they learned to demonstrate the reinforced response during training and continued to perform it during test. In a second experiment utilizing a similar procedure, two visual cues were placed consistently at the maze junction. Salamanders were reinforced for turning towards one cue. Cue placement was reversed during test. All learned the initial task, but executed the trained turn rather than turning towards the visual cue during test, evidencing response learning. In Experiment 3, we investigated whether a compound visual cue could control salamanders' behaviour when it was the only cue predictive of reinforcement in a cross-maze by varying start position and cue placement. All learned to turn in the direction indicated by the compound visual cue, indicating that visual cues can come to control their behaviour. Following training, testing revealed that salamanders attended to stimuli foreground over background features. Overall, these results suggest that salamanders learn to execute responses over learning to use visual cues but can use visual cues if required. Our success with this paradigm offers the potential in future studies to explore salamanders' cognition further, as well as to shed light on how features of the tiger salamanders' life history (e.g. hibernation and metamorphosis) impact cognition.

  3. Monitoring and regulation of learning in medical education: the need for predictive cues.

    Science.gov (United States)

    de Bruin, Anique B H; Dunlosky, John; Cavalcanti, Rodrigo B

    2017-06-01

    Being able to accurately monitor learning activities is a key element in self-regulated learning in all settings, including medical schools. Yet students' ability to monitor their progress is often limited, leading to inefficient use of study time. Interventions that improve the accuracy of students' monitoring can optimise self-regulated learning, leading to higher achievement. This paper reviews findings from cognitive psychology and explores potential applications in medical education, as well as areas for future research. Effective monitoring depends on students' ability to generate information ('cues') that accurately reflects their knowledge and skills. The ability of these 'cues' to predict achievement is referred to as 'cue diagnosticity'. Interventions that improve the ability of students to elicit predictive cues typically fall into two categories: (i) self-generation of cues and (ii) generation of cues that is delayed after self-study. Providing feedback and support is useful when cues are predictive but may be too complex to be readily used. Limited evidence exists about interventions to improve the accuracy of self-monitoring among medical students or trainees. Developing interventions that foster use of predictive cues can enhance the accuracy of self-monitoring, thereby improving self-study and clinical reasoning. First, insight should be gained into the characteristics of predictive cues used by medical students and trainees. Next, predictive cue prompts should be designed and tested to improve monitoring and regulation of learning. Finally, the use of predictive cues should be explored in relation to teaching and learning clinical reasoning. Improving self-regulated learning is important to help medical students and trainees efficiently acquire knowledge and skills necessary for clinical practice. Interventions that help students generate and use predictive cues hold the promise of improved self-regulated learning and achievement. This framework is

  4. Activation in brain energy regulation and reward centers by food cues varies with choice of visual stimulus.

    Science.gov (United States)

    Schur, E A; Kleinhans, N M; Goldberg, J; Buchwald, D; Schwartz, M W; Maravilla, K

    2009-06-01

    To develop a non-invasive method of studying brain mechanisms involved in energy homeostasis and appetite regulation in humans by using visual food cues that are relevant to individuals attempting weight loss. Functional magnetic resonance imaging (fMRI) was used to compare brain activation in regions of interest between groups of food photographs. Ten healthy, non-obese women who were not dieting for weight loss. Independent raters viewed food photographs and evaluated whether the foods depicted should be eaten by individuals attempting a calorically-restricted diet. Based on their responses, we categorized photographs into 'non-fattening' and 'fattening' food groups, the latter characterized by high-caloric content and usually also high-fat or high-sugar content. Blood oxygen level-dependent (BOLD) response was measured by fMRI while participants viewed photographs of 'fattening' food, 'non-fattening' food, and non-food objects. Viewing photographs of fattening food compared with non-food objects resulted in significantly greater activation in the brainstem; hypothalamus; left amygdala; left dorsolateral prefrontal cortex; left orbitofrontal cortex; right insular cortex; bilateral striatum, including the nucleus accumbens, caudate nucleus, and putamen; bilateral thalamus; and occipital lobe. By comparison, only the occipital region had greater activation by non-fattening food than by object photographs. Combining responses to all food types resulted in attenuation of activation in the brainstem, hypothalamus, and striatum. These findings suggest that, in non-obese women, neural circuits engaged in energy homeostasis and reward processing are selectively attuned to representations of high-calorie foods that are perceived as fattening. Studies to investigate hormonal action or manipulation of energy balance may benefit from fMRI protocols that contrast energy-rich food stimuli with non-food or low-calorie food stimuli.

  5. Ventral, but not dorsal, hippocampus inactivation impairs reward memory expression and retrieval in contexts defined by proximal cues.

    Science.gov (United States)

    Riaz, Sadia; Schumacher, Anett; Sivagurunathan, Seyon; Van Der Meer, Matthijs; Ito, Rutsuko

    2017-07-01

    The hippocampus (HPC) has been widely implicated in the contextual control of appetitive and aversive conditioning. However, whole hippocampal lesions do not invariably impair all forms of contextual processing, as in the case of complex biconditional context discrimination, leading to contention over the exact nature of the contribution of the HPC in contextual processing. Moreover, the increasingly well-established functional dissociation between the dorsal (dHPC) and ventral (vHPC) subregions of the HPC has been largely overlooked in the existing literature on hippocampal-based contextual memory processing in appetitively motivated tasks. Thus, the present study sought to investigate the individual roles of the dHPC and the vHPC in contextual biconditional discrimination (CBD) performance and memory retrieval. To this end, we examined the effects of transient post-acquisition pharmacological inactivation (using a combination of GABA A and GABA B receptor agonists muscimol and baclofen) of functionally distinct subregions of the HPC (CA1/CA3 subfields of the dHPC and vHPC) on CBD memory retrieval. Additional behavioral assays including novelty preference, light-dark box and locomotor activity test were also performed to confirm that the respective sites of inactivation were functionally silent. We observed robust deficits in CBD performance and memory retrieval following inactivation of the vHPC, but not the dHPC. Our data provides novel insight into the differential roles of the ventral and dorsal HPC in reward contextual processing, under conditions in which the context is defined by proximal cues. © 2017 Wiley Periodicals, Inc.

  6. No two cues are alike: Depth of learning during infancy is dependent on what orients attention.

    Science.gov (United States)

    Wu, Rachel; Kirkham, Natasha Z

    2010-10-01

    Human infants develop a variety of attentional mechanisms that allow them to extract relevant information from a cluttered multimodal world. We know that both social and nonsocial cues shift infants' attention, but not how these cues differentially affect learning of multimodal events. Experiment 1 used social cues to direct 8- and 4-month-olds' attention to two audiovisual events (i.e., animations of a cat or dog accompanied by particular sounds) while identical distractor events played in another location. Experiment 2 directed 8-month-olds' attention with colorful flashes to the same events. Experiment 3 measured baseline learning without attention cues both with the familiarization and test trials (no cue condition) and with only the test trials (test control condition). The 8-month-olds exposed to social cues showed specific learning of audiovisual events. The 4-month-olds displayed only general spatial learning from social cues, suggesting that specific learning of audiovisual events from social cues may be a function of experience. Infants cued with the colorful flashes looked indiscriminately to both cued locations during test (similar to the 4-month-olds learning from social cues) despite attending for equal duration to the training trials as the 8-month-olds with the social cues. Results from Experiment 3 indicated that the learning effects in Experiments 1 and 2 resulted from exposure to the different cues and multimodal events. We discuss these findings in terms of the perceptual differences and relevance of the cues. Copyright 2010 Elsevier Inc. All rights reserved.

  7. Beyond Rewards

    Science.gov (United States)

    Hall, Philip S.

    2009-01-01

    Using rewards to impact students' behavior has long been common practice. However, using reward systems to enhance student learning conveniently masks the larger and admittedly more difficult task of finding and implementing the structure and techniques that children with special needs require to learn. More important, rewarding the child for good…

  8. Single versus multiple impulse control disorders in Parkinson's disease: an ¹¹C-raclopride positron emission tomography study of reward cue-evoked striatal dopamine release.

    Science.gov (United States)

    Wu, Kit; Politis, Marios; O'Sullivan, Sean S; Lawrence, Andrew D; Warsi, Sarah; Bose, Subrata; Lees, Andrew J; Piccini, Paola

    2015-06-01

    Impulse control disorders (ICDs) are reported in Parkinson's disease (PD) in association with dopaminergic treatment. Approximately 25 % of patients with ICDs have multiple co-occurring ICDs (i.e. more than one diagnosed ICD). The extent to which dopaminergic neurotransmission in PD patients with multiple ICDs differs from those with only one diagnosed ICD is unknown. The aims of this study are: (1) to investigate dopamine neurotransmission in PD patients diagnosed with multiple ICDs, single ICDs and non-ICD controls in response to reward-related visual cues using positron emission tomography with (11)C-raclopride. (2) to compare clinical features of the above three groups. PD individuals with mulitple ICDs (n = 10), single ICD (n = 7) and no ICDs (n = 9) were recruited and underwent two positron emission tomography (PET) scans with (11)C-raclopride: one where they viewed neutral visual cues and the other where they viewed a range of visual cues related to different rewards. Individuals with both multiple ICDs and single ICDs showed significantly greater ventral striatal dopamine release compared to non-ICD PD individuals in response to reward cues, but the two ICD groups did not differ from each other in the extent of dopamine release. Subjects with multiple ICDs were, however, significantly more depressed, and had higher levels of impulsive sensation-seeking compared to subjects with single ICDs and without ICDs. This is the first study to compare dopamine neurotransmission using PET neuroimaging in PD subjects with multiple vs. single ICDs. Our results suggest that striatal dopamine neurotransmission is not directly related to the co-occurrence of ICDs in PD, potentially implicating non-dopaminergic mechanisms linked to depression; and suggest that physicians should be vigilant in managing depression in PD patients with ICDs.

  9. Errorful and errorless learning: The impact of cue-target constraint in learning from errors.

    Science.gov (United States)

    Bridger, Emma K; Mecklinger, Axel

    2014-08-01

    The benefits of testing on learning are well described, and attention has recently turned to what happens when errors are elicited during learning: Is testing nonetheless beneficial, or can errors hinder learning? Whilst recent findings have indicated that tests boost learning even if errors are made on every trial, other reports, emphasizing the benefits of errorless learning, have indicated that errors lead to poorer later memory performance. The possibility that this discrepancy is a function of the materials that must be learned-in particular, the relationship between the cues and targets-was addressed here. Cued recall after either a study-only errorless condition or an errorful learning condition was contrasted across cue-target associations, for which the extent to which the target was constrained by the cue was either high or low. Experiment 1 showed that whereas errorful learning led to greater recall for low-constraint stimuli, it led to a significant decrease in recall for high-constraint stimuli. This interaction is thought to reflect the extent to which retrieval is constrained by the cue-target association, as well as by the presence of preexisting semantic associations. The advantage of errorful retrieval for low-constraint stimuli was replicated in Experiment 2, and the interaction with stimulus type was replicated in Experiment 3, even when guesses were randomly designated as being either correct or incorrect. This pattern provides support for inferences derived from reports in which participants made errors on all learning trials, whilst highlighting the impact of material characteristics on the benefits and disadvantages that accrue from errorful learning in episodic memory.

  10. Evidence for the negative impact of reward on self-regulated learning.

    Science.gov (United States)

    Wehe, Hillary S; Rhodes, Matthew G; Seger, Carol A

    2015-01-01

    The undermining effect refers to the detrimental impact rewards can have on intrinsic motivation to engage in a behaviour. The current study tested the hypothesis that participants' self-regulated learning behaviours are susceptible to the undermining effect. Participants were assigned to learn a set of Swahili-English word pairs. Half of the participants were offered a reward for performance, and half were not offered a reward. After the initial study phase, participants were permitted to continue studying the words during a free period. The results were consistent with an undermining effect: Participants who were not offered a reward spent more time studying the words during the free period. The results suggest that rewards may negatively impact self-regulated learning behaviours and provide support for the encouragement of intrinsic motivation.

  11. Episodic Memory Encoding Interferes with Reward Learning and Decreases Striatal Prediction Errors

    Science.gov (United States)

    Braun, Erin Kendall; Daw, Nathaniel D.

    2014-01-01

    Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making. PMID:25378157

  12. Episodic memory encoding interferes with reward learning and decreases striatal prediction errors.

    Science.gov (United States)

    Wimmer, G Elliott; Braun, Erin Kendall; Daw, Nathaniel D; Shohamy, Daphna

    2014-11-05

    Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making. Copyright © 2014 the authors 0270-6474/14/3414901-12$15.00/0.

  13. Comparing the neural basis of monetary reward and cognitive feedback during information-integration category learning.

    Science.gov (United States)

    Daniel, Reka; Pollmann, Stefan

    2010-01-06

    The dopaminergic system is known to play a central role in reward-based learning (Schultz, 2006), yet it was also observed to be involved when only cognitive feedback is given (Aron et al., 2004). Within the domain of information-integration category learning, in which information from several stimulus dimensions has to be integrated predecisionally (Ashby and Maddox, 2005), the importance of contingent feedback is well established (Maddox et al., 2003). We examined the common neural correlates of reward anticipation and prediction error in this task. Sixteen subjects performed two parallel information-integration tasks within a single event-related functional magnetic resonance imaging session but received a monetary reward only for one of them. Similar functional areas including basal ganglia structures were activated in both task versions. In contrast, a single structure, the nucleus accumbens, showed higher activation during monetary reward anticipation compared with the anticipation of cognitive feedback in information-integration learning. Additionally, this activation was predicted by measures of intrinsic motivation in the cognitive feedback task and by measures of extrinsic motivation in the rewarded task. Our results indicate that, although all other structures implicated in category learning are not significantly affected by altering the type of reward, the nucleus accumbens responds to the positive incentive properties of an expected reward depending on the specific type of the reward.

  14. CLEANing the Reward: Counterfactual Actions to Remove Exploratory Action Noise in Multiagent Learning

    Science.gov (United States)

    HolmesParker, Chris; Taylor, Mathew E.; Tumer, Kagan; Agogino, Adrian

    2014-01-01

    Learning in multiagent systems can be slow because agents must learn both how to behave in a complex environment and how to account for the actions of other agents. The inability of an agent to distinguish between the true environmental dynamics and those caused by the stochastic exploratory actions of other agents creates noise in each agent's reward signal. This learning noise can have unforeseen and often undesirable effects on the resultant system performance. We define such noise as exploratory action noise, demonstrate the critical impact it can have on the learning process in multiagent settings, and introduce a reward structure to effectively remove such noise from each agent's reward signal. In particular, we introduce Coordinated Learning without Exploratory Action Noise (CLEAN) rewards and empirically demonstrate their benefits

  15. Cueing Complex Animations: Does Direction of Attention Foster Learning Processes?

    Science.gov (United States)

    Lowe, Richard; Boucheix, Jean-Michel

    2011-01-01

    The time course of learners' processing of a complex animation was studied using a dynamic diagram of a piano mechanism. Over successive repetitions of the material, two forms of cueing (standard colour cueing and anti-cueing) were administered either before or during the animated segment of the presentation. An uncued group and two other control…

  16. Value and probability coding in a feedback-based learning task utilizing food rewards.

    Science.gov (United States)

    Tricomi, Elizabeth; Lempert, Karolina M

    2015-01-01

    For the consequences of our actions to guide behavior, the brain must represent different types of outcome-related information. For example, an outcome can be construed as negative because an expected reward was not delivered or because an outcome of low value was delivered. Thus behavioral consequences can differ in terms of the information they provide about outcome probability and value. We investigated the role of the striatum in processing probability-based and value-based negative feedback by training participants to associate cues with food rewards and then employing a selective satiety procedure to devalue one food outcome. Using functional magnetic resonance imaging, we examined brain activity related to receipt of expected rewards, receipt of devalued outcomes, omission of expected rewards, omission of devalued outcomes, and expected omissions of an outcome. Nucleus accumbens activation was greater for rewarding outcomes than devalued outcomes, but activity in this region did not correlate with the probability of reward receipt. Activation of the right caudate and putamen, however, was largest in response to rewarding outcomes relative to expected omissions of reward. The dorsal striatum (caudate and putamen) at the time of feedback also showed a parametric increase correlating with the trialwise probability of reward receipt. Our results suggest that the ventral striatum is sensitive to the motivational relevance, or subjective value, of the outcome, while the dorsal striatum codes for a more complex signal that incorporates reward probability. Value and probability information may be integrated in the dorsal striatum, to facilitate action planning and allocation of effort. Copyright © 2015 the American Physiological Society.

  17. Diurnal rhythms in psychological reward functioning in healthy young men: 'Wanting', liking, and learning.

    Science.gov (United States)

    Byrne, Jamie E M; Murray, Greg

    2017-01-01

    A range of evidence suggests that human reward functioning is partly driven by the endogenous circadian system, generating 24-hour rhythms in behavioural measures of reward activation. Reward functioning is multifaceted but literature to date is largely limited to measures of self-reported positive mood states. The aim of this study was to advance the field by testing for hypothesised diurnal variation in previously unexplored components of psychological reward: 'wanting', liking, and learning using subjective and behavioural measures. Risky decision making (automatic Balloon Analogue Risk Task), affective responsivity to positive images (International Affective Pictures System), uncued self-reported discrete emotions, and learning-contingent reward (Iowa Gambling Task) were measured at 10.00 hours, 14.00 hours, and 19.00 hours in a counterbalanced repeated measures design with 50 healthy male participants (aged 18-30). As hypothesised, risky decision making (unconscious 'wanting') and ratings of arousal towards positive images (conscious wanting) exhibited a diurnal waveform with indices highest at 14.00 hours. No diurnal rhythm was observed for liking (pleasure ratings to positive images, discrete uncued positive emotions) or in a learning-contingent reward task. Findings reaffirm that diurnal variation in human reward functioning is most pronounced in the motivational 'wanting' components of reward.

  18. Learning to Cooperate: The Evolution of Social Rewards in Repeated Interactions.

    Science.gov (United States)

    Dridi, Slimane; Akçay, Erol

    2018-01-01

    Understanding the behavioral and psychological mechanisms underlying social behaviors is one of the major goals of social evolutionary theory. In particular, a persistent question about animal cooperation is to what extent it is supported by other-regarding preferences-the motivation to increase the welfare of others. In many situations, animals adjust their behaviors through learning by responding to the rewards they experience as a consequence of their actions. Therefore, we may ask whether learning in social situations can be driven by evolved other-regarding rewards. Here we develop a mathematical model in order to ask whether the mere act of cooperating with a social partner will evolve to be inherently rewarding. Individuals interact repeatedly in pairs and adjust their behaviors through reinforcement learning. We assume that individuals associate with each game outcome an internal reward value. These perceived rewards are genetically evolving traits. We find that conditionally cooperative rewards that value mutual cooperation positively but the sucker's outcome negatively tend to be evolutionarily stable. Purely other-regarding rewards can evolve only under special parameter combinations. On the other hand, selfish rewards that always lead to pure defection are also evolutionarily successful. These findings are consistent with empirical observations showing that humans tend to display conditionally cooperative behavior and also exhibit a diversity of preferences. Our model also demonstrates the need to further integrate multiple levels of biological causation of behavior.

  19. Nonparametric bayesian reward segmentation for skill discovery using inverse reinforcement learning

    CSIR Research Space (South Africa)

    Ranchod, P

    2015-10-01

    Full Text Available We present a method for segmenting a set of unstructured demonstration trajectories to discover reusable skills using inverse reinforcement learning (IRL). Each skill is characterised by a latent reward function which the demonstrator is assumed...

  20. Changes in expression of c-Fos protein following cocaine-cue extinction learning.

    Science.gov (United States)

    Nic Dhonnchadha, B Á; Lovascio, B F; Shrestha, N; Lin, A; Leite-Morris, K A; Man, H Y; Kaplan, G B; Kantak, K M

    2012-09-01

    Extinguishing abnormally strengthened learned responses to cues associated with drugs of abuse remains a key tactic for alleviating addiction. To assist in developing pharmacotherapies to augment exposure therapy for relapse prevention, investigation into neurobiological underpinnings of drug-cue extinction learning is needed. We used regional analyses of c-Fos and GluR2 protein expression to delineate neural activity and plasticity that may be associated with cocaine-cue extinction learning. Rats were trained to self-administer cocaine paired with a light cue, and later underwent a single 2h extinction session for which cocaine was withheld but response-contingent cues were presented (cocaine-cue extinction). Control groups consisted of rats yoked to animals self-administering cocaine and receiving saline non-contingently followed by an extinction session, or rats trained to self-administer cocaine followed by a no-extinction session for which levers were retracted, and cocaine and cues were withheld. Among 11 brain sites examined, extinction training increased c-Fos expression in basolateral amygdala and prelimbic prefrontal cortex of cocaine-cue extinguished rats relative to both control conditions. In dorsal subiculum and infralimbic prefrontal cortex, extinction training increased c-Fos expression in both cocaine-cue and saline-cue extinguished rats relative to the no-extinction control condition. GluR2 protein expression was not altered in any site examined after extinction or control training. Findings suggest that basolateral amygdala and prelimbic prefrontal cortex neurons are activated during acquisition of cocaine-cue extinction learning, a process that is independent of changes in GluR2 abundance. Other sites are implicated in processing the significance of cues that are present early in extinction training. Copyright © 2012 Elsevier B.V. All rights reserved.

  1. A neurogenetic dissociation between punishment-, reward- and relief-learning in Drosophila

    Directory of Open Access Journals (Sweden)

    ayse Yarali

    2010-12-01

    Full Text Available What is particularly worth remembering about a traumatic experience is what brought it about, and what made it cease. For example, fruit flies avoid an odour which during training had preceded electric shock punishment; on the other hand, if the odour had followed shock during training, it is later on approached as a signal for the relieving end of shock. We provide a neurogenetic analysis of such relief learning. Blocking, using UAS-shibirets1, the output from a particular set of dopaminergic neurons defined by the TH-Gal4 driver partially impaired punishment learning, but left relief learning intact. Thus, with respect to these particular neurons, relief learning differs from punishment learning. Targeting another set of dopaminergic/ serotonergic neurons defined by the DDC-Gal4 driver on the other hand affected neither punishment nor relief learning. As for the octopaminergic system, the tbhM18 mutation, compromising octopamine biosynthesis, partially impaired sugar-reward learning, but not relief learning. Thus, with respect to this particular mutation, relief learning and reward learning are dissociated. Finally, blocking output from the set of octopaminergic/ tyraminergic neurons defined by the TDC2-Gal4 driver affected neither reward, nor relief learning. We conclude that regarding the used genetic tools, relief learning is neurogenetically dissociated from both punishment and reward learning.

  2. Multi-layer network utilizing rewarded spike time dependent plasticity to learn a foraging task.

    Directory of Open Access Journals (Sweden)

    Pavel Sanda

    2017-09-01

    Full Text Available Neural networks with a single plastic layer employing reward modulated spike time dependent plasticity (STDP are capable of learning simple foraging tasks. Here we demonstrate advanced pattern discrimination and continuous learning in a network of spiking neurons with multiple plastic layers. The network utilized both reward modulated and non-reward modulated STDP and implemented multiple mechanisms for homeostatic regulation of synaptic efficacy, including heterosynaptic plasticity, gain control, output balancing, activity normalization of rewarded STDP and hard limits on synaptic strength. We found that addition of a hidden layer of neurons employing non-rewarded STDP created neurons that responded to the specific combinations of inputs and thus performed basic classification of the input patterns. When combined with a following layer of neurons implementing rewarded STDP, the network was able to learn, despite the absence of labeled training data, discrimination between rewarding patterns and the patterns designated as punishing. Synaptic noise allowed for trial-and-error learning that helped to identify the goal-oriented strategies which were effective in task solving. The study predicts a critical set of properties of the spiking neuronal network with STDP that was sufficient to solve a complex foraging task involving pattern classification and decision making.

  3. A Molecular Dissociation between Cued and Contextual Appetitive Learning

    Science.gov (United States)

    Kheirbek, Mazen A.; Beeler, Jeff A.; Chi, Wanhao; Ishikawa, Yoshihiro; Zhuang, Xiaoxi

    2010-01-01

    In appetitive Pavlovian learning, animals learn to associate discrete cues or environmental contexts with rewarding outcomes, and these cues and/or contexts can potentiate an ongoing instrumental response for reward. Although anatomical substrates underlying cued and contextual learning have been proposed, it remains unknown whether specific…

  4. Learned helplessness and learned prevalence: exploring the causal relations among perceived controllability, reward prevalence, and exploration.

    Science.gov (United States)

    Teodorescu, Kinneret; Erev, Ido

    2014-10-01

    Exposure to uncontrollable outcomes has been found to trigger learned helplessness, a state in which the agent, because of lack of exploration, fails to take advantage of regained control. Although the implications of this phenomenon have been widely studied, its underlying cause remains undetermined. One can learn not to explore because the environment is uncontrollable, because the average reinforcement for exploring is low, or because rewards for exploring are rare. In the current research, we tested a simple experimental paradigm that contrasts the predictions of these three contributors and offers a unified psychological mechanism that underlies the observed phenomena. Our results demonstrate that learned helplessness is not correlated with either the perceived controllability of one's environment or the average reward, which suggests that reward prevalence is a better predictor of exploratory behavior than the other two factors. A simple computational model in which exploration decisions were based on small samples of past experiences captured the empirical phenomena while also providing a cognitive basis for feelings of uncontrollability. © The Author(s) 2014.

  5. Enhancing a Multi-body Mechanism with Learning-Aided Cues in an Augmented Reality Environment

    International Nuclear Information System (INIS)

    Sidhu, Manjit Singh

    2013-01-01

    Augmented Reality (AR) is a potential area of research for education, covering issues such as tracking and calibration, and realistic rendering of virtual objects. The ability to augment real world with virtual information has opened the possibility of using AR technology in areas such as education and training as well. In the domain of Computer Aided Learning (CAL), researchers have long been looking into enhancing the effectiveness of the teaching and learning process by providing cues that could assist learners to better comprehend the materials presented. Although a number of works were done looking into the effectiveness of learning-aided cues, but none has really addressed this issue for AR-based learning solutions. This paper discusses the design and model of an AR based software that uses visual cues to enhance the learning process and the outcome perception results of the cues.

  6. Enhancing a Multi-body Mechanism with Learning-Aided Cues in an Augmented Reality Environment

    Science.gov (United States)

    Singh Sidhu, Manjit

    2013-06-01

    Augmented Reality (AR) is a potential area of research for education, covering issues such as tracking and calibration, and realistic rendering of virtual objects. The ability to augment real world with virtual information has opened the possibility of using AR technology in areas such as education and training as well. In the domain of Computer Aided Learning (CAL), researchers have long been looking into enhancing the effectiveness of the teaching and learning process by providing cues that could assist learners to better comprehend the materials presented. Although a number of works were done looking into the effectiveness of learning-aided cues, but none has really addressed this issue for AR-based learning solutions. This paper discusses the design and model of an AR based software that uses visual cues to enhance the learning process and the outcome perception results of the cues.

  7. Do reward-processing deficits in schizophrenia-spectrum disorders promote cannabis use? An investigation of physiological response to natural rewards and drug cues

    Science.gov (United States)

    Cassidy, Clifford M.; Brodeur, Mathieu B.; Lepage, Martin; Malla, Ashok

    2014-01-01

    Background Dysfunctional reward processing is present in individuals with schizophrenia-spectrum disorders (SSD) and may confer vulnerability to addiction. Our objective was to identify a deficit in patients with SSD on response to rewarding stimuli and determine whether this deficit predicts cannabis use. Methods We divided a group of patients with SSD and nonpsychotic controls into cannabis users and nonusers. Response to emotional and cannabis-associated visual stimuli was assessed using self-report, event-related potentials (using the late positive potential [LPP]), facial electromyography and skin-conductance response. Results Our sample comprised 35 patients with SSD and 35 nonpsychotic controls. Compared with controls, the patients with SSD showed blunted LPP response to pleasant stimuli (p = 0.003). Across measures, cannabis-using controls showed greater response to pleasant stimuli than to cannabis stimuli whereas cannabis-using patients showed little bias toward pleasant stimuli. Reduced LPP response to pleasant stimuli was predictive of more frequent subsequent cannabis use (β = −0.24, p = 0.034). Limitations It is not clear if the deficit associated with cannabis use is specific to rewarding stimuli or nonspecific to any kind of emotionally salient stimuli. Conclusion The LPP captures a reward-processing deficit in patients with SSD and shows potential as a biomarker for identifying patients at risk of heavy cannabis use. PMID:24913137

  8. Learning to Match Auditory and Visual Speech Cues: Social Influences on Acquisition of Phonological Categories

    Science.gov (United States)

    Altvater-Mackensen, Nicole; Grossmann, Tobias

    2015-01-01

    Infants' language exposure largely involves face-to-face interactions providing acoustic and visual speech cues but also social cues that might foster language learning. Yet, both audiovisual speech information and social information have so far received little attention in research on infants' early language development. Using a preferential…

  9. An explicit statistical model of learning lexical segmentation using multiple cues

    NARCIS (Netherlands)

    Çöltekin, Ça ̆grı; Nerbonne, John; Lenci, Alessandro; Padró, Muntsa; Poibeau, Thierry; Villavicencio, Aline

    2014-01-01

    This paper presents an unsupervised and incremental model of learning segmentation that combines multiple cues whose use by children and adults were attested by experimental studies. The cues we exploit in this study are predictability statistics, phonotactics, lexical stress and partial lexical

  10. Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning.

    Science.gov (United States)

    Marsh, Brandi T; Tarigoppula, Venkata S Aditya; Chen, Chen; Francis, Joseph T

    2015-05-13

    For decades, neurophysiologists have worked on elucidating the function of the cortical sensorimotor control system from the standpoint of kinematics or dynamics. Recently, computational neuroscientists have developed models that can emulate changes seen in the primary motor cortex during learning. However, these simulations rely on the existence of a reward-like signal in the primary sensorimotor cortex. Reward modulation of the primary sensorimotor cortex has yet to be characterized at the level of neural units. Here we demonstrate that single units/multiunits and local field potentials in the primary motor (M1) cortex of nonhuman primates (Macaca radiata) are modulated by reward expectation during reaching movements and that this modulation is present even while subjects passively view cursor motions that are predictive of either reward or nonreward. After establishing this reward modulation, we set out to determine whether we could correctly classify rewarding versus nonrewarding trials, on a moment-to-moment basis. This reward information could then be used in collaboration with reinforcement learning principles toward an autonomous brain-machine interface. The autonomous brain-machine interface would use M1 for both decoding movement intention and extraction of reward expectation information as evaluative feedback, which would then update the decoding algorithm as necessary. In the work presented here, we show that this, in theory, is possible. Copyright © 2015 the authors 0270-6474/15/357374-14$15.00/0.

  11. Scaling prediction errors to reward variability benefits error-driven learning in humans.

    Science.gov (United States)

    Diederen, Kelly M J; Schultz, Wolfram

    2015-09-01

    Effective error-driven learning requires individuals to adapt learning to environmental reward variability. The adaptive mechanism may involve decays in learning rate across subsequent trials, as shown previously, and rescaling of reward prediction errors. The present study investigated the influence of prediction error scaling and, in particular, the consequences for learning performance. Participants explicitly predicted reward magnitudes that were drawn from different probability distributions with specific standard deviations. By fitting the data with reinforcement learning models, we found scaling of prediction errors, in addition to the learning rate decay shown previously. Importantly, the prediction error scaling was closely related to learning performance, defined as accuracy in predicting the mean of reward distributions, across individual participants. In addition, participants who scaled prediction errors relative to standard deviation also presented with more similar performance for different standard deviations, indicating that increases in standard deviation did not substantially decrease "adapters'" accuracy in predicting the means of reward distributions. However, exaggerated scaling beyond the standard deviation resulted in impaired performance. Thus efficient adaptation makes learning more robust to changing variability. Copyright © 2015 the American Physiological Society.

  12. Combining Correlation-Based and Reward-Based Learning in Neural Control for Policy Improvement

    DEFF Research Database (Denmark)

    Manoonpong, Poramate; Kolodziejski, Christoph; Wörgötter, Florentin

    2013-01-01

    Classical conditioning (conventionally modeled as correlation-based learning) and operant conditioning (conventionally modeled as reinforcement learning or reward-based learning) have been found in biological systems. Evidence shows that these two mechanisms strongly involve learning about...... associations. Based on these biological findings, we propose a new learning model to achieve successful control policies for artificial systems. This model combines correlation-based learning using input correlation learning (ICO learning) and reward-based learning using continuous actor–critic reinforcement...... learning (RL), thereby working as a dual learner system. The model performance is evaluated by simulations of a cart-pole system as a dynamic motion control problem and a mobile robot system as a goal-directed behavior control problem. Results show that the model can strongly improve pole balancing control...

  13. A Plant Control Technology Using Reinforcement Learning Method with Automatic Reward Adjustment

    Science.gov (United States)

    Eguchi, Toru; Sekiai, Takaaki; Yamada, Akihiro; Shimizu, Satoru; Fukai, Masayuki

    A control technology using Reinforcement Learning (RL) and Radial Basis Function (RBF) Network has been developed to reduce environmental load substances exhausted from power and industrial plants. This technology consists of the statistic model using RBF Network, which estimates characteristics of plants with respect to environmental load substances, and RL agent, which learns the control logic for the plants using the statistic model. In this technology, it is necessary to design an appropriate reward function given to the agent immediately according to operation conditions and control goals to control plants flexibly. Therefore, we propose an automatic reward adjusting method of RL for plant control. This method adjusts the reward function automatically using information of the statistic model obtained in its learning process. In the simulations, it is confirmed that the proposed method can adjust the reward function adaptively for several test functions, and executes robust control toward the thermal power plant considering the change of operation conditions and control goals.

  14. From prediction error to incentive salience: mesolimbic computation of reward motivation

    Science.gov (United States)

    Berridge, Kent C.

    2011-01-01

    Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I will discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g., drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously-learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus a consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To comprehend these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience. PMID:22487042

  15. Macaque monkeys can learn token values from human models through vicarious reward.

    Science.gov (United States)

    Bevacqua, Sara; Cerasti, Erika; Falcone, Rossella; Cervelloni, Milena; Brunamonti, Emiliano; Ferraina, Stefano; Genovesio, Aldo

    2013-01-01

    Monkeys can learn the symbolic meaning of tokens, and exchange them to get a reward. Monkeys can also learn the symbolic value of a token by observing conspecifics but it is not clear if they can learn passively by observing other actors, e.g., humans. To answer this question, we tested two monkeys in a token exchange paradigm in three experiments. Monkeys learned token values through observation of human models exchanging them. We used, after a phase of object familiarization, different sets of tokens. One token of each set was rewarded with a bit of apple. Other tokens had zero value (neutral tokens). Each token was presented only in one set. During the observation phase, monkeys watched the human model exchange tokens and watched them consume rewards (vicarious rewards). In the test phase, the monkeys were asked to exchange one of the tokens for food reward. Sets of three tokens were used in the first experiment and sets of two tokens were used in the second and third experiments. The valuable token was presented with different probabilities in the observation phase during the first and second experiments in which the monkeys exchanged the valuable token more frequently than any of the neutral tokens. The third experiments examined the effect of unequal probabilities. Our results support the view that monkeys can learn from non-conspecific actors through vicarious reward, even a symbolic task like the token-exchange task.

  16. Influence of cue word perceptual information on metamemory accuracy in judgement of learning.

    Science.gov (United States)

    Hu, Xiao; Liu, Zhaomin; Li, Tongtong; Luo, Liang

    2016-01-01

    Previous studies have suggested that perceptual information regarding to-be-remembered words in the study phase affects the accuracy of judgement of learning (JOL). However, few have investigated whether the perceptual information in the JOL phase influences JOL accuracy. This study examined the influence of cue word perceptual information in the JOL phase on immediate and delayed JOL accuracy through changes in cue word font size. In Experiment 1, large-cue word pairs had significantly higher mean JOL magnitude than small-cue word pairs in immediate JOLs and higher relative accuracy than small-cue pairs in delayed JOLs, but font size had no influence on recall performance. Experiment 2 increased the JOL time, and mean JOL magnitude did not reliably differ for large-cue compared with small-cue pairs in immediate JOLs. However, the influence on relative accuracy still existed in delayed JOLs. Experiment 3 increased the familiarity of small-cue words in the delayed JOL phase by adding a lexical decision task. The results indicated that cue word font size no longer affected relative accuracy in delayed JOLs. The three experiments in our study indicated that the perceptual information regarding cue words in the JOL phase affects immediate and delayed JOLs in different ways.

  17. Dopamine selectively remediates 'model-based' reward learning: a computational approach.

    Science.gov (United States)

    Sharp, Madeleine E; Foerde, Karin; Daw, Nathaniel D; Shohamy, Daphna

    2016-02-01

    Patients with loss of dopamine due to Parkinson's disease are impaired at learning from reward. However, it remains unknown precisely which aspect of learning is impaired. In particular, learning from reward, or reinforcement learning, can be driven by two distinct computational processes. One involves habitual stamping-in of stimulus-response associations, hypothesized to arise computationally from 'model-free' learning. The other, 'model-based' learning, involves learning a model of the world that is believed to support goal-directed behaviour. Much work has pointed to a role for dopamine in model-free learning. But recent work suggests model-based learning may also involve dopamine modulation, raising the possibility that model-based learning may contribute to the learning impairment in Parkinson's disease. To directly test this, we used a two-step reward-learning task which dissociates model-free versus model-based learning. We evaluated learning in patients with Parkinson's disease tested ON versus OFF their dopamine replacement medication and in healthy controls. Surprisingly, we found no effect of disease or medication on model-free learning. Instead, we found that patients tested OFF medication showed a marked impairment in model-based learning, and that this impairment was remediated by dopaminergic medication. Moreover, model-based learning was positively correlated with a separate measure of working memory performance, raising the possibility of common neural substrates. Our results suggest that some learning deficits in Parkinson's disease may be related to an inability to pursue reward based on complete representations of the environment. © The Author (2015). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

  18. Modulation of spatial attention by goals, statistical learning, and monetary reward.

    Science.gov (United States)

    Jiang, Yuhong V; Sha, Li Z; Remington, Roger W

    2015-10-01

    This study documented the relative strength of task goals, visual statistical learning, and monetary reward in guiding spatial attention. Using a difficult T-among-L search task, we cued spatial attention to one visual quadrant by (i) instructing people to prioritize it (goal-driven attention), (ii) placing the target frequently there (location probability learning), or (iii) associating that quadrant with greater monetary gain (reward-based attention). Results showed that successful goal-driven attention exerted the strongest influence on search RT. Incidental location probability learning yielded a smaller though still robust effect. Incidental reward learning produced negligible guidance for spatial attention. The 95 % confidence intervals of the three effects were largely nonoverlapping. To understand these results, we simulated the role of location repetition priming in probability cuing and reward learning. Repetition priming underestimated the strength of location probability cuing, suggesting that probability cuing involved long-term statistical learning of how to shift attention. Repetition priming provided a reasonable account for the negligible effect of reward on spatial attention. We propose a multiple-systems view of spatial attention that includes task goals, search habit, and priming as primary drivers of top-down attention.

  19. Rule Learning in Autism: The Role of Reward Type and Social Context

    OpenAIRE

    Jones, E. J. H.; Webb, S. J.; Estes, A.; Dawson, G.

    2013-01-01

    Learning abstract rules is central to social and cognitive development. Across two experiments, we used Delayed Non-Matching to Sample tasks to characterize the longitudinal development and nature of rule-learning impairments in children with Autism Spectrum Disorder (ASD). Results showed that children with ASD consistently experienced more difficulty learning an abstract rule from a discrete physical reward than children with DD. Rule learning was facilitated by the provision of more concret...

  20. Contingency learning in alcohol dependence and pathological gambling: learning and unlearning reward contingencies

    NARCIS (Netherlands)

    Vanes, Lucy D.; van Holst, Ruth J.; Jansen, Jochem M.; van den Brink, Wim; Oosterlaan, Jaap; Goudriaan, Anna E.

    2014-01-01

    Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex mix of

  1. Contingency learning in alcohol dependence and pathological gambling: learning and unlearning reward contingencies

    NARCIS (Netherlands)

    Vanes, L.D.; Holst, R.J. van; Jansen, J.M.; Brink, W. van den; Oosterlaan, J.; Goudriaan, A.E.

    2014-01-01

    BACKGROUND: Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex

  2. Contingency Learning in Alcohol Dependence and Pathological Gambling: Learning and Unlearning Reward Contingencies

    NARCIS (Netherlands)

    Vanes, L.D.; Holst, R.; Jansen, J.D.; van den Brink, W.A.; Oosterlaan, J.; Goudriaan, A.E.

    2014-01-01

    Background: Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex

  3. A Computer-Assisted Learning Model Based on the Digital Game Exponential Reward System

    Science.gov (United States)

    Moon, Man-Ki; Jahng, Surng-Gahb; Kim, Tae-Yong

    2011-01-01

    The aim of this research was to construct a motivational model which would stimulate voluntary and proactive learning using digital game methods offering players more freedom and control. The theoretical framework of this research lays the foundation for a pedagogical learning model based on digital games. We analyzed the game reward system, which…

  4. Curiosity and reward: Valence predicts choice and information prediction errors enhance learning.

    Science.gov (United States)

    Marvin, Caroline B; Shohamy, Daphna

    2016-03-01

    Curiosity drives many of our daily pursuits and interactions; yet, we know surprisingly little about how it works. Here, we harness an idea implied in many conceptualizations of curiosity: that information has value in and of itself. Reframing curiosity as the motivation to obtain reward-where the reward is information-allows one to leverage major advances in theoretical and computational mechanisms of reward-motivated learning. We provide new evidence supporting 2 predictions that emerge from this framework. First, we find an asymmetric effect of positive versus negative information, with positive information enhancing both curiosity and long-term memory for information. Second, we find that it is not the absolute value of information that drives learning but, rather, the gap between the reward expected and reward received, an "information prediction error." These results support the idea that information functions as a reward, much like money or food, guiding choices and driving learning in systematic ways. (c) 2016 APA, all rights reserved).

  5. The role of reward in word learning and its implications for language acquisition.

    Science.gov (United States)

    Ripollés, Pablo; Marco-Pallarés, Josep; Hielscher, Ulrike; Mestres-Missé, Anna; Tempelmann, Claus; Heinze, Hans-Jochen; Rodríguez-Fornells, Antoni; Noesselt, Toemme

    2014-11-03

    The exact neural processes behind humans' drive to acquire a new language--first as infants and later as second-language learners--are yet to be established. Recent theoretical models have proposed that during human evolution, emerging language-learning mechanisms might have been glued to phylogenetically older subcortical reward systems, reinforcing human motivation to learn a new language. Supporting this hypothesis, our results showed that adult participants exhibited robust fMRI activation in the ventral striatum (VS)--a core region of reward processing--when successfully learning the meaning of new words. This activation was similar to the VS recruitment elicited using an independent reward task. Moreover, the VS showed enhanced functional and structural connectivity with neocortical language areas during successful word learning. Together, our results provide evidence for the neural substrate of reward and motivation during word learning. We suggest that this strong functional and anatomical coupling between neocortical language regions and the subcortical reward system provided a crucial advantage in humans that eventually enabled our lineage to successfully acquire linguistic skills. Copyright © 2014 Elsevier Ltd. All rights reserved.

  6. Spared internal but impaired external reward prediction error signals in major depressive disorder during reinforcement learning.

    Science.gov (United States)

    Bakic, Jasmina; Pourtois, Gilles; Jepma, Marieke; Duprat, Romain; De Raedt, Rudi; Baeken, Chris

    2017-01-01

    Major depressive disorder (MDD) creates debilitating effects on a wide range of cognitive functions, including reinforcement learning (RL). In this study, we sought to assess whether reward processing as such, or alternatively the complex interplay between motivation and reward might potentially account for the abnormal reward-based learning in MDD. A total of 35 treatment resistant MDD patients and 44 age matched healthy controls (HCs) performed a standard probabilistic learning task. RL was titrated using behavioral, computational modeling and event-related brain potentials (ERPs) data. MDD patients showed comparable learning rate compared to HCs. However, they showed decreased lose-shift responses as well as blunted subjective evaluations of the reinforcers used during the task, relative to HCs. Moreover, MDD patients showed normal internal (at the level of error-related negativity, ERN) but abnormal external (at the level of feedback-related negativity, FRN) reward prediction error (RPE) signals during RL, selectively when additional efforts had to be made to establish learning. Collectively, these results lend support to the assumption that MDD does not impair reward processing per se during RL. Instead, it seems to alter the processing of the emotional value of (external) reinforcers during RL, when additional intrinsic motivational processes have to be engaged. © 2016 Wiley Periodicals, Inc.

  7. Subliminal Cues While Teaching: HCI Technique for Enhanced Learning

    Directory of Open Access Journals (Sweden)

    Pierre Chalfoun

    2011-01-01

    Full Text Available This paper presents results from an empirical study conducted with a subliminal teaching technique aimed at enhancing learner's performance in Intelligent Systems through the use of physiological sensors. This technique uses carefully designed subliminal cues (positive and miscues (negative and projects them under the learner's perceptual visual threshold. A positive cue, called answer cue, is a hint aiming to enhance the learner's inductive reasoning abilities and projected in a way to help them figure out the solution faster but more importantly better. A negative cue, called miscue, is also used and aims at obviously at the opposite (distract the learner or lead them to the wrong conclusion. The latest obtained results showed that only subliminal cues, not miscues, could significantly increase learner performance and intuition in a logic-based problem-solving task. Nonintrusive physiological sensors (EEG for recording brainwaves, blood volume pressure to compute heart rate and skin response to record skin conductivity were used to record affective and cerebral responses throughout the experiment. The descriptive analysis, combined with the physiological data, provides compelling evidence for the positive impact of answer cues on reasoning and intuitive decision making in a logic-based problem-solving paradigm.

  8. Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior.

    Science.gov (United States)

    Garofalo, Sara; di Pellegrino, Giuseppe

    2015-01-01

    Pavlovian-to-instrumental transfer (PIT) refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking). In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses (CR) during Pavlovian conditioning: Sign-Trackers (ST) and Goal-Trackers (GT). Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect). This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction.

  9. Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior

    Directory of Open Access Journals (Sweden)

    Sara eGarofalo

    2015-06-01

    Full Text Available Pavlovian-to-instrumental transfer (PIT refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking. In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses during Pavlovian conditioning: Sign-Trackers (ST and Goal-Trackers (GT. Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect. This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction.

  10. Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior

    Science.gov (United States)

    Garofalo, Sara; di Pellegrino, Giuseppe

    2015-01-01

    Pavlovian-to-instrumental transfer (PIT) refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking). In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses (CR) during Pavlovian conditioning: Sign-Trackers (ST) and Goal-Trackers (GT). Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect). This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction. PMID:26157371

  11. Dopamine prediction errors in reward learning and addiction: from theory to neural circuitry

    Science.gov (United States)

    Keiflin, Ronald; Janak, Patricia H.

    2015-01-01

    Summary Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error-signaling and addiction can be formulated and tested. PMID:26494275

  12. Hedging Your Bets by Learning Reward Correlations in the Human Brain

    Science.gov (United States)

    Wunderlich, Klaus; Symmonds, Mkael; Bossaerts, Peter; Dolan, Raymond J.

    2011-01-01

    Summary Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling. PMID:21943609

  13. Bio-robots automatic navigation with graded electric reward stimulation based on Reinforcement Learning.

    Science.gov (United States)

    Zhang, Chen; Sun, Chao; Gao, Liqiang; Zheng, Nenggan; Chen, Weidong; Zheng, Xiaoxiang

    2013-01-01

    Bio-robots based on brain computer interface (BCI) suffer from the lack of considering the characteristic of the animals in navigation. This paper proposed a new method for bio-robots' automatic navigation combining the reward generating algorithm base on Reinforcement Learning (RL) with the learning intelligence of animals together. Given the graded electrical reward, the animal e.g. the rat, intends to seek the maximum reward while exploring an unknown environment. Since the rat has excellent spatial recognition, the rat-robot and the RL algorithm can convergent to an optimal route by co-learning. This work has significant inspiration for the practical development of bio-robots' navigation with hybrid intelligence.

  14. Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry.

    Science.gov (United States)

    Keiflin, Ronald; Janak, Patricia H

    2015-10-21

    Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error signaling and addiction can be formulated and tested. Copyright © 2015 Elsevier Inc. All rights reserved.

  15. One-trial spatial learning: wild hummingbirds relocate a reward after a single visit.

    Science.gov (United States)

    Flores-Abreu, I Nuri; Hurly, T Andrew; Healy, Susan D

    2012-07-01

    Beaconing to rewarded locations is typically achieved by visual recognition of the actual goal. Spatial recognition, on the other hand, can occur in the absence of the goal itself, relying instead on the landmarks surrounding the goal location. Although the duration or frequency of experiences that an animal needs to learn the landmarks surrounding a goal have been extensively studied with a variety of laboratory tasks, little is known about the way in which wild vertebrates use them in their natural environment. Here, we allowed hummingbirds to feed once only from a rewarding flower (goal) before it was removed. When we presented a similar flower at a different height in another location, birds frequently returned to the location the flower had previously occupied (spatial recognition) before flying to the flower itself (beaconing). After experiencing three rewarded flowers, each in a different location, they were more likely to beacon to the current visible flower than they were to return to previously rewarded locations (without a visible flower). These data show that hummingbirds can encode a rewarded location on the basis of the surrounding landmarks after a single visit. After multiple goal location manipulations, however, the birds changed their strategy to beaconing presumably because they had learned that the flower itself reliably signalled reward.

  16. Functional changes in the reward circuit in response to gaming-related cues after training with a commercial video game.

    Science.gov (United States)

    Gleich, Tobias; Lorenz, Robert C; Gallinat, Jürgen; Kühn, Simone

    2017-05-15

    In the present longitudinal study, we aimed to investigate video game training associated neuronal changes in reward processing using functional magnetic resonance imaging (fMRI). We recruited 48 healthy young participants which were assigned to one of 2 groups: A group in which participants were instructed to play a commercial video game ("Super Mario 64 DS") on a portable Nintendo DS handheld console at least 30minutes a day over a period of two months (video gaming group; VG) or to a matched passive control group (CG). Before and after the training phase, in both groups, fMRI imaging was conducted during passively viewing reward and punishment-related videos sequences recorded from the trained video game. The results show that video game training may lead to reward related decrease in neuronal activation in the dorsolateral prefrontal cortex (DLPFC) and increase in the hippocampus. Additionally, the decrease in DLPFC activation was associated with gaming related parameters experienced during playing. Specifically, we found that in the VG, gaming related parameters like performance, experienced fun and frustration (assessed during the training period) were correlated to decrease in reward related DLPFC activity. Thus, neuronal changes in terms of video game training seem to be highly related to the appetitive character and reinforcement schedule of the game. Those neuronal changes may also be related to the often reported video game associated improvements in cognitive functions. Copyright © 2017 Elsevier Inc. All rights reserved.

  17. Association of contextual cues with morphine reward increases neural and synaptic plasticity in the ventral hippocampus of rats

    NARCIS (Netherlands)

    Alvandi, M.S.; Bourmpoula, M.; Homberg, J.R.; Fathollahi, Y.

    2017-01-01

    Drug addiction is associated with aberrant memory and permanent functional changes in neural circuits. It is known that exposure to drugs like morphine is associated with positive emotional states and reward-related memory. However, the underlying mechanisms in terms of neural plasticity in the

  18. Bats without borders: Predators learn novel prey cues from other predatory species.

    Science.gov (United States)

    Patriquin, Krista J; Kohles, Jenna E; Page, Rachel A; Ratcliffe, John M

    2018-03-01

    Learning from others allows individuals to adapt rapidly to environmental change. Although conspecifics tend to be reliable models, heterospecifics with similar resource requirements may be suitable surrogates when conspecifics are few or unfamiliar with recent changes in resource availability. We tested whether Trachops cirrhosus , a gleaning bat that localizes prey using their mating calls, can learn about novel prey from conspecifics and the sympatric bat Lophostoma silvicolum. Specifically, we compared the rate for naïve T. cirrhosus to learn an unfamiliar tone from either a trained conspecific or heterospecific alone through trial and error or through social facilitation. T. cirrhosus learned this novel cue from L. silvicolum as quickly as from conspecifics. This is the first demonstration of social learning of a novel acoustic cue in bats and suggests that heterospecific learning may occur in nature. We propose that auditory-based social learning may help bats learn about unfamiliar prey and facilitate their adaptive radiation.

  19. Dual mechanisms governing reward-driven perceptual learning [version 1; referees: 2 approved

    Directory of Open Access Journals (Sweden)

    Dongho Kim

    2015-09-01

    Full Text Available In this review, we explore how reward signals shape perceptual learning in animals and humans. Perceptual learning is the well-established phenomenon by which extensive practice elicits selective improvement in one’s perceptual discrimination of basic visual features, such as oriented lines or moving stimuli. While perceptual learning has long been thought to rely on ‘top-down’ processes, such as attention and decision-making, a wave of recent findings suggests that these higher-level processes are, in fact, not necessary.  Rather, these recent findings indicate that reward signals alone, in the absence of the contribution of higher-level cognitive processes, are sufficient to drive the benefits of perceptual learning. Here, we will review the literature tying reward signals to perceptual learning. Based on these findings, we propose dual underlying mechanisms that give rise to perceptual learning: one mechanism that operates ‘automatically’ and is tied directly to reward signals, and another mechanism that involves more ‘top-down’, goal-directed computations.

  20. How motivation and reward learning modulate selective attention.

    Science.gov (United States)

    Bourgeois, A; Chelazzi, L; Vuilleumier, P

    2016-01-01

    Motivational stimuli such as rewards elicit adaptive responses and influence various cognitive functions. Notably, increasing evidence suggests that stimuli with particular motivational values can strongly shape perception and attention. These effects resemble both selective top-down and stimulus-driven attentional orienting, as they depend on internal states but arise without conscious will, yet they seem to reflect attentional systems that are functionally and anatomically distinct from those classically associated with frontoparietal cortical networks in the brain. Recent research in human and nonhuman primates has begun to reveal how reward can bias attentional selection, and where within the cognitive system the signals providing attentional priority are generated. This review aims at describing the different mechanisms sustaining motivational attention, their impact on different behavioral tasks, and current knowledge concerning the neural networks governing the integration of motivational influences on attentional behavior. © 2016 Elsevier B.V. All rights reserved.

  1. The issues of goal setting, interest, and reward in self-regulated learning

    OpenAIRE

    Okazaki, Makiko; 岡崎, 万紀子

    2011-01-01

    Self-regulated learning (SRL) is the theory developed in the field of educational psychology. SRL is based on the idea that behaviours are regulated by the self-concept. This self-concept is the basis of the social cognitive theory which does not depend on students' innate motivation (intrinsic motivation) at the initial stage of learning. This study examines three motivation-related factors in the concept of SRL: goals, interest, and rewards followed by the suggestion of a students' learning...

  2. Modeling effects of intrinsic and extrinsic rewards on the competition between striatal learning systems.

    Science.gov (United States)

    Boedecker, Joschka; Lampe, Thomas; Riedmiller, Martin

    2013-01-01

    A common assumption in psychology, economics, and other fields holds that higher performance will result if extrinsic rewards (such as money) are offered as an incentive. While this principle seems to work well for tasks that require the execution of the same sequence of steps over and over, with little uncertainty about the process, in other cases, especially where creative problem solving is required due to the difficulty in finding the optimal sequence of actions, external rewards can actually be detrimental to task performance. Furthermore, they have the potential to undermine intrinsic motivation to do an otherwise interesting activity. In this work, we extend a computational model of the dorsomedial and dorsolateral striatal reinforcement learning systems to account for the effects of extrinsic and intrinsic rewards. The model assumes that the brain employs both a goal-directed and a habitual learning system, and competition between both is based on the trade-off between the cost of the reasoning process and value of information. The goal-directed system elicits internal rewards when its models of the environment improve, while the habitual system, being model-free, does not. Our results account for the phenomena that initial extrinsic reward leads to reduced activity after extinction compared to the case without any initial extrinsic rewards, and that performance in complex task settings drops when higher external rewards are promised. We also test the hypothesis that external rewards bias the competition in favor of the computationally efficient, but cruder and less flexible habitual system, which can negatively influence intrinsic motivation and task performance in the class of tasks we consider.

  3. Modeling effects of intrinsic and extrinsic rewards on the competition between striatal learning systems

    Directory of Open Access Journals (Sweden)

    Joschka eBoedecker

    2013-10-01

    Full Text Available A common assumption in psychology, economics, and other fields holds that higher performance will result if extrinsic rewards (such as money are offered as an incentive. While this principle seems to work well for tasks that require the execution of the same sequence of steps over and over, with little uncertainty about the process, in other cases, especially where creative problem solving is required due to the difficulty in finding the optimal sequence of actions, external rewards can actually be detrimental to task performance. Furthermore, they have the potential to undermine intrinsic motivation to do an otherwise interesting activity. In this work, we extend a computational model of the prefrontal and dorsolateral striatal reinforcement learning systems to account for the effects of extrinsic and intrinsic rewards. The model assumes that the brain employs both a goal-directed and a habitual learning system, and competition between both is based on the trade-off between the cost of the reasoning process and value of information. The goal-directed system elicits internal rewards when its models of the environment improve, while the habitual system, being model-free, does not. Our results account for the phenomena that initial extrinsic reward leads to reduced activity after extinction compared to the case without any initial extrinsic rewards, and that performance in complex task settings drops when higher external rewards are promised. We also test the hypothesis that external rewards bias the competition in favor of the computationally efficient, but cruder and less flexible habitual system, which can negatively influence intrinsic motivation and task performance in the class of tasks we consider.

  4. The role of BDNF, leptin, and catecholamines in reward learning in bulimia nervosa.

    Science.gov (United States)

    Homan, Philipp; Grob, Simona; Milos, Gabriella; Schnyder, Ulrich; Eckert, Anne; Lang, Undine; Hasler, Gregor

    2014-12-07

    A relationship between bulimia nervosa and reward-related behavior is supported by several lines of evidence. The dopaminergic dysfunctions in the processing of reward-related stimuli have been shown to be modulated by the neurotrophin brain derived neurotrophic factor (BDNF) and the hormone leptin. Using a randomized, double-blind, placebo-controlled, crossover design, a reward learning task was applied to study the behavior of 20 female subjects with remitted bulimia nervosa and 27 female healthy controls under placebo and catecholamine depletion with alpha-methyl-para-tyrosine (AMPT). The plasma levels of BDNF and leptin were measured twice during the placebo and the AMPT condition, immediately before and 1 hour after a standardized breakfast. AMPT-induced differences in plasma BDNF levels were positively correlated with the AMPT-induced differences in reward learning in the whole sample (P=.05). Across conditions, plasma brain derived neurotrophic factor levels were higher in remitted bulimia nervosa subjects compared with controls (diagnosis effect; P=.001). Plasma BDNF and leptin levels were higher in the morning before compared with after a standardized breakfast across groups and conditions (time effect; Pbulimia nervosa and controls. A role of leptin in reward learning is not supported by this study. However, leptin levels were sensitive to a depletion of catecholamine stores in both remitted bulimia nervosa and controls. © The Author 2015. Published by Oxford University Press on behalf of CINP.

  5. The Effects of Attention Cueing on Visualizers' Multimedia Learning

    Science.gov (United States)

    Yang, Hui-Yu

    2016-01-01

    The present study examines how various types of attention cueing and cognitive preference affect learners' comprehension of a cardiovascular system and cognitive load. EFL learners were randomly assigned to one of four conditions: non-signal, static-blood-signal, static-blood-static-arrow-signal, and animation-signal. The results indicated that…

  6. WWC Quick Review of the Article "Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning" Revised

    Science.gov (United States)

    What Works Clearinghouse, 2010

    2010-01-01

    This paper presents an updated WWC (What Works Clearinghouse) Review of the Article "Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning". The study examined the effects of different reward systems used in group learning situations on the math skills of African-American and White students. The…

  7. WWC Review of the Article "Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning"

    Science.gov (United States)

    What Works Clearinghouse, 2010

    2010-01-01

    "Culture and the Interaction of Student Ethnicity with Reward Structure in Group Learning" examined the effects of different reward systems used in group learning situations on the math skills of African-American and white students. The study analyzed data on 75 African-American and 57 white fourth- and fifth-grade students from urban…

  8. COMT Val158Met genotype is associated with reward learning: A replication study and meta-analysis

    Science.gov (United States)

    Corral-Frías, Nadia S.; Pizzagalli, Diego A.; Carré, Justin; Michalski, Lindsay J; Nikolova, Yuliya S.; Perlis, Roy H.; Fagerness, Jesen; Lee, Mary R.; Conley, Emily Drabant; Lancaster, Thomas M.; Haddad, Stephen; Wolf, Aaron; Smoller, Jordan W.; Hariri, Ahmad R.; Bogdan, Ryan

    2016-01-01

    Identifying mechanisms through which individual differences in reward learning emerge offers an opportunity to understand both a fundamental form of adaptive responding as well as etiological pathways through which aberrant reward learning may contribute to maladaptive behaviors and psychopathology. One candidate mechanism through which individual differences in reward learning may emerge is variability in dopaminergic reinforcement signaling. A common functional polymorphism within the catechol-O-methyl transferase gene (COMT; rs4680, Val158Met) has been linked to reward learning where homozygosity for the Met allele (associated with heightened prefrontal dopamine function and decreased dopamine synthesis in the midbrain) has been associated with relatively increased reward learning. Here, we used a probabilistic reward learning task to asses response bias, a behavioral form of reward learning, across 3 separate samples that were combined for analyses (age: 21.80 ± 3.95; n=392; 268 female; European-American, n=208). We replicate prior reports that COMT rs4680 Met allele homozygosity is associated with increased reward learning in European-American participants (β=0.20, t= 2.75, p< 0.01; ΔR2= 0.04). Moreover, a meta-analysis of 4 studies, including the current one, confirmed the association between COMT rs4680 genotype and reward learning (95% CI −0.11 to −0.03; z=3.2; p<0.01). These results suggest that variability in dopamine signaling associated with COMT rs4680 influences individual differences in reward which may potentially contribute to psychopathology characterized by reward dysfunction. PMID:27138112

  9. The impact of effort-reward imbalance and learning motivation on teachers' sickness absence.

    Science.gov (United States)

    Derycke, Hanne; Vlerick, Peter; Van de Ven, Bart; Rots, Isabel; Clays, Els

    2013-02-01

    The aim of this study was to analyse the impact of the effort-reward imbalance and learning motivation on sickness absence duration and sickness absence frequency among beginning teachers in Flanders (Belgium). A total of 603 teachers, who recently graduated, participated in this study. Effort-reward imbalance and learning motivation were assessed by means of self-administered questionnaires. Prospective data of registered sickness absence during 12 months follow-up were collected. Multivariate logistic regression analyses were performed. An imbalance between high efforts and low rewards (extrinsic hypothesis) was associated with longer sickness absence duration and more frequent absences. A low level of learning motivation (intrinsic hypothesis) was not associated with longer sickness absence duration but was significantly positively associated with sickness absence frequency. No significant results were obtained for the interaction hypothesis between imbalance and learning motivation. Further research is needed to deepen our understanding of the impact of psychosocial work conditions and personal resources on both sickness absence duration and frequency. Specifically, attention could be given to optimizing or reducing efforts spent at work, increasing rewards and stimulating learning motivation to influence sickness absence. Copyright © 2012 John Wiley & Sons, Ltd.

  10. A reward optimization method based on action subrewards in hierarchical reinforcement learning.

    Science.gov (United States)

    Fu, Yuchen; Liu, Quan; Ling, Xionghong; Cui, Zhiming

    2014-01-01

    Reinforcement learning (RL) is one kind of interactive learning methods. Its main characteristics are "trial and error" and "related reward." A hierarchical reinforcement learning method based on action subrewards is proposed to solve the problem of "curse of dimensionality," which means that the states space will grow exponentially in the number of features and low convergence speed. The method can reduce state spaces greatly and choose actions with favorable purpose and efficiency so as to optimize reward function and enhance convergence speed. Apply it to the online learning in Tetris game, and the experiment result shows that the convergence speed of this algorithm can be enhanced evidently based on the new method which combines hierarchical reinforcement learning algorithm and action subrewards. The "curse of dimensionality" problem is also solved to a certain extent with hierarchical method. All the performance with different parameters is compared and analyzed as well.

  11. An Eye Tracking Comparison of External Pointing Cues and Internal Continuous Cues in Learning with Complex Animations

    Science.gov (United States)

    Boucheix, Jean-Michel; Lowe, Richard K.

    2010-01-01

    Two experiments used eye tracking to investigate a novel cueing approach for directing learner attention to low salience, high relevance aspects of a complex animation. In the first experiment, comprehension of a piano mechanism animation containing spreading-colour cues was compared with comprehension obtained with arrow cues or no cues. Eye…

  12. Monetary reward modulates task-irrelevant perceptual learning for invisible stimuli.

    Directory of Open Access Journals (Sweden)

    David Pascucci

    Full Text Available Task Irrelevant Perceptual Learning (TIPL shows that the brain's discriminative capacity can improve also for invisible and unattended visual stimuli. It has been hypothesized that this form of "unconscious" neural plasticity is mediated by an endogenous reward mechanism triggered by the correct task performance. Although this result has challenged the mandatory role of attention in perceptual learning, no direct evidence exists of the hypothesized link between target recognition, reward and TIPL. Here, we manipulated the reward value associated with a target to demonstrate the involvement of reinforcement mechanisms in sensory plasticity for invisible inputs. Participants were trained in a central task associated with either high or low monetary incentives, provided only at the end of the experiment, while subliminal stimuli were presented peripherally. Our results showed that high incentive-value targets induced a greater degree of perceptual improvement for the subliminal stimuli, supporting the role of reinforcement mechanisms in TIPL.

  13. Monetary reward modulates task-irrelevant perceptual learning for invisible stimuli.

    Science.gov (United States)

    Pascucci, David; Mastropasqua, Tommaso; Turatto, Massimo

    2015-01-01

    Task Irrelevant Perceptual Learning (TIPL) shows that the brain's discriminative capacity can improve also for invisible and unattended visual stimuli. It has been hypothesized that this form of "unconscious" neural plasticity is mediated by an endogenous reward mechanism triggered by the correct task performance. Although this result has challenged the mandatory role of attention in perceptual learning, no direct evidence exists of the hypothesized link between target recognition, reward and TIPL. Here, we manipulated the reward value associated with a target to demonstrate the involvement of reinforcement mechanisms in sensory plasticity for invisible inputs. Participants were trained in a central task associated with either high or low monetary incentives, provided only at the end of the experiment, while subliminal stimuli were presented peripherally. Our results showed that high incentive-value targets induced a greater degree of perceptual improvement for the subliminal stimuli, supporting the role of reinforcement mechanisms in TIPL.

  14. Towards a common theory for learning from reward, affect, and motivation: The SIMON framework

    Directory of Open Access Journals (Sweden)

    Christopher R Madan

    2013-10-01

    Full Text Available While the effects of reward, affect, and motivation on learning have each developed into their own fields of research, they largely have been investigated in isolation. As all three of these constructs are highly related, and use similar experimental procedures, an important advance in research would be to consider the interplay between these constructs. Here we first define each of the three constructs, and then discuss how they may influence each other within a common framework. Finally, we delineate several sources of evidence supporting the framework. By considering the constructs of reward, affect, and motivation within a single framework, we can develop a better understanding of the processes involved in learning and how they interplay, and work towards a comprehensive theory that encompasses reward, affect, and motivation.

  15. Toward a common theory for learning from reward, affect, and motivation: the SIMON framework.

    Science.gov (United States)

    Madan, Christopher R

    2013-10-07

    While the effects of reward, affect, and motivation on learning have each developed into their own fields of research, they largely have been investigated in isolation. As all three of these constructs are highly related, and use similar experimental procedures, an important advance in research would be to consider the interplay between these constructs. Here we first define each of the three constructs, and then discuss how they may influence each other within a common framework. Finally, we delineate several sources of evidence supporting the framework. By considering the constructs of reward, affect, and motivation within a single framework, we can develop a better understanding of the processes involved in learning and how they interplay, and work toward a comprehensive theory that encompasses reward, affect, and motivation.

  16. Rule learning in autism: the role of reward type and social context.

    Science.gov (United States)

    Jones, E J H; Webb, S J; Estes, A; Dawson, G

    2013-01-01

    Learning abstract rules is central to social and cognitive development. Across two experiments, we used Delayed Non-Matching to Sample tasks to characterize the longitudinal development and nature of rule-learning impairments in children with Autism Spectrum Disorder (ASD). Results showed that children with ASD consistently experienced more difficulty learning an abstract rule from a discrete physical reward than children with DD. Rule learning was facilitated by the provision of more concrete reinforcement, suggesting an underlying difficulty in forming conceptual connections. Learning abstract rules about social stimuli remained challenging through late childhood, indicating the importance of testing executive functions in both social and non-social contexts.

  17. Object-based implicit learning in visual search: perceptual segmentation constrains contextual cueing.

    Science.gov (United States)

    Conci, Markus; Müller, Hermann J; von Mühlenen, Adrian

    2013-07-09

    In visual search, detection of a target is faster when it is presented within a spatial layout of repeatedly encountered nontarget items, indicating that contextual invariances can guide selective attention (contextual cueing; Chun & Jiang, 1998). However, perceptual regularities may interfere with contextual learning; for instance, no contextual facilitation occurs when four nontargets form a square-shaped grouping, even though the square location predicts the target location (Conci & von Mühlenen, 2009). Here, we further investigated potential causes for this interference-effect: We show that contextual cueing can reliably occur for targets located within the region of a segmented object, but not for targets presented outside of the object's boundaries. Four experiments demonstrate an object-based facilitation in contextual cueing, with a modulation of context-based learning by relatively subtle grouping cues including closure, symmetry, and spatial regularity. Moreover, the lack of contextual cueing for targets located outside the segmented region was due to an absence of (latent) learning of contextual layouts, rather than due to an attentional bias towards the grouped region. Taken together, these results indicate that perceptual segmentation provides a basic structure within which contextual scene regularities are acquired. This in turn argues that contextual learning is constrained by object-based selection.

  18. The role of within-compound associations in learning about absent cues.

    Science.gov (United States)

    Witnauer, James E; Miller, Ralph R

    2011-05-01

    When two cues are reinforced together (in compound), most associative models assume that animals learn an associative network that includes direct cue-outcome associations and a within-compound association. All models of associative learning subscribe to the importance of cue-outcome associations, but most models assume that within-compound associations are irrelevant to each cue's subsequent behavioral control. In the present article, we present an extension of Van Hamme and Wasserman's (Learning and Motivation 25:127-151, 1994) model of retrospective revaluation based on learning about absent cues that are retrieved through within-compound associations. The model was compared with a model lacking retrieval through within-compound associations. Simulations showed that within-compound associations are necessary for the model to explain higher-order retrospective revaluation and the observed greater retrospective revaluation after partial reinforcement than after continuous reinforcement alone. These simulations suggest that the associability of an absent stimulus is determined by the extent to which the stimulus is activated through the within-compound association.

  19. Conflict Adaptation and Cue Competition during Learning in an Eriksen Flanker Task

    Science.gov (United States)

    Ghinescu, Rodica; Ramsey, Ashley K.; Gratton, Gabriele; Fabiani, Monica

    2016-01-01

    Two experiments investigated competition between cues that predicted the correct target response to a target stimulus in a response conflict procedure using a flanker task. Subjects received trials with five-character arrays with a central target character and distractor flanker characters that matched (compatible) or did not match (incompatible) the central target. Subjects’ expectancies for compatible and incompatible trials were manipulated by presenting pre-trial cues that signaled the occurrence of compatible or incompatible trials. On some trials, a single cue predicted the target stimulus and the required target response. On other trials, a second redundant, predictive cue was also present on such trials. The results showed an effect of competition between cues for control over strategic responding to the target stimuli, a finding that is predicted by associative learning theories. The finding of competition between pre-trial cues that predict incompatible trials, but not cues that predict compatible trials, suggests that different strategic processes may occur during adaptation to conflict when different kinds of trials are expected. PMID:27941977

  20. Sensory Responsiveness and the Effects of Equal Subjective Rewards on Tactile Learning and Memory of Honeybees

    Science.gov (United States)

    Scheiner, Ricarda; Kuritz-Kaiser, Anthea; Menzel, Randolf; Erber, Joachim

    2005-01-01

    In tactile learning, sucrose is the unconditioned stimulus and reward, which is usually applied to the antenna to elicit proboscis extension and which the bee can drink when it is subsequently applied to the extended proboscis. The conditioned stimulus is a tactile object that the bee can scan with its antennae. In this paper we describe the…

  1. Two spatiotemporally distinct value systems shape reward-based learning in the human brain.

    Science.gov (United States)

    Fouragnan, Elsa; Retzler, Chris; Mullinger, Karen; Philiastides, Marios G

    2015-09-08

    Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions. Yet, the neural underpinnings of the value systems that encode different decision-outcomes remain elusive. Here coupling single-trial electroencephalography with simultaneously acquired functional magnetic resonance imaging, we uncover the spatiotemporal dynamics of two separate but interacting value systems encoding decision-outcomes. Consistent with a role in regulating alertness and switching behaviours, an early system is activated only by negative outcomes and engages arousal-related and motor-preparatory brain structures. Consistent with a role in reward-based learning, a later system differentially suppresses or activates regions of the human reward network in response to negative and positive outcomes, respectively. Following negative outcomes, the early system interacts and downregulates the late system, through a thalamic interaction with the ventral striatum. Critically, the strength of this coupling predicts participants' switching behaviour and avoidance learning, directly implicating the thalamostriatal pathway in reward-based learning.

  2. Fostering participation in learning networks by using reward systems and face-to-face meetings

    NARCIS (Netherlands)

    Hummel, Hans; Tattersall, Colin; Burgos, Daniel; Brouns, Francis; Koper, Rob

    2006-01-01

    Hummel, H. G. K., Tattersall, C., Burgos, D., Brouns, F. M. R., & Koper, E. J. R. (Submitted). Fostering participation in learning networks by using reward systems and face-to-face meetings. In Proceedings of ICALT 2006 Conference. July 5-7, Kerkrade, The Netherlands

  3. Construction of a Learning Agent Handling Its Rewards According to Environmental Situations

    Science.gov (United States)

    Moriyama, Koichi; Numao, Masayuki

    The authors aim at constructing an agent which learns appropriate actions in a Multi-Agent environment with and without social dilemmas. For this aim, the agent must have nonrationality that makes it give up its own profit when it should do that. Since there are many studies on rational learning that brings more and more profit, it is desirable to utilize them for constructing the agent. Therefore, we use a reward-handling manner that makes internal evaluation from the agent's rewards, and then the agent learns actions by a rational learning method with the internal evaluation. If the agent has only a fixed manner, however, it does not act well in the environment with and without dilemmas. Thus, the authors equip the agent with several reward-handling manners and criteria for selecting an effective one for the environmental situation. In the case of humans, what generates the internal evaluation is usually called emotion. Hence, this study also aims at throwing light on emotional activities of humans from a constructive view. In this paper, we divide a Multi-Agent environment into three situations and construct an agent having the reward-handling manners and the criteria. We observe that the agent acts well in all the three Multi-Agent situations composed of homogeneous agents.

  4. A review of reward processing and motivational impairment in schizophrenia.

    Science.gov (United States)

    Strauss, Gregory P; Waltz, James A; Gold, James M

    2014-03-01

    This article reviews and synthesizes research on reward processing in schizophrenia, which has begun to provide important insights into the cognitive and neural mechanisms associated with motivational impairments. Aberrant cortical-striatal interactions may be involved with multiple reward processing abnormalities, including: (1) dopamine-mediated basal ganglia systems that support reinforcement learning and the ability to predict cues that lead to rewarding outcomes; (2) orbitofrontal cortex-driven deficits in generating, updating, and maintaining value representations; (3) aberrant effort-value computations, which may be mediated by disrupted anterior cingulate cortex and midbrain dopamine functioning; and (4) altered activation of the prefrontal cortex, which is important for generating exploratory behaviors in environments where reward outcomes are uncertain. It will be important for psychosocial interventions targeting negative symptoms to account for abnormalities in each of these reward processes, which may also have important interactions; suggestions for novel behavioral intervention strategies that make use of external cues, reinforcers, and mobile technology are discussed.

  5. Attentional Bias for Uncertain Cues of Shock in Human Fear Conditioning: Evidence for Attentional Learning Theory

    Directory of Open Access Journals (Sweden)

    Stephan Koenig

    2017-05-01

    Full Text Available We conducted a human fear conditioning experiment in which three different color cues were followed by an aversive electric shock on 0, 50, and 100% of the trials, and thus induced low (L, partial (P, and high (H shock expectancy, respectively. The cues differed with respect to the strength of their shock association (L < P < H and the uncertainty of their prediction (L < P > H. During conditioning we measured pupil dilation and ocular fixations to index differences in the attentional processing of the cues. After conditioning, the shock-associated colors were introduced as irrelevant distracters during visual search for a shape target while shocks were no longer administered and we analyzed the cues’ potential to capture and hold overt attention automatically. Our findings suggest that fear conditioning creates an automatic attention bias for the conditioned cues that depends on their correlation with the aversive outcome. This bias was exclusively linked to the strength of the cues’ shock association for the early attentional processing of cues in the visual periphery, but additionally was influenced by the uncertainty of the shock prediction after participants fixated on the cues. These findings are in accord with attentional learning theories that formalize how associative learning shapes automatic attention.

  6. Attentional Bias for Uncertain Cues of Shock in Human Fear Conditioning: Evidence for Attentional Learning Theory

    Science.gov (United States)

    Koenig, Stephan; Uengoer, Metin; Lachnit, Harald

    2017-01-01

    We conducted a human fear conditioning experiment in which three different color cues were followed by an aversive electric shock on 0, 50, and 100% of the trials, and thus induced low (L), partial (P), and high (H) shock expectancy, respectively. The cues differed with respect to the strength of their shock association (L H). During conditioning we measured pupil dilation and ocular fixations to index differences in the attentional processing of the cues. After conditioning, the shock-associated colors were introduced as irrelevant distracters during visual search for a shape target while shocks were no longer administered and we analyzed the cues’ potential to capture and hold overt attention automatically. Our findings suggest that fear conditioning creates an automatic attention bias for the conditioned cues that depends on their correlation with the aversive outcome. This bias was exclusively linked to the strength of the cues’ shock association for the early attentional processing of cues in the visual periphery, but additionally was influenced by the uncertainty of the shock prediction after participants fixated on the cues. These findings are in accord with attentional learning theories that formalize how associative learning shapes automatic attention. PMID:28588466

  7. The prelimbic cortex directs attention toward predictive cues during fear learning.

    Science.gov (United States)

    Sharpe, Melissa J; Killcross, Simon

    2015-06-01

    The prelimbic cortex is argued to promote conditioned fear expression, at odds with appetitive research implicating this region in attentional processing. Consistent with an attentional account, we report that the effect of prelimbic lesions on fear expression depends on the degree of competition between contextual and discrete cues. Further, when competition from contextual cues is low, we found that PL inactivation resulted in animals expressing fear toward irrelevant discrete cues; an effect selective to inactivation during the learning phase and not during retrieval. These data demonstrate that the prelimbic cortex modulates attention toward cues to preferentially direct fear responding on the basis of their predictive value. © 2015 Sharpe and Killcross; Published by Cold Spring Harbor Laboratory Press.

  8. Cingulate neglect in humans: disruption of contralesional reward learning in right brain damage.

    Science.gov (United States)

    Lecce, Francesca; Rotondaro, Francesca; Bonnì, Sonia; Carlesimo, Augusto; Thiebaut de Schotten, Michel; Tomaiuolo, Francesco; Doricchi, Fabrizio

    2015-01-01

    Motivational valence plays a key role in orienting spatial attention. Nonetheless, clinical documentation and understanding of motivationally based deficits of spatial orienting in the human is limited. Here in a series of one group-study and two single-case studies, we have examined right brain damaged patients (RBD) with and without left spatial neglect in a spatial reward-learning task, in which the motivational valence of the left contralesional and the right ipsilesional space was contrasted. In each trial two visual boxes were presented, one to the left and one to the right of central fixation. In one session monetary rewards were released more frequently in the box on the left side (75% of trials) whereas in another session they were released more frequently on the right side. In each trial patients were required to: 1) point to each one of the two boxes; 2) choose one of the boxes for obtaining monetary reward; 3) report explicitly the position of reward and whether this position matched or not the original choice. Despite defective spontaneous allocation of attention toward the contralesional space, RBD patients with left spatial neglect showed preserved contralesional reward learning, i.e., comparable to ipsilesional learning and to reward learning displayed by patients without neglect. A notable exception in the group of neglect patients was L.R., who showed no sign of contralesional reward learning in a series of 120 consecutive trials despite being able of reaching learning criterion in only 20 trials in the ipsilesional space. L.R. suffered a cortical-subcortical brain damage affecting the anterior components of the parietal-frontal attentional network and, compared with all other neglect and non-neglect patients, had additional lesion involvement of the medial anterior cingulate cortex (ACC) and of the adjacent sectors of the corpus callosum. In contrast to his lateralized motivational learning deficit, L.R. had no lateral bias in the early phases of

  9. Gaze-contingent reinforcement learning reveals incentive value of social signals in young children and adults.

    Science.gov (United States)

    Vernetti, Angélina; Smith, Tim J; Senju, Atsushi

    2017-03-15

    While numerous studies have demonstrated that infants and adults preferentially orient to social stimuli, it remains unclear as to what drives such preferential orienting. It has been suggested that the learned association between social cues and subsequent reward delivery might shape such social orienting. Using a novel, spontaneous indication of reinforcement learning (with the use of a gaze contingent reward-learning task), we investigated whether children and adults' orienting towards social and non-social visual cues can be elicited by the association between participants' visual attention and a rewarding outcome. Critically, we assessed whether the engaging nature of the social cues influences the process of reinforcement learning. Both children and adults learned to orient more often to the visual cues associated with reward delivery, demonstrating that cue-reward association reinforced visual orienting. More importantly, when the reward-predictive cue was social and engaging, both children and adults learned the cue-reward association faster and more efficiently than when the reward-predictive cue was social but non-engaging. These new findings indicate that social engaging cues have a positive incentive value. This could possibly be because they usually coincide with positive outcomes in real life, which could partly drive the development of social orienting. © 2017 The Authors.

  10. Heads for learning, tails for memory: Reward, reinforcement and a role of dopamine in determining behavioural relevance across multiple timescales

    Directory of Open Access Journals (Sweden)

    Mathieu eBaudonnat

    2013-10-01

    Full Text Available Dopamine has long been tightly associated with aspects of reinforcement learning and motivation in simple situations where there are a limited number of stimuli to guide behaviour and constrained range of outcomes. In naturalistic situations, however, there are many potential cues and foraging strategies that could be adopted, and it is critical that animals determine what might be behaviourally relevant in such complex environments. This requires not only detecting discrepancies with what they have recently experienced, but also identifying similarities with past experiences stored in memory. Here, we review what role dopamine might play in determining how and when to learn about the world, and how to develop choice policies appropriate to the situation faced. We discuss evidence that dopamine is shaped by motivation and memory and in turn shapes reward-based memory formation. In particular, we suggest that hippocampal-striatal-dopamine networks may interact to determine how surprising the world is and to either inhibit or promote actions at time of behavioural uncertainty.

  11. Tactile learning and the individual evaluation of the reward in honey bees (Apis mellifera L.).

    Science.gov (United States)

    Scheiner, R; Erber, J; Page, R E

    1999-07-01

    Using the proboscis extension response we conditioned pollen and nectar foragers of the honey bee (Apis mellifera L.) to tactile patterns under laboratory conditions. Pollen foragers demonstrated better acquisition, extinction, and reversal learning than nectar foragers. We tested whether the known differences in response thresholds to sucrose between pollen and nectar foragers could explain the observed differences in learning and found that nectar foragers with low response thresholds performed better during acquisition and extinction than ones with higher thresholds. Conditioning pollen and nectar foragers with similar response thresholds did not yield differences in their learning performance. These results suggest that differences in the learning performance of pollen and nectar foragers are a consequence of differences in their perception of sucrose. Furthermore, we analysed the effect which the perception of sucrose reward has on associative learning. Nectar foragers with uniform low response thresholds were conditioned using varying concentrations of sucrose. We found significant positive correlations between the concentrations of the sucrose rewards and the performance during acquisition and extinction. The results are summarised in a model which describes the relationships between learning performance, response threshold to sucrose, concentration of sucrose and the number of rewards.

  12. The role of high-frequency oscillatory activity in reward processing and learning.

    Science.gov (United States)

    Marco-Pallarés, Josep; Münte, Thomas F; Rodríguez-Fornells, Antoni

    2015-02-01

    Oscillatory activity has been proposed as a key mechanism in the integration of brain activity of distant structures. Particularly, high frequency brain oscillatory activity in the beta and gamma range has received increasing interest in the domains of attention and memory. In addition, a number of recent studies have revealed an increase of beta-gamma activity (20-35 Hz) after unexpected or relevant positive reward outcomes. In the present manuscript we review the literature on this phenomenon and we propose that this activity is a brain signature elicited by unexpected positive outcomes in order to transmit a fast motivational value signal to the reward network. In addition, we hypothesize that beta-gamma oscillatory activity indexes the interaction between attentional and emotional systems, and that it directly reflects the appearance of unexpected positive rewards in learning-related contexts. Copyright © 2014 Elsevier Ltd. All rights reserved.

  13. Reward-based learning under hardware constraints - Using a RISC processor embedded in a neuromorphic substrate

    Directory of Open Access Journals (Sweden)

    Simon eFriedmann

    2013-09-01

    Full Text Available In this study, we propose and analyze in simulations a new, highly flexible method of imple-menting synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. Thestudy focuses on globally modulated STDP, as a special use-case of this method. Flexibility isachieved by embedding a general-purpose processor dedicated to plasticity into the wafer. Toevaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spiketrain learning task. A single layer of neurons is trained to fire at specific points in time withonly the reward as feedback. This model is simulated to measure its performance, i.e. the in-crease in received reward after learning. Using this performance as baseline, we then simulatethe model with various constraints imposed by the proposed implementation and compare theperformance. The simulated constraints include discretized synaptic weights, a restricted inter-face between analog synapses and embedded processor, and mismatch of analog circuits. Wefind that probabilistic updates can increase the performance of low-resolution weights, a simpleinterface between analog synapses and processor is sufficient for learning, and performance isinsensitive to mismatch. Further, we consider communication latency between wafer and theconventional control computer system that is simulating the environment. This latency increasesthe delay, with which the reward is sent to the embedded processor. Because of the time continu-ous operation of the analog synapses, delay can cause a deviation of the updates as compared tothe not delayed situation. We find that for highly accelerated systems latency has to be kept to aminimum. This study demonstrates the suitability of the proposed implementation to emulatethe selected reward modulated STDP learning rule. It is therefore an ideal candidate for imple-mentation in an upgraded version of the wafer-scale system developed within the BrainScaleSproject.

  14. Visible spatial contiguity of social information and reward affects social learning in brown capuchins (Sapajus apella) and children (Homo sapiens).

    Science.gov (United States)

    Wood, Lara A; Whiten, Andrew

    2017-11-01

    Animal social learning is typically studied experimentally by the presentation of artificial foraging tasks. Although productive, results are often variable even for the same species. We present and test the hypothesis that one cause of variation is that spatial distance between rewards and the means of reward release causes conflicts for participants' attentional focus. We investigated whether spatial contiguity between a visible reward and the means of release would affect behavioral responses that evidence social learning, testing 21 brown capuchins ( Sapajus apella ), a much-studied species with variant evidence for social learning, and one hundred eighty 2- to 4-year-old human children ( Homo sapiens ), a benchmark species known for a strong social learning disposition. Participants were presented with a novel transparent apparatus where a reward was either proximal or distal to a demonstrated means of releasing it. A distal reward location decreased attention toward the location of the demonstration and impaired subsequent success in gaining rewards. Generally, the capuchins produced the alternative method to that demonstrated, whereas children copied the method demonstrated, although a distal reward location reduced copying in younger children. We conclude that some design features in common social learning tasks may significantly degrade the evidence for social learning. We have demonstrated this for 2 different primates but suggest that it is a significant factor to control for in social learning research across all taxa. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

  15. Wild, free-living rufous hummingbirds do not use geometric cues in a spatial task.

    Science.gov (United States)

    Hornsby, Mark A W; Hurly, T Andrew; Hamilton, Caitlin E; Pritchard, David J; Healy, Susan D

    2014-10-01

    In the laboratory, many species orient themselves using the geometric properties of an enclosure or array and geometric information is often preferred over visual cues. Whether animals use geometric cues when relocating rewarded locations in the wild, however, has rarely been investigated. We presented free-living rufous hummingbirds with a rectangular array of four artificial flowers to investigate learning of rewarded locations using geometric cues. In one treatment, we rewarded two of four flowers at diagonally opposite corners. In a second treatment, we provided a visual cue to the rewarded flower by connecting the flowers with "walls" consisting of four dowels (three white, one blue) laid on the ground connecting each of the flowers. Neither treatment elicited classical geometry results; instead, hummingbirds typically chose one particular flower over all others. When we exchanged that flower with another, hummingbirds tended to visit the original flower. These results suggest that (1) hummingbirds did not use geometric cues, but instead may have used a visually derived cue on the flowers themselves, and (2) using geometric cues may have been more difficult than using visual characteristics. Although hummingbirds typically prefer spatial over visual information, we hypothesize that they will not use geometric cues over stable visual features but that they make use of small, flower-specific visual cues. Such cues may play a more important role in foraging decisions than previously thought. Copyright © 2014 Elsevier B.V. All rights reserved.

  16. Association of contextual cues with morphine reward increases neural and synaptic plasticity in the ventral hippocampus of rats.

    Science.gov (United States)

    Alvandi, Mina Sadighi; Bourmpoula, Maria; Homberg, Judith R; Fathollahi, Yaghoub

    2017-11-01

    Drug addiction is associated with aberrant memory and permanent functional changes in neural circuits. It is known that exposure to drugs like morphine is associated with positive emotional states and reward-related memory. However, the underlying mechanisms in terms of neural plasticity in the ventral hippocampus, a region involved in associative memory and emotional behaviors, are not fully understood. Therefore, we measured adult neurogenesis, dendritic spine density and brain-derived neurotrophic factor (BDNF) and TrkB mRNA expression as parameters for synaptic plasticity in the ventral hippocampus. Male Sprague Dawley rats were subjected to the CPP (conditioned place preference) paradigm and received 10 mg/kg morphine. Half of the rats were used to evaluate neurogenesis by immunohistochemical markers Ki67 and doublecortin (DCX). The other half was used for Golgi staining to measure spine density and real-time quantitative reverse transcription-polymerase chain reaction to assess BDNF/TrkB expression levels. We found that morphine-treated rats exhibited more place conditioning as compared with saline-treated rats and animals that were exposed to the CPP without any injections. Locomotor activity did not change significantly. Morphine-induced CPP significantly increased the number of Ki67 and DCX-labeled cells in the ventral dentate gyrus. Additionally, we found increased dendritic spine density in both CA1 and dentate gyrus and an enhancement of BDNF/TrkB mRNA levels in the whole ventral hippocampus. Ki67, DCX and spine density were significantly correlated with CPP scores. In conclusion, we show that morphine-induced reward-related memory is associated with neural and synaptic plasticity changes in the ventral hippocampus. Such neural changes could underlie context-induced drug relapse. © 2017 Society for the Study of Addiction.

  17. Amygdala Contributions to Stimulus–Reward Encoding in the Macaque Medial and Orbital Frontal Cortex during Learning

    Science.gov (United States)

    Averbeck, Bruno B.

    2017-01-01

    Orbitofrontal cortex (OFC), medial frontal cortex (MFC), and amygdala mediate stimulus–reward learning, but the mechanisms through which they interact are unclear. Here, we investigated how neurons in macaque OFC and MFC signaled rewards and the stimuli that predicted them during learning with and without amygdala input. Macaques performed a task that required them to evaluate two stimuli and then choose one to receive the reward associated with that option. Four main findings emerged. First, amygdala lesions slowed the acquisition and use of stimulus–reward associations. Further analyses indicated that this impairment was due, at least in part, to ineffective use of negative feedback to guide subsequent decisions. Second, the activity of neurons in OFC and MFC rapidly evolved to encode the amount of reward associated with each stimulus. Third, amygdalectomy reduced encoding of stimulus–reward associations during the evaluation of different stimuli. Reward encoding of anticipated and received reward after choices were made was not altered. Fourth, amygdala lesions led to an increase in the proportion of neurons in MFC, but not OFC, that encoded the instrumental response that monkeys made on each trial. These correlated changes in behavior and neural activity after amygdala lesions strongly suggest that the amygdala contributes to the ability to learn stimulus–reward associations rapidly by shaping encoding within OFC and MFC. SIGNIFICANCE STATEMENT Altered functional interactions among orbital frontal cortex (OFC), medial frontal cortex (MFC), and amygdala are thought to underlie several psychiatric conditions, many related to reward learning. Here, we investigated the causal contribution of the amygdala to the development of neuronal activity in macaque OFC and MFC related to rewards and the stimuli that predict them during learning. Without amygdala inputs, neurons in both OFC and MFC showed decreased encoding of stimulus–reward associations. MFC also

  18. Amygdala Contributions to Stimulus-Reward Encoding in the Macaque Medial and Orbital Frontal Cortex during Learning.

    Science.gov (United States)

    Rudebeck, Peter H; Ripple, Joshua A; Mitz, Andrew R; Averbeck, Bruno B; Murray, Elisabeth A

    2017-02-22

    Orbitofrontal cortex (OFC), medial frontal cortex (MFC), and amygdala mediate stimulus-reward learning, but the mechanisms through which they interact are unclear. Here, we investigated how neurons in macaque OFC and MFC signaled rewards and the stimuli that predicted them during learning with and without amygdala input. Macaques performed a task that required them to evaluate two stimuli and then choose one to receive the reward associated with that option. Four main findings emerged. First, amygdala lesions slowed the acquisition and use of stimulus-reward associations. Further analyses indicated that this impairment was due, at least in part, to ineffective use of negative feedback to guide subsequent decisions. Second, the activity of neurons in OFC and MFC rapidly evolved to encode the amount of reward associated with each stimulus. Third, amygdalectomy reduced encoding of stimulus-reward associations during the evaluation of different stimuli. Reward encoding of anticipated and received reward after choices were made was not altered. Fourth, amygdala lesions led to an increase in the proportion of neurons in MFC, but not OFC, that encoded the instrumental response that monkeys made on each trial. These correlated changes in behavior and neural activity after amygdala lesions strongly suggest that the amygdala contributes to the ability to learn stimulus-reward associations rapidly by shaping encoding within OFC and MFC. SIGNIFICANCE STATEMENT Altered functional interactions among orbital frontal cortex (OFC), medial frontal cortex (MFC), and amygdala are thought to underlie several psychiatric conditions, many related to reward learning. Here, we investigated the causal contribution of the amygdala to the development of neuronal activity in macaque OFC and MFC related to rewards and the stimuli that predict them during learning. Without amygdala inputs, neurons in both OFC and MFC showed decreased encoding of stimulus-reward associations. MFC also showed

  19. Evolving Neural Turing Machines for Reward-based Learning

    DEFF Research Database (Denmark)

    Greve, Rasmus Boll; Jacobsen, Emil Juul; Risi, Sebastian

    2016-01-01

    An unsolved problem in neuroevolution (NE) is to evolve artificial neural networks (ANN) that can store and use information to change their behavior online. While plastic neural networks have shown promise in this context, they have difficulties retaining information over longer periods of time...... version of the double T-Maze, a complex reinforcement-like learning problem. In the T-Maze learning task the agent uses the memory bank to display adaptive behavior that normally requires a plastic ANN, thereby suggesting a complementary and effective mechanism for adaptive behavior in NE....

  20. Data from ‘Placebo Enhances Reward Learning in Healthy Individuals’

    Directory of Open Access Journals (Sweden)

    Zsolt Turi

    2018-04-01

    Full Text Available This dataset contains three repeated measures of a standard reward-based reinforcement-learning task from 29 healthy male individuals who participated in three experimental sessions exploring cognitive placebo effects on reward learning. The dataset includes behavioural data (accuracy, reaction times during learning and transfer, estimates of model-free computational analysis, self-reported arousal values, and expectations about the interventions’ efficacy. The data were collected in 2014 at the Department of Clinical Neurophysiology, University Medical Center Goettingen, Germany. The data collection and formal analysis used a triple-blind study design as participants, operator and analyst were unaware of conditions. A github repository contains data and analyses for the paper “Placebo Intervention Enhances Reward Learning in Healthy Individuals”. The dataset can be used for further analysis, reference, validation studies, teaching purposes, and collaborative research. Funding statement: This study was supported by the DFG (PA 419/15-1 awarded to WP. The preparation of this manuscript was supported by the “Research program, University Medical Center, University of Goettingen” awarded to Z.T.

  1. A tribute to Charlie Chaplin: Induced positive affect improves reward-based decision-learning in Parkinson’s Disease

    Directory of Open Access Journals (Sweden)

    K. Richard eRidderinkhof

    2012-06-01

    Full Text Available Reward-based decision-learning refers to the process of learning to select those actions that lead to rewards while avoiding actions that lead to punishments. This process, known to rely on dopaminergic activity in striatal brain regions, is compromised in Parkinson’s disease (PD. We hypothesized that such decision-learning deficits are alleviated by induced positive affect, which is thought to incur transient boosts in midbrain and striatal dopaminergic activity. Computational measures of probabilistic reward-based decision-learning were determined for 51 patients diagnosed with PD. Previous work has shown these measures to rely on the nucleus caudatus (outcome evaluation during the early phases of learning and the putamen (reward prediction during later phases of learning. We observed that induced positive affect facilitated learning, through its effects on reward prediction rather than outcome evaluation. Viewing a few minutes of comedy clips served to remedy dopamine-related problems in putamen-based frontostriatal circuitry and, consequently, in learning to predict which actions will yield reward.

  2. A learning rule that explains how rewards teach attention

    NARCIS (Netherlands)

    Rombouts, Jaldert O.; Bohte, Sander M.; Martinez-Trujillo, Julio; Roelfsema, Pieter R.

    2015-01-01

    Many theories propose that top-down attentional signals control processing in sensory cortices by modulating neural activity. But who controls the controller? Here we investigate how a biologically plausible neural reinforcement learning scheme can create higher order representations and top-down

  3. A learning rule that explains how rewards teach attention

    NARCIS (Netherlands)

    J.O. Rombouts (Jaldert); S.M. Bohte (Sander); J. Martinez-Trujillo; P.R. Roelfsema

    2015-01-01

    htmlabstractMany theories propose that top-down attentional signals control processing in sensory cortices by modulating neural activity. But who controls the controller? Here we investigate how a biologically plausible neural reinforcement learning scheme can create higher order representations and

  4. Pedunculopontine tegmental nucleus lesions impair stimulus--reward learning in autoshaping and conditioned reinforcement paradigms.

    Science.gov (United States)

    Inglis, W L; Olmstead, M C; Robbins, T W

    2000-04-01

    The role of the pedunculopontine tegmental nucleus (PPTg) in stimulus-reward learning was assessed by testing the effects of PPTg lesions on performance in visual autoshaping and conditioned reinforcement (CRf) paradigms. Rats with PPTg lesions were unable to learn an association between a conditioned stimulus (CS) and a primary reward in either paradigm. In the autoshaping experiment, PPTg-lesioned rats approached the CS+ and CS- with equal frequency, and the latencies to respond to the two stimuli did not differ. PPTg lesions also disrupted discriminated approaches to an appetitive CS in the CRf paradigm and completely abolished the acquisition of responding with CRf. These data are discussed in the context of a possible cognitive function of the PPTg, particularly in terms of lesion-induced disruptions of attentional processes that are mediated by the thalamus.

  5. Toward a common theory for learning from reward, affect, and motivation: the SIMON framework

    OpenAIRE

    Madan, Christopher R.

    2013-01-01

    While the effects of reward, affect, and motivation on learning have each developed into their own fields of research, they largely have been investigated in isolation. As all three of these constructs are highly related, and use similar experimental procedures, an important advance in research would be to consider the interplay between these constructs. Here we first define each of the three constructs, and then discuss how they may influence each other within a common framework. Finally, we...

  6. Impaired reward learning and intact motivation after serotonin depletion in rats.

    Science.gov (United States)

    Izquierdo, Alicia; Carlos, Kathleen; Ostrander, Serena; Rodriguez, Danilo; McCall-Craddolph, Aaron; Yagnik, Gargey; Zhou, Feimeng

    2012-08-01

    Aside from the well-known influence of serotonin (5-hydroxytryptamine, 5-HT) on emotional regulation, more recent investigations have revealed the importance of this monoamine in modulating cognition. Parachlorophenylalanine (PCPA) depletes 5-HT by inhibiting tryptophan hydroxylase, the enzyme required for 5-HT synthesis and, if administered at sufficiently high doses, can result in a depletion of at least 90% of the brain's 5-HT levels. The present study assessed the long-lasting effects of widespread 5-HT depletions on two tasks of cognitive flexibility in Long Evans rats: effort discounting and reversal learning. We assessed performance on these tasks after administration of either 250 or 500 mg/kg PCPA or saline (SAL) on two consecutive days. Consistent with a previous report investigating the role of 5-HT on effort discounting, pretreatment with either dose of PCPA resulted in normal effortful choice: All rats continued to climb tall barriers to obtain large rewards and were not work-averse. Additionally, rats receiving the lower dose of PCPA displayed normal reversal learning. However, despite intact motivation to work for food rewards, rats receiving the largest dose of PCPA were unexpectedly impaired relative to SAL rats on the pretraining stages leading up to reversal learning, ultimately failing to approach and respond to the stimuli associated with reward. High performance liquid chromatography (HPLC) with electrochemical detection confirmed 5-HT, and not dopamine, levels in the ventromedial frontal cortex were correlated with this measure of associative reward learning. Copyright © 2012 Elsevier B.V. All rights reserved.

  7. Rewards modulate saccade latency but not exogenous spatial attention.

    Directory of Open Access Journals (Sweden)

    Stephen eDunne

    2015-07-01

    Full Text Available The eye movement system is sensitive to reward. However, whilst the eye movement system is extremely flexible, the extent to which changes to oculomotor behaviour induced by reward paradigms persist beyond the training period or transfer to other oculomotor tasks is unclear. To address these issues we examined the effects of presenting feedback that represented small monetary rewards to spatial locations on the latency of saccadic eye movements, the time-course of learning and extinction of the effects of rewarding saccades on exogenous spatial attention and oculomotor IOR. Reward feedback produced a relative facilitation of saccadic latency in a stimulus driven saccade task which persisted for 3 blocks of extinction trials. However this hemifield-specific effect failed to transfer to peripheral cueing tasks. We conclude that rewarding specific spatial locations is unlikely to induce long-term, systemic changes to the human oculomotor or attention systems.

  8. Rewards modulate saccade latency but not exogenous spatial attention.

    Science.gov (United States)

    Dunne, Stephen; Ellison, Amanda; Smith, Daniel T

    2015-01-01

    The eye movement system is sensitive to reward. However, whilst the eye movement system is extremely flexible, the extent to which changes to oculomotor behavior induced by reward paradigms persist beyond the training period or transfer to other oculomotor tasks is unclear. To address these issues we examined the effects of presenting feedback that represented small monetary rewards to spatial locations on the latency of saccadic eye movements, the time-course of learning and extinction of the effects of rewarding saccades on exogenous spatial attention and oculomotor inhibition of return. Reward feedback produced a relative facilitation of saccadic latency in a stimulus driven saccade task which persisted for three blocks of extinction trials. However, this hemifield-specific effect failed to transfer to peripheral cueing tasks. We conclude that rewarding specific spatial locations is unlikely to induce long-term, systemic changes to the human oculomotor or attention systems.

  9. External validity of individual differences in multiple cue probability learning: The case of pilot training

    Directory of Open Access Journals (Sweden)

    Nadine Matton

    2013-09-01

    Full Text Available Individuals differ in their ability to deal with unpredictable environments. Could impaired performances on learning an unpredictable cue-criteria relationship in a laboratory task be associated with impaired learning of complex skills in a natural setting? We focused on a multiple-cue probability learning (MCPL laboratory task and on the natural setting of pilot training. We used data from three selection sessions and from the three corresponding selected pilot student classes of a national airline pilot selection and training system. First, applicants took an MCPL task at the selection stage (N=556; N=701; N=412. Then, pilot trainees selected from the applicant pools (N=44; N=60; N=28 followed the training for 2.5 to 3 yrs. Differences in final MCPL performance were associated with pilot training difficulties. Indeed, poor MCPL performers experienced almost twice as many pilot training difficulties as better MCPL performers (44.0% and 25.0%, respectively.

  10. Repeated nicotine exposure enhances reward-related learning in the rat.

    Science.gov (United States)

    Olausson, Peter; Jentsch, J David; Taylor, Jane R

    2003-07-01

    Repeated exposure to addictive drugs causes neuroadaptive changes in cortico-limbic-striatal circuits that may underlie alterations in incentive-motivational processes and reward-related learning. Such drug-induced alterations may be relevant to drug addiction because enhanced incentive motivation and increased control over behavior by drug-associated stimuli may contribute to aspects of compulsive drug-seeking and drug-taking behaviors. This study investigated the consequences of repeated nicotine treatment on the acquisition and performance of Pavlovian discriminative approach behavior, a measure of reward-related learning, in male rats. Water-restricted rats were trained to associate a compound conditioned stimulus (tone+light) with the availability of water (the unconditioned stimulus) in 15 consecutive daily sessions. In separate experiments, rats were repeatedly treated with nicotine (0.35 mg/kg, s.c.) either (1) prior to the onset of training, (2) after each daily training session was completed (ie postsession injections), or (3) received nicotine both before the onset of training as well as after each daily training session. In this study, all nicotine treatment schedules increased Pavlovian discriminative approach behavior and, thus, prior repeated exposure to nicotine, repeated postsession nicotine injections, or both, facilitated reward-related learning.

  11. Probability cueing of distractor locations: both intertrial facilitation and statistical learning mediate interference reduction.

    Science.gov (United States)

    Goschy, Harriet; Bakos, Sarolta; Müller, Hermann J; Zehetleitner, Michael

    2014-01-01

    Targets in a visual search task are detected faster if they appear in a probable target region as compared to a less probable target region, an effect which has been termed "probability cueing." The present study investigated whether probability cueing cannot only speed up target detection, but also minimize distraction by distractors in probable distractor regions as compared to distractors in less probable distractor regions. To this end, three visual search experiments with a salient, but task-irrelevant, distractor ("additional singleton") were conducted. Experiment 1 demonstrated that observers can utilize uneven spatial distractor distributions to selectively reduce interference by distractors in frequent distractor regions as compared to distractors in rare distractor regions. Experiments 2 and 3 showed that intertrial facilitation, i.e., distractor position repetitions, and statistical learning (independent of distractor position repetitions) both contribute to the probability cueing effect for distractor locations. Taken together, the present results demonstrate that probability cueing of distractor locations has the potential to serve as a strong attentional cue for the shielding of likely distractor locations.

  12. Retrieval cues that trigger reconsolidation of associative fear memory are not necessarily an exact replica of the original learning experience.

    Science.gov (United States)

    Soeter, Marieke; Kindt, Merel

    2015-01-01

    Disrupting the process of memory reconsolidation may point to a novel therapeutic strategy for the permanent reduction of fear in patients suffering from anxiety disorders. However both in animal and human studies the retrieval cue typically involves a re-exposure to the original fear-conditioned stimulus (CS). A relevant question is whether abstract cues not directly associated with the threat event also trigger reconsolidation, given that anxiety disorders often result from vicarious or unobtrusive learning for which no explicit memory exists. Insofar as the fear memory involves a flexible representation of the original learning experience, we hypothesized that the process of memory reconsolidation may also be triggered by abstract cues. We addressed this hypothesis by using a differential human fear-conditioning procedure in two distinct fear-learning groups. We predicted that if fear learning involves discrimination on basis of perceptual cues within one semantic category (i.e., the perceptual-learning group, n = 15), the subsequent ambiguity of the abstract retrieval cue would not trigger memory reconsolidation. In contrast, if fear learning involves discriminating between two semantic categories (i.e., categorical-learning group, n = 15), an abstract retrieval cue would unequivocally reactivate the fear memory and might subsequently trigger memory reconsolidation. Here we show that memory reconsolidation may indeed be triggered by another cue than the one that was present during the original learning occasion, but this effect depends on the learning history. Evidence for fear memory reconsolidation was inferred from the fear-erasing effect of one pill of propranolol (40 mg) systemically administered upon exposure to the abstract retrieval cue. Our finding that reconsolidation of a specific fear association does not require exposure to the original retrieval cue supports the feasibility of reconsolidation-based interventions for emotional disorders.

  13. Retrieval cues that trigger reconsolidation of associative fear memory are not necessarily an exact replica of the original learning experience

    Directory of Open Access Journals (Sweden)

    Marieke eSoeter

    2015-05-01

    Full Text Available Disrupting the process of memory reconsolidation may point to a novel therapeutic strategy for the permanent reduction of fear in patients suffering from anxiety disorders. However both in animal and human studies the retrieval cue typically involves a re-exposure to the original fear-conditioned stimulus. A relevant question is whether abstract cues not directly associated with the threat event also trigger reconsolidation, given that anxiety disorders often result from vicarious or unobtrusive learning for which no explicit memory exists. Insofar as the fear memory involves a flexible representation of the original learning experience, we hypothesized that the process of memory reconsolidation may also be triggered by abstract cues. We addressed this hypothesis by using a differential human fear-conditioning procedure in two distinct fear-learning groups. We predicted that if fear learning involves discrimination on basis of perceptual cues within one semantic category (i.e., the perceptual-learning group, n = 15, the subsequent ambiguity of the abstract retrieval cue would not trigger memory reconsolidation. In contrast, if fear learning involves discriminating between two semantic categories (i.e., categorical-learning group, n = 15, an abstract retrieval cue would unequivocally reactivate the fear memory and might subsequently trigger memory reconsolidation. Here we show that memory reconsolidation may indeed be triggered by another cue than the one that was present during the original learning occasion, but this effect depends on the learning history. Evidence for fear memory reconsolidation was inferred from the fear-erasing effect of one pill of propranolol (40 mg systemically administered upon exposure to the abstract retrieval cue. Our finding that reconsolidation of a specific fear association does not require exposure to the original retrieval cue supports the feasibility of reconsolidation-based interventions for emotional disorders.

  14. Information search with situation-specific reward functions

    Directory of Open Access Journals (Sweden)

    Bjorn Meder

    2012-03-01

    Full Text Available can strongly conflict with the goal of obtaining information for improving payoffs. Two environments with such a conflict were identified through computer optimization. Three subsequent experiments investigated people's search behavior in these environments. Experiments 1 and 2 used a multiple-cue probabilistic category-learning task to convey environmental probabilities. In a subsequent search task subjects could query only a single feature before making a classification decision. The crucial manipulation concerned the search-task reward structure. The payoffs corresponded either to accuracy, with equal rewards associated with the two categories, or to an asymmetric payoff function, with different rewards associated with each category. In Experiment 1, in which learning-task feedback corresponded to the true category, people later preferentially searched the accuracy-maximizing feature, whether or not this would improve monetary rewards. In Experiment 2, an asymmetric reward structure was used during learning. Subjects searched the reward-maximizing feature when asymmetric payoffs were preserved in the search task. However, if search-task payoffs corresponded to accuracy, subjects preferentially searched a feature that was suboptimal for reward and accuracy alike. Importantly, this feature would have been most useful, under the learning-task payoff structure. Experiment 3 found that, if words and numbers are used to convey environmental probabilities, neither reward nor accuracy consistently predicts search. These findings emphasize the necessity of taking into account people's goals and search-and-decision processes during learning, thereby challenging current models of information search.

  15. Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex.

    Science.gov (United States)

    Klein-Flügge, Miriam Cornelia; Barron, Helen Catharine; Brodersen, Kay Henning; Dolan, Raymond J; Behrens, Timothy Edward John

    2013-02-13

    A dominant focus in studies of learning and decision-making is the neural coding of scalar reward value. This emphasis ignores the fact that choices are strongly shaped by a rich representation of potential rewards. Here, using fMRI adaptation, we demonstrate that responses in the human orbitofrontal cortex (OFC) encode a representation of the specific type of food reward predicted by a visual cue. By controlling for value across rewards and by linking each reward with two distinct stimuli, we could test for representations of reward-identity that were independent of associative information. Our results show reward-identity representations in a medial-caudal region of OFC, independent of the associated predictive stimulus. This contrasts with a more rostro-lateral OFC region encoding reward-identity representations tied to the predicate stimulus. This demonstration of adaptation in OFC to reward specific representations opens an avenue for investigation of more complex decision mechanisms that are not immediately accessible in standard analyses, which focus on correlates of average activity.

  16. Reward-dependent learning in neuronal networks for planning and decision making.

    Science.gov (United States)

    Dehaene, S; Changeux, J P

    2000-01-01

    Neuronal network models have been proposed for the organization of evaluation and decision processes in prefrontal circuitry and their putative neuronal and molecular bases. The models all include an implementation and simulation of an elementary reward mechanism. Their central hypothesis is that tentative rules of behavior, which are coded by clusters of active neurons in prefrontal cortex, are selected or rejected based on an evaluation by this reward signal, which may be conveyed, for instance, by the mesencephalic dopaminergic neurons with which the prefrontal cortex is densely interconnected. At the molecular level, the reward signal is postulated to be a neurotransmitter such as dopamine, which exerts a global modulatory action on prefrontal synaptic efficacies, either via volume transmission or via targeted synaptic triads. Negative reinforcement has the effect of destabilizing the currently active rule-coding clusters; subsequently, spontaneous activity varies again from one cluster to another, giving the organism the chance to discover and learn a new rule. Thus, reward signals function as effective selection signals that either maintain or suppress currently active prefrontal representations as a function of their current adequacy. Simulations of this variation-selection have successfully accounted for the main features of several major tasks that depend on prefrontal cortex integrity, such as the delayed-response test, the Wisconsin card sorting test, the Tower of London test and the Stroop test. For the more complex tasks, we have found it necessary to supplement the external reward input with a second mechanism that supplies an internal reward; it consists of an auto-evaluation loop which short-circuits the reward input from the exterior. This allows for an internal evaluation of covert motor intentions without actualizing them as behaviors, by simply testing them covertly by comparison with memorized former experiences. This element of architecture

  17. Learning to maximize reward rate: a model based on semi-Markov decision processes.

    Science.gov (United States)

    Khodadadi, Arash; Fakhari, Pegah; Busemeyer, Jerome R

    2014-01-01

    WHEN ANIMALS HAVE TO MAKE A NUMBER OF DECISIONS DURING A LIMITED TIME INTERVAL, THEY FACE A FUNDAMENTAL PROBLEM: how much time they should spend on each decision in order to achieve the maximum possible total outcome. Deliberating more on one decision usually leads to more outcome but less time will remain for other decisions. In the framework of sequential sampling models, the question is how animals learn to set their decision threshold such that the total expected outcome achieved during a limited time is maximized. The aim of this paper is to provide a theoretical framework for answering this question. To this end, we consider an experimental design in which each trial can come from one of the several possible "conditions." A condition specifies the difficulty of the trial, the reward, the penalty and so on. We show that to maximize the expected reward during a limited time, the subject should set a separate value of decision threshold for each condition. We propose a model of learning the optimal value of decision thresholds based on the theory of semi-Markov decision processes (SMDP). In our model, the experimental environment is modeled as an SMDP with each "condition" being a "state" and the value of decision thresholds being the "actions" taken in those states. The problem of finding the optimal decision thresholds then is cast as the stochastic optimal control problem of taking actions in each state in the corresponding SMDP such that the average reward rate is maximized. Our model utilizes a biologically plausible learning algorithm to solve this problem. The simulation results show that at the beginning of learning the model choses high values of decision threshold which lead to sub-optimal performance. With experience, however, the model learns to lower the value of decision thresholds till finally it finds the optimal values.

  18. From prediction error to incentive salience: mesolimbic computation of reward motivation.

    Science.gov (United States)

    Berridge, Kent C

    2012-04-01

    Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience. © 2012 The Author. European Journal of Neuroscience © 2012 Federation of European Neuroscience Societies and Blackwell Publishing Ltd.

  19. Valence of facial cues influences sheep learning in a visual discrimination task

    OpenAIRE

    Bellegarde, Lucille; Erhard, Hans; Weiss, A.; Boissy, Alain; Haskell, M.J.

    2017-01-01

    Sheep are one of the most studied farm species in terms of their ability to process information from faces, but little is known about their face-based emotion recognition abilities. We investigated (a) whether sheep could use images of sheep faces taken in situation of varying valence as cues in a simultaneous discrimination task and (b) whether the valence of the situation affects their learning performance. To accomplish this, we photographed faces of sheep in three situations inducing emot...

  20. Valence of Facial Cues Influences Sheep Learning in a Visual Discrimination Task

    OpenAIRE

    Lucille G. A. Bellegarde; Lucille G. A. Bellegarde; Lucille G. A. Bellegarde; Hans W. Erhard; Alexander Weiss; Alain Boissy; Marie J. Haskell

    2017-01-01

    Sheep are one of the most studied farm species in terms of their ability to process information from faces, but little is known about their face-based emotion recognition abilities. We investigated (a) whether sheep could use images of sheep faces taken in situation of varying valence as cues in a simultaneous discrimination task and (b) whether the valence of the situation affects their learning performance. To accomplish this, we photographed faces of sheep in three situations inducing emot...

  1. Cue competition in evaluative conditioning as a function of the learning process.

    Science.gov (United States)

    Kattner, Florian; Green, C Shawn

    2015-11-01

    Evaluative conditioning (EC) is the change in the valence of a stimulus resulting from pairings with an affective (unconditioned) stimulus (US). With some exceptions, previous work has indicated that this form of conditioning might be insensitive to cue competition effects such as blocking and overshadowing. Here we assessed whether the extent of cue competition in EC depends upon the type of contingency learning during conditioning. Specifically, we contrasted a learning task that biased participants toward cognitive/inferential learning (i.e., predicting the US) with a learning task that prevented prolonged introspection (i.e., a rapid response made to the US). In all cases, standard EC effects were observed, with the subjective liking of stimuli changed in the direction of the valence of the US. More importantly, when inferential learning was likely, larger EC effects occurred for isolated stimuli than for compounds (indicating overshadowing). No blocking effects on explicit evaluations were observed for either learning task. Contingency judgments and implicit evaluations, however, were sensitive to blocking, indicating that the absence of a blocking effect on explicit evaluations might be due to inferences that occur during testing.

  2. 'You see?' Teaching and learning how to interpret visual cues during surgery.

    Science.gov (United States)

    Cope, Alexandra C; Bezemer, Jeff; Kneebone, Roger; Lingard, Lorelei

    2015-11-01

    The ability to interpret visual cues is important in many medical specialties, including surgery, in which poor outcomes are largely attributable to errors of perception rather than poor motor skills. However, we know little about how trainee surgeons learn to make judgements in the visual domain. We explored how trainees learn visual cue interpretation in the operating room. A multiple case study design was used. Participants were postgraduate surgical trainees and their trainers. Data included observer field notes, and integrated video- and audio-recordings from 12 cases representing more than 11 hours of observation. A constant comparative methodology was used to identify dominant themes. Visual cue interpretation was a recurrent feature of trainer-trainee interactions and was achieved largely through the pedagogic mechanism of co-construction. Co-construction was a dialogic sequence between trainer and trainee in which they explored what they were looking at together to identify and name structures or pathology. Co-construction took two forms: 'guided co-construction', in which the trainer steered the trainee to see what the trainer was seeing, and 'authentic co-construction', in which neither trainer nor trainee appeared certain of what they were seeing and pieced together the information collaboratively. Whether the co-construction activity was guided or authentic appeared to be influenced by case difficulty and trainee seniority. Co-construction was shown to occur verbally, through discussion, and also through non-verbal exchanges in which gestures made with laparoscopic instruments contributed to the co-construction discourse. In the training setting, learning visual cue interpretation occurs in part through co-construction. Co-construction is a pedagogic phenomenon that is well recognised in the context of learning to interpret verbal information. In articulating the features of co-construction in the visual domain, this work enables the development of

  3. Dopamine reward prediction error coding

    OpenAIRE

    Schultz, Wolfram

    2016-01-01

    Reward prediction errors consist of the differences between received and predicted rewards. They are crucial for basic forms of learning about rewards and make us strive for more rewards?an evolutionary beneficial trait. Most dopamine neurons in the midbrain of humans, monkeys, and rodents signal a reward prediction error; they are activated by more reward than predicted (positive prediction error), remain at baseline activity for fully predicted rewards, and show depressed activity with less...

  4. Ventromedial Prefrontal Cortex Activation Is Associated with Memory Formation for Predictable Rewards

    Science.gov (United States)

    Bialleck, Katharina A.; Schaal, Hans-Peter; Kranz, Thorsten A.; Fell, Juergen; Elger, Christian E.; Axmacher, Nikolai

    2011-01-01

    During reinforcement learning, dopamine release shifts from the moment of reward consumption to the time point when the reward can be predicted. Previous studies provide consistent evidence that reward-predicting cues enhance long-term memory (LTM) formation of these items via dopaminergic projections to the ventral striatum. However, it is less clear whether memory for items that do not precede a reward but are directly associated with reward consumption is also facilitated. Here, we investigated this question in an fMRI paradigm in which LTM for reward-predicting and neutral cues was compared to LTM for items presented during consumption of reliably predictable as compared to less predictable rewards. We observed activation of the ventral striatum and enhanced memory formation during reward anticipation. During processing of less predictable as compared to reliably predictable rewards, the ventral striatum was activated as well, but items associated with less predictable outcomes were remembered worse than items associated with reliably predictable outcomes. Processing of reliably predictable rewards activated the ventromedial prefrontal cortex (vmPFC), and vmPFC BOLD responses were associated with successful memory formation of these items. Taken together, these findings show that consumption of reliably predictable rewards facilitates LTM formation and is associated with activation of the vmPFC. PMID:21326612

  5. Who Deserves My Trust? Cue-Elicited Feedback Negativity Tracks Reputation Learning in Repeated Social Interactions.

    Science.gov (United States)

    Li, Diandian; Meng, Liang; Ma, Qingguo

    2017-01-01

    Trust and trustworthiness contribute to reciprocal behavior and social relationship development. To make better decisions, people need to evaluate others' trustworthiness. They often assess this kind of reputation by learning through repeated social interactions. The present event-related potential (ERP) study explored the reputation learning process in a repeated trust game where subjects made multi-round decisions of investment to different partners. We found that subjects gradually learned to discriminate trustworthy partners from untrustworthy ones based on how often their partners reciprocated the investment, which was indicated by their own investment decisions. Besides, electrophysiological data showed that the faces of the untrustworthy partners induced larger feedback negativity (FN) amplitude than those of the trustworthy partners, but only in the late phase of the game. The ERP results corresponded with the behavioral pattern and revealed that the learned trustworthiness differentiation was coded by the cue-elicited FN component. Consistent with previous research, our findings suggest that the anterior cue-elicited FN reflects the reputation appraisal and tracks the reputation learning process in social interactions.

  6. Who Deserves My Trust? Cue-Elicited Feedback Negativity Tracks Reputation Learning in Repeated Social Interactions

    Directory of Open Access Journals (Sweden)

    Diandian Li

    2017-06-01

    Full Text Available Trust and trustworthiness contribute to reciprocal behavior and social relationship development. To make better decisions, people need to evaluate others’ trustworthiness. They often assess this kind of reputation by learning through repeated social interactions. The present event-related potential (ERP study explored the reputation learning process in a repeated trust game where subjects made multi-round decisions of investment to different partners. We found that subjects gradually learned to discriminate trustworthy partners from untrustworthy ones based on how often their partners reciprocated the investment, which was indicated by their own investment decisions. Besides, electrophysiological data showed that the faces of the untrustworthy partners induced larger feedback negativity (FN amplitude than those of the trustworthy partners, but only in the late phase of the game. The ERP results corresponded with the behavioral pattern and revealed that the learned trustworthiness differentiation was coded by the cue-elicited FN component. Consistent with previous research, our findings suggest that the anterior cue-elicited FN reflects the reputation appraisal and tracks the reputation learning process in social interactions.

  7. Nicotine disrupts safety learning by enhancing fear associated with a safety cue via the dorsal hippocampus.

    Science.gov (United States)

    Connor, David A; Kutlu, Munir G; Gould, Thomas J

    2017-07-01

    Learned safety, a learning process in which a cue becomes associated with the absence of threat, is disrupted in individuals with post-traumatic stress disorder (PTSD). A bi-directional relationship exists between smoking and PTSD and one potential explanation is that nicotine-associated changes in cognition facilitate PTSD emotional dysregulation by disrupting safety associations. Therefore, we investigated whether nicotine would disrupt learned safety by enhancing fear associated with a safety cue. In the present study, C57BL/6 mice were administered acute or chronic nicotine and trained over three days in a differential backward trace conditioning paradigm consisting of five trials of a forward conditioned stimulus (CS)+ (Light) co-terminating with a footshock unconditioned stimulus followed by a backward CS- (Tone) presented 20 s after cessation of the unconditioned stimulus. Summation testing found that acute nicotine disrupted learned safety, but chronic nicotine had no effect. Another group of animals administered acute nicotine showed fear when presented with the backward CS (Light) alone, indicating the formation of a maladaptive fear association with the backward CS. Finally, we investigated the brain regions involved by administering nicotine directly into the dorsal hippocampus, ventral hippocampus, and prelimbic cortex. Infusion of nicotine into the dorsal hippocampus disrupted safety learning.

  8. Maximize Producer Rewards in Distributed Windmill Environments: A Q-Learning Approach

    Directory of Open Access Journals (Sweden)

    Bei Li

    2015-03-01

    Full Text Available In Smart Grid environments, homes equipped with windmills are encouraged to generate energy and sell it back to utilities. Time of Use pricing and the introduction of storage devices would greatly influence a user in deciding when to sell back energy and how much to sell. Therefore, a study of sequential decision making algorithms that can optimize the total pay off for the user is necessary. In this paper, reinforcement learning is used to tackle this optimization problem. The problem of determining when to sell back energy is formulated as a Markov decision process and the model is learned adaptively using Q-learning. Experiments are done with varying sizes of storage capacities and under periodic energy generation rates of different levels of fluctuations. The results show a notable increase in discounted total rewards from selling back energy with the proposed approach.

  9. Functional Specialization within the Striatum along Both the Dorsal/Ventral and Anterior/Posterior Axes during Associative Learning via Reward and Punishment

    Science.gov (United States)

    Mattfeld, Aaron T.; Gluck, Mark A.; Stark, Craig E. L.

    2011-01-01

    The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what…

  10. Enhancing inhibitory learning to reduce overeating: Design and rationale of a cue exposure therapy trial in overweight and obese women.

    Science.gov (United States)

    van den Akker, Karolien; Schyns, Ghislaine; Jansen, Anita

    2016-07-01

    The prevalence of overweight and obesity has increased substantially over the last decades. Weight loss attempts in overweight individuals are common, though they seldom result in successful long-term weight loss. One very promising treatment is food cue exposure therapy, during which overweight individuals are repeatedly exposed to food-associated cues (e.g., the sight, smell and taste of high-calorie foods, overeating environments) without eating in order to extinguish cue-elicited appetitive responses to food cues. However, only few studies have tested the effectiveness of cue exposure, especially with regards to weight loss. For exposure treatment of anxiety disorders, it has been proposed that inhibitory learning is critical for exposure to be effective. In this RCT, we translated techniques proposed by Craske et al. (2014) to the appetitive domain and developed a novel cue exposure therapy for overeating aimed at maximizing inhibitory learning. The current RCT tested the effectiveness of this 8-session cue exposure intervention relative to a control intervention in 45 overweight adult (aged 18-60) females at post-treatment and 3-month follow-up, of which 39 participants completed the study. Weight loss, eating psychopathology, food cue reactivity, and snacking behaviour were studied as main treatment outcomes, and mediators and moderators of treatment effects were studied. The presented study design represents an innovative effort to provide valuable clinical recommendations for the treatment of overeating and obesity. Copyright © 2016 Elsevier Inc. All rights reserved.

  11. Implication of Dopaminergic Modulation in Operant Reward Learning and the Induction of Compulsive-Like Feeding Behavior in "Aplysia"

    Science.gov (United States)

    Bedecarrats, Alexis; Cornet, Charles; Simmers, John; Nargeot, Romuald

    2013-01-01

    Feeding in "Aplysia" provides an amenable model system for analyzing the neuronal substrates of motivated behavior and its adaptability by associative reward learning and neuromodulation. Among such learning processes, appetitive operant conditioning that leads to a compulsive-like expression of feeding actions is known to be associated…

  12. Sex-specific associative learning cues and inclusive fitness benefits in the Seychelles warbler.

    Science.gov (United States)

    Richardson, D S; Burke, T; Komdeurs, J

    2003-09-01

    In cooperative breeding vertebrates, indirect fitness benefits would be maximized by subordinates that accurately assess their relatedness to group offspring and preferentially help more closely related kin. In the Seychelles warbler (Acrocephalus sechellensis), we found a positive relationship between subordinate-nestling kinship (determined using microsatellite marker genotypes) and provisioning rates, but only for female subordinates. Female subordinates that helped were significantly more related to the nestlings than were nonhelpers, and the decision to help appears to be based on associative learning cues. High levels of female infidelity means that subordinates cannot trust their legitimacy through the male line, consequently they appear to use the continued presence of the primary female, but not the primary male, as a reliable cue to determine when to feed nestlings. By using effective discrimination, female subordinates are able to maximize the indirect benefits gained within a cooperative breeding system otherwise driven primarily by direct breeding benefits.

  13. Food and drug cues activate similar brain regions: a meta-analysis of functional MRI studies.

    Science.gov (United States)

    Tang, D W; Fellows, L K; Small, D M; Dagher, A

    2012-06-06

    In healthy individuals, food cues can trigger hunger and feeding behavior. Likewise, smoking cues can trigger craving and relapse in smokers. Brain imaging studies report that structures involved in appetitive behaviors and reward, notably the insula, striatum, amygdala and orbital frontal cortex, tend to be activated by both visual food and smoking cues. Here, by carrying out a meta-analysis of human neuro-imaging studies, we investigate the neural network activated by: 1) food versus neutral cues (14 studies, 142 foci) 2) smoking versus neutral cues (15 studies, 176 foci) 3) smoking versus neutral cues when correlated with craving scores (7 studies, 108 foci). PubMed was used to identify cue-reactivity imaging studies that compared brain response to visual food or smoking cues to neutral cues. Fourteen articles were identified for the food meta-analysis and fifteen articles were identified for the smoking meta-analysis. Six articles were identified for the smoking cue correlated with craving analysis. Meta-analyses were carried out using activation likelihood estimation. Food cues were associated with increased blood oxygen level dependent (BOLD) response in the left amygdala, bilateral insula, bilateral orbital frontal cortex, and striatum. Smoking cues were associated with increased BOLD signal in the same areas, with the exception of the insula. However, the smoking meta-analysis of brain maps correlating cue-reactivity with subjective craving did identify the insula, suggesting that insula activation is only found when craving levels are high. The brain areas identified here are involved in learning, memory and motivation, and their cue-induced activity is an index of the incentive salience of the cues. Using meta-analytic techniques to combine a series of studies, we found that food and smoking cues activate comparable brain networks. There is significant overlap in brain regions responding to conditioned cues associated with natural and drug rewards

  14. 15-month-olds' transfer of learning between touch screen and real-world displays: language cues and cognitive loads.

    Science.gov (United States)

    Zack, Elizabeth; Gerhardstein, Peter; Meltzoff, Andrew N; Barr, Rachel

    2013-02-01

    Infants have difficulty transferring information between 2D and 3D sources. The current study extends Zack, Barr, Gerhardstein, Dickerson & Meltzoff's (2009) touch screen imitation task to examine whether the addition of specific language cues significantly facilitates 15-month-olds' transfer of learning between touch screens and real-world 3D objects. The addition of two kinds of linguistic cues (object label plus verb or nonsense name) did not elevate action imitation significantly above levels observed when such language cues were not used. Language cues hindered infants' performance in the 3D→2D direction of transfer, but only for the object label plus verb condition. The lack of a facilitative effect of language is discussed in terms of competing cognitive loads imposed by conjointly transferring information across dimensions and processing linguistic cues in an action imitation task at this age. © 2012 The Authors. Scandinavian Journal of Psychology © 2012 The Scandinavian Psychological Associations.

  15. 15-Month-Olds’ Transfer of Learning between Touch Screen and Real-World Displays: Language Cues and Cognitive Loads

    Science.gov (United States)

    Zack, Elizabeth; Gerhardstein, Peter; Meltzoff, Andrew N.; Barr, Rachel

    2012-01-01

    Infants have difficulty transferring information between 2D and 3D sources. The current study extends Zack et al.’s (2009) touch screen imitation task to examine whether the addition of specific language cues significantly facilitates 15-month-olds’ transfer of learning between touch screens and real-world 3D objects. The addition of two kinds of linguistic cues (object label plus verb or nonsense name) did not elevate action imitation significantly above levels observed when such language cues were not used. Language cues hindered infants’ performance in the 3D→2D direction of transfer, but only for the object label plus verb condition. The lack of a facilitative effect of language is discussed in terms of competing cognitive loads imposed by conjointly transferring information across dimensions and processing linguistic cues in an action imitation task at this age. PMID:23121508

  16. Chronic mitragynine (kratom) enhances punishment resistance in natural reward seeking and impairs place learning in mice.

    Science.gov (United States)

    Ismail, Nurul Iman W; Jayabalan, Nanthini; Mansor, Sharif Mahsufi; Müller, Christian P; Muzaimi, Mustapha

    2017-07-01

    Kratom (Mitragyna speciosa) is a widely abused herbal drug preparation in Southeast Asia. It is often consumed as a substitute for heroin, but imposing itself unknown harms and addictive burdens. Mitragynine is the major psychostimulant constituent of kratom that has recently been reported to induce morphine-like behavioural and cognitive effects in rodents. The effects of chronic consumption on non-drug related behaviours are still unclear. In the present study, we investigated the effects of chronic mitragynine treatment on spontaneous activity, reward-related behaviour and cognition in mice in an IntelliCage® system, and compared them with those of morphine and Δ-9-tetrahydrocannabinol (THC). We found that chronic mitragynine treatment significantly potentiated horizontal exploratory activity. It enhanced spontaneous sucrose preference and also its persistence when the preference had aversive consequences. Furthermore, mitragynine impaired place learning and its reversal. Thereby, mitragynine effects closely resembled that of morphine and THC sensitisation. These findings suggest that chronic mitragynine exposure enhances spontaneous locomotor activity and the preference for natural rewards, but impairs learning and memory. These findings confirm pleiotropic effects of mitragynine (kratom) on human lifestyle, but may also support the recognition of the drug's harm potential. © 2016 Society for the Study of Addiction.

  17. Feature extraction and learning using context cue and Rényi entropy based mutual information

    DEFF Research Database (Denmark)

    Pan, Hong; Olsen, Søren Ingvor; Zhu, Yaping

    2015-01-01

    information. In particular, for feature extraction, we develop a new set of kernel descriptors−Context Kernel Descriptors (CKD), which enhance the original KDES by embedding the spatial context into the descriptors. Context cues contained in the context kernel enforce some degree of spatial consistency, thus...... improving the robustness of CKD. For feature learning and reduction, we propose a novel codebook learning method, based on a Rényi quadratic entropy based mutual information measure called Cauchy-Schwarz Quadratic Mutual Information (CSQMI), to learn a compact and discriminative CKD codebook. Projecting...... as the information about the underlying labels of the CKD using CSQMI. Thus the resulting codebook and reduced CKD are discriminative. We verify the effectiveness of our method on several public image benchmark datasets such as YaleB, Caltech-101 and CIFAR-10, as well as a challenging chicken feet dataset of our own...

  18. Learning to Produce Syllabic Speech Sounds via Reward-Modulated Neural Plasticity

    Science.gov (United States)

    Warlaumont, Anne S.; Finnegan, Megan K.

    2016-01-01

    At around 7 months of age, human infants begin to reliably produce well-formed syllables containing both consonants and vowels, a behavior called canonical babbling. Over subsequent months, the frequency of canonical babbling continues to increase. How the infant’s nervous system supports the acquisition of this ability is unknown. Here we present a computational model that combines a spiking neural network, reinforcement-modulated spike-timing-dependent plasticity, and a human-like vocal tract to simulate the acquisition of canonical babbling. Like human infants, the model’s frequency of canonical babbling gradually increases. The model is rewarded when it produces a sound that is more auditorily salient than sounds it has previously produced. This is consistent with data from human infants indicating that contingent adult responses shape infant behavior and with data from deaf and tracheostomized infants indicating that hearing, including hearing one’s own vocalizations, is critical for canonical babbling development. Reward receipt increases the level of dopamine in the neural network. The neural network contains a reservoir with recurrent connections and two motor neuron groups, one agonist and one antagonist, which control the masseter and orbicularis oris muscles, promoting or inhibiting mouth closure. The model learns to increase the number of salient, syllabic sounds it produces by adjusting the base level of muscle activation and increasing their range of activity. Our results support the possibility that through dopamine-modulated spike-timing-dependent plasticity, the motor cortex learns to harness its natural oscillations in activity in order to produce syllabic sounds. It thus suggests that learning to produce rhythmic mouth movements for speech production may be supported by general cortical learning mechanisms. The model makes several testable predictions and has implications for our understanding not only of how syllabic vocalizations develop

  19. Differential roles of nonsynaptic and synaptic plasticity in operant reward learning-induced compulsive behavior.

    Science.gov (United States)

    Sieling, Fred; Bédécarrats, Alexis; Simmers, John; Prinz, Astrid A; Nargeot, Romuald

    2014-05-05

    Rewarding stimuli in associative learning can transform the irregularly and infrequently generated motor patterns underlying motivated behaviors into output for accelerated and stereotyped repetitive action. This transition to compulsive behavioral expression is associated with modified synaptic and membrane properties of central neurons, but establishing the causal relationships between cellular plasticity and motor adaptation has remained a challenge. We found previously that changes in the intrinsic excitability and electrical synapses of identified neurons in Aplysia's central pattern-generating network for feeding are correlated with a switch to compulsive-like motor output expression induced by in vivo operant conditioning. Here, we used specific computer-simulated ionic currents in vitro to selectively replicate or suppress the membrane and synaptic plasticity resulting from this learning. In naive in vitro preparations, such experimental manipulation of neuronal membrane properties alone increased the frequency but not the regularity of feeding motor output found in preparations from operantly trained animals. On the other hand, changes in synaptic strength alone switched the regularity but not the frequency of feeding output from naive to trained states. However, simultaneously imposed changes in both membrane and synaptic properties reproduced both major aspects of the motor plasticity. Conversely, in preparations from trained animals, experimental suppression of the membrane and synaptic plasticity abolished the increase in frequency and regularity of the learned motor output expression. These data establish direct causality for the contributions of distinct synaptic and nonsynaptic adaptive processes to complementary facets of a compulsive behavior resulting from operant reward learning. Copyright © 2014 Elsevier Ltd. All rights reserved.

  20. On the limits of statistical learning: Intertrial contextual cueing is confined to temporally close contingencies.

    Science.gov (United States)

    Thomas, Cyril; Didierjean, André; Maquestiaux, François; Goujon, Annabelle

    2018-04-12

    Since the seminal study by Chun and Jiang (Cognitive Psychology, 36, 28-71, 1998), a large body of research based on the contextual-cueing paradigm has shown that the cognitive system is capable of extracting statistical contingencies from visual environments. Most of these studies have focused on how individuals learn regularities found within an intratrial temporal window: A context predicts the target position within a given trial. However, Ono, Jiang, and Kawahara (Journal of Experimental Psychology, 31, 703-712, 2005) provided evidence of an intertrial implicit-learning effect when a distractor configuration in preceding trials N - 1 predicted the target location in trials N. The aim of the present study was to gain further insight into this effect by examining whether it occurs when predictive relationships are impeded by interfering task-relevant noise (Experiments 2 and 3) or by a long delay (Experiments 1, 4, and 5). Our results replicated the intertrial contextual-cueing effect, which occurred in the condition of temporally close contingencies. However, there was no evidence of integration across long-range spatiotemporal contingencies, suggesting a temporal limitation of statistical learning.

  1. Differential, but not opponent, effects of L -DOPA and citalopram on action learning with reward and punishment.

    Science.gov (United States)

    Guitart-Masip, Marc; Economides, Marcos; Huys, Quentin J M; Frank, Michael J; Chowdhury, Rumana; Duzel, Emrah; Dayan, Peter; Dolan, Raymond J

    2014-03-01

    Decision-making involves two fundamental axes of control namely valence, spanning reward and punishment, and action, spanning invigoration and inhibition. We recently exploited a go/no-go task whose contingencies explicitly decouple valence and action to show that these axes are inextricably coupled during learning. This results in a disadvantage in learning to go to avoid punishment and in learning to no-go to obtain a reward. The neuromodulators dopamine and serotonin are likely to play a role in these asymmetries: Dopamine signals anticipation of future rewards and is also involved in an invigoration of motor responses leading to reward, but it also arbitrates between different forms of control. Conversely, serotonin is implicated in motor inhibition and punishment processing. To investigate the role of dopamine and serotonin in the interaction between action and valence during learning.Methods We combined computational modeling with pharmacological manipulation in 90 healthy human volunteers, using levodopa and citalopram to affect dopamine and serotonin, respectively. We found that, after administration of levodopa,action learning was less affected by outcome valence when compared with the placebo and citalopram groups. This highlights in this context a predominant effect of levodopa in controlling the balance between different forms of control.Citalopram had distinct effects, increasing participants'tendency to perform active responses independent of outcome valence, consistent with a role in decreasing motor inhibition. Our findings highlight the rich complexities of the roles played by dopamine and serotonin during instrumental learning.

  2. Pigeons learn stimulus identity and stimulus relations when both serve as redundant, relevant cues during same-different discrimination training.

    Science.gov (United States)

    Gibson, Brett M; Wasserman, Edward A

    2003-01-01

    The authors taught pigeons to discriminate displays of 16 identical items from displays of 16 nonidentical items. Unlike most same-different discrimination studies--where only stimulus relations could serve a discriminative function--both the identity of the items and the relations among the items were discriminative features of the displays. The pigeons learned about both stimulus identity and stimulus relations when these 2 sources of information served as redundant, relevant cues. In tests of associative competition, identity cues exerted greater stimulus control than relational cues. These results suggest that the pigeon can respond to both specific stimuli and general relations in the environment.

  3. Acupuncture inhibits cue-induced heroin craving and brain activation.

    Science.gov (United States)

    Cai, Xinghui; Song, Xiaoge; Li, Chuanfu; Xu, Chunsheng; Li, Xiliang; Lu, Qi

    2012-11-25

    Previous research using functional MRI has shown that specific brain regions associated with drug dependence and cue-elicited heroin craving are activated by environmental cues. Craving is an important trigger of heroin relapse, and acupuncture may inhibit craving. In this study, we performed functional MRI in heroin addicts and control subjects. We compared differences in brain activation between the two groups during heroin cue exposure, heroin cue exposure plus acupuncture at the Zusanli point (ST36) without twirling of the needle, and heroin cue exposure plus acupuncture at the Zusanli point with twirling of the needle. Heroin cue exposure elicited significant activation in craving-related brain regions mainly in the frontal lobes and callosal gyri. Acupuncture without twirling did not significantly affect the range of brain activation induced by heroin cue exposure, but significantly changed the extent of the activation in the heroin addicts group. Acupuncture at the Zusanli point with twirling of the needle significantly decreased both the range and extent of activation induced by heroin cue exposure compared with heroin cue exposure plus acupuncture without twirling of the needle. These experimental findings indicate that presentation of heroin cues can induce activation in craving-related brain regions, which are involved in reward, learning and memory, cognition and emotion. Acupuncture at the Zusanli point can rapidly suppress the activation of specific brain regions related to craving, supporting its potential as an intervention for drug craving.

  4. The role of reward and reward uncertainty in episodic memory

    OpenAIRE

    Mason, Alice; Farrell, Simon; Howard-Jones, Paul; Ludwig, Casimir

    2017-01-01

    Declarative memory has been found to be sensitive to reward-related changes in the environment. The reward signal can be broken down into information regarding the expected value of the reward, reward uncertainty and the prediction error. Research has established that high as opposed to low reward values enhance declarative memory. Research in neuroscience suggests that high uncertainty activates the reward system, which could lead to enhanced learning and memory. Here we present the results ...

  5. Probabilistically-Cued Patterns Trump Perfect Cues in Statistical Language Learning.

    Science.gov (United States)

    Lany, Jill; Gómez, Rebecca L

    2013-01-01

    Probabilistically-cued co-occurrence relationships between word categories are common in natural languages but difficult to acquire. For example, in English, determiner-noun and auxiliary-verb dependencies both involve co-occurrence relationships, but determiner-noun relationships are more reliably marked by correlated distributional and phonological cues, and appear to be learned more readily. We tested whether experience with co-occurrence relationships that are more reliable promotes learning those that are less reliable using an artificial language paradigm. Prior experience with deterministically-cued contingencies did not promote learning of less reliably-cued structure, nor did prior experience with relationships instantiated in the same vocabulary. In contrast, prior experience with probabilistically-cued co-occurrence relationships instantiated in different vocabulary did enhance learning. Thus, experience with co-occurrence relationships sharing underlying structure but not vocabulary may be an important factor in learning grammatical patterns. Furthermore, experience with probabilistically-cued co-occurrence relationships, despite their difficultly for naïve learners, lays an important foundation for learning novel probabilistic structure.

  6. DISRUPTION OF CONDITIONED REWARD ASSOCIATION BY TYPICAL AND ATYPICAL ANTIPSYCHOTICS

    Science.gov (United States)

    Danna, C.L.; Elmer, G.I.

    2013-01-01

    Antipsychotic drugs are broadly classified into typical and atypical compounds; they vary in their pharmacological profile however a common component is their antagonist effects at the D2 dopamine receptors (DRD2). Unfortunately, diminished DRD2 activation is generally thought to be associated with the severity of neuroleptic-induced anhedonia. The purpose of this study was to determine the effect of the atypical antipsychotic olanzapine and typical antipsychotic haloperidol in a paradigm that reflects the learned transfer of incentive motivational properties to previously neutral stimuli, namely autoshaping. In order to provide a dosing comparison to a therapeutically relevant endpoint, both drugs were tested against amphetamine-induced disruption of prepulse inhibition as well. In the autoshaping task, rats were exposed to repeated pairings of stimuli that were differentially predictive of reward delivery. Conditioned approach to the reward predictive cue (sign-tracking) and to the reward (goal-tracking) increased during repeated pairings in the vehicle treated rats. Haloperidol and olanzapine completely abolished this behavior at relatively low doses (100 μg/kg). This same dose was the threshold dose for each drug to antagonize the sensorimotor gating deficits produced by amphetamine. At lower doses (3–30 μg/kg) both drugs produced a dose-dependent decrease in conditioned approach to the reward predictive cue. There was no difference between drugs at this dose range which indicates that olanzapine disrupts autoshaping at a significantly lower proposed DRD2 receptor occupancy. Interestingly, neither drug disrupted conditioned approach to the reward at the same dose range that disrupted conditioned approach to the reward predictive cue. Thus, haloperidol and olanzapine, at doses well below what is considered therapeutically relevant, disrupts the attribution of incentive motivational value to previously neutral cues. Drug effects on this dimension of reward

  7. Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation.

    Science.gov (United States)

    Dayan, Peter; Berridge, Kent C

    2014-06-01

    Evidence supports at least two methods for learning about reward and punishment and making predictions for guiding actions. One method, called model-free, progressively acquires cached estimates of the long-run values of circumstances and actions from retrospective experience. The other method, called model-based, uses representations of the environment, expectations, and prospective calculations to make cognitive predictions of future value. Extensive attention has been paid to both methods in computational analyses of instrumental learning. By contrast, although a full computational analysis has been lacking, Pavlovian learning and prediction has typically been presumed to be solely model-free. Here, we revise that presumption and review compelling evidence from Pavlovian revaluation experiments showing that Pavlovian predictions can involve their own form of model-based evaluation. In model-based Pavlovian evaluation, prevailing states of the body and brain influence value computations, and thereby produce powerful incentive motivations that can sometimes be quite new. We consider the consequences of this revised Pavlovian view for the computational landscape of prediction, response, and choice. We also revisit differences between Pavlovian and instrumental learning in the control of incentive motivation.

  8. Distributed hippocampal patterns that discriminate reward context are associated with enhanced associative binding.

    Science.gov (United States)

    Wolosin, Sasha M; Zeithamova, Dagmar; Preston, Alison R

    2013-11-01

    Recent research indicates that reward-based motivation impacts medial temporal lobe (MTL) encoding processes, leading to enhanced memory for rewarded events. In particular, previous functional magnetic resonance imaging (fMRI) studies of motivated learning have shown that MTL activation is greater for highly rewarded events, with the degree of reward-related activation enhancement tracking the corresponding behavioral memory advantage. These studies, however, do not directly address leading theoretical perspectives that propose such reward-based enhancements in MTL encoding activation reflect enhanced discrimination of the motivational context of specific events. In this study, a high-value or low-value monetary cue preceded a pair of objects, indicating the future reward for successfully remembering the pair. Using representational similarity analysis and high-resolution fMRI, we show that MTL activation patterns are more similar for encoding trials preceded by the same versus different reward cues, indicating a distributed code in this region that distinguishes between motivational contexts. Moreover, we show that activation patterns in hippocampus and parahippocampal cortex (PHc) that differentiate reward conditions during anticipatory cues and object pairs relate to successful associative memory. Additionally, the degree to which patterns differentiate reward contexts in dentate gyrus/CA2,3 and PHc is related to individual differences in reward modulation of memory. Collectively, these findings suggest that distributed activation patterns in the human hippocampus and PHc reflect the rewards associated with individual events. Furthermore, we show that these activation patterns-which discriminate between reward conditions--may influence memory through the incorporation of information about motivational contexts into stored memory representations. PsycINFO Database Record (c) 2013 APA, all rights reserved.

  9. FNDC5/irisin, a molecular target for boosting reward-related learning and motivation.

    Science.gov (United States)

    Zsuga, Judit; Tajti, Gabor; Papp, Csaba; Juhasz, Bela; Gesztelyi, Rudolf

    2016-05-01

    neurotropic factor that increases neuronal dopamine content, modulates dopamine release relevant for neuronal plasticity and increased neuronal survival as well as learning and memory. Further linking BDNF to dopaminergic function is BDNF's ability to activate tropomyosin-related kinase B receptor that shares signalization with presynaptic dopamine-3 receptors in the ventral tegmental area. Summarizing, we propose that the skeletal muscle derived irisin may be the link between physical activity and reward-related processes and motivation. Moreover alteration of this axis may contribute to sedentary lifestyle and subsequent non-communicable diseases. Preclinical and clinical experimental models to test this hypothesis are also proposed. Copyright © 2016 Elsevier Ltd. All rights reserved.

  10. Deep brain stimulation of the subthalamic nucleus improves reward-based decision-learning in Parkinson's disease

    NARCIS (Netherlands)

    van Wouwe, N.C.; Ridderinkhof, K.R.; van den Wildenberg, W.P.M.; Band, G.P.H.; Abisogun, A.; Elias, W.J.; Frysinger, R.; Wylie, S.A.

    2011-01-01

    Recently, the subthalamic nucleus (STN) has been shown to be critically involved in decision-making, action selection, and motor control. Here we investigate the effect of deep brain stimulation (DBS) of the STN on reward-based decision-learning in patients diagnosed with Parkinson's disease (PD).

  11. Deep brain stimulation of the subthalamic nucleus improves reward-based decision-learning in Parkinson’s disease

    NARCIS (Netherlands)

    Wouwe, N.C. van; Ridderinkhof, K.R.; Wildenberg, W.P.M. van den; Band, G.P.H.; Abisogun, A.; Elias, W.J.; Frysinger, R.; Wylie, S.A.

    2011-01-01

    Recently, the subthalamic nucleus (STN) has been shown to be critically involved in decision-making, action selection, and motor control. Here we investigate the effect of deep brain stimulation (DBS) of the STN on reward-based decision-learning in patients diagnosed with Parkinson’s disease (PD).

  12. A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping.

    Science.gov (United States)

    Takikawa, Yoriko; Kawagoe, Reiko; Hikosaka, Okihide

    2004-10-01

    Dopamine (DA) neurons respond to sensory stimuli that predict reward. To understand how DA neurons acquire such ability, we trained monkeys on a one-direction-rewarded version of memory-guided saccade task (1DR) only when we recorded from single DA neurons. In 1DR, position-reward mapping was changed across blocks of trials. In the early stage of training of 1DR, DA neurons responded to reward delivery; in the later stages, they responded predominantly to the visual cue that predicted reward or no reward (reward predictor) differentially. We found that such a shift of activity from reward to reward predictor also occurred within a block of trials after position-reward mapping was altered. A main effect of long-term training was to accelerate the within-block reward-to-predictor shift of DA neuronal responses. The within-block shift appeared first in the intermediate stage, but was slow, and DA neurons often responded to the cue that indicated reward in the preceding block. In the advanced stage, the reward-to-predictor shift occurred quickly such that the DA neurons' responses to visual cues faithfully matched the current position-reward mapping. Changes in the DA neuronal responses co-varied with the reward-predictive differentiation of saccade latency both in short-term (within-block) and long-term adaptation. DA neurons' response to the fixation point also underwent long-term changes until it occurred predominantly in the first trial within a block. This might trigger a switch between the learned sets. These results suggest that midbrain DA neurons play an essential role in adapting oculomotor behavior to frequent switches in position-reward mapping.

  13. Neuromodulatory Adaptive Combination of Correlation-based Learning in Cerebellum and Reward-based Learning in Basal Ganglia for Goal-directed Behavior Control

    DEFF Research Database (Denmark)

    Dasgupta, Sakyasingha; Wörgötter, Florentin; Manoonpong, Poramate

    2014-01-01

    Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational...... and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role...... in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We...

  14. Obese individuals with more components of the metabolic syndrome and/or prediabetes demonstrate decreased activation of reward-related brain centers in response to food cues in both the fed and fasting states: a preliminary fMRI study.

    Science.gov (United States)

    Farr, O M; Mantzoros, C S

    2017-03-01

    It remains unknown whether obese individuals with more components of the metabolic syndrome and/or prediabetes demonstrate altered activation of brain centers in response to food cues. We examined obese individuals with prediabetes (n=26) vs obese individuals without prediabetes (n=11) using fMRI. We also performed regression analyses on the basis of the number of MetS components per subject. Obese individuals with prediabetes have decreased activation of the reward-related putamen in the fasting state and decreased activation of the salience- and reward-related insula after eating. Obese individuals with more components of MetS demonstrate decreased activation of the putamen while fasting. All these activations remain significant when corrected for BMI, waist circumference (WC), HbA1c and gender. Decreased activation in the reward-related central nervous system areas among the obese is more pronounced in subjects with prediabetes and MetS. Prospective studies are needed to quantify their contributions to the development of prediabetes/MetS and to study whether they may predispose to the exacerbation of obesity and the development of comorbidities over time.

  15. Natural Variation in "Drosophila" Larval Reward Learning and Memory Due to a cGMP-Dependent Protein Kinase

    Science.gov (United States)

    Kaun, Karla R.; Hendel, Thomas; Gerber, Bertram; Sokolowski, Marla B.

    2007-01-01

    Animals must be able to find and evaluate food to ensure survival. The ability to associate a cue with the presence of food is advantageous because it allows an animal to quickly identify a situation associated with a good, bad, or even harmful food. Identifying genes underlying these natural learned responses is essential to understanding this…

  16. Translation of associative learning models into extinction reminders delivered via mobile phones during cue exposure interventions for substance use.

    Science.gov (United States)

    Rosenthal, M Zachary; Kutlu, Munir G

    2014-09-01

    Despite experimental findings and some treatment research supporting the use of cues as a means to induce and extinguish cravings, interventions using cue exposure have not been well integrated into contemporary substance abuse treatments. A primary problem with exposure-based interventions for addiction is that after learning not to use substances in the presence of addiction cues inside the clinic (i.e., extinction), stimuli in the naturalistic setting outside the clinic may continue to elicit craving, drug use, or other maladaptive conditioned responses. For exposure-based substance use interventions to be efficacious, new approaches are needed that can prevent relapse by directly generalizing learning from the therapeutic setting into naturalistic settings associated with a high risk for relapse. Basic research suggests that extinction reminders (ERs) can be paired with the context of learning new and more adaptive conditioned responses to substance abuse cues in exposure therapies for addiction. Using mobile phones and automated dialing and data collection software, ERs can be delivered in everyday high-risk settings to inhibit conditioned responses to substance-use-related stimuli. In this review, we describe how associative learning mechanisms (e.g., conditioned inhibition) can inform how ERs are conceptualized, learned, and implemented to prevent substance use when delivered via mobile phones. This approach, exposure with portable reminders of extinction, is introduced as an adjunctive intervention that uses brief automated ERs between clinic visits when individuals are in high-risk settings for drug use.

  17. Dopamine reward prediction error coding.

    Science.gov (United States)

    Schultz, Wolfram

    2016-03-01

    Reward prediction errors consist of the differences between received and predicted rewards. They are crucial for basic forms of learning about rewards and make us strive for more rewards-an evolutionary beneficial trait. Most dopamine neurons in the midbrain of humans, monkeys, and rodents signal a reward prediction error; they are activated by more reward than predicted (positive prediction error), remain at baseline activity for fully predicted rewards, and show depressed activity with less reward than predicted (negative prediction error). The dopamine signal increases nonlinearly with reward value and codes formal economic utility. Drugs of addiction generate, hijack, and amplify the dopamine reward signal and induce exaggerated, uncontrolled dopamine effects on neuronal plasticity. The striatum, amygdala, and frontal cortex also show reward prediction error coding, but only in subpopulations of neurons. Thus, the important concept of reward prediction errors is implemented in neuronal hardware.

  18. Reward-based learning under hardware constraints-using a RISC processor embedded in a neuromorphic substrate.

    Science.gov (United States)

    Friedmann, Simon; Frémaux, Nicolas; Schemmel, Johannes; Gerstner, Wulfram; Meier, Karlheinz

    2013-01-01

    In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e., the increase in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted interface between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continuous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for implementation in an upgraded version of the wafer-scale system developed within the BrainScaleS project.

  19. Testing a cue outside the training context increases attention to the contexts and impairs performance in human predictive learning.

    Science.gov (United States)

    Aristizabal, José A; Ramos-Álvarez, Manuel M; Callejas-Aguilera, José E; Rosas, Juan M

    2017-12-01

    One experiment in human predictive learning explored the impact of a context change on attention to contexts and predictive ratings controlled by the cue. In Context A: cue X was paired with an outcome four times, while cue Y was presented without an outcome four times in Context B:. In both contexts filler cues were presented without the outcome. During the test, target cues X and Y were presented either in the context where they were trained, or in the alternative context. With the context change expectation of the outcome X, expressed as predictive ratings, decreased in the presence of X and increased in the presence of Y. Looking at the contexts, expressed as a percentage of the overall gaze dwell time on a trial, was high across the four training trials, and increased with the context change. Results suggest that the presentation of unexpected information leads to increases in attention to contextual cues. Implications for contextual control of behavior are discussed. Copyright © 2017 Elsevier B.V. All rights reserved.

  20. The Effects of Social Cue Principles on Cognitive Load, Situational Interest, Motivation, and Achievement in Pedagogical Agent Multimedia Learning

    Science.gov (United States)

    Park, Sanghoon

    2015-01-01

    Animated pedagogical agents have become popular in multimedia learning with combined delivery of verbal and non-verbal forms of information. In order to reduce unnecessary cognitive load caused by such multiple forms of information and also to foster generative cognitive processing, multimedia design principles with social cues are suggested…

  1. Use of Emotional Cues for Lexical Learning: A Comparison of Autism Spectrum Disorder and Fragile X Syndrome

    Science.gov (United States)

    Thurman, Angela John; McDuffie, Andrea; Kover, Sara T.; Hagerman, Randi; Channell, Marie Moore; Mastergeorge, Ann; Abbeduto, Leonard

    2015-01-01

    The present study evaluated the ability of males with fragile X syndrome (FXS), nonsyndromic autism spectrum disorder (ASD), or typical development to learn new words by using as a cue to the intended referent an emotional reaction indicating a successful (excitement) or unsuccessful (disappointment) search for a novel object. Performance for all…

  2. The impact of reward and punishment on skill learning depends on task demands.

    Science.gov (United States)

    Steel, Adam; Silson, Edward H; Stagg, Charlotte J; Baker, Chris I

    2016-10-27

    Reward and punishment motivate behavior, but it is unclear exactly how they impact skill performance and whether the effect varies across skills. The present study investigated the effect of reward and punishment in both a sequencing skill and a motor skill context. Participants trained on either a sequencing skill (serial reaction time task) or a motor skill (force-tracking task). Skill knowledge was tested immediately after training, and again 1 hour, 24-48 hours, and 30 days after training. We found a dissociation of the effects of reward and punishment on the tasks, primarily reflecting the impact of punishment. While punishment improved serial reaction time task performance, it impaired force-tracking task performance. In contrast to prior literature, neither reward nor punishment benefitted memory retention, arguing against the common assumption that reward ubiquitously benefits skill retention. Collectively, these results suggest that punishment impacts skilled behavior more than reward in a complex, task dependent fashion.

  3. A reward-centred model of anorexia nervosa: a focussed narrative review of the neurological and psychophysiological literature.

    Science.gov (United States)

    O'Hara, Caitlin B; Campbell, Iain C; Schmidt, Ulrike

    2015-05-01

    This focussed narrative review examines neurobiological and psychophysiological evidence supporting a role for altered reward processes in the development and maintenance of anorexia nervosa (AN). In AN, there does not appear to be a generalised inability to experience reward. Rather, data suggest that a reluctance to gain weight leads to an aversive appraisal of food- and taste-related stimuli. As a result, cues compatible with this aberrant mode of thinking become rewarding for the individual. Evidence also suggests that attribution of motivational salience to such cues promotes anorectic behaviours. These findings are consistent with models in which interactions between cognition and reward are important in eliciting the anorectic "habit". A model is proposed which is consistent with elements of other theoretical frameworks, but differs in that its emphasis is towards neural overlaps between AN and addiction. It is consistent with AN being a reward-based learned behaviour in which aberrant cognitions related to eating and shape alter functioning of central reward systems. It proposes that the primary neural problem responsible for the development, maintenance, and treatment resistance is centred in the striatal reward system. This helps shift the emphasis of aetiological models towards reward processing, particularly in the context of illness-compatible cues. Furthermore, it suggests that continuing to explore the utility and valued nature of AN in the patient's life would be a useful inclusion in treatment and prevention models. Copyright © 2015. Published by Elsevier Ltd.

  4. Beyond negative valence: 2-week administration of a serotonergic antidepressant enhances both reward and effort learning signals.

    Directory of Open Access Journals (Sweden)

    Jacqueline Scholl

    2017-02-01

    Full Text Available To make good decisions, humans need to learn about and integrate different sources of appetitive and aversive information. While serotonin has been linked to value-based decision-making, its role in learning is less clear, with acute manipulations often producing inconsistent results. Here, we show that when the effects of a selective serotonin reuptake inhibitor (SSRI, citalopram are studied over longer timescales, learning is robustly improved. We measured brain activity with functional magnetic resonance imaging (fMRI in volunteers as they performed a concurrent appetitive (money and aversive (effort learning task. We found that 2 weeks of citalopram enhanced reward and effort learning signals in a widespread network of brain regions, including ventromedial prefrontal and anterior cingulate cortex. At a behavioral level, this was accompanied by more robust reward learning. This suggests that serotonin can modulate the ability to learn via a mechanism that is independent of stimulus valence. Such effects may partly underlie SSRIs' impact in treating psychological illnesses. Our results highlight both a specific function in learning for serotonin and the importance of studying its role across longer timescales.

  5. Ventromedial Frontal Cortex Is Critical for Guiding Attention to Reward-Predictive Visual Features in Humans.

    Science.gov (United States)

    Vaidya, Avinash R; Fellows, Lesley K

    2015-09-16

    Adaptively interacting with our environment requires extracting information that will allow us to successfully predict reward. This can be a challenge, particularly when there are many candidate cues, and when rewards are probabilistic. Recent work has demonstrated that visual attention is allocated to stimulus features that have been associated with reward on previous trials. The ventromedial frontal lobe (VMF) has been implicated in learning in dynamic environments of this kind, but the mechanism by which this region influences this process is not clear. Here, we hypothesized that the VMF plays a critical role in guiding attention to reward-predictive stimulus features based on feedback. We tested the effects of VMF damage in human subjects on a visual search task in which subjects were primed to attend to task-irrelevant colors associated with different levels of reward, incidental to the search task. Consistent with previous work, we found that distractors had a greater influence on reaction time when they appeared in colors associated with high reward in the previous trial compared with colors associated with low reward in healthy control subjects and patients with prefrontal damage sparing the VMF. However, this reward modulation of attentional priming was absent in patients with VMF damage. Thus, an intact VMF is necessary for directing attention based on experience with cue-reward associations. We suggest that this region plays a role in selecting reward-predictive cues to facilitate future learning. There has been a swell of interest recently in the ventromedial frontal cortex (VMF), a brain region critical to associative learning. However, the underlying mechanism by which this region guides learning is not well understood. Here, we tested the effects of damage to this region in humans on a task in which rewards were linked incidentally to visual features, resulting in trial-by-trial attentional priming. Controls and subjects with prefrontal damage

  6. Attention Cueing and Activity Equally Reduce False Alarm Rate in Visual-Auditory Associative Learning through Improving Memory.

    Science.gov (United States)

    Nikouei Mahani, Mohammad-Ali; Haghgoo, Hojjat Allah; Azizi, Solmaz; Nili Ahmadabadi, Majid

    2016-01-01

    In our daily life, we continually exploit already learned multisensory associations and form new ones when facing novel situations. Improving our associative learning results in higher cognitive capabilities. We experimentally and computationally studied the learning performance of healthy subjects in a visual-auditory sensory associative learning task across active learning, attention cueing learning, and passive learning modes. According to our results, the learning mode had no significant effect on learning association of congruent pairs. In addition, subjects' performance in learning congruent samples was not correlated with their vigilance score. Nevertheless, vigilance score was significantly correlated with the learning performance of the non-congruent pairs. Moreover, in the last block of the passive learning mode, subjects significantly made more mistakes in taking non-congruent pairs as associated and consciously reported lower confidence. These results indicate that attention and activity equally enhanced visual-auditory associative learning for non-congruent pairs, while false alarm rate in the passive learning mode did not decrease after the second block. We investigated the cause of higher false alarm rate in the passive learning mode by using a computational model, composed of a reinforcement learning module and a memory-decay module. The results suggest that the higher rate of memory decay is the source of making more mistakes and reporting lower confidence in non-congruent pairs in the passive learning mode.

  7. A Selective Role for Dopamine in Learning to Maximize Reward But Not to Minimize Effort: Evidence from Patients with Parkinson's Disease.

    Science.gov (United States)

    Skvortsova, Vasilisa; Degos, Bertrand; Welter, Marie-Laure; Vidailhet, Marie; Pessiglione, Mathias

    2017-06-21

    Instrumental learning is a fundamental process through which agents optimize their choices, taking into account various dimensions of available options such as the possible reward or punishment outcomes and the costs associated with potential actions. Although the implication of dopamine in learning from choice outcomes is well established, less is known about its role in learning the action costs such as effort. Here, we tested the ability of patients with Parkinson's disease (PD) to maximize monetary rewards and minimize physical efforts in a probabilistic instrumental learning task. The implication of dopamine was assessed by comparing performance ON and OFF prodopaminergic medication. In a first sample of PD patients ( n = 15), we observed that reward learning, but not effort learning, was selectively impaired in the absence of treatment, with a significant interaction between learning condition (reward vs effort) and medication status (OFF vs ON). These results were replicated in a second, independent sample of PD patients ( n = 20) using a simplified version of the task. According to Bayesian model selection, the best account for medication effects in both studies was a specific amplification of reward magnitude in a Q-learning algorithm. These results suggest that learning to avoid physical effort is independent from dopaminergic circuits and strengthen the general idea that dopaminergic signaling amplifies the effects of reward expectation or obtainment on instrumental behavior. SIGNIFICANCE STATEMENT Theoretically, maximizing reward and minimizing effort could involve the same computations and therefore rely on the same brain circuits. Here, we tested whether dopamine, a key component of reward-related circuitry, is also implicated in effort learning. We found that patients suffering from dopamine depletion due to Parkinson's disease were selectively impaired in reward learning, but not effort learning. Moreover, anti-parkinsonian medication restored the

  8. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making.

    Science.gov (United States)

    Schönberg, Tom; Daw, Nathaniel D; Joel, Daphna; O'Doherty, John P

    2007-11-21

    The computational framework of reinforcement learning has been used to forward our understanding of the neural mechanisms underlying reward learning and decision-making behavior. It is known that humans vary widely in their performance in decision-making tasks. Here, we used a simple four-armed bandit task in which subjects are almost evenly split into two groups on the basis of their performance: those who do learn to favor choice of the optimal action and those who do not. Using models of reinforcement learning we sought to determine the neural basis of these intrinsic differences in performance by scanning both groups with functional magnetic resonance imaging. We scanned 29 subjects while they performed the reward-based decision-making task. Our results suggest that these two groups differ markedly in the degree to which reinforcement learning signals in the striatum are engaged during task performance. While the learners showed robust prediction error signals in both the ventral and dorsal striatum during learning, the nonlearner group showed a marked absence of such signals. Moreover, the magnitude of prediction error signals in a region of dorsal striatum correlated significantly with a measure of behavioral performance across all subjects. These findings support a crucial role of prediction error signals, likely originating from dopaminergic midbrain neurons, in enabling learning of action selection preferences on the basis of obtained rewards. Thus, spontaneously observed individual differences in decision making performance demonstrate the suggested dependence of this type of learning on the functional integrity of the dopaminergic striatal system in humans.

  9. Rewarding Foreign Language Learning: Effects of the Swedish Grade Point Average Enhancement Initiative on Students' Motivation to Learn French

    Science.gov (United States)

    Henry, Alastair

    2017-01-01

    In order to reinstate interest and motivation for learning foreign languages (FLs) other than English, the Swedish government has recently reformed the system for admission to higher education. Upper secondary students who continue with the FL learnt in secondary school are rewarded with extra credits that considerably enhance their grade point…

  10. The Effects of Reward, Punishment, and Knowledge of Results on Children's Discrimination Learning

    Science.gov (United States)

    Donohue, Barbara; Ratliff, Richard G.

    1976-01-01

    The differential effects of contingent reward (candy), punishment (loss of candy), and knowledge of results (KOR) were investigated in eighty 9- to 10-year-old males. Level of performance of groups receiving KOR was significantly higher than performance on groups rewarded or punished with candy. (MS)

  11. The opportunities and rewards of distance learning programs offered by the university of Newcastle

    International Nuclear Information System (INIS)

    Lyall, D.

    2002-01-01

    Full text: The University of Newcastle offers, via the distance learning mode of study, three innovative postgraduate programs designed for Nuclear Medicine Scientists and Practitioners. The Graduate Diploma in Nuclear Medicine is designed for Nuclear Medicine Scientists who have been absent from the work force for an extended period of time or have completed a qualification in medical imaging, who are currently working as a Nuclear Medicine Scientist. The Graduate Diploma fosters and encourages a greater depth of knowledge of Nuclear Medicine through the understanding and application of clinical skills. The Master of Nuclear Medicine is designed to develop within Nuclear Medicine Scientists and Practitioners a greater depth of knowledge of Nuclear Medicine through the acquisition and application of research skills. The Master of Applied Management (Health) is designed for Nuclear Medicine Scientists, practitioners, health managers and administrators who desire to acquire essential management skills while expanding their clinical skills. Graduates of the Graduate Diploma are eligible to sit the certification examination for the Nuclear Medicine Technology Certification Board in the United States of America. They may also eligible to apply to the Australian and New Zealand Society of Nuclear Medicine Accreditation Board for interim accreditation. Graduates of the Masters of Nuclear Medicine enjoy the same opportunities as per the Graduate Diploma. They also enjoy a greater success with promotion and have the skills necessary to develop a rewarding career in research. Copyright (2002) The Australian and New Zealand Society of Nuclear Medicine Inc

  12. Dopaminergic control of motivation and reinforcement learning: a closed-circuit account for reward-oriented behavior.

    Science.gov (United States)

    Morita, Kenji; Morishima, Mieko; Sakai, Katsuyuki; Kawaguchi, Yasuo

    2013-05-15

    Humans and animals take actions quickly when they expect that the actions lead to reward, reflecting their motivation. Injection of dopamine receptor antagonists into the striatum has been shown to slow such reward-seeking behavior, suggesting that dopamine is involved in the control of motivational processes. Meanwhile, neurophysiological studies have revealed that phasic response of dopamine neurons appears to represent reward prediction error, indicating that dopamine plays central roles in reinforcement learning. However, previous attempts to elucidate the mechanisms of these dopaminergic controls have not fully explained how the motivational and learning aspects are related and whether they can be understood by the way the activity of dopamine neurons itself is controlled by their upstream circuitries. To address this issue, we constructed a closed-circuit model of the corticobasal ganglia system based on recent findings regarding intracortical and corticostriatal circuit architectures. Simulations show that the model could reproduce the observed distinct motivational effects of D1- and D2-type dopamine receptor antagonists. Simultaneously, our model successfully explains the dopaminergic representation of reward prediction error as observed in behaving animals during learning tasks and could also explain distinct choice biases induced by optogenetic stimulation of the D1 and D2 receptor-expressing striatal neurons. These results indicate that the suggested roles of dopamine in motivational control and reinforcement learning can be understood in a unified manner through a notion that the indirect pathway of the basal ganglia represents the value of states/actions at a previous time point, an empirically driven key assumption of our model.

  13. Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control.

    Science.gov (United States)

    Dasgupta, Sakyasingha; Wörgötter, Florentin; Manoonpong, Poramate

    2014-01-01

    Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.

  14. Adolescents, adults and rewards: comparing motivational neurocircuitry recruitment using fMRI.

    Directory of Open Access Journals (Sweden)

    James M Bjork

    Full Text Available BACKGROUND: Adolescent risk-taking, including behaviors resulting in injury or death, has been attributed in part to maturational differences in mesolimbic incentive-motivational neurocircuitry, including ostensible oversensitivity of the nucleus accumbens (NAcc to rewards. METHODOLOGY/PRINCIPAL FINDINGS: To test whether adolescents showed increased NAcc activation by cues for rewards, or by delivery of rewards, we scanned 24 adolescents (age 12-17 and 24 adults age (22-42 with functional magnetic resonance imaging while they performed a monetary incentive delay (MID task. The MID task was configured to temporally disentangle potential reward or potential loss anticipation-related brain signal from reward or loss notification-related signal. Subjects saw cues signaling opportunities to win or avoid losing $0, $.50, or $5 for responding quickly to a subsequent target. Subjects then viewed feedback of their trial success after a variable interval from cue presentation of between 6 to 17 s. Adolescents showed reduced NAcc recruitment by reward-predictive cues compared to adult controls in a linear contrast with non-incentive cues, and in a volume-of-interest analysis of signal change in the NAcc. In contrast, adolescents showed little difference in striatal and frontocortical responsiveness to reward deliveries compared to adults. CONCLUSIONS/SIGNIFICANCE: In light of divergent developmental difference findings between neuroimaging incentive paradigms (as well as at different stages within the same task, these data suggest that maturational differences in incentive-motivational neurocircuitry: 1 may be sensitive to nuances of incentive tasks or stimuli, such as behavioral or learning contingencies, and 2 may be specific to the component of the instrumental behavior (such as anticipation versus notification.

  15. Hemispheric Asymmetries in Striatal Reward Responses Relate to Approach-Avoidance Learning and Encoding of Positive-Negative Prediction Errors in Dopaminergic Midbrain Regions.

    Science.gov (United States)

    Aberg, Kristoffer Carl; Doell, Kimberly C; Schwartz, Sophie

    2015-10-28

    Some individuals are better at learning about rewarding situations, whereas others are inclined to avoid punishments (i.e., enhanced approach or avoidance learning, respectively). In reinforcement learning, action values are increased when outcomes are better than predicted (positive prediction errors [PEs]) and decreased for worse than predicted outcomes (negative PEs). Because actions with high and low values are approached and avoided, respectively, individual differences in the neural encoding of PEs may influence the balance between approach-avoidance learning. Recent correlational approaches also indicate that biases in approach-avoidance learning involve hemispheric asymmetries in dopamine function. However, the computational and neural mechanisms underpinning such learning biases remain unknown. Here we assessed hemispheric reward asymmetry in striatal activity in 34 human participants who performed a task involving rewards and punishments. We show that the relative difference in reward response between hemispheres relates to individual biases in approach-avoidance learning. Moreover, using a computational modeling approach, we demonstrate that better encoding of positive (vs negative) PEs in dopaminergic midbrain regions is associated with better approach (vs avoidance) learning, specifically in participants with larger reward responses in the left (vs right) ventral striatum. Thus, individual dispositions or traits may be determined by neural processes acting to constrain learning about specific aspects of the world. Copyright © 2015 the authors 0270-6474/15/3514491-10$15.00/0.

  16. Aversive counterconditioning attenuates reward signalling in the ventral striatum

    Directory of Open Access Journals (Sweden)

    Anne Marije Kaag

    2016-08-01

    Full Text Available Appetitive conditioning refers to the process of learning cue-reward associations and is mediated by the mesocorticolimbic system. Appetitive conditioned responses are difficult to extinguish, especially for highly salient rewards such as food and drugs. We investigate whether aversive counterconditioning can alter reward reinstatement in the ventral striatum in healthy volunteers using functional Magnetic Resonance Imaging (fMRI. In the initial conditioning phase, two different stimuli were reinforced with a monetary reward. In the subsequent counterconditioning phase, one of these stimuli was paired with an aversive shock to the wrist. In the following extinction phase, none of the stimuli were reinforced. In the final reinstatement phase, reward was reinstated by informing the participants that the monetary gain could be doubled. Our fMRI data revealed that reward signalling in the ventral striatum and ventral tegmental area following reinstatement was smaller for the stimulus that was counterconditioned with an electrical shock, compared to the non-counterconditioned stimulus. A functional connectivity analysis showed that aversive counterconditioning strengthened striatal connectivity with the hippocampus and insula. These results suggest that reward signalling in the ventral striatum can be attenuated through aversive counterconditioning, possibly by concurrent retrieval of the aversive association through enhanced connectivity with hippocampus and insula.

  17. The amygdala, reward and emotion.

    Science.gov (United States)

    Murray, Elisabeth A

    2007-11-01

    Recent research provides new insights into amygdala contributions to positive emotion and reward. Studies of neuronal activity in the monkey amygdala and of autonomic responses mediated by the monkey amygdala show that, contrary to a widely held view, the amygdala is just as important for processing positive reward and reinforcement as it is for negative. In addition, neuropsychological studies reveal that the amygdala is essential for only a fraction of what might be considered 'stimulus-reward processing', and that the neural substrates for emotion and reward are partially nonoverlapping. Finally, evidence suggests that two systems within the amygdala, operating in parallel, enable reward-predicting cues to influence behavior; one mediates a general, arousing effect of reward and the other links the sensory properties of reward to emotion.

  18. Competition between learned reward and error outcome predictions in anterior cingulate cortex.

    Science.gov (United States)

    Alexander, William H; Brown, Joshua W

    2010-02-15

    The anterior cingulate cortex (ACC) is implicated in performance monitoring and cognitive control. Non-human primate studies of ACC show prominent reward signals, but these are elusive in human studies, which instead show mainly conflict and error effects. Here we demonstrate distinct appetitive and aversive activity in human ACC. The error likelihood hypothesis suggests that ACC activity increases in proportion to the likelihood of an error, and ACC is also sensitive to the consequence magnitude of the predicted error. Previous work further showed that error likelihood effects reach a ceiling as the potential consequences of an error increase, possibly due to reductions in the average reward. We explored this issue by independently manipulating reward magnitude of task responses and error likelihood while controlling for potential error consequences in an Incentive Change Signal Task. The fMRI results ruled out a modulatory effect of expected reward on error likelihood effects in favor of a competition effect between expected reward and error likelihood. Dynamic causal modeling showed that error likelihood and expected reward signals are intrinsic to the ACC rather than received from elsewhere. These findings agree with interpretations of ACC activity as signaling both perceptions of risk and predicted reward. Copyright 2009 Elsevier Inc. All rights reserved.

  19. Vascular Risk Factors and Diseases Modulate Deficits of Reward-Based Reversal Learning in Acute Basal Ganglia Stroke.

    Directory of Open Access Journals (Sweden)

    Ulla K Seidel

    Full Text Available Besides motor function, the basal ganglia have been implicated in feedback learning. In patients with chronic basal ganglia infarcts, deficits in reward-based reversal learning have previously been described.We re-examined the acquisition and reversal of stimulus-stimulus-reward associations and acquired equivalence in eleven patients with acute basal ganglia stroke (8 men, 3 women; 57.8±13.3 years, whose performance was compared eleven healthy subjects of comparable age, sex distribution and education, who were recruited outside the hospital. Eleven hospitalized patients with a similar vascular risk profile as the stroke patients but without stroke history served as clinical control group.In a neuropsychological assessment 7±3 days post-stroke, verbal and spatial short-term and working memory and inhibition control did not differ between groups. Compared with healthy subjects, control patients with vascular risk factors exhibited significantly reduced performance in the reversal phase (F[2,30] = 3.47; p = 0.044; post-hoc comparison between risk factor controls and healthy controls: p = 0.030, but not the acquisition phase (F[2,30] = 1.01; p = 0.376 and the acquired equivalence (F[2,30] = 1.04; p = 0.367 tasks. In all tasks, the performance of vascular risk factor patients closely resembled that of basal ganglia stroke patients. Correlation studies revealed a significant association of the number of vascular risk factors with reversal learning (r = -0.33, p = 0.012, but not acquisition learning (r = -0.20, p = 0.121 or acquired equivalence (r = -0.22, p = 0.096.The previously reported impairment of reward-based learning may be attributed to vascular risk factors and associated diseases, which are enriched in stroke patients. This study emphasizes the necessity of appropriate control subjects in cognition studies.

  20. A Dynamic Connectome Supports the Emergence of Stable Computational Function of Neural Circuits through Reward-Based Learning.

    Science.gov (United States)

    Kappel, David; Legenstein, Robert; Habenschuss, Stefan; Hsieh, Michael; Maass, Wolfgang

    2018-01-01

    Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapse-autonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.

  1. An Attempt at Blocking of Position Learning by Training with Reward-Memory Associations

    Science.gov (United States)

    Burns, Richard A.; Johnson, Kendra S.

    2006-01-01

    Rats were runway trained with sequences of rewards that changed in 3 phases. In Phase 1 (24 days), the sequences were NP', SNP', and P'SNP' (n = 3), or NS', PNS', and S'PNS', where P and P' refer to 4 and 8 plain Noyes pellets, and S and S' are 4 and 8 sucrose pellets. N was a 30-s confinement in the goal without reward. In Phase 2 (14 days) the…

  2. Reward, Context, and Human Behaviour

    Directory of Open Access Journals (Sweden)

    Clare L. Blaukopf

    2007-01-01

    Full Text Available Animal models of reward processing have revealed an extensive network of brain areas that process different aspects of reward, from expectation and prediction to calculation of relative value. These results have been confirmed and extended in human neuroimaging to encompass secondary rewards more unique to humans, such as money. The majority of the extant literature covers the brain areas associated with rewards whilst neglecting analysis of the actual behaviours that these rewards generate. This review strives to redress this imbalance by illustrating the importance of looking at the behavioural outcome of rewards and the context in which they are produced. Following a brief review of the literature of reward-related activity in the brain, we examine the effect of reward context on actions. These studies reveal how the presence of reward vs. reward and punishment, or being conscious vs. unconscious of reward-related actions, differentially influence behaviour. The latter finding is of particular importance given the extent to which animal models are used in understanding the reward systems of the human mind. It is clear that further studies are needed to learn about the human reaction to reward in its entirety, including any distinctions between conscious and unconscious behaviours. We propose that studies of reward entail a measure of the animal's (human or nonhuman knowledge of the reward and knowledge of its own behavioural outcome to achieve that reward.

  3. The impact of napping on memory for future-relevant stimuli: Prioritization among multiple salience cues.

    Science.gov (United States)

    Bennion, Kelly A; Payne, Jessica D; Kensinger, Elizabeth A

    2016-06-01

    Prior research has demonstrated that sleep enhances memory for future-relevant information, including memory for information that is salient due to emotion, reward, or knowledge of a later memory test. Although sleep has been shown to prioritize information with any of these characteristics, the present study investigates the novel question of how sleep prioritizes information when multiple salience cues exist. Participants encoded scenes that were future-relevant based on emotion (emotional vs. neutral), reward (rewarded vs. unrewarded), and instructed learning (intentionally vs. incidentally encoded), preceding a delay consisting of a nap, an equivalent time period spent awake, or a nap followed by wakefulness (to control for effects of interference). Recognition testing revealed that when multiple dimensions of future relevance co-occur, sleep prioritizes top-down, goal-directed cues (instructed learning, and to a lesser degree, reward) over bottom-up, stimulus-driven characteristics (emotion). Further, results showed that these factors interact; the effect of a nap on intentionally encoded information was especially strong for neutral (relative to emotional) information, suggesting that once one cue for future relevance is present, there are diminishing returns with additional cues. Sleep may binarize information based on whether it is future-relevant or not, preferentially consolidating memory for the former category. Potential neural mechanisms underlying these selective effects and the implications of this research for educational and vocational domains are discussed. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

  4. Persistent effects of prior chronic exposure to corticosterone on reward-related learning and motivation in rodents.

    Science.gov (United States)

    Olausson, Peter; Kiraly, Drew D; Gourley, Shannon L; Taylor, Jane R

    2013-02-01

    Repeated or prolonged exposure to stress has profound effects on a wide spectrum of behavioral and neurobiological processes and has been associated with the pathophysiology of depression. The multifaceted nature of this disorder includes despair, anhedonia, diminished motivation, and disrupted cognition, and it has been proposed that depression is also associated with reduced reward-motivated learning. We have previously reported that prior chronic corticosterone exposure to mice produces a lasting depressive-like state that can be reversed by chronic antidepressant treatment. In the present study, we tested the effects of prior chronic exposure to corticosterone (50 μg/ml) administered to rats or to mice in drinking water for 14 days followed by dose-tapering over 9 days. The exposure to corticosterone produced lasting deficits in the acquisition of reward-related learning tested on a food-motivated instrumental task conducted 10-20 days after the last day of full dose corticosterone exposure. Rats exposed to corticosterone also displayed reduced responding on a progressive ratio schedule of reinforcement when tested on day 21 after exposure. Amitriptyline (200 mg/ml in drinking water) exposure for 14 days to mice produced the opposite effect, enhancing food-motivated instrumental acquisition and performance. Repeated treatment with amitriptyline (5 mg/kg, intraperitoneally; bid) subsequent to corticosterone exposure also prevented the corticosterone-induced deficits in rats. These results are consistent with aberrant reward-related learning and motivational processes in depressive states and provide new evidence that stress-induced neuroadaptive alterations in cortico-limbic-striatal brain circuits involved in learning and motivation may play a critical role in aspects of mood disorders.

  5. A configural dominant account of contextual cueing: Configural cues are stronger than colour cues.

    Science.gov (United States)

    Kunar, Melina A; John, Rebecca; Sweetman, Hollie

    2014-01-01

    Previous work has shown that reaction times to find a target in displays that have been repeated are faster than those for displays that have never been seen before. This learning effect, termed "contextual cueing" (CC), has been shown using contexts such as the configuration of the distractors in the display and the background colour. However, it is not clear how these two contexts interact to facilitate search. We investigated this here by comparing the strengths of these two cues when they appeared together. In Experiment 1, participants searched for a target that was cued by both colour and distractor configural cues, compared with when the target was only predicted by configural information. The results showed that the addition of a colour cue did not increase contextual cueing. In Experiment 2, participants searched for a target that was cued by both colour and distractor configuration compared with when the target was only cued by colour. The results showed that adding a predictive configural cue led to a stronger CC benefit. Experiments 3 and 4 tested the disruptive effects of removing either a learned colour cue or a learned configural cue and whether there was cue competition when colour and configural cues were presented together. Removing the configural cue was more disruptive to CC than removing colour, and configural learning was shown to overshadow the learning of colour cues. The data support a configural dominant account of CC, where configural cues act as the stronger cue in comparison to colour when they are presented together.

  6. Spatial midsession reversal learning in rats: Effects of egocentric Cue use and memory.

    Science.gov (United States)

    Rayburn-Reeves, Rebecca M; Moore, Mary K; Smith, Thea E; Crafton, Daniel A; Marden, Kelly L

    2018-07-01

    The midsession reversal task has been used to investigate behavioral flexibility and cue use in non-human animals, with results indicating differences in the degree of control by environmental cues across species. For example, time-based control has been found in rats only when tested in a T-maze apparatus and under specific conditions in which position and orientation (i.e., egocentric) cues during the intertrial interval could not be used to aid performance. Other research in an operant setting has shown that rats often produce minimal errors around the reversal location, demonstrating response patterns similar to patterns exhibited by humans and primates in this task. The current study aimed to reduce, but not eliminate, the ability for rats to utilize egocentric cues by placing the response levers on the opposite wall of the chamber in relation to the pellet dispenser. Results showed that rats made minimal errors prior to the reversal, suggesting time-based cues were not controlling responses, and that they switched to the second correct stimulus within a few trials after the reversal event. Video recordings also revealed highly structured patterns of behavior by the majority of rats, which often differed depending on which response was reinforced. We interpret these findings as evidence that rats are adept at utilizing their own egocentric cues and that these cues, along with memory for the recent response-reinforcement contingencies, aid in maximizing reinforcement over the session. Copyright © 2018 Elsevier B.V. All rights reserved.

  7. Mindfulness meditation modulates reward prediction errors in the striatum in a passive conditioning task

    Directory of Open Access Journals (Sweden)

    Ulrich eKirk

    2015-02-01

    Full Text Available Reinforcement learning models have demonstrated that phasic activity of dopamine neurons during reward expectation encodes information about the predictability of rewards and cues that predict reward. Evidence indicates that mindfulness-based approaches reduce reward anticipation signal in the striatum to negative and positive incentives suggesting the hypothesis that such training influence basic reward processing. Using a passive conditioning task and fMRI in a group of experienced mindfulness meditators and age-matched controls, we tested the hypothesis that mindfulness meditation influence reward and reward prediction error signals. We found diminished positive and negative prediction error-related blood-oxygen level-dependent (BOLD responses in the putamen in meditators compared with controls. In the meditators, this decrease in striatal BOLD responses to reward prediction was paralleled by increased activity in posterior insula, a primary interoceptive region. Critically, responses in the putamen during early trials of the conditioning procedure (run 1 were elevated in both meditators and controls. These results provide evidence that experienced mindfulness meditators show attenuated reward prediction signals to valenced stimuli, which may be related to interoceptive processes encoded in the posterior insula.

  8. Heterogeneity of reward mechanisms.

    Science.gov (United States)

    Lajtha, A; Sershen, H

    2010-06-01

    The finding that many drugs that have abuse potential and other natural stimuli such as food or sexual activity cause similar chemical changes in the brain, an increase in extracellular dopamine (DA) in the shell of the nucleus accumbens (NAccS), indicated some time ago that the reward mechanism is at least very similar for all stimuli and that the mechanism is relatively simple. The presently available information shows that the mechanisms involved are more complex and have multiple elements. Multiple brain regions, multiple receptors, multiple distinct neurons, multiple transmitters, multiple transporters, circuits, peptides, proteins, metabolism of transmitters, and phosphorylation, all participate in reward mechanisms. The system is variable, is changed during development, is sex-dependent, and is influenced by genetic differences. Not all of the elements participate in the reward of all stimuli. Different set of mechanisms are involved in the reward of different drugs of abuse, yet different mechanisms in the reward of natural stimuli such as food or sexual activity; thus there are different systems that distinguish different stimuli. Separate functions of the reward system such as anticipation, evaluation, consummation and identification; all contain function-specific elements. The level of the stimulus also influences the participation of the elements of the reward system, there are possible reactions to even below threshold stimuli, and excessive stimuli can change reward to aversion involving parts of the system. Learning and memory of past reward is an important integral element of reward and addictive behavior. Many of the reward elements are altered by repeated or chronic stimuli, and chronic exposure to one drug is likely to alter the response to another stimulus. To evaluate and identify the reward stimulus thus requires heterogeneity of the reward components in the brain.

  9. Impaired capacity of cerebellar patients to perceive and learn two-dimensional shapes based on kinesthetic cues.

    Science.gov (United States)

    Shimansky, Y; Saling, M; Wunderlich, D A; Bracha, V; Stelmach, G E; Bloedel, J R

    1997-01-01

    This study addresses the issue of the role of the cerebellum in the processing of sensory information by determining the capability of cerebellar patients to acquire and use kinesthetic cues received via the active or passive tracing of an irregular shape while blindfolded. Patients with cerebellar lesions and age-matched healthy controls were tested on four tasks: (1) learning to discriminate a reference shape from three others through the repeated tracing of the reference template; (2) reproducing the reference shape from memory by drawing blindfolded; (3) performing the same task with vision; and (4) visually recognizing the reference shape. The cues used to acquire and then to recognize the reference shape were generated under four conditions: (1) "active kinesthesia," in which cues were acquired by the blindfolded subject while actively tracing a reference template; (2) "passive kinesthesia," in which the tracing was performed while the hand was guided passively through the template; (3) "sequential vision," in which the shape was visualized by the serial exposure of small segments of its outline; and (4) "full vision," in which the entire shape was visualized. The sequential vision condition was employed to emulate the sequential way in which kinesthetic information is acquired while tracing the reference shape. The results demonstrate a substantial impairment of cerebellar patients in their capability to perceive two-dimensional irregular shapes based only on kinesthetic cues. There also is evidence that this deficit in part relates to a reduced capacity to integrate temporal sequences of sensory cues into a complete image useful for shape discrimination tasks or for reproducing the shape through drawing. Consequently, the cerebellum has an important role in this type of sensory information processing even when it is not directly associated with the execution of movements.

  10. Oxytocin attenuates trust as a subset of more general reinforcement learning, with altered reward circuit functional connectivity in males.

    Science.gov (United States)

    Ide, Jaime S; Nedic, Sanja; Wong, Kin F; Strey, Shmuel L; Lawson, Elizabeth A; Dickerson, Bradford C; Wald, Lawrence L; La Camera, Giancarlo; Mujica-Parodi, Lilianne R

    2018-07-01

    Oxytocin (OT) is an endogenous neuropeptide that, while originally thought to promote trust, has more recently been found to be context-dependent. Here we extend experimental paradigms previously restricted to de novo decision-to-trust, to a more realistic environment in which social relationships evolve in response to iterative feedback over twenty interactions. In a randomized, double blind, placebo-controlled within-subject/crossover experiment of human adult males, we investigated the effects of a single dose of intranasal OT (40 IU) on Bayesian expectation updating and reinforcement learning within a social context, with associated brain circuit dynamics. Subjects participated in a neuroeconomic task (Iterative Trust Game) designed to probe iterative social learning while their brains were scanned using ultra-high field (7T) fMRI. We modeled each subject's behavior using Bayesian updating of belief-states ("willingness to trust") as well as canonical measures of reinforcement learning (learning rate, inverse temperature). Behavioral trajectories were then used as regressors within fMRI activation and connectivity analyses to identify corresponding brain network functionality affected by OT. Behaviorally, OT reduced feedback learning, without bias with respect to positive versus negative reward. Neurobiologically, reduced learning under OT was associated with muted communication between three key nodes within the reward circuit: the orbitofrontal cortex, amygdala, and lateral (limbic) habenula. Our data suggest that OT, rather than inspiring feelings of generosity, instead attenuates the brain's encoding of prediction error and therefore its ability to modulate pre-existing beliefs. This effect may underlie OT's putative role in promoting what has typically been reported as 'unjustified trust' in the face of information that suggests likely betrayal, while also resolving apparent contradictions with regard to OT's context-dependent behavioral effects. Copyright

  11. Oxytocin selectively facilitates learning with social feedback and increases activity and functional connectivity in emotional memory and reward processing regions.

    Science.gov (United States)

    Hu, Jiehui; Qi, Song; Becker, Benjamin; Luo, Lizhu; Gao, Shan; Gong, Qiyong; Hurlemann, René; Kendrick, Keith M

    2015-06-01

    In male Caucasian subjects, learning is facilitated by receipt of social compared with non-social feedback, and the neuropeptide oxytocin (OXT) facilitates this effect. In this study, we have first shown a cultural difference in that male Chinese subjects actually perform significantly worse in the same reinforcement associated learning task with social (emotional faces) compared with non-social feedback. Nevertheless, in two independent double-blind placebo (PLC) controlled between-subject design experiments we found OXT still selectively facilitated learning with social feedback. Similar to Caucasian subjects this OXT effect was strongest with feedback using female rather than male faces. One experiment performed in conjunction with functional magnetic resonance imaging showed that during the response, but not feedback phase of the task, OXT selectively increased activity in the amygdala, hippocampus, parahippocampal gyrus and putamen during the social feedback condition, and functional connectivity between the amygdala and insula and caudate. Therefore, OXT may be increasing the salience and reward value of anticipated social feedback. In the PLC group, response times and state anxiety scores during social feedback were associated with signal changes in these same regions but not in the OXT group. OXT may therefore have also facilitated learning by reducing anxiety in the social feedback condition. Overall our results provide the first evidence for cultural differences in social facilitation of learning per se, but a similar selective enhancement of learning with social feedback under OXT. This effect of OXT may be associated with enhanced responses and functional connectivity in emotional memory and reward processing regions. © 2015 Wiley Periodicals, Inc.

  12. Metacognitive Monitoring and Dementia: How Intrinsic and Extrinsic Cues Influence Judgments of Learning in People with Alzheimer’s disease

    Science.gov (United States)

    Thomas, Ayanna K.; Lee, Meeyeon; Balota, David A.

    2014-01-01

    Objective The present research compared metamemorial monitoring processes among younger adults, non-demented older adults, and older adults diagnosed with early stage Dementia of the Alzheimer’s Type (DAT). Method In three experiments we examined the influence of intrinsic and extrinsic cues on Judgment of Learning (JOL) accuracy. Changes in association strength between cue-target word pairs served as our intrinsic manipulation in Experiments 1 and 2. Changes in encoding orientation served as our extrinsic manipulation in Experiment 3. Results Across all experiments we found that young adults, non-demented older adults, and individuals in the early stages of DAT effectively used intrinsic and extrinsic factors to guide JOL predictions. Conclusions We conclude that while certain aspects of metacognition may be impaired in both the normal and demented older populations, these groups remain able to use theory-based processing, or general knowledge about how memory works, to make metamemory monitoring predictions. PMID:23876118

  13. Early Years Education: Are Young Students Intrinsically or Extrinsically Motivated Towards School Activities? A Discussion about the Effects of Rewards on Young Children's Learning

    Science.gov (United States)

    Theodotou, Evgenia

    2014-01-01

    Rewards can reinforce and at the same time forestall young children's willingness to learn. However, they are broadly used in the field of education, especially in early years settings, to stimulate children towards learning activities. This paper reviews the theoretical and research literature related to intrinsic and extrinsic motivational…

  14. The neural dynamics of reward value and risk coding in the human orbitofrontal cortex.

    Science.gov (United States)

    Li, Yansong; Vanni-Mercier, Giovanna; Isnard, Jean; Mauguière, François; Dreher, Jean-Claude

    2016-04-01

    The orbitofrontal cortex is known to carry information regarding expected reward, risk and experienced outcome. Yet, due to inherent limitations in lesion and neuroimaging methods, the neural dynamics of these computations has remained elusive in humans. Here, taking advantage of the high temporal definition of intracranial recordings, we characterize the neurophysiological signatures of the intact orbitofrontal cortex in processing information relevant for risky decisions. Local field potentials were recorded from the intact orbitofrontal cortex of patients suffering from drug-refractory partial epilepsy with implanted depth electrodes as they performed a probabilistic reward learning task that required them to associate visual cues with distinct reward probabilities. We observed three successive signals: (i) around 400 ms after cue presentation, the amplitudes of the local field potentials increased with reward probability; (ii) a risk signal emerged during the late phase of reward anticipation and during the outcome phase; and (iii) an experienced value signal appeared at the time of reward delivery. Both the medial and lateral orbitofrontal cortex encoded risk and reward probability while the lateral orbitofrontal cortex played a dominant role in coding experienced value. The present study provides the first evidence from intracranial recordings that the human orbitofrontal cortex codes reward risk both during late reward anticipation and during the outcome phase at a time scale of milliseconds. Our findings offer insights into the rapid mechanisms underlying the ability to learn structural relationships from the environment. © The Author (2016). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

  15. Effect of reference frames and number of cues available on the spatial orientation of males and females in a virtual memory task.

    Science.gov (United States)

    Cánovas, Rosa; García, Rubén Fernández; Cimadevilla, Jose Manuel

    2011-01-01

    The aim of this study was to examine the influence of the number of cues and cue location in human spatial learning. To assess their importance, subjects performed variants of a virtual task called "The Boxes Room". Participants were trained to locate, in a computer-generated environment with 16 boxes, the rewarded boxes through 8 trials. In experiment I, the number of distal cues available was zero, one, two or the standard arrangement (seven cues). In experiment II, place navigation was compared based on distal landmarks (extra-maze cues placed on the walls) and proximal landmarks (proximal cues placed between the boxes). The results of experiment I demonstrated that one cue in the room is enough to obtain a good performance in the task. Experiment II showed that groups using proximal cues were slower and less accurate than groups using distal cues. In addition, our data suggest that men are better navigators than women, as they found the rewarded boxes sooner and committed fewer errors in both studies. These results indicate that performance can change depending on the number and location of available cues. Copyright © 2010 Elsevier B.V. All rights reserved.

  16. Adolescent development of context-dependent stimulus-reward association memory and its neural correlates.

    Science.gov (United States)

    Voss, Joel L; O'Neil, Jonathan T; Kharitonova, Maria; Briggs-Gowan, Margaret J; Wakschlag, Lauren S

    2015-01-01

    Expression of learned stimulus-reward associations based on context is essential for regulation of behavior to meet situational demands. Contextual regulation improves during development, although the developmental progression of relevant neural and cognitive processes is not fully specified. We therefore measured neural correlates of flexible, contextual expression of stimulus-reward associations in pre/early-adolescent children (ages 9-13 years) and young adults (ages 19-22 years). After reinforcement learning using standard parameters, a contextual reversal manipulation was used whereby contextual cues indicated that stimulus-reward associations were the same as previously reinforced for some trials (consistent trials) or were reversed on other trials (inconsistent trials). Subjects were thus required to respond according to original stimulus-reward associations vs. reversed associations based on trial-specific contextual cues. Children and young adults did not differ in reinforcement learning or in relevant functional magnetic resonance imaging (fMRI) correlates. In contrast, adults outperformed children during contextual reversal, with better performance specifically for inconsistent trials. fMRI signals corresponding to this selective advantage included greater activity in lateral prefrontal cortex (LPFC), hippocampus, and dorsal striatum for young adults relative to children. Flexible expression of stimulus-reward associations based on context thus improves via adolescent development, as does recruitment of brain regions involved in reward learning and contextual expression of memory. HighlightsEarly-adolescent children and young adults were equivalent in reinforcement learning.Adults outperformed children in contextual expression of stimulus-reward associations.Adult advantages correlated with increased activity of relevant brain regions.Specific neurocognitive developmental changes support better contextual regulation.

  17. How Food as a Reward Is Detrimental to Children's Health, Learning, and Behavior

    Science.gov (United States)

    Fedewa, Alicia L.; Davis, Matthew Cody

    2015-01-01

    Background: Despite small- and wide-scale prevention efforts to curb obesity, the percentage of children classified as overweight and obese has remained relatively consistent in the last decade. As school personnel are increasingly pressured to enhance student performance, many educators use food as a reward to motivate and reinforce positive…

  18. Why is obesity such a problem in the 21st century? The intersection of palatable food, cues and reward pathways, stress, and cognition.

    Science.gov (United States)

    Morris, Margaret J; Beilharz, Jessica E; Maniam, Jayanthi; Reichelt, Amy C; Westbrook, R Frederick

    2015-11-01

    Changes in food composition and availability have contributed to the dramatic increase in obesity over the past 30-40 years in developed and, increasingly, in developing countries. The brain plays a critical role in regulating energy balance. Some human studies have demonstrated increased preference for high fat and high sugar foods in people reporting greater stress exposure. We have examined neurochemical changes in the brain in rodent models during the development of obesity, including the impact of obesity on cognition, reward neurocircuitry and stress responsiveness. Using supermarket foods high in fat and sugar, we showed that such a diet leads to changes in neurotransmitters involved in the hedonic appraisal of foods, indicative of an addiction-like capacity of foods high in fat and/or sugar. Importantly, withdrawal of the palatable diet led to a stress-like response. Furthermore, access to this palatable diet attenuated the physiological effects of acute stress (restraint), indicating that it could act as a comfort food. In more chronic studies, the diet also attenuated anxiety-like behavior in rats exposed to stress (maternal separation) early in life, but these rats may suffer greater metabolic harm than rats exposed to the early life stressor but not provided with the palatable diet. Impairments in cognitive function have been associated with obesity in both people and rodents. However, as little as 1 week of exposure to a high fat, high sugar diet selectively impaired place but not object recognition memory in the rat. Excess sugar alone had similar effects, and both diets were linked to increased inflammatory markers in the hippocampus, a critical region involved in memory. Obesity-related inflammatory changes have been found in the human brain. Ongoing work examines interventions to prevent or reverse diet-induced cognitive impairments. These data have implications for minimizing harm caused by unhealthy eating. Copyright © 2014 Elsevier Ltd. All

  19. Time to learn: evidence for two types of attentional guidance in contextual cueing.

    Science.gov (United States)

    Ogawa, Hirokazu; Watanabe, Katsumi

    2010-01-01

    Repetition of the same spatial configurations of a search display implicitly facilitates performance of a visual-search task when the target location in the display is fixed. The improvement of performance is referred to as contextual cueing. We examined whether the association process between target location and surrounding configuration of distractors occurs during active search or at the instant the target is found. To dissociate these two processes, we changed the surrounding configuration of the distractors at the instant of target detection so that the layout where the participants had searched for the target and the layout presented at the instant of target detection differed. The results demonstrated that both processes are responsible for the contextual-cueing effect, but they differ in the accuracies of attentional guidance and their time courses, suggesting that two different types of attentional-guidance processes may be involved in contextual cueing.

  20. The influence of attention and reward on the learning of stimulus-response associations

    NARCIS (Netherlands)

    Vartak, Devavrat; Jeurissen, Danique; Self, Matthew W; Roelfsema, Pieter R

    2017-01-01

    We can learn new tasks by listening to a teacher, but we can also learn by trial-and-error. Here, we investigate the factors that determine how participants learn new stimulus-response mappings by trial-and-error. Does learning in human observers comply with reinforcement learning theories, which

  1. TARGETED SEQUENTIAL DESIGN FOR TARGETED LEARNING INFERENCE OF THE OPTIMAL TREATMENT RULE AND ITS MEAN REWARD.

    Science.gov (United States)

    Chambaz, Antoine; Zheng, Wenjing; van der Laan, Mark J

    2017-01-01

    This article studies the targeted sequential inference of an optimal treatment rule (TR) and its mean reward in the non-exceptional case, i.e. , assuming that there is no stratum of the baseline covariates where treatment is neither beneficial nor harmful, and under a companion margin assumption. Our pivotal estimator, whose definition hinges on the targeted minimum loss estimation (TMLE) principle, actually infers the mean reward under the current estimate of the optimal TR. This data-adaptive statistical parameter is worthy of interest on its own. Our main result is a central limit theorem which enables the construction of confidence intervals on both mean rewards under the current estimate of the optimal TR and under the optimal TR itself. The asymptotic variance of the estimator takes the form of the variance of an efficient influence curve at a limiting distribution, allowing to discuss the efficiency of inference. As a by product, we also derive confidence intervals on two cumulated pseudo-regrets, a key notion in the study of bandits problems. A simulation study illustrates the procedure. One of the corner-stones of the theoretical study is a new maximal inequality for martingales with respect to the uniform entropy integral.

  2. Olfactory memory formation and the influence of reward pathway during appetitive learning by honey bees.

    Science.gov (United States)

    Wright, Geraldine A; Mustard, Julie A; Kottcamp, Sonya M; Smith, Brian H

    2007-11-01

    Animals possess the ability to assess food quality via taste and via changes in state that occur after ingestion. Here, we investigate the extent to which a honey bee's ability to assess food quality affected the formation of association with an odor stimulus and the retention of olfactory memories associated with reward. We used three different conditioning protocols in which the unconditioned stimulus (food) was delivered as sucrose stimulation to the proboscis (mouthparts), the antennae or to both proboscis and antennae. All means of delivery of the unconditioned stimulus produced robust associative conditioning with an odor. However, the memory of a conditioned odor decayed at a significantly greater rate for subjects experiencing antennal-only stimulation after either multiple- or single-trial conditioning. Finally, to test whether the act of feeding on a reward containing sucrose during conditioning affected olfactory memory formation, we conditioned honey bees to associate an odor with antennal stimulation with sucrose followed by feeding on a water droplet. We observed that a honey bee's ability to recall the conditioned odor was not significantly different from that of subjects conditioned with an antennal-only sucrose stimulus. Our results show that stimulation of the sensory receptors on the proboscis and/or ingestion of the sucrose reward during appetitive olfactory conditioning are necessary for long-term memory formation.

  3. Effects of an acute therapeutic or rewarding dose of amphetamine on acquisition of Pavlovian autoshaping and ventral striatal dopamine signaling.

    Science.gov (United States)

    Schuweiler, D R; Athens, J M; Thompson, J M; Vazhayil, S T; Garris, P A

    2018-01-15

    Rewarding doses of amphetamine increase the amplitude, duration, and frequency of dopamine transients in the ventral striatum. Debate continues at the behavioral level about which component of reward, learning or incentive salience, is signaled by these dopamine transients and thus altered in addiction. The learning hypothesis proposes that rewarding drugs result in pathological overlearning of drug-predictive cues, while the incentive sensitization hypothesis suggests that rewarding drugs result in sensitized attribution of incentive salience to drug-predictive cues. Therapeutic doses of amphetamine, such as those used to treat attention-deficit hyperactivity disorder, are hypothesized to enhance the ventral striatal dopamine transients that are critical for reward-related learning and to enhance Pavlovian learning. However, the effects of therapeutic doses of amphetamine on Pavlovian learning are poorly understood, and the effects on dopamine transients are completely unknown. We determined the effects of an acute pre-training therapeutic or rewarding amphetamine injection on the acquisition of Pavlovian autoshaping in the intact rat. We also determined the effects of these doses on electrically evoked transient-like dopamine signals using fast-scan cyclic voltammetry in the anesthetized rat. The rewarding dose enhanced the amplitude and duration of DA signals, caused acute task disengagement, impaired learning for several days, and triggered incentive sensitization. The therapeutic dose produced smaller enhancements in DA signals but did not have similar behavioral effects. These results underscore the necessity of more studies using therapeutic doses, and suggest a hybrid learning/incentive sensitization model may be required to explain the development of addiction. Copyright © 2017 Elsevier B.V. All rights reserved.

  4. The Prelimbic Cortex Directs Attention toward Predictive Cues during Fear Learning

    Science.gov (United States)

    Sharpe, Melissa J.; Killcross, Simon

    2015-01-01

    The prelimbic cortex is argued to promote conditioned fear expression, at odds with appetitive research implicating this region in attentional processing. Consistent with an attentional account, we report that the effect of prelimbic lesions on fear expression depends on the degree of competition between contextual and discrete cues. Further, when…

  5. Evidence for a shared representation of sequential cues that engage sign-tracking.

    Science.gov (United States)

    Smedley, Elizabeth B; Smith, Kyle S

    2018-06-19

    Sign-tracking is a phenomenon whereby cues that predict rewards come to acquire their own motivational value (incentive salience) and attract appetitive behavior. Typically, sign-tracking paradigms have used single auditory, visual, or lever cues presented prior to a reward delivery. Yet, real world examples of events often can be predicted by a sequence of cues. We have shown that animals will sign-track to multiple cues presented in temporal sequence, and with time develop a bias in responding toward a reward distal cue over a reward proximal cue. Further, extinction of responding to the reward proximal cue directly decreases responding to the reward distal cue. One possible explanation of this result is that serial cues become representationally linked with one another. Here we provide further support of this by showing that extinction of responding to a reward distal cue directly reduces responding to a reward proximal cue. We suggest that the incentive salience of one cue can influence the incentive salience of the other cue. Copyright © 2018. Published by Elsevier B.V.

  6. AN EXTENDED REINFORCEMENT LEARNING MODEL OF BASAL GANGLIA TO UNDERSTAND THE CONTRIBUTIONS OF SEROTONIN AND DOPAMINE IN RISK-BASED DECISION MAKING, REWARD PREDICTION, AND PUNISHMENT LEARNING

    Directory of Open Access Journals (Sweden)

    Pragathi Priyadharsini Balasubramani

    2014-04-01

    Full Text Available Although empirical and neural studies show that serotonin (5HT plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL-framework. The model depicts the roles of dopamine (DA and serotonin (5HT in Basal Ganglia (BG. In this model, the DA signal is represented by the temporal difference error (δ, while the 5HT signal is represented by a parameter (α that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: 1 Risk-sensitive decision making, where 5HT controls risk assessment, 2 Temporal reward prediction, where 5HT controls time-scale of reward prediction, and 3 Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.

  7. Premotor and Motor Cortices Encode Reward.

    Directory of Open Access Journals (Sweden)

    Pavan Ramkumar

    Full Text Available Rewards associated with actions are critical for motivation and learning about the consequences of one's actions on the world. The motor cortices are involved in planning and executing movements, but it is unclear whether they encode reward over and above limb kinematics and dynamics. Here, we report a categorical reward signal in dorsal premotor (PMd and primary motor (M1 neurons that corresponds to an increase in firing rates when a trial was not rewarded regardless of whether or not a reward was expected. We show that this signal is unrelated to error magnitude, reward prediction error, or other task confounds such as reward consumption, return reach plan, or kinematic differences across rewarded and unrewarded trials. The availability of reward information in motor cortex is crucial for theories of reward-based learning and motivational influences on actions.

  8. Teachers' Motives for Learning in Networks: Costs, Rewards and Community Interest

    Science.gov (United States)

    van den Beemt, Antoine; Ketelaar, Evelien; Diepstraten, Isabelle; de Laat, Maarten

    2018-01-01

    Background: This paper discusses teachers' perspectives on learning networks and their motives for participating in these networks. Although it is widely held that teachers' learning may be developed through learning networks, not all teachers participate in such networks. Purpose: The theme of reciprocity, central to studies in the area of…

  9. Integrating Service Learning into Public Relations Coursework: Applications, Implications, Challenges, and Rewards

    Science.gov (United States)

    Gleason, James P.; Violette, Jayne L.

    2012-01-01

    Drawing on a theoretical framework based on "use-inspired" applied research and service learning practice (Honnet-Porter & Poulsen, 1989), this paper argues the relationship between a service-learning approach and Public Relations coursework is a natural and highly desirable fit. Through examination of the goals of both service-learning and public…

  10. Teachers’ motives for learning in networks : costs, rewards and community interest

    NARCIS (Netherlands)

    van den Beemt, A.A.J.; Ketelaar, E.; Diepstraten, I.; de Laat, M.

    2018-01-01

    Background: This paper discusses teachers’ perspectives on learning networks and their motives for participating in these networks. Although it is widely held that teachers’ learning may be developed through learning networks, not all teachers participate in such networks. Purpose: The theme of

  11. First-Pass Processing of Value Cues in the Ventral Visual Pathway.

    Science.gov (United States)

    Sasikumar, Dennis; Emeric, Erik; Stuphorn, Veit; Connor, Charles E

    2018-02-19

    Real-world value often depends on subtle, continuously variable visual cues specific to particular object categories, like the tailoring of a suit, the condition of an automobile, or the construction of a house. Here, we used microelectrode recording in behaving monkeys to test two possible mechanisms for category-specific value-cue processing: (1) previous findings suggest that prefrontal cortex (PFC) identifies object categories, and based on category identity, PFC could use top-down attentional modulation to enhance visual processing of category-specific value cues, providing signals to PFC for calculating value, and (2) a faster mechanism would be first-pass visual processing of category-specific value cues, immediately providing the necessary visual information to PFC. This, however, would require learned mechanisms for processing the appropriate cues in a given object category. To test these hypotheses, we trained monkeys to discriminate value in four letter-like stimulus categories. Each category had a different, continuously variable shape cue that signified value (liquid reward amount) as well as other cues that were irrelevant. Monkeys chose between stimuli of different reward values. Consistent with the first-pass hypothesis, we found early signals for category-specific value cues in area TE (the final stage in monkey ventral visual pathway) beginning 81 ms after stimulus onset-essentially at the start of TE responses. Task-related activity emerged in lateral PFC approximately 40 ms later and consisted mainly of category-invariant value tuning. Our results show that, for familiar, behaviorally relevant object categories, high-level ventral pathway cortex can implement rapid, first-pass processing of category-specific value cues. Copyright © 2018 Elsevier Ltd. All rights reserved.

  12. Rewards and Performance Incentives.

    Science.gov (United States)

    Zigon, Jack

    1994-01-01

    Discusses rewards and performance incentives for employees, including types of rewards; how rewards help in managing; dysfunctional awards; selecting the right reward; how to find rewards that fit; and delivering rewards effectively. Examples are included. (three references) (LRW)

  13. The more total cognitive load is reduced by cues, the better retention and transfer of multimedia learning: A meta-analysis and two meta-regression analyses.

    Science.gov (United States)

    Xie, Heping; Wang, Fuxing; Hao, Yanbin; Chen, Jiaxue; An, Jing; Wang, Yuxin; Liu, Huashan

    2017-01-01

    Cueing facilitates retention and transfer of multimedia learning. From the perspective of cognitive load theory (CLT), cueing has a positive effect on learning outcomes because of the reduction in total cognitive load and avoidance of cognitive overload. However, this has not been systematically evaluated. Moreover, what remains ambiguous is the direct relationship between the cue-related cognitive load and learning outcomes. A meta-analysis and two subsequent meta-regression analyses were conducted to explore these issues. Subjective total cognitive load (SCL) and scores on a retention test and transfer test were selected as dependent variables. Through a systematic literature search, 32 eligible articles encompassing 3,597 participants were included in the SCL-related meta-analysis. Among them, 25 articles containing 2,910 participants were included in the retention-related meta-analysis and the following retention-related meta-regression, while there were 29 articles containing 3,204 participants included in the transfer-related meta-analysis and the transfer-related meta-regression. The meta-analysis revealed a statistically significant cueing effect on subjective ratings of cognitive load (d = -0.11, 95% CI = [-0.19, -0.02], p < 0.05), retention performance (d = 0.27, 95% CI = [0.08, 0.46], p < 0.01), and transfer performance (d = 0.34, 95% CI = [0.12, 0.56], p < 0.01). The subsequent meta-regression analyses showed that dSCL for cueing significantly predicted dretention for cueing (β = -0.70, 95% CI = [-1.02, -0.38], p < 0.001), as well as dtransfer for cueing (β = -0.60, 95% CI = [-0.92, -0.28], p < 0.001). Thus in line with CLT, adding cues in multimedia materials can indeed reduce SCL and promote learning outcomes, and the more SCL is reduced by cues, the better retention and transfer of multimedia learning.

  14. Reward positivity: Reward prediction error or salience prediction error?

    Science.gov (United States)

    Heydari, Sepideh; Holroyd, Clay B

    2016-08-01

    The reward positivity is a component of the human ERP elicited by feedback stimuli in trial-and-error learning and guessing tasks. A prominent theory holds that the reward positivity reflects a reward prediction error signal that is sensitive to outcome valence, being larger for unexpected positive events relative to unexpected negative events (Holroyd & Coles, 2002). Although the theory has found substantial empirical support, most of these studies have utilized either monetary or performance feedback to test the hypothesis. However, in apparent contradiction to the theory, a recent study found that unexpected physical punishments also elicit the reward positivity (Talmi, Atkinson, & El-Deredy, 2013). The authors of this report argued that the reward positivity reflects a salience prediction error rather than a reward prediction error. To investigate this finding further, in the present study participants navigated a virtual T maze and received feedback on each trial under two conditions. In a reward condition, the feedback indicated that they would either receive a monetary reward or not and in a punishment condition the feedback indicated that they would receive a small shock or not. We found that the feedback stimuli elicited a typical reward positivity in the reward condition and an apparently delayed reward positivity in the punishment condition. Importantly, this signal was more positive to the stimuli that predicted the omission of a possible punishment relative to stimuli that predicted a forthcoming punishment, which is inconsistent with the salience hypothesis. © 2016 Society for Psychophysiological Research.

  15. Reward and punishment.

    Science.gov (United States)

    Sigmund, K; Hauert, C; Nowak, M A

    2001-09-11

    Minigames capturing the essence of Public Goods experiments show that even in the absence of rationality assumptions, both punishment and reward will fail to bring about prosocial behavior. This result holds in particular for the well-known Ultimatum Game, which emerges as a special case. But reputation can induce fairness and cooperation in populations adapting through learning or imitation. Indeed, the inclusion of reputation effects in the corresponding dynamical models leads to the evolution of economically productive behavior, with agents contributing to the public good and either punishing those who do not or rewarding those who do. Reward and punishment correspond to two types of bifurcation with intriguing complementarity. The analysis suggests that reputation is essential for fostering social behavior among selfish agents, and that it is considerably more effective with punishment than with reward.

  16. Attention Cueing as a Means to Enhance Learning from an Animation

    NARCIS (Netherlands)

    B.B. de Koning (Björn); H.K. Tabbers (Huib); R.M.J.P. Rikers (Remy); G.W.C. Paas (Fred)

    2007-01-01

    textabstractThe question how animations should be designed so that learning is optimised, is still under discussion. Animations are often cognitively very demanding, resulting in decreased learning outcomes. In this study, we tried to prevent cognitive overload and foster learning by focusing the

  17. Differential encoding of factors influencing predicted reward value in monkey rostral anterior cingulate cortex.

    Science.gov (United States)

    Toda, Koji; Sugase-Miyamoto, Yasuko; Mizuhiki, Takashi; Inaba, Kiyonori; Richmond, Barry J; Shidara, Munetaka

    2012-01-01

    The value of a predicted reward can be estimated based on the conjunction of both the intrinsic reward value and the length of time to obtain it. The question we addressed is how the two aspects, reward size and proximity to reward, influence the responses of neurons in rostral anterior cingulate cortex (rACC), a brain region thought to play an important role in reward processing. We recorded from single neurons while two monkeys performed a multi-trial reward schedule task. The monkeys performed 1-4 sequential color discrimination trials to obtain a reward of 1-3 liquid drops. There were two task conditions, a valid cue condition, where the number of trials and reward amount were associated with visual cues, and a random cue condition, where the cue was picked from the cue set at random. In the valid cue condition, the neuronal firing is strongly modulated by the predicted reward proximity during the trials. Information about the predicted reward amount is almost absent at those times. In substantial subpopulations, the neuronal responses decreased or increased gradually through schedule progress to the predicted outcome. These two gradually modulating signals could be used to calculate the effect of time on the perception of reward value. In the random cue condition, little information about the reward proximity or reward amount is encoded during the course of the trial before reward delivery, but when the reward is actually delivered the responses reflect both the reward proximity and reward amount. Our results suggest that the rACC neurons encode information about reward proximity and amount in a manner that is dependent on utility of reward information. The manner in which the information is represented could be used in the moment-to-moment calculation of the effect of time and amount on predicted outcome value.

  18. Learned Helplessness in High School Students Following Experience of Noncontingent Rewards.

    Science.gov (United States)

    Buys, Nicholas J.; Winefield, Anthony H.

    1982-01-01

    Compared high-school students differing in achievement motivation in a learned helplessness experiment. A strong helplessness effect was observed in both high- and low-achievement motivation groups. Results show a strong learned helplessness effect unrelated to individual differences in achievement motivation and refute claims that helplessness…

  19. Brain Circuits Encoding Reward from Pain Relief.

    Science.gov (United States)

    Navratilova, Edita; Atcherley, Christopher W; Porreca, Frank

    2015-11-01

    Relief from pain in humans is rewarding and pleasurable. Primary rewards, or reward-predictive cues, are encoded in brain reward/motivational circuits. While considerable advances have been made in our understanding of reward circuits underlying positive reinforcement, less is known about the circuits underlying the hedonic and reinforcing actions of pain relief. We review findings from electrophysiological, neuroimaging, and behavioral studies supporting the concept that the rewarding effect of pain relief requires opioid signaling in the anterior cingulate cortex (ACC), activation of midbrain dopamine neurons, and the release of dopamine in the nucleus accumbens (NAc). Understanding of circuits that govern the reward of pain relief may allow the discovery of more effective and satisfying therapies for patients with acute or chronic pain.

  20. Reward components of feeding behavior are preserved during mouse aging

    Directory of Open Access Journals (Sweden)

    Mazen R. Harb

    2014-09-01

    Full Text Available Eating behavior depends on associations between the sensory and energetic properties of foods. Healthful balance of these factors is a challenge for industrialized societies that have an abundance of food, food choices and food-related cues. Here, we were interested in whether appetitive conditioning changes as a function of age. Operant and pavlovian conditioning experiments (rewarding stimulus was a palatable food in male mice (aged 3, 6 and 15 months showed that implicit (non-declarative memory remains intact during aging. Two other essential components of eating behavior, motivation and hedonic preference for rewarding foods, were also found not to be altered in aging mice. Specifically, hedonic responding by satiated mice to isocaloric foods of differing sensory properties (sucrose, milk was similar in all age groups; importantly, however, this paradigm disclosed that older animals adjust their energy intake according to energetic need. Based on the assumption that the mechanisms that control feeding are conserved across species, it would appear that overeating and obesity in humans reflects a mismatch between ancient physiological mechanisms and today’s cue-laden environment. The implication of the present results showing that aging does not impair the ability to learn stimulus-food associations is that the risk of overeating in response to food cues is maintained through to old age.

  1. Reward components of feeding behavior are preserved during mouse aging.

    Science.gov (United States)

    Harb, Mazen R; Sousa, Nuno; Zihl, Joseph; Almeida, Osborne F X

    2014-01-01

    Eating behavior depends on associations between the sensory and energetic properties of foods. Healthful balance of these factors is a challenge for industrialized societies that have an abundance of food, food choices and food-related cues. Here, we were interested in whether appetitive conditioning changes as a function of age. Operant and pavlovian conditioning experiments (rewarding stimulus was a palatable food) in male mice (aged 3, 6, and 15 months) showed that implicit (non-declarative) memory remains intact during aging. Two other essential components of eating behavior, motivation and hedonic preference for rewarding foods, were also found not to be altered in aging mice. Specifically, hedonic responding by satiated mice to isocaloric foods of differing sensory properties (sucrose, milk) was similar in all age groups; importantly, however, this paradigm disclosed that older animals adjust their energy intake according to energetic need. Based on the assumption that the mechanisms that control feeding are conserved across species, it would appear that overeating and obesity in humans reflects a mismatch between ancient physiological mechanisms and today's cue-laden environment. The implication of the present results showing that aging does not impair the ability to learn stimulus-food associations is that the risk of overeating in response to food cues is maintained through to old age.

  2. The Effective Use of Symbols in Teaching Word Recognition to Children with Severe Learning Difficulties: A Comparison of Word Alone, Integrated Picture Cueing and the Handle Technique.

    Science.gov (United States)

    Sheehy, Kieron

    2002-01-01

    A comparison is made between a new technique (the Handle Technique), Integrated Picture Cueing, and a Word Alone Method. Results show using a new combination of teaching strategies enabled logographic symbols to be used effectively in teaching word recognition to 12 children with severe learning difficulties. (Contains references.) (Author/CR)

  3. Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia.

    Directory of Open Access Journals (Sweden)

    Erin C Dowd

    Full Text Available Reward processing abnormalities have been implicated in the pathophysiology of negative symptoms such as anhedonia and avolition in schizophrenia. However, studies examining neural responses to reward anticipation and receipt have largely relied on instrumental tasks, which may confound reward processing abnormalities with deficits in response selection and execution. 25 chronic, medicated outpatients with schizophrenia and 20 healthy controls underwent functional magnetic resonance imaging using a pavlovian reward prediction paradigm with no response requirements. Subjects passively viewed cues that predicted subsequent receipt of monetary reward or non-reward, and blood-oxygen-level-dependent signal was measured at the time of cue presentation and receipt. At the group level, neural responses to both reward anticipation and receipt were largely similar between groups. At the time of cue presentation, striatal anticipatory responses did not differ between patients and controls. Right anterior insula demonstrated greater activation for nonreward than reward cues in controls, and for reward than nonreward cues in patients. At the time of receipt, robust responses to receipt of reward vs. nonreward were seen in striatum, midbrain, and frontal cortex in both groups. Furthermore, both groups demonstrated responses to unexpected versus expected outcomes in cortical areas including bilateral dorsolateral prefrontal cortex. Individual difference analyses in patients revealed an association between physical anhedonia and activity in ventral striatum and ventromedial prefrontal cortex during anticipation of reward, in which greater anhedonia severity was associated with reduced activation to money versus no-money cues. In ventromedial prefrontal cortex, this relationship held among both controls and patients, suggesting a relationship between anticipatory activity and anhedonia irrespective of diagnosis. These findings suggest that in the absence of

  4. Expected reward value and reward uncertainty have temporally dissociable effects on memory formation

    OpenAIRE

    Adcock, R; Clement, Nathaniel; Chiew, Kimberly; Dickerson, Kathryn; Stanek, Jessica

    2018-01-01

    Anticipating rewards has been shown to enhance memory formation. While substantial evidence implicates dopamine in this behavioral effect, the precise mechanisms remain ambiguous. Because dopamine nuclei show two distinct physiological signatures of reward prediction, we hypothesized two dissociable effects on memory formation. These two signatures are a phasic dopamine response immediately following a reward cue that encodes its expected value, and a sustained, ramping dopamine response that...

  5. The hypocretins and the reward function: what have we learned so far?

    Directory of Open Access Journals (Sweden)

    Benjamin eBoutrel

    2013-06-01

    Full Text Available A general consensus acknowledges that drug consumption (including alcohol, tobacco and illicit drugs constitutes the leading cause of preventable death worldwide. Dramatically, drug abuse is not only a major cause of mortality. The comorbid long-term debilitating effects also significantly deteriorates the quality of life of individuals suffering from addiction disorders. Despite the large body of evidence delineating the cellular and molecular adaptations induced by chronic drug consumption, the brain mechanisms responsible for drug craving and relapse remain insufficiently understood, and even the most recent developments in the field have not brought significant improvement in the management of drug dependence. Though, recent preclinical evidence suggests that disrupting the hypocretin (orexin system may serve as an anticraving medication therapy. Here, we discuss how the hypocretins, which orchestrate normal wakefulness, metabolic health and the execution of goal-oriented behaviors, may be compromised and contribute to elicit compulsive drug seeking. We propose an overview on the most recent studies demonstrating an important role for the hypocretin neuropeptide system in the regulation of drug reward and the prevention of drug relapse, and we question the relevance of disrupting the hypocretin system to alleviate symptoms of drug addiction.

  6. Action Speaks Louder than Words: Young Children Differentially Weight Perceptual, Social, and Linguistic Cues to Learn Verbs

    Science.gov (United States)

    Brandone, Amanda C.; Pence, Khara L.; Golinkoff, Roberta Michnick; Hirsh-Pasek, Kathy

    2007-01-01

    This paper explores how children use two possible solutions to the verb-mapping problem: attention to perceptually salient actions and attention to social and linguistic information (speaker cues). Twenty-two-month-olds attached a verb to one of two actions when perceptual cues (presence/absence of a result) coincided with speaker cues but not…

  7. Bilingual children weigh speaker’s referential cues and word-learning heuristics differently in different language contexts when interpreting a speaker’s intent

    Directory of Open Access Journals (Sweden)

    Wan-Yu eHung

    2015-06-01

    Full Text Available Past research has investigated how children use different sources of information such as social cues and word-learning heuristics to infer referential intents. The present research explored how children weigh and use some of these cues to make referential inferences. Specifically, we examined how switching between languages known (familiar or unknown (unfamiliar to a child would influence his or her choice of cue to interpret a novel label in a challenging disambiguation task, where a pointing cue was pitted against the mutual exclusivity (ME principle. Forty-eight 3-and 4-year-old English-Mandarin bilingual children listened to a story told either in English only (No-Switch, English and Mandarin (Familiar-Switch, English and Japanese (Unfamiliar-Switch, or English and English-sounding nonsense sentences (Nonsense-Switch. They were then asked to select an object (from a pair of familiar and novel objects after hearing a novel label paired with the speaker’s point at the familiar object, e.g., Can you give me the blicket? Results showed that children in the Familiar-Switch condition were more willing to relax ME to follow the speaker’s point to pick the familiar object than those in the Unfamiliar-Switch condition, who were more likely to pick the novel object. No significant differences were found between the other conditions. Further analyses revealed that children in the Unfamiliar-Switch condition looked at the speaker longer than children in the other conditions when the switch happened. Our findings suggest that children weigh speakers’ referential cues and word-learning heuristics differently in different language contexts while taking into account their communicative history with the speaker. There are important implications for general education and other learning efforts, such as designing learning games so that the history of credibility with the user is maintained and how learning may be best scaffolded in a helpful and trusting

  8. Are Distal and Proximal Visual Cues Equally Important during Spatial Learning in Mice? A Pilot Study of Overshadowing in the Spatial Domain

    Directory of Open Access Journals (Sweden)

    Marie Hébert

    2017-06-01

    Full Text Available Animals use distal and proximal visual cues to accurately navigate in their environment, with the possibility of the occurrence of associative mechanisms such as cue competition as previously reported in honey-bees, rats, birds and humans. In this pilot study, we investigated one of the most common forms of cue competition, namely the overshadowing effect, between visual landmarks during spatial learning in mice. To this end, C57BL/6J × Sv129 mice were given a two-trial place recognition task in a T-maze, based on a novelty free-choice exploration paradigm previously developed to study spatial memory in rodents. As this procedure implies the use of different aspects of the environment to navigate (i.e., mice can perceive from each arm of the maze, we manipulated the distal and proximal visual landmarks during both the acquisition and retrieval phases. Our prospective findings provide a first set of clues in favor of the occurrence of an overshadowing between visual cues during a spatial learning task in mice when both types of cues are of the same modality but at varying distances from the goal. In addition, the observed overshadowing seems to be non-reciprocal, as distal visual cues tend to overshadow the proximal ones when competition occurs, but not vice versa. The results of the present study offer a first insight about the occurrence of associative mechanisms during spatial learning in mice, and may open the way to promising new investigations in this area of research. Furthermore, the methodology used in this study brings a new, useful and easy-to-use tool for the investigation of perceptive, cognitive and/or attentional deficits in rodents.

  9. Participatory cues and program familiarity predict young children’s learning from educational television

    NARCIS (Netherlands)

    Piotrowski, J.

    2014-01-01

    The capacity model is designed to predict young children's learning from educational television. It posits that select program features and individual child characteristics can support this learning either by increasing total working memory allocated to the program or altering the allocation of

  10. Drug-sensitive reward in crayfish: an invertebrate model system for the study of SEEKING, reward, addiction, and withdrawal.

    Science.gov (United States)

    Huber, Robert; Panksepp, Jules B; Nathaniel, Thomas; Alcaro, Antonio; Panksepp, Jaak

    2011-10-01

    In mammals, rewarding properties of drugs depend on their capacity to activate appetitive motivational states. With the underlying mechanisms strongly conserved in evolution, invertebrates have recently emerged as a powerful new model in addiction research. In crayfish natural reward has proven surprisingly sensitive to human drugs of abuse, opening an unlikely avenue of research into the basic biological mechanisms of drug addiction. In a series of studies we first examined the presence of natural reward systems in crayfish, then characterized its sensitivity to a wide range of human drugs of abuse. A conditioned place preference (CPP) paradigm was used to demonstrate that crayfish seek out those environments that had previously been paired with the psychostimulants cocaine and amphetamine, and the opioid morphine. The administration of amphetamine exerted its effects at a number of sites, including the stimulation of circuits for active exploratory behaviors (i.e., SEEKING). A further study examined morphine-induced reward, extinction and reinstatement in crayfish. Repeated intra-circulatory infusions of morphine served as a reward when paired with distinct visual or tactile cues. Morphine-induced CPP was extinguished after repeated saline injections. Following this extinction phase, morphine-experienced crayfish were once again challenged with the drug. The priming injections of morphine reinstated CPP at all tested doses, suggesting that morphine-induced CPP is unrelenting. In an exploration of drug-associated behavioral sensitization in crayfish we concurrently mapped measures of locomotion and rewarding properties of morphine. Single and repeated intra-circulatory infusions of morphine resulted in persistent locomotory sensitization, even 5 days following the infusion. Moreover, a single dose of morphine was sufficient to induce long-term behavioral sensitization. CPP for morphine and context-dependent cues could not be disrupted over a drug free period of 5

  11. “Liking” and “Wanting” Linked to Reward Deficiency Syndrome (RDS): Hypothesizing Differential Responsivity in Brain Reward Circuitry

    OpenAIRE

    Blum, Kenneth; Gardner, Eliot; Oscar-Berman, Marlene; Gold, Mark

    2012-01-01

    In an attempt to resolve controversy regarding the causal contributions of mesolimbic dopamine (DA) systems to reward, we evaluate the three main competing explanatory categories: “liking,” “learning,” and “wanting” [1]. That is, DA may mediate (a) the hedonic impact of reward (liking), (b) learned predictions about rewarding effects (learning), or (c) the pursuit of rewards by attributing incentive salience to reward-related stimuli (wanting). We evaluate these hypotheses, especially as they...

  12. Sleep spindles during a nap correlate with post sleep memory performance for highly rewarded word-pairs.

    Science.gov (United States)

    Studte, Sara; Bridger, Emma; Mecklinger, Axel

    2017-04-01

    The consolidation of new associations is thought to depend in part on physiological processes engaged during non-REM (NREM) sleep, such as slow oscillations and sleep spindles. Moreover, NREM sleep is thought to selectively benefit associations that are adaptive for the future. In line with this, the current study investigated whether different reward cues at encoding are associated with changes in sleep physiology and memory retention. Participants' associative memory was tested after learning a list of arbitrarily paired words both before and after taking a 90-min nap. During learning, word-pairs were preceded by a cue indicating either a high or a low reward for correct memory performance at test. The motivation manipulation successfully impacted retention such that memory declined to a greater extent from pre- to post sleep for low rewarded than for high rewarded word-pairs. In line with previous studies, positive correlations between spindle density during NREM sleep and general memory performance pre- and post-sleep were found. In addition to this, however, a selective positive relationship between memory performance for highly rewarded word-pairs at posttest and spindle density during NREM sleep was also observed. These results support the view that motivationally salient memories are preferentially consolidated and that sleep spindles may be an important underlying mechanism for selective consolidation. Copyright © 2016 Elsevier Inc. All rights reserved.

  13. Repeated blockade of mineralocorticoid receptors, but not of glucocorticoid receptors impairs food rewarded spatial learning

    NARCIS (Netherlands)

    Douma, B. R.; Korte, S. M.; Buwalda, B.; La Fleur, S. E.; Bohus, B.; Luiten, P. G.

    1998-01-01

    Corticosteroids from the adrenal cortex influence a variety of behaviours including cognition, learning and memory. These hormones act via two intracellular receptors, the mineralo-corticoid receptor (MR) and the glucocorticoid receptor (GR). These two receptor types display a high concentration and

  14. Repeated blockade of mineralocorticoid receptors, but not of glucocorticoid receptors impairs food rewarded spatial learning

    NARCIS (Netherlands)

    Douma, BRK; Korte, SM; Buwalda, B; la Fleur, SE; Bohus, B; Luiten, PGM

    Corticosteroids from the adrenal cortex influence a variety of behaviours including cognition, learning and memory. These hormones act via two intracellular receptors, the mineralo-corticoid receptor (MR) and the glucocorticoid receptor (GR). These two receptor types display a high concentration and

  15. Bribes or Rewards.

    Science.gov (United States)

    Megyeri, Kathy A.

    Small tangible rewards for student progress, such as candy bars, pens, or ribbons, add potency to the verbal and written praise offered by the teacher, thus increasing student motivation. Giving students small prizes enhances the cooperative atmosphere of learning, especially for those who do not normally do well. Research indicates that low…

  16. Human strategies for solving a time-place learning task: the role of counting and following verbal cues.

    Science.gov (United States)

    García-Gallardo, Daniel; Aguilar, Francisco; Armenta, Benjamín; Carpio, Claudio

    2015-04-01

    Two experiments were conducted to assess the emergence of time-place learning in humans. In experiment 1, a computer based software was designed in which participants had to choose to enter one of four rooms in an abandoned house search for a zombie every 3-15s. Zombies could be found in only one of these rooms every trial in 3 min periods during the 12 min sessions. After 4 training sessions, participants were exposed to a probe session in which zombies could be found in any room on every trial. Almost all participants behaved as if they were timing the availability intervals: they anticipated the changes in the location of the zombie and they persisted in their performance patterns during the probe session; however, verbal reports revealed that they were counting the number of trials in each period in order to decide when to switch between rooms. In the second experiment, the task was modified in two ways: counting was made harder by using three different intertrial ranges within each session: 2-6s, 2-11s and 2-16s. Second, labels were displaced during the final session to assess whether participants learned to click on a given place or to follow a set of verbal cues. We found that participants did not notice the label changes suggesting that they learned to click on a given place, and that a win/stay-lose/shift strategy was clearly used to decide when to switch rooms in the second experiment. The implications of verbal behavior when assessing time-place learning with humans and the possible differences in this process between humans and animals are discussed. Copyright © 2015 Elsevier B.V. All rights reserved.

  17. Instant transformation of learned repulsion into motivational "wanting".

    Science.gov (United States)

    Robinson, Mike J F; Berridge, Kent C

    2013-02-18

    Learned cues for pleasant reward often elicit desire, which, in addicts, may become compulsive. According to the dominant view in addiction neuroscience and reinforcement modeling, such desires are the simple products of learning, coming from a past association with reward outcome. We demonstrate that cravings are more than merely the products of accumulated pleasure memories-even a repulsive learned cue for unpleasantness can become suddenly desired via the activation of mesocorticolimbic circuitry. Rats learned repulsion toward a Pavlovian cue (a briefly-inserted metal lever) that always predicted an unpleasant Dead Sea saltiness sensation. Yet, upon first reencounter in a novel sodium-depletion state to promote mesocorticolimbic reactivity (reflected by elevated Fos activation in ventral tegmentum, nucleus accumbens, ventral pallidum, and the orbitofrontal prefrontal cortex), the learned cue was instantly transformed into an attractive and powerful motivational magnet. Rats jumped and gnawed on the suddenly attractive Pavlovian lever cue, despite never having tasted intense saltiness as anything other than disgusting. Instant desire transformation of a learned cue contradicts views that Pavlovian desires are essentially based on previously learned values (e.g., prediction error or temporal difference models). Instead desire is recomputed at reencounter by integrating Pavlovian information with the current brain/physiological state. This powerful brain transformation reverses strong learned revulsion into avid attraction. When applied to addiction, related mesocorticolimbic transformations (e.g., drugs or neural sensitization) of cues for already-pleasant drug experiences could create even more intense cravings. This cue/state transformation helps define what it means to say that addiction hijacks brain limbic circuits of natural reward. Copyright © 2013 Elsevier Ltd. All rights reserved.

  18. Psychological processes in chronic pain: Influences of reward and fear learning as key mechanisms - Behavioral evidence, neural circuits, and maladaptive changes.

    Science.gov (United States)

    Nees, Frauke; Becker, Susanne

    2017-09-07

    In the understanding of chronic pain, hypotheses derived from psychological theories, together with insights from physiological assessments and brain imaging, highlight the importance of mechanistically driven approaches. Physical system changes, for example following injury, can result in alterations of psychological processes and are accompanied by changes in corticolimbic circuits, which have been shown to be essential in emotional learning and memory, as well as reward processing and related behavior. In the present review, we thus highlight the importance of motivational, reward/pain relief, and fear learning processes in the context of chronic pain and discuss the potential of a mechanistic understanding of chronic pain within a clinical perspective, for example for the development of therapeutic strategies. We argue that changes in these mechanisms are not only characteristic for chronic pain, reflecting consequences of the disorder, but are also critically involved in the transition from acute to chronic pain states. Copyright © 2017 IBRO. Published by Elsevier Ltd. All rights reserved.

  19. Comparing the Distance Learning-Related Course Development Approach and Faculty Support and Rewards Structure at AACSB Accredited Institutions between 2001 and 2006

    Directory of Open Access Journals (Sweden)

    Heidi Perreault, Ed.D.

    2008-07-01

    Full Text Available The study compared the support and rewards provided faculty members for online course teaching and the development approaches used at business schools accredited by AACSB between 2001 and 2006. Data were collected from 81 professors in 2001 and 140 professors in 2006. The professors were involved in developing or teaching online courses at AACSB business schools across the United States. The findings indicate that faculty members received limited support and are not taking advantage of training options. Faculty members are most likely rewarded for their involvement in distance learning through stipends based on the number of online sections taught. Little has changed during the five-year period in regards to course development. Faculty members continue to use an individual instead of a team approach to course development and most faculty members learned online course development and delivery techniques on their own.

  20. DO SEMANTIC CONTEXTUAL CUES FACILITATE TRANSFER LEARNING FROM VIDEO IN TODDLERS?

    Directory of Open Access Journals (Sweden)

    Laura eZimmermann

    2015-05-01

    Full Text Available Young children typically demonstrate a transfer deficit, learning less from video than live presentations. Semantically meaningful context has been demonstrated to enhance learning in young children. We examined the effect of a semantically meaningful context on toddlers’ imitation performance. Two- and 2.5-year-olds participated in a puzzle imitation task to examine learning from either a live or televised model. The model demonstrated how to assemble a three-piece puzzle to make a fish or a boat, with the puzzle demonstration occurring against a semantically meaningful background context (ocean or a yellow background (no context. Participants in the video condition performed significantly worse than participants in the live condition, demonstrating the typical transfer deficit effect. While the context helped improve overall levels of imitation, especially for the boat puzzle, only individual differences in the ability to self-generate a stimulus label were related to a reduction in the transfer deficit.

  1. Do semantic contextual cues facilitate transfer learning from video in toddlers?

    Science.gov (United States)

    Zimmermann, Laura; Moser, Alecia; Grenell, Amanda; Dickerson, Kelly; Yao, Qianwen; Gerhardstein, Peter; Barr, Rachel

    2015-01-01

    Young children typically demonstrate a transfer deficit, learning less from video than live presentations. Semantically meaningful context has been demonstrated to enhance learning in young children. We examined the effect of a semantically meaningful context on toddlers' imitation performance. Two- and 2.5-year-olds participated in a puzzle imitation task to examine learning from either a live or televised model. The model demonstrated how to assemble a three-piece puzzle to make a fish or a boat, with the puzzle demonstration occurring against a semantically meaningful background context (ocean) or a yellow background (no context). Participants in the video condition performed significantly worse than participants in the live condition, demonstrating the typical transfer deficit effect. While the context helped improve overall levels of imitation, especially for the boat puzzle, only individual differences in the ability to self-generate a stimulus label were associated with a reduction in the transfer deficit.

  2. Abnormal Striatal BOLD Responses to Reward Anticipation and Reward Delivery in ADHD

    Science.gov (United States)

    Furukawa, Emi; Bado, Patricia; Tripp, Gail; Mattos, Paulo; Wickens, Jeff R.; Bramati, Ivanei E.; Alsop, Brent; Ferreira, Fernanda Meireles; Lima, Debora; Tovar-Moll, Fernanda; Sergeant, Joseph A.; Moll, Jorge

    2014-01-01

    Altered reward processing has been proposed to contribute to the symptoms of attention deficit hyperactivity disorder (ADHD). The neurobiological mechanism underlying this alteration remains unclear. We hypothesize that the transfer of dopamine release from reward to reward-predicting cues, as normally observed in animal studies, may be deficient in ADHD. Functional magnetic resonance imaging (fMRI) was used to investigate striatal responses to reward-predicting cues and reward delivery in a classical conditioning paradigm. Data from 14 high-functioning and stimulant-naïve young adults with elevated lifetime symptoms of ADHD (8 males, 6 females) and 15 well-matched controls (8 males, 7 females) were included in the analyses. During reward anticipation, increased blood-oxygen-level-dependent (BOLD) responses in the right ventral and left dorsal striatum were observed in controls, but not in the ADHD group. The opposite pattern was observed in response to reward delivery; the ADHD group demonstrated significantly greater BOLD responses in the ventral striatum bilaterally and the left dorsal striatum relative to controls. In the ADHD group, the number of current hyperactivity/impulsivity symptoms was inversely related to ventral striatal responses during reward anticipation and positively associated with responses to reward. The BOLD response patterns observed in the striatum are consistent with impaired predictive dopamine signaling in ADHD, which may explain altered reward-contingent behaviors and symptoms of ADHD. PMID:24586543

  3. Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention.

    Science.gov (United States)

    Chun, Marvin M.; Jiang, Yuhong

    1998-01-01

    Six experiments involving a total of 112 college students demonstrate that a robust memory for visual context exists to guide spatial attention. Results show how implicit learning and memory of visual context can guide spatial attention toward task-relevant aspects of a scene. (SLD)

  4. The Timing Effects of Reward, Business Longevity, and Involvement on Consumers’ Responses to a Reward Program

    Directory of Open Access Journals (Sweden)

    Badri Munir Sukoco

    2015-06-01

    Full Text Available Managers could elicit customers’ repeat purchase behavior through a well-designed reward program. This study examines two extrinsic cues - business longevity and timing effects of reward – to determine the consumers’ perceived risk and intention to participate in this kind of program. Moreover, this study discusses how different levels of involvement might interact with these two cues. An experiment with a 2 (business longevity: long vs. short x 2 (timing of reward: delayed vs. immediate x 2 (involvement: high vs. low between-subject factorial design is conducted to validate the proposed research hypotheses. The results show that an immediate reward offered by an older, more established, firm for a highly-involved product, make loyalty programs less risky and consequently attract consumers to participate. Interestingly, immediate rewards that are offered by older firms for a product that customers are less involved in has the opposite effects. Managerial and academic implications are further presented in this study.

  5. The Computational Development of Reinforcement Learning during Adolescence.

    Directory of Open Access Journals (Sweden)

    Stefano Palminteri

    2016-06-01

    Full Text Available Adolescence is a period of life characterised by changes in learning and decision-making. Learning and decision-making do not rely on a unitary system, but instead require the coordination of different cognitive processes that can be mathematically formalised as dissociable computational modules. Here, we aimed to trace the developmental time-course of the computational modules responsible for learning from reward or punishment, and learning from counterfactual feedback. Adolescents and adults carried out a novel reinforcement learning paradigm in which participants learned the association between cues and probabilistic outcomes, where the outcomes differed in valence (reward versus punishment and feedback was either partial or complete (either the outcome of the chosen option only, or the outcomes of both the chosen and unchosen option, were displayed. Computational strategies changed during development: whereas adolescents' behaviour was better explained by a basic reinforcement learning algorithm, adults' behaviour integrated increasingly complex computational features, namely a counterfactual learning module (enabling enhanced performance in the presence of complete feedback and a value contextualisation module (enabling symmetrical reward and punishment learning. Unlike adults, adolescent performance did not benefit from counterfactual (complete feedback. In addition, while adults learned symmetrically from both reward and punishment, adolescents learned from reward but were less likely to learn from punishment. This tendency to rely on rewards and not to consider alternative consequences of actions might contribute to our understanding of decision-making in adolescence.

  6. What is the reward? Medical students’ learning and personal development during a research project course

    Science.gov (United States)

    Möller, Riitta; Shoshan, Maria; Heikkilä, Kristiina

    2015-01-01

    Background Until recently, the outcome of medical students’ research projects has mainly been assessed in terms of scientific publications, whereas other results important for students’ development have been less studied. The aim of this study was to investigate medical students’ experiences of learning as an outcome of the research project course. Method Written reflections of 50 students were analyzed by manifest inductive content analysis. Results Three categories emerged: ‘thinking as a scientist’, ‘working as a scientist’, and ‘personal development’. Students became more aware about the nature of knowledge, how to generate new knowledge, and developed skills in scientific thinking and critical appraisal. Unexpectedly, effects on personal characteristics, such as self-confidence, self-discipline, independence, and time management skills were also acknowledged. Conclusions We conclude that individual research projects enhance research-specific skills and competencies needed in evidence-based clinical work and are beneficial for personal and professional development. PMID:26344390

  7. What is the reward? Medical students’ learning and personal development during a research project course

    Directory of Open Access Journals (Sweden)

    Riitta Möller

    2015-09-01

    Full Text Available Background: Until recently, the outcome of medical students’ research projects has mainly been assessed in terms of scientific publications, whereas other results important for students’ development have been less studied. The aim of this study was to investigate medical students’ experiences of learning as an outcome of the research project course. Method: Written reflections of 50 students were analyzed by manifest inductive content analysis. Results: Three categories emerged: ‘thinking as a scientist’, ‘working as a scientist’, and ‘personal development’. Students became more aware about the nature of knowledge, how to generate new knowledge, and developed skills in scientific thinking and critical appraisal. Unexpectedly, effects on personal characteristics, such as self-confidence, self-discipline, independence, and time management skills were also acknowledged. Conclusions: We conclude that individual research projects enhance research-specific skills and competencies needed in evidence-based clinical work and are beneficial for personal and professional development.

  8. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis.

    Science.gov (United States)

    Glimcher, Paul W

    2011-09-13

    A number of recent advances have been achieved in the study of midbrain dopaminergic neurons. Understanding these advances and how they relate to one another requires a deep understanding of the computational models that serve as an explanatory framework and guide ongoing experimental inquiry. This intertwining of theory and experiment now suggests very clearly that the phasic activity of the midbrain dopamine neurons provides a global mechanism for synaptic modification. These synaptic modifications, in turn, provide the mechanistic underpinning for a specific class of reinforcement learning mechanisms that now seem to underlie much of human and animal behavior. This review describes both the critical empirical findings that are at the root of this conclusion and the fantastic theoretical advances from which this conclusion is drawn.

  9. Functional architecture of reward learning in mushroom body extrinsic neurons of larval Drosophila.

    Science.gov (United States)

    Saumweber, Timo; Rohwedder, Astrid; Schleyer, Michael; Eichler, Katharina; Chen, Yi-Chun; Aso, Yoshinori; Cardona, Albert; Eschbach, Claire; Kobler, Oliver; Voigt, Anne; Durairaja, Archana; Mancini, Nino; Zlatic, Marta; Truman, James W; Thum, Andreas S; Gerber, Bertram

    2018-03-16

    The brain adaptively integrates present sensory input, past experience, and options for future action. The insect mushroom body exemplifies how a central brain structure brings about such integration. Here we use a combination of systematic single-cell labeling, connectomics, transgenic silencing, and activation experiments to study the mushroom body at single-cell resolution, focusing on the behavioral architecture of its input and output neurons (MBINs and MBONs), and of the mushroom body intrinsic APL neuron. Our results reveal the identity and morphology of almost all of these 44 neurons in stage 3 Drosophila larvae. Upon an initial screen, functional analyses focusing on the mushroom body medial lobe uncover sparse and specific functions of its dopaminergic MBINs, its MBONs, and of the GABAergic APL neuron across three behavioral tasks, namely odor preference, taste preference, and associative learning between odor and taste. Our results thus provide a cellular-resolution study case of how brains organize behavior.

  10. Action learning for health system governance: the reward and challenge of co-production.

    Science.gov (United States)

    Lehmann, Uta; Gilson, Lucy

    2015-10-01

    Health policy and systems research (HPSR) is centrally concerned with people, their relationships and the actions and practices they can implement towards better health systems. These concerns suggest that HPS researchers must work in direct engagement with the practitioners and practice central to the inquiry, acknowledging their tacit knowledge and drawing it into generating new insights into health system functioning. Social science perspectives are of particular importance in this field because health policies and health systems are themselves social and political constructs. However, how can social science methodologies such as action research and narrative and appreciative enquiry enable such research, and how can methodologies from different disciplines be woven together to construct and make meaning of evidence for 'this' field? This article seeks to present 'methodological musings' on these points, to prompt wider discussion on the practice of HPSR. It draws on one long-term collaborative action learning research project being undertaken in Cape Town, South Africa. The District Innovation and Action Learning for Health System Development project is an action research partnership between two South African academic institutions and two health authorities focused, ultimately, on strengthening governance in primary health care.Drawing on this experience, the article considers three interrelated issues: The diversity and complexities of practitioner and research actors involved in co-producing HPSR; The nature of co-production and the importance of providing space to grapple across different systems of meaning;The character of evidence and data in co-production. There is much to be learnt from research traditions outside the health sector, but HPSR must work out its own practices--through collaboration and innovation among researchers and practitioners. In this article, we provide one set of experiences to prompt wider reflection and stimulate engagement on the

  11. Dissociable effects of 5-HT2C receptor antagonism and genetic inactivation on perseverance and learned non-reward in an egocentric spatial reversal task.

    Directory of Open Access Journals (Sweden)

    Simon R O Nilsson

    Full Text Available Cognitive flexibility can be assessed in reversal learning tests, which are sensitive to modulation of 5-HT2C receptor (5-HT2CR function. Successful performance in these tests depends on at least two dissociable cognitive mechanisms which may separately dissipate associations of previous positive and negative valence. The first is opposed by perseverance and the second by learned non-reward. The current experiments explored the effect of reducing function of the 5-HT2CR on the cognitive mechanisms underlying egocentric reversal learning in the mouse. Experiment 1 used the 5-HT2CR antagonist SB242084 (0.5 mg/kg in a between-groups serial design and Experiment 2 used 5-HT2CR KO mice in a repeated measures design. Animals initially learned to discriminate between two egocentric turning directions, only one of which was food rewarded (denoted CS+, CS-, in a T- or Y-maze configuration. This was followed by three conditions; (1 Full reversal, where contingencies reversed; (2 Perseverance, where the previous CS+ became CS- and the previous CS- was replaced by a novel CS+; (3 Learned non-reward, where the previous CS- became CS+ and the previous CS+ was replaced by a novel CS-. SB242084 reduced perseverance, observed as a decrease in trials and incorrect responses to criterion, but increased learned non-reward, observed as an increase in trials to criterion. In contrast, 5-HT2CR KO mice showed increased perseverance. 5-HT2CR KO mice also showed retarded egocentric discrimination learning. Neither manipulation of 5-HT2CR function affected performance in the full reversal test. These results are unlikely to be accounted for by increased novelty attraction, as SB242084 failed to affect performance in an unrewarded novelty task. In conclusion, acute 5-HT2CR antagonism and constitutive loss of the 5-HT2CR have opposing effects on perseverance in egocentric reversal learning in mice. It is likely that this difference reflects the broader impact of 5HT2CR loss

  12. [Sucrose reward promotes rats' motivation for cocaine].

    Science.gov (United States)

    Li, Yan-Qing; LE, Qiu-Min; Yu, Xiang-Chen; Ma, Lan; Wang, Fei-Fei

    2016-06-25

    Caloric diet, such as fat and sugar intake, has rewarding effects, and has been indicated to affect the responses to addictive substances in animal experiments. However, the possible association between sucrose reward and the motivation for addictive drugs remains to be elucidated. Thus, we carried out behavioral tests after sucrose self-administration training to determine the effects of sucrose experience on rats' motivation for cocaine, locomotor sensitivity to cocaine, basal locomotor activity, anxiety level, and associative learning ability. The sucrose-experienced (sucrose) group exhibited higher lever press, cocaine infusion and break point, as well as upshift of cocaine dose-response curve in cocaine self-administration test, as compared with the control (chow) group. Additionally, despite similar locomotor activity in open field test and comparable score in cocaine-induced conditioned place preference, the sucrose group showed higher cocaine-induced locomotor sensitivity as compared with the chow group. The anxiety level and the performance in vocal-cue induced fear memory were similar between these two groups in elevated plus maze and fear conditioning tests, respectively. Taken together, our work indicates that sucrose experience promotes the rats' motivation for cocaine.

  13. Memory Consolidation and Neural Substrate of Reward

    Directory of Open Access Journals (Sweden)

    Redolar-Ripoll, Diego

    2012-08-01

    Full Text Available The aim of this report is to analyze the relationships between reward and learning and memory processes. Different studies have described how information about rewards influences behavior and how the brain uses this reward information to control learning and memory processes. Reward nature seems to be processed in different ways by neurons in different brain structures, ranging from the detection and perception of rewards to the use of information about predicted rewards for the control of goal-directed behavior. The neural substrate underling this processing of reward information is a reliable way of improving learning and memory processes. Evidence from several studies indicates that this neural system can facilitate memory consolidation in a wide variety of learning tasks. From a molecular perspective, certain cardinal features of reward have been described as forms of memory. Studies of human addicts and studies in animal models of addiction show that chronic drug exposure produces stable changes in the brain at the cellular and molecular levels that underlie the long-lasting behavioral plasticity associated with addiction. These molecular and cellular adaptations involved in addiction are also implicated in learning and memory processes. Dopamine seems to be a critical common signal to activate different genetic mechanisms that ultimately remodel synapses and circuits. Despite memory is an active and complex process mediated by different brain areas, the neural substrate of reward is able to improve memory consolidation in a several paradigms. We believe that there are many equivalent traits between reward and learning and memory processes.

  14. Distinct Roles for the Amygdala and Orbitofrontal Cortex in Representing the Relative Amount of Expected Reward.

    Science.gov (United States)

    Saez, Rebecca A; Saez, Alexandre; Paton, Joseph J; Lau, Brian; Salzman, C Daniel

    2017-07-05

    The same reward can possess different motivational meaning depending upon its magnitude relative to other rewards. To study the neurophysiological mechanisms mediating assignment of motivational meaning, we recorded the activity of neurons in the amygdala and orbitofrontal cortex (OFC) of monkeys during a Pavlovian task in which the relative amount of liquid reward associated with one conditioned stimulus (CS) was manipulated by changing the reward amount associated with a second CS. Anticipatory licking tracked relative reward magnitude, implying that monkeys integrated information about recent rewards to adjust the motivational meaning of a CS. Upon changes in relative reward magnitude, neural responses to reward-predictive cues updated more rapidly in OFC than amygdala, and activity in OFC but not the amygdala was modulated by recent reward history. These results highlight a distinction between the amygdala and OFC in assessing reward history to support the flexible assignment of motivational meaning to sensory cues. Copyright © 2017 Elsevier Inc. All rights reserved.

  15. The impact of memory load and perceptual cues on puzzle learning by 24-month olds.

    Science.gov (United States)

    Barr, Rachel; Moser, Alecia; Rusnak, Sylvia; Zimmermann, Laura; Dickerson, Kelly; Lee, Herietta; Gerhardstein, Peter

    2016-11-01

    Early childhood is characterized by memory capacity limitations and rapid perceptual and motor development [Rovee-Collier (1996). Infant Behavior & Development, 19, 385-400]. The present study examined 2-year olds' reproduction of a sliding action to complete an abstract fish puzzle under different levels of memory load and perceptual feature support. Experimental groups were compared to baseline controls to assess spontaneous rates of production of the target actions; baseline production was low across all experiments. Memory load was manipulated in Exp. 1 by adding pieces to the puzzle, increasing sequence length from 2 to 3 items, and to 3 items plus a distractor. Although memory load did not influence how toddlers learned to manipulate the puzzle pieces, it did influence toddlers' achievement of the goal-constructing the fish. Overall, girls were better at constructing the puzzle than boys. In Exp. 2, the perceptual features of the puzzle were altered by changing shape boundaries to create a two-piece horizontally cut puzzle (displaying bilateral symmetry), and by adding a semantically supportive context to the vertically cut puzzle (iconic). Toddlers were able to achieve the goal of building the fish equally well across the 2-item puzzle types (bilateral symmetry, vertical, iconic), but how they learned to manipulate the puzzle pieces varied as a function of the perceptual features. Here, as in Exp. 1, girls showed a different pattern of performance from the boys. This study demonstrates that changes in memory capacity and perceptual processing influence both goal-directed imitation learning and motoric performance. © 2016 Wiley Periodicals, Inc.

  16. The Responses of Young Domestic Horses to Human-Given Cues

    Science.gov (United States)

    Proops, Leanne; Rayner, Jenny; Taylor, Anna M.; McComb, Karen

    2013-01-01

    It has been suggested that the process of domestication, at least in some species, has led to an innate predisposition to be skilled at reading human communicative and attentional cues. Adult domestic horses (Equus caballus) are highly sensitive to subtle bodily cues when determining if a person is attending to them but they are less adept at using human communicative cues in object choice tasks. Here we provide the first study into the ontogeny of such skills in order to gain insights into the mechanisms underlying these abilities. Compared with adult horses, youngsters under the age of three could use body orientation but not more subtle cues such as head movement and open/closed eyes to correctly choose an attentive person to approach for food. Across two object choice experiments, the performance of young horses was comparable to that of adult horses – subjects were able to correctly choose a rewarded bucket using marker placement, pointing and touching cues but could not use body orientation, gaze, elbow pointing or tapping cues. Taken together these results do not support the theory that horses possess an innate predisposition to be particularly skilled at using human cues. Horses' ability to determine whether humans are attending to them using subtle body cues appears to require significant experience to fully develop and their perhaps less remarkable use of limited cues in object choice tasks, although present at a much earlier age, is likely to reflect a more general learning ability related to stimulus enhancement rather than a specific ‘human-reading’ skill. PMID:23840572

  17. Memory for location and visual cues in white-eared hummingbirds Hylocharis leucotis

    Directory of Open Access Journals (Sweden)

    Guillermo PÉREZ, Carlos LARA, José VICCON-PALE, Martha SIGNORET-POILLON

    2011-08-01

    Full Text Available In nature hummingbirds face floral resources whose availability, quality and quantity can vary spatially and temporally. Thus, they must constantly make foraging decisions about which patches, plants and flowers to visit, partly as a function of the nectar reward. The uncertainty of these decisions would possibly be reduced if an individual could remember locations or use visual cues to avoid revisiting recently depleted flowers. In the present study, we carried out field experiments with white-eared hummingbirds Hylocharis leucotis, to evaluate their use of locations or visual cues when foraging on natural flowers Penstemon roseus. We evaluated the use of spatial memory by observing birds while they were foraging between two plants and within a single plant. Our results showed that hummingbirds prefer to use location when foraging in two plants, but they also use visual cues to efficiently locate unvisited rewarded flowers when they feed on a single plant. However, in absence of visual cues, in both experiments birds mainly used the location of previously visited flowers to make subsequent visits. Our data suggest that hummingbirds are capable of learning and employing this flexibility depending on the faced environmental conditions and the information acquired in previous visits [Current Zoology 57 (4: 468–476, 2011].

  18. Long-term memory of relative reward values.

    Science.gov (United States)

    Soldati, Francesca; Burman, Oliver H P; John, Elizabeth A; Pike, Thomas W; Wilkinson, Anna

    2017-02-01

    Long-term memory can be adaptive as it allows animals to retain information that is crucial for survival, such as the appearance and location of key resources. This is generally examined by comparing choices of stimuli that have value to the animal with those that do not; however, in nature choices are rarely so clear cut. Animals are able to assess the relative value of a resource via direct comparison, but it remains unclear whether they are able to retain this information for a biologically meaningful amount of time. To test this, captive red-footed tortoises (Chelonoidis carbonaria) were first trained to associate visual cues with specific qualities and quantities of food, and their preferences for the different reward values determined. They were then retested after an interval of 18 months. We found that the tortoises were able to retain the information they had learned about the cues as indicators of relative reward values over this interval, demonstrating a memory for the relative quantity and quality of food over an extended period of time. This is likely to impact directly on an animal's foraging decisions, such as the exploitation of seasonally varying resources, with obvious fitness implications for the individual; however, the implications may also extend to the ecological interactions in which the animal is involved, affecting processes such as herbivory and seed dispersal. © 2017 The Author(s).

  19. Biased Competition in Visual Processing Hierarchies: A Learning Approach Using Multiple Cues.

    Science.gov (United States)

    Gepperth, Alexander R T; Rebhan, Sven; Hasler, Stephan; Fritsch, Jannik

    2011-03-01

    In this contribution, we present a large-scale hierarchical system for object detection fusing bottom-up (signal-driven) processing results with top-down (model or task-driven) attentional modulation. Specifically, we focus on the question of how the autonomous learning of invariant models can be embedded into a performing system and how such models can be used to define object-specific attentional modulation signals. Our system implements bi-directional data flow in a processing hierarchy. The bottom-up data flow proceeds from a preprocessing level to the hypothesis level where object hypotheses created by exhaustive object detection algorithms are represented in a roughly retinotopic way. A competitive selection mechanism is used to determine the most confident hypotheses, which are used on the system level to train multimodal models that link object identity to invariant hypothesis properties. The top-down data flow originates at the system level, where the trained multimodal models are used to obtain space- and feature-based attentional modulation signals, providing biases for the competitive selection process at the hypothesis level. This results in object-specific hypothesis facilitation/suppression in certain image regions which we show to be applicable to different object detection mechanisms. In order to demonstrate the benefits of this approach, we apply the system to the detection of cars in a variety of challenging traffic videos. Evaluating our approach on a publicly available dataset containing approximately 3,500 annotated video images from more than 1 h of driving, we can show strong increases in performance and generalization when compared to object detection in isolation. Furthermore, we compare our results to a late hypothesis rejection approach, showing that early coupling of top-down and bottom-up information is a favorable approach especially when processing resources are constrained.

  20. Distinct Motivational Effects of Contingent and Noncontingent Rewards.

    Science.gov (United States)

    Manohar, Sanjay G; Finzi, Rebecca Dawn; Drew, Daniel; Husain, Masud

    2017-07-01

    When rewards are available, people expend more energy, increasing their motivational vigor. In theory, incentives might drive behavior for two distinct reasons: First, they increase expected reward; second, they increase the difference in subjective value between successful and unsuccessful performance, which increases contingency-the degree to which action determines outcome. Previous studies of motivational vigor have never compared these directly. Here, we indexed motivational vigor by measuring the speed of eye movements toward a target after participants heard a cue indicating how outcomes would be determined. Eye movements were faster when the cue indicated that monetary rewards would be contingent on performance than when the cue indicated that rewards would be random. But even when the cue indicated that a reward was guaranteed regardless of speed, movement was still faster than when no reward was available. Motivation by contingent and certain rewards was uncorrelated across individuals, which suggests that there are two separable, independent components of motivation. Contingent motivation generated autonomic arousal, and unlike noncontingent motivation, was effective with penalties as well as rewards.

  1. Fear Conditioning Effects on Sensitivity to Drug Reward

    Science.gov (United States)

    2010-06-01

    motivational responses and self-administration behaviors (Robbins et al., 2008). Pavlovian conditioning mechanisms link unconditioned drug responses...model. Induction of fear conditioning is followed by measurement of sensitivity to drug reward using a conditioned place preference (CPP) model to...morphine. Conditioned drug reward is a relevant model in addiction because environmental cues (e.g. a barroom) induce craving and persistent

  2. Reward and punishment

    OpenAIRE

    Sigmund, Karl; Hauert, Christoph; Nowak, Martin A.

    2001-01-01

    Minigames capturing the essence of Public Goods experiments show that even in the absence of rationality assumptions, both punishment and reward will fail to bring about prosocial behavior. This result holds in particular for the well-known Ultimatum Game, which emerges as a special case. But reputation can induce fairness and cooperation in populations adapting through learning or imitation. Indeed, the inclusion of reputation effects in the corresponding dynamical models leads to the evolut...

  3. Intersection of reward and memory in monkey rhinal cortex.

    Science.gov (United States)

    Clark, Andrew M; Bouret, Sebastien; Young, Adrienne M; Richmond, Barry J

    2012-05-16

    In humans and other animals, the vigor with which a reward is pursued depends on its desirability, that is, on the reward's predicted value. Predicted value is generally context-dependent, varying according to the value of rewards obtained in the recent and distant past. Signals related to reward prediction and valuation are believed to be encoded in a circuit centered around midbrain dopamine neurons and their targets in the prefrontal cortex and basal ganglia. Notably absent from this hypothesized reward pathway are dopaminergic targets in the medial temporal lobe. Here we show that a key part of the medial temporal lobe memory system previously reported to be important for sensory mnemonic and perceptual processing, the rhinal cortex (Rh), is required for using memories of previous reward values to predict the value of forthcoming rewards. We tested monkeys with bilateral Rh lesions on a task in which reward size varied across blocks of uncued trials. In this experiment, the only cues for predicting current reward value are the sizes of rewards delivered in previous blocks. Unexpectedly, monkeys with Rh ablations, but not intact controls, were insensitive to differences in predicted reward, responding as if they expected all rewards to be of equal magnitude. Thus, it appears that Rh is critical for using memory of previous rewards to predict the value of forthcoming rewards. These results are in agreement with accumulating evidence that Rh is critical for establishing the relationships between temporally interleaved events, which is a key element of episodic memory.

  4. Acute and chronic effects of cannabinoids on effort-related decision-making and reward learning: an evaluation of the cannabis 'amotivational' hypotheses.

    Science.gov (United States)

    Lawn, Will; Freeman, Tom P; Pope, Rebecca A; Joye, Alyssa; Harvey, Lisa; Hindocha, Chandni; Mokrysz, Claire; Moss, Abigail; Wall, Matthew B; Bloomfield, Michael Ap; Das, Ravi K; Morgan, Celia Ja; Nutt, David J; Curran, H Valerie

    2016-10-01

    Anecdotally, both acute and chronic cannabis use have been associated with apathy, amotivation, and other reward processing deficits. To date, empirical support for these effects is limited, and no previous studies have assessed both acute effects of Δ-9-tetrahydrocannabinol (THC) and cannabidiol (CBD), as well as associations with cannabis dependence. The objectives of this study were (1) to examine acute effects of cannabis with CBD (Cann + CBD) and without CBD (Cann-CBD) on effort-related decision-making and (2) to examine associations between cannabis dependence, effort-related decision-making and reward learning. In study 1, 17 participants each received three acute vaporized treatments, namely Cann-CBD (8 mg THC), Cann + CBD (8 mg THC + 10 mg CBD) and matched placebo, followed by a 50 % dose top-up 1.5 h later, and completed the Effort Expenditure for Rewards Task (EEfRT). In study 2, 20 cannabis-dependent participants were compared with 20 non-dependent, drug-using control participants on the EEfRT and the Probabilistic Reward Task (PRT) in a non-intoxicated state. Cann-CBD reduced the likelihood of high-effort choices relative to placebo (p = 0.042) and increased sensitivity to expected value compared to both placebo (p = 0.014) and Cann + CBD (p = 0.006). The cannabis-dependent and control groups did not differ on the EEfRT. However, the cannabis-dependent group exhibited a weaker response bias than the control group on the PRT (p = 0.007). Cannabis acutely induced a transient amotivational state and CBD influenced the effects of THC on expected value. In contrast, cannabis dependence was associated with preserved motivation alongside impaired reward learning, although confounding factors, including depression, cannot be disregarded. This is the first well powered, fully controlled study to objectively demonstrate the acute amotivational effects of THC.

  5. A balance of activity in brain control and reward systems predicts self-regulatory outcomes

    OpenAIRE

    Lopez, Richard B.; Chen, Pin-Hao A.; Huckins, Jeremy F.; Hofmann, Wilhelm; Kelley, William M.; Heatherton, Todd F.

    2017-01-01

    Abstract Previous neuroimaging work has shown that increased reward-related activity following exposure to food cues is predictive of self-control failure. The balance model suggests that self-regulation failures result from an imbalance in reward and executive control mechanisms. However, an open question is whether the relative balance of activity in brain systems associated with executive control (vs reward) supports self-regulatory outcomes when people encounter tempting cues in daily lif...

  6. Point Reward System: A Method of Assessment That Accommodates a Diversity of Student Abilities and Interests and Enhances Learning

    Science.gov (United States)

    Derado, Josip; Garner, Mary L.; Tran, Thu-Hang

    2016-01-01

    Students' abilities and interests vary dramatically in the college mathematics classroom. How do we teach all of these students effectively? In this paper, we present the Point Reward System (PRS), a new method of assessment that addresses this problem. We designed the PRS with three main goals in mind: to increase the retention rates; to keep all…

  7. Effects of Ginsenoside Rg1 on Learning and Memory in a Reward-directed Instrumental Conditioning Task in Chronic Restraint Stressed Rats.

    Science.gov (United States)

    Kezhu, Wang; Pan, Xu; Cong, Lu; Liming, Dong; Beiyue, Zhang; Jingwei, Lu; Yanyan, Yang; Xinmin, Liu

    2017-01-01

    Ginsenoside Rg1 is one of the major active ingredients of Panax ginseng and has showed notable improving learning and memory effects in several behavioral tasks, such as water maze, shuttle-box, and step-through, based on avoidance. However, there was no report about the role of Rg1 on the performance of reward-directed instrumental conditioning, which could reflect the adaptive capacity to ever-changing environments. Thus, in this study, the reward devaluation test and conditional visual discrimination task were conducted to study the ameliorating effects of Rg1 on cognitive deficits, especially the loss of adaptation capacity in chronic restraint stress (CRS) rat model. Our results showed that rat subjected to CRS became insensitive to the changes in outcome value, and it significantly harmed the rat's performance in conditional visual discrimination task. Moreover, the levels of BDNF, TrkB, and Erk phosphorylation were decreased in the prefrontal cortex of CRS rats. However, these changes were effectively reversed by Rg1 (5 and 10 mg/kg, i.p.). Therefore, it demonstrated that Rg1 has a good ability to improve learning and memory and also ameliorate impaired adaptive capacity induced by CRS. This amelioration effect of Rg1 might be mediated partially by BDNF/TrkB/Erk pathway in prefrontal cortex. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

  8. How partial reinforcement of food cues affects the extinction and reacquisition of appetitive responses. A new model for dieting success?

    Science.gov (United States)

    van den Akker, Karolien; Havermans, Remco C; Bouton, Mark E; Jansen, Anita

    2014-10-01

    Animals and humans can easily learn to associate an initially neutral cue with food intake through classical conditioning, but extinction of learned appetitive responses can be more difficult. Intermittent or partial reinforcement of food cues causes especially persistent behaviour in animals: after exposure to such learning schedules, the decline in responding that occurs during extinction is slow. After extinction, increases in responding with renewed reinforcement of food cues (reacquisition) might be less rapid after acquisition with partial reinforcement. In humans, it may be that the eating behaviour of some individuals resembles partial reinforcement schedules to a greater extent, possibly affecting dieting success by interacting with extinction and reacquisition. Furthermore, impulsivity has been associated with less successful dieting, and this association might be explained by impulsivity affecting the learning and extinction of appetitive responses. In the present two studies, the effects of different reinforcement schedules and impulsivity on the acquisition, extinction, and reacquisition of appetitive responses were investigated in a conditioning paradigm involving food rewards in healthy humans. Overall, the results indicate both partial reinforcement schedules and, possibly, impulsivity to be associated with worse extinction performance. A new model of dieting success is proposed: learning histories and, perhaps, certain personality traits (impulsivity) can interfere with the extinction and reacquisition of appetitive responses to food cues and they may be causally related to unsuccessful dieting. Copyright © 2014 Elsevier Ltd. All rights reserved.

  9. Decision Utility, Incentive Salience, and Cue-Triggered "Wanting"

    Science.gov (United States)

    Berridge, Kent C; Aldridge, J Wayne

    2009-01-01

    This chapter examines brain mechanisms of reward utility operating at particular decision moments in life-moments such as when one encounters an image, sound, scent, or other cue associated in the past with a particular reward or perhaps just when one vividly imagines that cue. Such a cue can often trigger a sudden motivational urge to pursue its reward and sometimes a decision to do so. Drawing on a utility taxonomy that distinguishes among subtypes of reward utility-predicted utility, decision utility, experienced utility, and remembered utility-it is shown how cue-triggered cravings, such as an addict's surrender to relapse, can hang on special transformations by brain mesolimbic systems of one utility subtype, namely, decision utility. The chapter focuses on a particular form of decision utility called incentive salience, a type of "wanting" for rewards that is amplified by brain mesolimbic systems. Sudden peaks of intensity of incentive salience, caused by neurobiological mechanisms, can elevate the decision utility of a particular reward at the moment its cue occurs. An understanding of what happens at such moments leads to a better understanding of the mechanisms at work in decision making in general.

  10. The CB1 Receptor as an Important Mediator of Hedonic Reward Processing

    Science.gov (United States)

    Friemel, Chris M; Zimmer, Andreas; Schneider, Miriam

    2014-01-01

    The endocannabinoid (ECB) system has emerged recently as a key mediator for reward processing. It is well known that cannabinoids affect appetitive learning processes and can induce reinforcing and rewarding effects. However, the involvement of the ECB system in hedonic aspects of reward-related behavior is not completely understood. With the present study, we investigated the modulatory role of the ECB system on hedonic perception, measured by the pleasure attenuated startle (PAS) paradigm for a palatable food reward. Here, a conditioned odor is thought to induce a pleasant affective state that attenuates an aversive reflex—the acoustic startle response. Modulatory effects of the CB1 receptor antagonist/inverse agonist SR1411716 and the cannabinoid agonist WIN 55 212-2 on PAS were examined in rats. PAS was also measured in CB1 receptor knockout (KO) and wild-type (WT) mice. Pharmacological inhibition as well as the absence of CB1 receptors was found to reduce PAS, whereas WIN 55 212-2 administration increased PAS. Finally, presentation of a conditioned reward cue was found to induce striatal FosB/ΔFosB expression in WT mice, but not in KO mice, indicating a reduced stimulation of reward-related brain regions in conditioned KO mice by odor presentation. We here show that in addition to our previous studies in rats, PAS may also serve as a valuable and suitable measure to assess hedonic processing in mice. Our data further indicate that the ECB system, and in particular CB1 receptor signaling, appears to be highly important for the mediation of hedonic aspects of reward processing. PMID:24718372

  11. Retrieval cues that trigger reconsolidation of associative fear memory are not necessarily an exact replica of the original learning experience

    NARCIS (Netherlands)

    Soeter, M.; Kindt, M.

    2015-01-01

    Disrupting the process of memory reconsolidation may point to a novel therapeutic strategy for the permanent reduction of fear in patients suffering from anxiety disorders. However both in animal and human studies the retrieval cue typically involves a re-exposure to the original fear-conditioned

  12. Procedural Learning and Associative Memory Mechanisms Contribute to Contextual Cueing: Evidence from fMRI and Eye-Tracking

    Science.gov (United States)

    Manelis, Anna; Reder, Lynne M.

    2012-01-01

    Using a combination of eye tracking and fMRI in a contextual cueing task, we explored the mechanisms underlying the facilitation of visual search for repeated spatial configurations. When configurations of distractors were repeated, greater activation in the right hippocampus corresponded to greater reductions in the number of saccades to locate…

  13. Addiction: beyond dopamine reward circuitry.

    Science.gov (United States)

    Volkow, Nora D; Wang, Gene-Jack; Fowler, Joanna S; Tomasi, Dardo; Telang, Frank

    2011-09-13

    Dopamine (DA) is considered crucial for the rewarding effects of drugs of abuse, but its role in addiction is much less clear. This review focuses on studies that used PET to characterize the brain DA system in addicted subjects. These studies have corroborated in humans the relevance of drug-induced fast DA increases in striatum [including nucleus accumbens (NAc)] in their rewarding effects but have unexpectedly shown that in addicted subjects, drug-induced DA increases (as well as their subjective reinforcing effects) are markedly blunted compared with controls. In contrast, addicted subjects show significant DA increases in striatum in response to drug-conditioned cues that are associated with self-reports of drug craving and appear to be of a greater magnitude than the DA responses to the drug. We postulate that the discrepancy between the expectation for the drug effects (conditioned responses) and the blunted pharmacological effects maintains drug taking in an attempt to achieve the expected reward. Also, whether tested during early or protracted withdrawal, addicted subjects show lower levels of D2 receptors in striatum (including NAc), which are associated with decreases in baseline activity in frontal brain regions implicated in salience attribution (orbitofrontal cortex) and inhibitory control (anterior cingulate gyrus), whose disruption results in compulsivity and impulsivity. These results point to an imbalance between dopaminergic circuits that underlie reward and conditioning and those that underlie executive function (emotional control and decision making), which we postulate contributes to the compulsive drug use and loss of control in addiction.

  14. Addiction: Beyond dopamine reward circuitry

    International Nuclear Information System (INIS)

    Volkow, N.D.; Wang, G.-J.; Fowler, J.S.; Tomasi, D.; Telang, F.

    2011-01-01

    Dopamine (DA) is considered crucial for the rewarding effects of drugs of abuse, but its role in addiction is much less clear. This review focuses on studies that used PET to characterize the brain DA system in addicted subjects. These studies have corroborated in humans the relevance of drug-induced fast DA increases in striatum [including nucleus accumbens (NAc)] in their rewarding effects but have unexpectedly shown that in addicted subjects, drug-induced DA increases (as well as their subjective reinforcing effects) are markedly blunted compared with controls. In contrast, addicted subjects show significant DA increases in striatum in response to drug-conditioned cues that are associated with self-reports of drug craving and appear to be of a greater magnitude than the DA responses to the drug. We postulate that the discrepancy between the expectation for the drug effects (conditioned responses) and the blunted pharmacological effects maintains drug taking in an attempt to achieve the expected reward. Also, whether tested during early or protracted withdrawal, addicted subjects show lower levels of D2 receptors in striatum (including NAc), which are associated with decreases in baseline activity in frontal brain regions implicated in salience attribution (orbitofrontal cortex) and inhibitory control (anterior cingulate gyrus), whose disruption results in compulsivity and impulsivity. These results point to an imbalance between dopaminergic circuits that underlie reward and conditioning and those that underlie executive function (emotional control and decision making), which we postulate contributes to the compulsive drug use and loss of control in addiction.

  15. Addiction: Beyond dopamine reward circuitry

    Energy Technology Data Exchange (ETDEWEB)

    Volkow, N.D.; Wang, G.; Volkow, N.D.; Wang, G.-J.; Fowler, J.S.; Tomasi, D.; Telang, F.

    2011-09-13

    Dopamine (DA) is considered crucial for the rewarding effects of drugs of abuse, but its role in addiction is much less clear. This review focuses on studies that used PET to characterize the brain DA system in addicted subjects. These studies have corroborated in humans the relevance of drug-induced fast DA increases in striatum [including nucleus accumbens (NAc)] in their rewarding effects but have unexpectedly shown that in addicted subjects, drug-induced DA increases (as well as their subjective reinforcing effects) are markedly blunted compared with controls. In contrast, addicted subjects show significant DA increases in striatum in response to drug-conditioned cues that are associated with self-reports of drug craving and appear to be of a greater magnitude than the DA responses to the drug. We postulate that the discrepancy between the expectation for the drug effects (conditioned responses) and the blunted pharmacological effects maintains drug taking in an attempt to achieve the expected reward. Also, whether tested during early or protracted withdrawal, addicted subjects show lower levels of D2 receptors in striatum (including NAc), which are associated with decreases in baseline activity in frontal brain regions implicated in salience attribution (orbitofrontal cortex) and inhibitory control (anterior cingulate gyrus), whose disruption results in compulsivity and impulsivity. These results point to an imbalance between dopaminergic circuits that underlie reward and conditioning and those that underlie executive function (emotional control and decision making), which we postulate contributes to the compulsive drug use and loss of control in addiction.

  16. Motivation and reward systems

    NARCIS (Netherlands)

    van Eerde, W.; Vodosek, M.; den Hartog, D.N.; McNett, J.M.

    2014-01-01

    Reward systems are identified as one of the human resource management (HRM) practices that may impact motivation. Reward systems may consist of several components, including financial and nonfinancial rewards, in fixed and variable amounts. Reinforcement, expectancy, and equity principles are

  17. Reward action in the initiation of smooth pursuit eye movements

    OpenAIRE

    Joshua, Mati; Lisberger, Stephen G.

    2012-01-01

    Reward has a powerful influence on motor behavior. To probe how and where reward systems alter motor behavior, we studied smooth pursuit eye movements in monkeys trained to associate the color of a visual cue with the size of the reward to be issued at the end of the target motion. When the tracking task presented two different colored targets that moved orthogonally, monkeys biased the initiation of pursuit towards the direction of motion of the target that led to larger reward. The bias was...

  18. Moderation of the Relationship Between Reward Expectancy and Prediction Error-Related Ventral Striatal Reactivity by Anhedonia in Unmedicated Major Depressive Disorder: Findings From the EMBARC Study

    Science.gov (United States)

    Greenberg, Tsafrir; Chase, Henry W.; Almeida, Jorge R.; Stiffler, Richelle; Zevallos, Carlos R.; Aslam, Haris A.; Deckersbach, Thilo; Weyandt, Sarah; Cooper, Crystal; Toups, Marisa; Carmody, Thomas; Kurian, Benji; Peltier, Scott; Adams, Phillip; McInnis, Melvin G.; Oquendo, Maria A.; McGrath, Patrick J.; Fava, Maurizio; Weissman, Myrna; Parsey, Ramin; Trivedi, Madhukar H.; Phillips, Mary L.

    2016-01-01

    Objective Anhedonia, disrupted reward processing, is a core symptom of major depressive disorder. Recent findings demonstrate altered reward-related ventral striatal reactivity in depressed individuals, but the extent to which this is specific to anhedonia remains poorly understood. The authors examined the effect of anhedonia on reward expectancy (expected outcome value) and prediction error-(discrepancy between expected and actual outcome) related ventral striatal reactivity, as well as the relationship between these measures. Method A total of 148 unmedicated individuals with major depressive disorder and 31 healthy comparison individuals recruited for the multisite EMBARC (Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care) study underwent functional MRI during a well-validated reward task. Region of interest and whole-brain data were examined in the first- (N=78) and second- (N=70) recruited cohorts, as well as the total sample, of depressed individuals, and in healthy individuals. Results Healthy, but not depressed, individuals showed a significant inverse relationship between reward expectancy and prediction error-related right ventral striatal reactivity. Across all participants, and in depressed individuals only, greater anhedonia severity was associated with a reduced reward expectancy-prediction error inverse relationship, even after controlling for other symptoms. Conclusions The normal reward expectancy and prediction error-related ventral striatal reactivity inverse relationship concords with conditioning models, predicting a shift in ventral striatal responding from reward outcomes to reward cues. This study shows, for the first time, an absence of this relationship in two cohorts of unmedicated depressed individuals and a moderation of this relationship by anhedonia, suggesting reduced reward-contingency learning with greater anhedonia. These findings help elucidate neural mechanisms of anhedonia, as a step toward

  19. Moderation of the Relationship Between Reward Expectancy and Prediction Error-Related Ventral Striatal Reactivity by Anhedonia in Unmedicated Major Depressive Disorder: Findings From the EMBARC Study.

    Science.gov (United States)

    Greenberg, Tsafrir; Chase, Henry W; Almeida, Jorge R; Stiffler, Richelle; Zevallos, Carlos R; Aslam, Haris A; Deckersbach, Thilo; Weyandt, Sarah; Cooper, Crystal; Toups, Marisa; Carmody, Thomas; Kurian, Benji; Peltier, Scott; Adams, Phillip; McInnis, Melvin G; Oquendo, Maria A; McGrath, Patrick J; Fava, Maurizio; Weissman, Myrna; Parsey, Ramin; Trivedi, Madhukar H; Phillips, Mary L

    2015-09-01

    Anhedonia, disrupted reward processing, is a core symptom of major depressive disorder. Recent findings demonstrate altered reward-related ventral striatal reactivity in depressed individuals, but the extent to which this is specific to anhedonia remains poorly understood. The authors examined the effect of anhedonia on reward expectancy (expected outcome value) and prediction error- (discrepancy between expected and actual outcome) related ventral striatal reactivity, as well as the relationship between these measures. A total of 148 unmedicated individuals with major depressive disorder and 31 healthy comparison individuals recruited for the multisite EMBARC (Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care) study underwent functional MRI during a well-validated reward task. Region of interest and whole-brain data were examined in the first- (N=78) and second- (N=70) recruited cohorts, as well as the total sample, of depressed individuals, and in healthy individuals. Healthy, but not depressed, individuals showed a significant inverse relationship between reward expectancy and prediction error-related right ventral striatal reactivity. Across all participants, and in depressed individuals only, greater anhedonia severity was associated with a reduced reward expectancy-prediction error inverse relationship, even after controlling for other symptoms. The normal reward expectancy and prediction error-related ventral striatal reactivity inverse relationship concords with conditioning models, predicting a shift in ventral striatal responding from reward outcomes to reward cues. This study shows, for the first time, an absence of this relationship in two cohorts of unmedicated depressed individuals and a moderation of this relationship by anhedonia, suggesting reduced reward-contingency learning with greater anhedonia. These findings help elucidate neural mechanisms of anhedonia, as a step toward identifying potential biosignatures

  20. Maladaptive Reward-Learning and Impulse Control Disorders in Patients with Parkinson’s Disease: A Clinical Overview and Pathophysiology Update

    Directory of Open Access Journals (Sweden)

    Jee-Young Lee

    2014-10-01

    Full Text Available Impulse control disorders (ICD in Parkinson’s disease (PD are a disabling non-motor symptom with frequencies of 13–35% among patients receiving dopamine replacement therapy. ICD in PD is strongly associated with dopaminergic drug use, especially non-ergot dopamine agonists (DA. However, individual susceptibility and disease-related neural changes are also important contributors to the development of ICD. Discrepancies between nigrostriatal and mesolimbic dopaminergic degeneration and non-physiological administration of dopaminergic drugs may induce abnormal ’hyperstimulation’ of the mesolimbic system, which alters reward-learning behaviors in PD patients. In addition, DA can make patients more impulsive during decision-making and seek risk-taking behaviors. DA intake is also related to the biased representation of rewards. Ultimately, loss of negative feedback control due to dysfunctional frontostriatal connections is necessary for the establishment of ICD in PD. The subsequent behavioral and neural changes are affected by PD treatment and disease progression; thus, proper treatment guidelines for physicians are needed to prevent the development of ICD. Future studies aimed at producing novel therapeutics to control the risk factors for ICD or treat ICD behaviors in PD are warranted. This review summarizes recent advances from epidemiological and pathophysiological studies on ICD in PD. Management principles and limitations of current therapeutics are briefly discussed.

  1. Low and high gamma oscillations in rat ventral striatum have distinct relationships to behavior, reward, and spiking activity on a learned spatial decision task

    Directory of Open Access Journals (Sweden)

    Matthijs A A Van Der Meer

    2009-06-01

    Full Text Available Local field potential (LFP oscillations in the brain reflect organization thought to be important for perception, attention, movement, and memory. In the basal ganglia, including dorsal striatum, dysfunctional LFP states are associated with Parkinson’s disease, while in healthy subjects, dorsal striatal LFPs have been linked to decision-making processes. However, LFPs in ventral striatum have been less studied. We report that in rats running a spatial decision task, prominent gamma-50 (45-55 Hz and gamma-80 (70-85 Hz oscillations in ventral striatum had distinct relationships to behavior, task events, and spiking activity. Gamma-50 power increased sharply following reward delivery and before movement initiation, while in contrast, gamma-80 power ramped up gradually to reward locations. Gamma-50 power was low and contained little structure during early learning, but rapidly developed a stable pattern, while gamma-80 power was initially high before returning to a stable level within a similar timeframe. Putative fast-spiking interneurons (FSIs showed phase, firing rate, and coherence relationships with gamma-50 and gamma-80, indicating that the observed LFP patterns are locally relevant. Furthermore, in a number of FSIs such relationships were specific to gamma-50 or gamma-80, suggesting that partially distinct FSI populations mediate the effects of gamma-50 and gamma-80.

  2. Theta and beta oscillatory dynamics in the dentate gyrus reveal a shift in network processing state during cue encounters

    Directory of Open Access Journals (Sweden)

    Lara Maria Rangel

    2015-07-01

    Full Text Available The hippocampus is an important structure for learning and memory processes, and has strong rhythmic activity. Although a large amount of research has been dedicated towards understanding the rhythmic activity in the hippocampus during exploratory behaviors, specifically in the theta (5-10 Hz frequency range, few studies have examined the temporal interplay of theta and other frequencies during the presentation of meaningful cues. We obtained in vivo electrophysiological recordings of local field potentials (LFP in the dentate gyrus (DG of the hippocampus as rats performed three different associative learning tasks. In each task, cue presentations elicited pronounced decrements in theta amplitude in conjunction with increases in beta (15-30Hz amplitude. These changes were often transient but were sustained from the onset of cue encounters until the occurrence of a reward outcome. This oscillatory profile shifted in time to precede cue encounters over the course of the session, and was not present during similar behavior in the absence of task relevant stimuli. The observed decreases in theta amplitude and increases in beta amplitude in the dentate gyrus may thus reflect a shift in processing state that occurs when encountering meaningful cues.

  3. Introduction: Addiction and Brain Reward and Anti-Reward Pathways

    Science.gov (United States)

    Gardner, Eliot L.

    2013-01-01

    bio-psycho-social” model of etiology holds very well for addiction. Addiction appears to correlate with a hypo-dopaminergic dysfunctional state within the reward circuitry of the brain. Neuroimaging studies in humans add credence to this hypothesis. Credible evidence also implicates serotonergic, opioid, endocannabinoid, GABAergic, and glutamatergic mechanisms in addiction. Critically, drug addiction progresses from occasional recreational use to impulsive use to habitual compulsive use. This correlates with a progression from reward-driven to habit-driven drug-seeking behavior. This behavioral progression correlates with a neuroanatomical progression from ventral striatal (nucleus accumbens) to dorsal striatal control over drug-seeking behavior. The three classical sets of craving and relapse triggers are a) re-exposure to addictive drugs, b) stress, and c) re-exposure to environmental cues (“people, places, things”) previously associated with drug-taking behavior. Drug-triggered relapse involves the nucleus accumbens and the neurotransmitter dopamine. Stress-triggered relapse involves a) the central nucleus of the amygdala, the bed nucleus of the stria terminalis, and the neurotransmitter CRF; and b) the lateral tegmental noradrenergic nuclei of the brain stem and the neurotransmitter norepinephrine. Cue-triggered relapse involves the basolateral nucleus of the amygdala, the hippocampus, and the neurotransmitter glutamate. Knowledge of the neuroanatomy, neurophysiology, neurochemistry, and neuropharmacology of addictive drug action in the brain is currently producing a variety of strategies for pharmacotherapeutic treatment of drug addiction, some of which appear promising. PMID:21508625

  4. Mapping of olfactory memory circuits: region-specific c-fos activation after odor-reward associative learning or after its retrieval.

    Science.gov (United States)

    Tronel, Sophie; Sara, Susan J

    2002-01-01

    Although there is growing knowledge about intracellular mechanisms underlying neuronal plasticity and memory consolidation and reconsolidation after retrieval, information concerning the interaction among brain areas during formation and retrieval of memory is relatively sparse and fragmented. Addressing this question requires simultaneous monitoring of activity in multiple brain regions during learning, the post-acquisition consolidation period, and retrieval and subsequent reconsolidation. Immunoreaction to the immediate early gene c-fos is a powerful tool to mark neuronal activation of specific populations of neurons. Using this method, we are able to report, for the first time, post-training activation of a network of closely related brain regions, particularly in the frontal cortex and the basolateral amygdala (BLA), that is specific to the learning of an odor-reward association. On the other hand, retrieval of a well-established associative memory trace does not seem to differentially activate the same regions. The amygdala, in particular, is not engaged after retrieval, whereas the lateral habenula (LHab) shows strong activation that is restricted to animals having previously learned the association. Although intracellular mechanisms may be similar during consolidation and reconsolidation, this study indicates that different brain circuits are involved in the two processes, at least with respect to a rapidly learned olfactory task.

  5. Temporal dynamics of reward anticipation in the human brain.

    Science.gov (United States)

    Zhang, Yuanyuan; Li, Qi; Wang, Zhao; Liu, Xun; Zheng, Ya

    2017-09-01

    Reward anticipation is a complex process including cue evaluation, motor preparation, and feedback anticipation. The present study investigated whether these psychological processes were dissociable on neural dynamics in terms of incentive valence and approach motivation. We recorded EEG when participants were performing a monetary incentive delay task, and found a cue-P3 during the cue-evaluation stage, a contingent negative variation (CNV) during the motor-preparation stage, and a stimulus-preceding negativity (SPN) during the feedback-anticipation stage. Critically, both the cue-P3 and SPN exhibited an enhanced sensitivity to gain versus loss anticipation, which was not observed for the CNV. Moreover, both the cue-P3 and SPN, instead of the CNV, for gain anticipation selectively predicted the participants' approach motivation as measured in a following effort expenditure for rewards task, particularly when reward uncertainty was maximal. Together, these results indicate that reward anticipation consists of several sub-stages, each with distinct functional significance, thus providing implications for neuropsychiatric diseases characterized by dysfunction in anticipatory reward processing. Copyright © 2017 Elsevier B.V. All rights reserved.

  6. Eating beyond metabolic need: how environmental cues influence feeding behavior.

    Science.gov (United States)

    Johnson, Alexander W

    2013-02-01

    Animals use current, past, and projected future states of the organism and the world in a finely tuned system to control ingestion. They must not only deal effectively with current nutrient deficiencies, but also manage energy resources to meet future needs, all within the constraints of the mechanisms of metabolism. Many recent approaches to understanding the control of ingestive behavior distinguish between homeostatic mechanisms concerned with energy balance, and hedonic and incentive processes based on palatability and reward characteristics of food. In this review, I consider how learning about environmental cues influences homeostatic and hedonic brain signals, which may lead to increases in the affective taste properties of food and desire to over consume. Understanding these mechanisms may be critical for elucidating the etiology of the obesity epidemic. Copyright © 2013 Elsevier Ltd. All rights reserved.

  7. Global Repetition Influences Contextual Cueing

    Science.gov (United States)

    Zang, Xuelian; Zinchenko, Artyom; Jia, Lina; Li, Hong

    2018-01-01

    Our visual system has a striking ability to improve visual search based on the learning of repeated ambient regularities, an effect named contextual cueing. Whereas most of the previous studies investigated contextual cueing effect with the same number of repeated and non-repeated search displays per block, the current study focused on whether a global repetition frequency formed by different presentation ratios between the repeated and non-repeated configurations influence contextual cueing effect. Specifically, the number of repeated and non-repeated displays presented in each block was manipulated: 12:12, 20:4, 4:20, and 4:4 in Experiments 1–4, respectively. The results revealed a significant contextual cueing effect when the global repetition frequency is high (≥1:1 ratio) in Experiments 1, 2, and 4, given that processing of repeated displays was expedited relative to non-repeated displays. Nevertheless, the contextual cueing effect reduced to a non-significant level when the repetition frequency reduced to 4:20 in Experiment 3. These results suggested that the presentation frequency of repeated relative to the non-repeated displays could influence the strength of contextual cueing. In other words, global repetition statistics could be a crucial factor to mediate contextual cueing effect. PMID:29636716

  8. Retrieval-induced forgetting and interference between cues:Training a cue-outcome association attenuates retrieval by alternative cues

    OpenAIRE

    Ortega-Castro, Nerea; Vadillo Nistal, Miguel

    2013-01-01

    Some researchers have attempted to determine whether situations in which a single cue is paired with several outcomes (A-B, A-C interference or interference between outcomes) involve the same learning and retrieval mechanisms as situations in which several cues are paired with a single outcome (A-B, C-B interference or interference between cues). Interestingly, current research on a related effect, which is known as retrieval-induced forgetting, can illuminate this debate. Most retrieval-indu...

  9. Monetary reward speeds up voluntary saccades.

    Science.gov (United States)

    Chen, Lewis L; Chen, Y Mark; Zhou, Wu; Mustain, William D

    2014-01-01

    Past studies have shown that reward contingency is critical for sensorimotor learning, and reward expectation speeds up saccades in animals. Whether monetary reward speeds up saccades in human remains unknown. Here we addressed this issue by employing a conditional saccade task, in which human subjects performed a series of non-reflexive, visually-guided horizontal saccades. The subjects were (or were not) financially compensated for making a saccade in response to a centrally-displayed visual congruent (or incongruent) stimulus. Reward modulation of saccadic velocities was quantified independently of the amplitude-velocity coupling. We found that reward expectation significantly sped up voluntary saccades up to 30°/s, and the reward modulation was consistent across tests. These findings suggest that monetary reward speeds up saccades in human in a fashion analogous to how juice reward sped up saccades in monkeys. We further noticed that the idiosyncratic nasal-temporal velocity asymmetry was highly consistent regardless of test order, and its magnitude was not correlated with the magnitude of reward modulation. This suggests that reward modulation and the intrinsic velocity asymmetry may be governed by separate mechanisms that regulate saccade generation.

  10. Increasing the efficacy of cue exposure treatment in preventing relapse of addictive behavior.

    Science.gov (United States)

    Havermans, Remco C; Jansen, Anita T M

    2003-07-01

    Theoretically, cue exposure treatment should be able to prevent relapse by extinguishing conditioned drug responding (e.g. cue-elicited craving). According to contemporary learning theory, though, extinction does not eliminate conditioned responding. Analogous cue exposure with response prevention (CERP) as a treatment of addictive behavior might not eliminate the learned relation between drug-related cues and drug use. This does not necessarily mean that cue exposure cannot successfully prevent relapse. Various suggestions for increasing the efficacy of cue exposure treatment are being discussed from a contemporary learning theory perspective. It is suggested that cue exposure treatment incorporating retrieval cues can be a beneficial treatment in preventing relapse of addictive behavior.

  11. Effects of motivation on reward and attentional networks: an fMRI study.

    Science.gov (United States)

    Ivanov, Iliyan; Liu, Xun; Clerkin, Suzanne; Schulz, Kurt; Friston, Karl; Newcorn, Jeffrey H; Fan, Jin

    2012-11-01

    Existing evidence suggests that reward and attentional networks function in concert and that activation in one system influences the other in a reciprocal fashion; however, the nature of these influences remains poorly understood. We therefore developed a three-component task to assess the interaction effects of reward anticipation and conflict resolution on the behavioral performance and the activation of brain reward and attentional systems. Sixteen healthy adult volunteers aged 21-45 years were scanned with functional magnetic resonance imaging (fMRI) while performing the task. A two-way repeated measures analysis of variance (ANOVA) with cue (reward vs. non-reward) and target (congruent vs. incongruent) as within-subjects factors was used to test for main and interaction effects. Neural responses to anticipation, conflict, and reward outcomes were tested. Behaviorally there were main effects of both reward cue and target congruency on reaction time. Neuroimaging results showed that reward anticipation and expected reward outcomes activated components of the attentional networks, including the inferior parietal and occipital cortices, whereas surprising non-rewards activated the frontoinsular cortex bilaterally and deactivated the ventral striatum. In turn, conflict activated a broad network associated with cognitive control and motor functions. Interaction effects showed decreased activity in the thalamus, anterior cingulated gyrus, and middle frontal gyrus bilaterally when difficult conflict trials (e.g., incongruent targets) were preceded by reward cues; in contrast, the ventral striatum and orbitofrontal cortex showed greater activation during congruent targets preceded by reward cues. These results suggest that reward anticipation is associated with lower activation in attentional networks, possibly due to increased processing efficiency, whereas more difficult, conflict trials are associated with lower activity in regions of the reward system, possibly

  12. Reward contingencies and the recalibration of task monitoring and reward systems: a high-density electrical mapping study.

    Science.gov (United States)

    Morie, K P; De Sanctis, P; Foxe, J J

    2014-07-25

    Task execution almost always occurs in the context of reward-seeking or punishment-avoiding behavior. As such, ongoing task-monitoring systems are influenced by reward anticipation systems. In turn, when a task has been executed either successfully or unsuccessfully, future iterations of that task will be re-titrated on the basis of the task outcome. Here, we examined the neural underpinnings of the task-monitoring and reward-evaluation systems to better understand how they govern reward-seeking behavior. Twenty-three healthy adult participants performed a task where they accrued points that equated to real world value (gift cards) by responding as rapidly as possible within an allotted timeframe, while success rate was titrated online by changing the duration of the timeframe dependent on participant performance. Informative cues initiated each trial, indicating the probability of potential reward or loss (four levels from very low to very high). We manipulated feedback by first informing participants of task success/failure, after which a second feedback signal indicated actual magnitude of reward/loss. High-density electroencephalography (EEG) recordings allowed for examination of event-related potentials (ERPs) to the informative cues and in turn, to both feedback signals. Distinct ERP components associated with reward cues, task-preparatory and task-monitoring processes, and reward feedback processes were identified. Unsurprisingly, participants displayed increased ERP amplitudes associated with task-preparatory processes following cues that predicted higher chances of reward. They also rapidly updated reward and loss prediction information dependent on task performance after the first feedback signal. Finally, upon reward receipt, initial reward probability was no longer taken into account. Rather, ERP measures suggested that only the magnitude of actual reward or loss was now processed. Reward and task-monitoring processes are clearly dissociable, but

  13. A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex.

    Science.gov (United States)

    Huertas, Marco A; Hussain Shuler, Marshall G; Shouval, Harel Z

    2015-09-16

    Many actions performed by animals and humans depend on an ability to learn, estimate, and produce temporal intervals of behavioral relevance. Exemplifying such learning of cued expectancies is the observation of reward-timing activity in the primary visual cortex (V1) of rodents, wherein neural responses to visual cues come to predict the time of future reward as behaviorally experienced in the past. These reward-timing responses exhibit significant heterogeneity in at least three qualitatively distinct classes: sustained increase or sustained decrease in firing rate until the time of expected reward, and a class of cells that reach a peak in firing at the expected delay. We elaborate upon our existing model by including inhibitory and excitatory units while imposing simple connectivity rules to demonstrate what role these inhibitory elements and the simple architectures play in sculpting the response dynamics of the network. We find that simply adding inhibition is not sufficient for obtaining the different distinct response classes, and that a broad distribution of inhibitory projections is necessary for obtaining peak-type responses. Furthermore, although changes in connection strength that modulate the effects of inhibition onto excitatory units have a strong impact on the firing rate profile of these peaked responses, the network exhibits robustness in its overall ability to predict the expected time of reward. Finally, we demonstrate how the magnitude of expected reward can be encoded at the expected delay in the network and how peaked responses express this reward expectancy. Heterogeneity in single-neuron responses is a common feature of neuronal systems, although sometimes, in theoretical approaches, it is treated as a nuisance and seldom considered as conveying a different aspect of a signal. In this study, we focus on the heterogeneous responses in the primary visual cortex of rodents trained with a predictable delayed reward time. We describe under what

  14. Wild rufous hummingbirds use local landmarks to return to rewarded locations.

    Science.gov (United States)

    Pritchard, David J; Scott, Renee D; Healy, Susan D; Hurly, Andrew T

    2016-01-01

    Animals may remember an important location with reference to one or more visual landmarks. In the laboratory, birds and mammals often preferentially use landmarks near a goal ("local landmarks") to return to that location at a later date. Although we know very little about how animals in the wild use landmarks to remember locations, mammals in the wild appear to prefer to use distant landmarks to return to rewarded locations. To examine what cues wild birds use when returning to a goal, we trained free-living hummingbirds to search for a reward at a location that was specified by three nearby visual landmarks. Following training we expanded the landmark array to test the extent that the birds relied on the local landmarks to return to the reward. During the test the hummingbirds' search was best explained by the birds having used the experimental landmarks to remember the reward location. How the birds used the landmarks was not clear and seemed to change over the course of each test. These wild hummingbirds, then, can learn locations in reference to nearby visual landmarks. Copyright © 2015 Elsevier B.V. All rights reserved.

  15. Measuring and modeling the interaction among reward size, delay to reward, and satiation level on motivation in monkeys.

    Science.gov (United States)

    Minamimoto, Takafumi; La Camera, Giancarlo; Richmond, Barry J

    2009-01-01

    Motivation is usually inferred from the likelihood or the intensity with which behavior is carried out. It is sensitive to external factors (e.g., the identity, amount, and timing of a rewarding outcome) and internal factors (e.g., hunger or thirst). We trained macaque monkeys to perform a nonchoice instrumental task (a sequential red-green color discrimination) while manipulating two external factors: reward size and delay-to-reward. We also inferred the state of one internal factor, level of satiation, by monitoring the accumulated reward. A visual cue indicated the forthcoming reward size and delay-to-reward in each trial. The fraction of trials completed correctly by the monkeys increased linearly with reward size and was hyperbolically discounted by delay-to-reward duration, relations that are similar to those found in free operant and choice tasks. The fraction of correct trials also decreased progressively as a function of the satiation level. Similar (albeit noiser) relations were obtained for reaction times. The combined effect of reward size, delay-to-reward, and satiation level on the proportion of correct trials is well described as a multiplication of the effects of the single factors when each factor is examined alone. These results provide a quantitative account of the interaction of external and internal factors on instrumental behavior, and allow us to extend the concept of subjective value of a rewarding outcome, usually confined to external factors, to account also for slow changes in the internal drive of the subject.

  16. Attentional Bias for Reward and Punishment in Overweight and Obesity: The TRAILS Study.

    Science.gov (United States)

    Jonker, Nienke C; Glashouwer, Klaske A; Ostafin, Brian D; van Hemel-Ruiter, Madelon E; Smink, Frédérique R E; Hoek, Hans W; de Jong, Peter J

    2016-01-01

    More than 80% of obese adolescents will become obese adults, and it is therefore important to enhance insight into characteristics that underlie the development and maintenance of overweight and obesity at a young age. The current study is the first to focus on attentional biases towards rewarding and punishing cues as potentially important factors. Participants were young adolescents (N = 607) who were followed from the age of 13 until the age of 19, and completed a motivational game indexing the attentional bias to general cues of reward and punishment. Additionally, self-reported reward and punishment sensitivity was measured. This study showed that attentional biases to cues that signal reward or punishment and self-reported reward and punishment sensitivity were not related to body mass index or the change in body mass index over six years in adolescents. Thus, attentional bias to cues of reward and cues of punishment, and self-reported reward and punishment sensitivity, do not seem to be crucial factors in the development and maintenance of overweight and obesity in adolescents. Exploratory analyses of the current study suggest that the amount of effort to gain reward and to avoid punishment may play a role in the development and maintenance of overweight and obesity. However, since the effort measure was a construct based on face validity and has not been properly validated, more studies are necessary before firm conclusions can be drawn.

  17. Cue conflicts in context

    DEFF Research Database (Denmark)

    Boeg Thomsen, Ditte; Poulsen, Mads

    2015-01-01

    When learning their first language, children develop strategies for assigning semantic roles to sentence structures, depending on morphosyntactic cues such as case and word order. Traditionally, comprehension experiments have presented transitive clauses in isolation, and crosslinguistically...... preschoolers. However, object-first clauses may be context-sensitive structures, which are infelicitous in isolation. In a second act-out study we presented OVS clauses in supportive and unsupportive discourse contexts and in isolation and found that five-to-six-year-olds’ OVS comprehension was enhanced...

  18. Introspective responses to cues and motivation to reduce cigarette smoking influence state and behavioral responses to cue exposure.

    Science.gov (United States)

    Veilleux, Jennifer C; Skinner, Kayla D

    2016-09-01

    In the current study, we aimed to extend smoking cue-reactivity research by evaluating delay discounting as an outcome of cigarette cue exposure. We also separated introspection in response to cues (e.g., self-reporting craving and affect) from cue exposure alone, to determine if introspection changes behavioral responses to cigarette cues. Finally, we included measures of quit motivation and resistance to smoking to assess motivational influences on cue exposure. Smokers were invited to participate in an online cue-reactivity study. Participants were randomly assigned to view smoking images or neutral images, and were randomized to respond to cues with either craving and affect questions (e.g., introspection) or filler questions. Following cue exposure, participants completed a delay discounting task and then reported state affect, craving, and resistance to smoking, as well as an assessment of quit motivation. We found that after controlling for trait impulsivity, participants who introspected on craving and affect showed higher delay discounting, irrespective of cue type, but we found no effect of response condition on subsequent craving (e.g., craving reactivity). We also found that motivation to quit interacted with experimental conditions to predict state craving and state resistance to smoking. Although asking about craving during cue exposure did not increase later craving, it resulted in greater delaying of discounted rewards. Overall, our findings suggest the need to further assess the implications of introspection and motivation on behavioral outcomes of cue exposure. Copyright © 2016 Elsevier Ltd. All rights reserved.

  19. Cue Reliance in L2 Written Production

    Science.gov (United States)

    Wiechmann, Daniel; Kerz, Elma

    2014-01-01

    Second language learners reach expert levels in relative cue weighting only gradually. On the basis of ensemble machine learning models fit to naturalistic written productions of German advanced learners of English and expert writers, we set out to reverse engineer differences in the weighting of multiple cues in a clause linearization problem. We…

  20. Reward inference by primate prefrontal and striatal neurons.

    Science.gov (United States)

    Pan, Xiaochuan; Fan, Hongwei; Sawa, Kosuke; Tsuda, Ichiro; Tsukada, Minoru; Sakagami, Masamichi

    2014-01-22

    The brain contains multiple yet distinct systems involved in reward prediction. To understand the nature of these processes, we recorded single-unit activity from the lateral prefrontal cortex (LPFC) and the striatum in monkeys performing a reward inference task using an asymmetric reward schedule. We found that neurons both in the LPFC and in the striatum predicted reward values for stimuli that had been previously well experienced with set reward quantities in the asymmetric reward task. Importantly, these LPFC neurons could predict the reward value of a stimulus using transitive inference even when the monkeys had not yet learned the stimulus-reward association directly; whereas these striatal neurons did not show such an ability. Nevertheless, because there were two set amounts of reward (large and small), the selected striatal neurons were able to exclusively infer the reward value (e.g., large) of one novel stimulus from a pair after directly experiencing the alternative stimulus with the other reward value (e.g., small). Our results suggest that although neurons that predict reward value for old stimuli in the LPFC could also do so for new stimuli via transitive inference, those in the striatum could only predict reward for new stimuli via exclusive inference. Moreover, the striatum showed more complex functions than was surmised previously for model-free learning.

  1. Enriched Encoding: Reward Motivation Organizes Cortical Networks for Hippocampal Detection of Unexpected Events

    OpenAIRE

    Murty, Vishnu P.; Adcock, R. Alison

    2013-01-01

    Learning how to obtain rewards requires learning about their contexts and likely causes. How do long-term memory mechanisms balance the need to represent potential determinants of reward outcomes with the computational burden of an over-inclusive memory? One solution would be to enhance memory for salient events that occur during reward anticipation, because all such events are potential determinants of reward. We tested whether reward motivation enhances encoding of salient events like expec...

  2. Effects of cue-exposure treatment on neural cue reactivity in alcohol dependence: a randomized trial.

    Science.gov (United States)

    Vollstädt-Klein, Sabine; Loeber, Sabine; Kirsch, Martina; Bach, Patrick; Richter, Anne; Bühler, Mira; von der Goltz, Christoph; Hermann, Derik; Mann, Karl; Kiefer, Falk

    2011-06-01

    In alcohol-dependent patients, alcohol-associated cues elicit brain activation in mesocorticolimbic networks involved in relapse mechanisms. Cue-exposure based extinction training (CET) has been shown to be efficacious in the treatment of alcoholism; however, it has remained unexplored whether CET mediates its therapeutic effects via changes of activity in mesolimbic networks in response to alcohol cues. In this study, we assessed CET treatment effects on cue-induced responses using functional magnetic resonance imaging (fMRI). In a randomized controlled trial, abstinent alcohol-dependent patients were randomly assigned to a CET group (n = 15) or a control group (n = 15). All patients underwent an extended detoxification treatment comprising medically supervised detoxification, health education, and supportive therapy. The CET patients additionally received nine CET sessions over 3 weeks, exposing the patient to his/her preferred alcoholic beverage. Cue-induced fMRI activation to alcohol cues was measured at pretreatment and posttreatment. Compared with pretreatment, fMRI cue-reactivity reduction was greater in the CET relative to the control group, especially in the anterior cingulate gyrus and the insula, as well as limbic and frontal regions. Before treatment, increased cue-induced fMRI activation was found in limbic and reward-related brain regions and in visual areas. After treatment, the CET group showed less activation than the control group in the left ventral striatum. The study provides first evidence that an exposure-based psychotherapeutic intervention in the treatment of alcoholism impacts on brain areas relevant for addiction memory and attentional focus to alcohol-associated cues and affects mesocorticolimbic reward pathways suggested to be pathophysiologically involved in addiction. Copyright © 2011 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

  3. Interaction between scene-based and array-based contextual cueing.

    Science.gov (United States)

    Rosenbaum, Gail M; Jiang, Yuhong V

    2013-07-01

    Contextual cueing refers to the cueing of spatial attention by repeated spatial context. Previous studies have demonstrated distinctive properties of contextual cueing by background scenes and by an array of search items. Whereas scene-based contextual cueing reflects explicit learning of the scene-target association, array-based contextual cueing is supported primarily by implicit learning. In this study, we investigated the interaction between scene-based and array-based contextual cueing. Participants searched for a target that was predicted by both the background scene and the locations of distractor items. We tested three possible patterns of interaction: (1) The scene and the array could be learned independently, in which case cueing should be expressed even when only one cue was preserved; (2) the scene and array could be learned jointly, in which case cueing should occur only when both cues were preserved; (3) overshadowing might occur, in which case learning of the stronger cue should preclude learning of the weaker cue. In several experiments, we manipulated the nature of the contextual cues present during training and testing. We also tested explicit awareness of scenes, scene-target associations, and arrays. The results supported the overshadowing account: Specifically, scene-based contextual cueing precluded array-based contextual cueing when both were predictive of the location of a search target. We suggest that explicit, endogenous cues dominate over implicit cues in guiding spatial attention.

  4. Integrated wireless fast-scan cyclic voltammetry recording and electrical stimulation for reward-predictive learning in awake, freely moving rats

    Science.gov (United States)

    Li, Yu-Ting; Wickens, Jeffery R.; Huang, Yi-Ling; Pan, Wynn H. T.; Chen, Fu-Yu Beverly; Chen, Jia-Jin Jason

    2013-08-01

    Objective. Fast-scan cyclic voltammetry (FSCV) is commonly used to monitor phasic dopamine release, which is usually performed using tethered recording and for limited types of animal behavior. It is necessary to design a wireless dopamine sensing system for animal behavior experiments. Approach. This study integrates a wireless FSCV system for monitoring the dopamine signal in the ventral striatum with an electrical stimulator that induces biphasic current to excite dopaminergic neurons in awake freely moving rats. The measured dopamine signals are unidirectionally transmitted from the wireless FSCV module to the host unit. To reduce electrical artifacts, an optocoupler and a separate power are applied to isolate the FSCV system and electrical stimulator, which can be activated by an infrared controller. Main results. In the validation test, the wireless backpack system has similar performance in comparison with a conventional wired system and it does not significantly affect the locomotor activity of the rat. In the cocaine administration test, the maximum electrically elicited dopamine signals increased to around 230% of the initial value 20 min after the injection of 10 mg kg-1 cocaine. In a classical conditioning test, the dopamine signal in response to a cue increased to around 60 nM over 50 successive trials while the electrically evoked dopamine concentration decreased from about 90 to 50 nM in the maintenance phase. In contrast, the cue-evoked dopamine concentration progressively decreased and the electrically evoked dopamine was eliminated during the extinction phase. In the histological evaluation, there was little damage to brain tissue after five months chronic implantation of the stimulating electrode. Significance. We have developed an integrated wireless voltammetry system for measuring dopamine concentration and providing electrical stimulation. The developed wireless FSCV system is proven to be a useful experimental tool for the continuous

  5. Expected reward modulates encoding-related theta activity before an event.

    Science.gov (United States)

    Gruber, Matthias J; Watrous, Andrew J; Ekstrom, Arne D; Ranganath, Charan; Otten, Leun J

    2013-01-01

    Oscillatory brain activity in the theta frequency range (4-8 Hz) before the onset of an event has been shown to affect the likelihood of successfully encoding the event into memory. Recent work has also indicated that frontal theta activity might be modulated by reward, but it is not clear how reward expectancy, anticipatory theta activity, and memory formation might be related. Here, we used scalp electroencephalography (EEG) to assess the relationship between these factors. EEG was recorded from healthy adults while they memorized a series of words. Each word was preceded by a cue that indicated whether a high or low monetary reward would be earned if the word was successfully remembered in a later recognition test. Frontal theta power between the presentation of the reward cue and the onset of a word was predictive of later memory for the word, but only in the high reward condition. No theta differences were observed before word onset following low reward cues. The magnitude of prestimulus encoding-related theta activity in the high reward condition was correlated with the number of high reward words that were later confidently recognized. These findings provide strong evidence for a link between reward expectancy, theta activity, and memory encoding. Theta activity before event onset seems to be especially important for the encoding of motivationally significant stimuli. One possibility is that dopaminergic activity during reward anticipation mediates frontal theta activity related to memory. Copyright © 2012 Elsevier Inc. All rights reserved.

  6. Imbalance in the sensitivity to different types of rewards in pathological gambling.

    Science.gov (United States)

    Sescousse, Guillaume; Barbalat, Guillaume; Domenech, Philippe; Dreher, Jean-Claude

    2013-08-01

    Pathological gambling is an addictive disorder characterized by a persistent and compulsive desire to engage in gambling activities. This maladaptive behaviour has been suggested to result from a decreased sensitivity to experienced rewards, regardless of reward type. Alternatively, pathological gambling might reflect an imbalance in the sensitivity to monetary versus non-monetary incentives. To directly test these two hypotheses, we examined how the brain reward circuit of pathological gamblers responds to different types of rewards. Using functional magnetic resonance imaging, we compared the brain responses of 18 pathological gamblers and 20 healthy control subjects while they engaged in a simple incentive task manipulating both monetary and visual erotic rewards. During reward anticipation, the ventral striatum of pathological gamblers showed a differential response to monetary versus erotic cues, essentially driven by a blunted reactivity to cues predicting erotic stimuli. This differential response correlated with the severity of gambling symptoms and was paralleled by a reduced behavioural motivation for erotic rewards. During reward outcome, a posterior orbitofrontal cortex region, responding to erotic rewards in both groups, was further recruited by monetary gains in pathological gamblers but not in control subjects. Moreover, while ventral striatal activity correlated with subjective ratings assigned to monetary and erotic rewards in control subjects, it only correlated with erotic ratings in gamblers. Our results point to a differential sensitivity to monetary versus non-monetary rewards in pathological gambling, both at the motivational and hedonic levels. Such an imbalance might create a bias towards monetary rewards, potentially promoting addictive gambling behaviour.

  7. Intranasal oxytocin enhances socially-reinforced learning in rhesus monkeys

    Directory of Open Access Journals (Sweden)

    Lisa A Parr

    2014-09-01

    Full Text Available There are currently no drugs approved for the treatment of social deficits associated with autism spectrum disorders (ASD. One hypothesis for these deficits is that individuals with ASD lack the motivation to attend to social cues because those cues are not implicitly rewarding. Therefore, any drug that could enhance the rewarding quality of social stimuli could have a profound impact on the treatment of ASD, and other social disorders. Oxytocin (OT is a neuropeptide that has been effective in enhancing social cognition and social reward in humans. The present study examined the ability of OT to selectively enhance learning after social compared to nonsocial reward in rhesus monkeys, an important species for modeling the neurobiology of social behavior in humans. Monkeys were required to learn an implicit visual matching task after receiving either intranasal (IN OT or Placebo (saline. Correct trials were rewarded with the presentation of positive and negative social (play faces/threat faces or nonsocial (banana/cage locks stimuli, plus food. Incorrect trials were not rewarded. Results demonstrated a strong effect of socially-reinforced learning, monkeys’ performed significantly better when reinforced with social versus nonsocial stimuli. Additionally, socially-reinforced learning was significantly better and occurred faster after IN-OT compared to placebo treatment. Performance in the IN-OT, but not Placebo, condition was also significantly better when the reinforcement stimuli were emotionally positive compared to negative facial expressions. These data support the hypothesis that OT may function to enhance prosocial behavior in primates by increasing the rewarding quality of emotionally positive, social compared to emotionally negative or nonsocial images. These data also support the use of the rhesus monkey as a model for exploring the neurobiological basis of social behavior and its impairment.

  8. Regulating task-monitoring systems in response to variable reward contingencies and outcomes in cocaine addicts.

    Science.gov (United States)

    Morie, Kristen P; De Sanctis, Pierfilippo; Garavan, Hugh; Foxe, John J

    2016-03-01

    We investigated anticipatory and consummatory reward processing in cocaine addiction. In addition, we set out to assess whether task-monitoring systems were appropriately recalibrated in light of variable reward schedules. We also examined neural measures of task-monitoring and reward processing as a function of hedonic tone, since anhedonia is a vulnerability marker for addiction that is obviously germane in the context of reward processing. High-density event-related potentials were recorded while participants performed a speeded response task that systematically varied anticipated probabilities of reward receipt. The paradigm dissociated feedback regarding task success (or failure) from feedback regarding the value of reward (or loss), so that task-monitoring and reward processing could be examined in partial isolation. Twenty-three active cocaine abusers and 23 age-matched healthy controls participated. Cocaine abusers showed amplified anticipatory responses to reward predictive cues, but crucially, these responses were not as strongly modulated by reward probability as in controls. Cocaine users also showed blunted responses to feedback about task success or failure and did not use this information to update predictions about reward. In turn, they showed clearly blunted responses to reward feedback. In controls and users, measures of anhedonia were associated with reward motivation. In cocaine users, anhedonia was also associated with diminished monitoring and reward feedback responses. Findings imply that reward anticipation and monitoring deficiencies in addiction are associated with increased responsiveness to reward cues but impaired ability to predict reward in light of task contingencies, compounded by deficits in responding to actual reward outcomes.

  9. "Liking" and "wanting" linked to Reward Deficiency Syndrome (RDS): hypothesizing differential responsivity in brain reward circuitry.

    Science.gov (United States)

    Blum, Kenneth; Gardner, Eliot; Oscar-Berman, Marlene; Gold, Mark

    2012-01-01

    In an attempt to resolve controversy regarding the causal contributions of mesolimbic dopamine (DA) systems to reward, we evaluate the three main competing explanatory categories: "liking,"learning," and "wanting" [1]. That is, DA may mediate (a) the hedonic impact of reward (liking), (b) learned predictions about rewarding effects (learning), or (c) the pursuit of rewards by attributing incentive salience to reward-related stimuli (wanting). We evaluate these hypotheses, especially as they relate to the Reward Deficiency Syndrome (RDS), and we find that the incentive salience or "wanting" hypothesis of DA function is supported by a majority of the evidence. Neuroimaging studies have shown that drugs of abuse, palatable foods, and anticipated behaviors such as sex and gaming affect brain regions involving reward circuitry, and may not be unidirectional. Drugs of abuse enhance DA signaling and sensitize mesolimbic mechanisms that evolved to attribute incentive salience to rewards. Addictive drugs have in common that they are voluntarily selfadministered, they enhance (directly or indirectly) dopaminergic synaptic function in the nucleus accumbens (NAC), and they stimulate the functioning of brain reward circuitry (producing the "high" that drug users seek). Although originally believed simply to encode the set point of hedonic tone, these circuits now are believed to be functionally more complex, also encoding attention, reward expectancy, disconfirmation of reward expectancy, and incentive motivation. Elevated stress levels, together with polymorphisms of dopaminergic genes and other neurotransmitter genetic variants, may have a cumulative effect on vulnerability to addiction. The RDS model of etiology holds very well for a variety of chemical and behavioral addictions.

  10. Prelude to passion: limbic activation by "unseen" drug and sexual cues.

    Directory of Open Access Journals (Sweden)

    Anna Rose Childress

    2008-01-01

    Full Text Available The human brain responds to recognizable signals for sex and for rewarding drugs of abuse by activation of limbic reward circuitry. Does the brain respond in similar way to such reward signals even when they are "unseen", i.e., presented in a way that prevents their conscious recognition? Can the brain response to "unseen" reward cues predict the future affective response to recognizable versions of such cues, revealing a link between affective/motivational processes inside and outside awareness?We exploited the fast temporal resolution of event-related functional magnetic resonance imaging (fMRI to test the brain response to "unseen" (backward-masked cocaine, sexual, aversive and neutral cues of 33 milliseconds duration in male cocaine patients (n = 22. Two days after scanning, the affective valence for visible versions of each cue type was determined using an affective bias (priming task. We demonstrate, for the first time, limbic brain activation by "unseen" drug and sexual cues of only 33 msec duration. Importantly, increased activity in an large interconnected ventral pallidum/amygdala cluster to the "unseen" cocaine cues strongly predicted future positive affect to visible versions of the same cues in subsequent off-magnet testing, pointing both to the functional significance of the rapid brain response, and to shared brain substrates for appetitive motivation within and outside awareness.These findings represent the first evidence that brain reward circuitry responds to drug and sexual cues presented outside awareness. The results underscore the sensitivity of the brain to "unseen" reward signals and may represent the brain's primordial signature for desire. The limbic brain response to reward cues outside awareness may represent a potential vulnerability in disorders (e.g., the addictions for whom poorly-controlled appetitive motivation is a central feature.

  11. Reward eliminates retrieval-induced forgetting.

    Science.gov (United States)

    Imai, Hisato; Kim, Dongho; Sasaki, Yuka; Watanabe, Takeo

    2014-12-02

    Although it is well known that reward enhances learning and memory, how extensively such enhancement occurs remains unclear. To address this question, we examined how reward influences retrieval-induced forgetting (RIF) in which the retrieval of a nonpracticed item under the same category as a practiced item is worse than the retrieval of a nonpracticed item outside the category. Subjects were asked to try to encode category-exemplar pairs (e.g., FISH-salmon). Then, they were presented with a category name and a two-letter word stem (e.g., FISH-sa) and were asked to complete an encoded word (retrieval practice). For a correct response, apple juice was given as a reward in the reward condition and a beeping sound was presented in the no-reward condition. Finally, subjects were asked to report whether each exemplar had been presented in the first phase. RIF was replicated in the no-reward condition. However, in the reward condition, RIF was eliminated. These results suggest that reward enhances processing of retrieval of unpracticed members by mechanisms such as spreading activation within the same category, irrespective of whether items were practiced or not.

  12. Ventral striatal activity links adversity and reward processing in children

    NARCIS (Netherlands)

    Kamkar, N.H.; Lewis, D.J.; van den Bos, W.; Morton, J.B.

    2017-01-01

    Adversity impacts many aspects of psychological and physical development including reward-based learning and decision-making. Mechanisms relating adversity and reward processing in children, however, remain unclear. Here, we show that adversity is associated with potentiated learning from positive

  13. The Rewards of Research at an Undergraduate Institution and Lessons Learned in Building Detector Systems that Work

    Science.gov (United States)

    Isenhower, Donald

    2015-04-01

    This talk addresses primary lessons learned during 28 years of work leading to the awarding of this prize for work on designing, building and operating detectors, with most of the work involving over 150 undergraduates during this time period. There are a wide range of skills and knowledge to be learned if a young scientist is interested in following this career route, so the most important subset of these will be described. Part will be how to involve undergraduate students at their fullest potential, and important differences of ACU from many programs, which has led to collaborators to make inquiries as to when will the ``ACU Army'' arrive so that they can time when their detector components will be shipped to the experiments for the testing and setup to be handed over to these students. The size of the detectors constructed have varied from small hodoscopes to the world's largest active cathode strip chambers. The science knowledge needed for detector construction is extremely multidisciplinary, and this must be learned by the professor directing the work as they will not have an engineering or support staff to lean on usually. This will include fields often considered unimportant to physics; however, ignorance of them can lead to failure. Knowing the primary question to ask will show where a significant area of concern will lie in what is being done by a person, group or company on a subsystem for a detector. Textbook descriptions of detectors, electronics, and materials can lead young experimenters astray. It has been learning the correct, fundamental physical processes that determine actual detector performance that has allowed the awardee to make his most important contributions over many years of research. A final lesson to be described will be how to make your undergraduate research program self-sustaining, so that critical knowledge is not lost as students graduate. Research supported in part by grants from the U.S. DOE Office of Science, the NSF, and

  14. Acute Social Stress Engages Synergistic Activity of Stress Mediators in the VTA to Promote Pavlovian Reward Learning

    OpenAIRE

    Kan, Russell; Pomrenze, Matthew; Tovar-Diaz, Jorge; Morikawa, Hitoshi; Drew, Michael; Pahlavan, Bahram

    2017-01-01

    Stressful events rapidly trigger activity-dependent synaptic plasticity in certain brain areas, driving the formation of aversive memories. However, it remains unclear how stressful experience affects plasticity mechanisms to regulate learning of appetitive events, such as intake of addictive drugs or palatable foods. Using rats, we show that two acute stress mediators, corticotropin-releasing factor (CRF) and norepinephrine (NE), enhance plasticity of NMDA receptor-mediated glutamatergic tra...

  15. Theta-band phase locking of orbitofrontal neurons during reward expectancy

    NARCIS (Netherlands)

    van Wingerden, M.; Vinck, M.; Lankelma, J.; Pennartz, C.M.A.

    2010-01-01

    The expectancy of a rewarding outcome following actions and cues is coded by a network of brain structures including the orbitofrontal cortex. Thus far, predicted reward was considered to be coded by time-averaged spike rates of neurons. However, besides firing rate, the precise timing of action

  16. Reward and Punishment in Minigames

    OpenAIRE

    Sigmund, K.; Hauert, C.; Nowak, M.A.

    2001-01-01

    Minigames capturing the essence of Public Goods experiments show that even in the absence of rationality assumptions, both punishment and reward will fail to bring about prosocial behavior. This result holds in particular for the well-known Ultimatum Game, which emerges as a special case. But reputation can induce fairness and cooperation in populations adapting through learning or imitation. Indeed, the inclusion of reputation effects in the corresponding dynamical models leads to the evolut...

  17. Slave to habit? Obesity is associated with decreased behavioural sensitivity to reward devaluation.

    Science.gov (United States)

    Horstmann, Annette; Dietrich, Anja; Mathar, David; Pössel, Maria; Villringer, Arno; Neumann, Jane

    2015-04-01

    The motivational value of food is lower during satiety compared to fasting. Dynamic changes in motivational value promote food seeking or meal cessation. In obesity this mechanism might be compromised since obese subjects ingest energy beyond homeostatic needs. Thus, lower adaptation of eating behaviour with respect to changes in motivational value might cause food overconsumption in obesity. To test this hypothesis, we implemented a selective satiation procedure to investigate the relationship between obesity and the size of the behavioural devaluation effect in humans. Lean to obese men (mean age 25.9, range 19-30 years; mean BMI 29.1, range 19.2-45.1 kg/m(2)) were trained on a free operant paradigm and learned to associate cues with the possibility to win different food rewards by pressing a button. After the initial training phase, one of the rewards was devalued by consumption. Response rates for and wanting of the different rewards were measured pre and post devaluation. Behavioural sensitivity to reward devaluation, measured as the magnitude of difference between pre and post responses, was regressed against BMI. Results indicate that (1) higher BMI compared to lower BMI in men led to an attenuated behavioural adjustment to reward devaluation, and (2) the decrease in motivational value was associated with the decrease in response rate between pre and post. Change in explicitly reported motivational value, however, was not affected by BMI. Thus, we conclude that high BMI in men is associated with lower behavioural adaptation with respect to changes in motivational value of food, possibly resulting in automatic overeating patterns that are hard to control in daily life. Copyright © 2014 Elsevier Ltd. All rights reserved.

  18. Cannabinoid CB1 receptor antagonist rimonabant disrupts nicotine reward-associated memory in rats.

    Science.gov (United States)

    Fang, Qin; Li, Fang-Qiong; Li, Yan-Qin; Xue, Yan-Xue; He, Ying-Ying; Liu, Jian-Feng; Lu, Lin; Wang, Ji-Shi

    2011-10-01

    Exposure to cues previously associated with drug intake leads to relapse by activating previously acquired memories. Based on previous findings, in which cannabinoid CB(1) receptors were found to be critically involved in specific aspects of learning and memory, we investigated the role of CB(1) receptors in nicotine reward memory using a rat conditioned place preference (CPP) model. In Experiment 1, rats were trained for CPP with alternating injections of nicotine (0.5mg/kg, s.c.) and saline to acquire the nicotine-conditioned memory. To examine the effects of rimonabant on the reconsolidation of nicotine reward memory, rats were administered rimonabant (0, 0.3, and 3.0mg/kg, i.p.) immediately after reexposure to the drug-paired context. In Experiment 2, rats were trained for CPP similarly to Experiment 1. To examine the effects of rimonabant on the reinstatement of nicotine reward memory, rimonabant (0, 0.3, and 3.0mg/kg, i.p.) was administered before the test of nicotine-induced CPP reinstatement. In Experiment 3, to evaluate whether rimonabant itself produces a reward memory, rats were trained for CPP with alternating injections of different doses of rimonabant (0, 0.3, and 3.0mg/kg) and saline. Rimonabant at a dose of 3.0mg/kg significantly disrupted the reconsolidation of nicotine memory and significantly blocked the reinstatement of nicotine-induced CPP. However, rimonabant itself did not produce CPP. These findings provide clear evidence that CB(1) receptors play a role in nicotine reward memory, suggesting that CB(1) receptor antagonists may be a potential target for managing nicotine addiction. Copyright © 2011 Elsevier Inc. All rights reserved.

  19. Overt and covert attention to location-based reward.

    Science.gov (United States)

    McCoy, Brónagh; Theeuwes, Jan

    2018-01-01

    Recent research on the impact of location-based reward on attentional orienting has indicated that reward factors play an influential role in spatial priority maps. The current study investigated whether and how reward associations based on spatial location translate from overt eye movements to covert attention. If reward associations can be tied to locations in space, and if overt and covert attention rely on similar overlapping neuronal populations, then both overt and covert attentional measures should display similar spatial-based reward learning. Our results suggest that location- and reward-based changes in one attentional domain do not lead to similar changes in the other. Specifically, although we found similar improvements at differentially rewarded locations during overt attentional learning, this translated to the least improvement at a highly rewarded location during covert attention. We interpret this as the result of an increased motivational link between the high reward location and the trained eye movement response acquired during learning, leading to a relative slowing during covert attention when the eyes remained fixated and the saccade response was suppressed. In a second experiment participants were not required to keep fixated during the covert attention task and we no longer observed relative slowing at the high reward location. Furthermore, the second experiment revealed no covert spatial priority of rewarded locations. We conclude that the transfer of location-based reward associations is intimately linked with the reward-modulated motor response employed during learning, and alternative attentional and task contexts may interfere with learned spatial priorities. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.

  20. Pervasive competition between threat and reward in the brain.

    Science.gov (United States)

    Choi, Jong Moon; Padmala, Srikanth; Spechler, Philip; Pessoa, Luiz

    2014-06-01

    In the current functional MRI study, we investigated interactions between reward and threat processing. Visual cues at the start of each trial informed participants about the chance of winning monetary reward and/or receiving a mild aversive shock. We tested two competing hypothesis: according to the 'salience hypothesis', in the condition involving both reward and threat, enhanced activation would be observed because of increased salience; according to the 'competition hypothesis', the processing of reward and threat would trade-off against each other, leading to reduced activation. Analysis of skin conductance data during a delay phase revealed an interaction between reward and threat processing, such that the effect of reward was reduced during threat and the effect of threat was reduced during reward. Analysis of imaging data during the same task phase revealed interactions between reward and threat processing in several regions, including the midbrain/ventral tegmental area, caudate, putamen, bed nucleus of the stria terminalis, anterior insula, middle frontal gyrus and dorsal anterior cingulate cortex. Taken together, our findings reveal conditions during which reward and threat trade-off against each other across multiple sites. Such interactions are suggestive of competitive processes and may reflect the organization of opponent systems in the brain. © The Author (2013). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

  1. Ventral striatal activity links adversity and reward processing in children.

    Science.gov (United States)

    Kamkar, Niki H; Lewis, Daniel J; van den Bos, Wouter; Morton, J Bruce

    2017-08-01

    Adversity impacts many aspects of psychological and physical development including reward-based learning and decision-making. Mechanisms relating adversity and reward processing in children, however, remain unclear. Here, we show that adversity is associated with potentiated learning from positive outcomes and impulsive decision-making, but unrelated to learning from negative outcomes. We then show via functional magnetic resonance imaging that the link between adversity and reward processing is partially mediated by differences in ventral striatal response to rewards. The findings suggest that early-life adversity is associated with alterations in the brain's sensitivity to rewards accounting, in part, for the link between adversity and altered reward processing in children. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.

  2. Ventral striatal activity links adversity and reward processing in children

    Directory of Open Access Journals (Sweden)

    Niki H. Kamkar

    2017-08-01

    Full Text Available Adversity impacts many aspects of psychological and physical development including reward-based learning and decision-making. Mechanisms relating adversity and reward processing in children, however, remain unclear. Here, we show that adversity is associated with potentiated learning from positive outcomes and impulsive decision-making, but unrelated to learning from negative outcomes. We then show via functional magnetic resonance imaging that the link between adversity and reward processing is partially mediated by differences in ventral striatal response to rewards. The findings suggest that early-life adversity is associated with alterations in the brain’s sensitivity to rewards accounting, in part, for the link between adversity and altered reward processing in children.

  3. Largely overlapping neuronal substrates of reactivity to drug, gambling, food and sexual cues: A comprehensive meta-analysis.

    Science.gov (United States)

    Noori, Hamid R; Cosa Linan, Alejandro; Spanagel, Rainer

    2016-09-01

    Cue reactivity to natural and social rewards is essential for motivational behavior. However, cue reactivity to drug rewards can also elicit craving in addicted subjects. The degree to which drug and natural rewards share neural substrates is not known. The objective of this study is to conduct a comprehensive meta-analysis of neuroimaging studies on drug, gambling and natural stimuli (food and sex) to identify the common and distinct neural substrates of cue reactivity to drug and natural rewards. Neural cue reactivity studies were selected for the meta-analysis by means of activation likelihood estimations, followed by sensitivity and clustering analyses of averaged neuronal response patterns. Data from 176 studies (5573 individuals) suggests largely overlapping neural response patterns towards all tested reward modalities. Common cue reactivity to natural and drug rewards was expressed by bilateral neural responses within anterior cingulate gyrus, insula, caudate head, inferior frontal gyrus, middle frontal gyrus and cerebellum. However, drug cues also generated distinct activation patterns in medial frontal gyrus, middle temporal gyrus, posterior cingulate gyrus, caudate body and putamen. Natural (sexual) reward cues induced unique activation of the pulvinar in thalamus. Neural substrates of cue reactivity to alcohol, drugs of abuse, food, sex and gambling are largely overlapping and comprise a network that processes reward, emotional responses and habit formation. This suggests that cue-mediated craving involves mechanisms that are not exclusive for addictive disorders but rather resemble the intersection of information pathways for processing reward, emotional responses, non-declarative memory and obsessive-compulsive behavior. Copyright © 2016 Elsevier B.V. and ECNP. All rights reserved.

  4. Using Virtual Microscopy to Scaffold Learning of Pathology: A Naturalistic Experiment on the Role of Visual and Conceptual Cues

    Science.gov (United States)

    Nivala, Markus; Saljo, Roger; Rystedt, Hans; Kronqvist, Pauliina; Lehtinen, Erno

    2012-01-01

    New representational technologies, such as virtual microscopy, create new affordances for medical education. In the article, a study on the following