Episodic memory encoding interferes with reward learning and decreases striatal prediction errors.

Science.gov (United States)

Wimmer, G Elliott; Braun, Erin Kendall; Daw, Nathaniel D; Shohamy, Daphna

2014-11-05

Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making. Copyright © 2014 the authors 0270-6474/14/3414901-12$15.00/0.

Working Memory Load Strengthens Reward Prediction Errors.

Science.gov (United States)

Collins, Anne G E; Ciullo, Brittany; Frank, Michael J; Badre, David

2017-04-19

Reinforcement learning (RL) in simple instrumental tasks is usually modeled as a monolithic process in which reward prediction errors (RPEs) are used to update expected values of choice options. This modeling ignores the different contributions of different memory and decision-making systems thought to contribute even to simple learning. In an fMRI experiment, we investigated how working memory (WM) and incremental RL processes interact to guide human learning. WM load was manipulated by varying the number of stimuli to be learned across blocks. Behavioral results and computational modeling confirmed that learning was best explained as a mixture of two mechanisms: a fast, capacity-limited, and delay-sensitive WM process together with slower RL. Model-based analysis of fMRI data showed that striatum and lateral prefrontal cortex were sensitive to RPE, as shown previously, but, critically, these signals were reduced when the learning problem was within capacity of WM. The degree of this neural interaction related to individual differences in the use of WM to guide behavioral learning. These results indicate that the two systems do not process information independently, but rather interact during learning. SIGNIFICANCE STATEMENT Reinforcement learning (RL) theory has been remarkably productive at improving our understanding of instrumental learning as well as dopaminergic and striatal network function across many mammalian species. However, this neural network is only one contributor to human learning and other mechanisms such as prefrontal cortex working memory also play a key role. Our results also show that these other players interact with the dopaminergic RL system, interfering with its key computation of reward prediction errors. Copyright © 2017 the authors 0270-6474/17/374332-11$15.00/0.

Dopamine prediction errors in reward learning and addiction: from theory to neural circuitry

Science.gov (United States)

Keiflin, Ronald; Janak, Patricia H.

2015-01-01

Summary Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error-signaling and addiction can be formulated and tested. PMID:26494275

Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry.

Science.gov (United States)

Keiflin, Ronald; Janak, Patricia H

2015-10-21

Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error signaling and addiction can be formulated and tested. Copyright © 2015 Elsevier Inc. All rights reserved.

Signed reward prediction errors drive declarative learning.

Directory of Open Access Journals (Sweden)

Esther De Loof

Full Text Available Reward prediction errors (RPEs are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning. However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.

Signed reward prediction errors drive declarative learning.

Science.gov (United States)

De Loof, Esther; Ergo, Kate; Naert, Lien; Janssens, Clio; Talsma, Durk; Van Opstal, Filip; Verguts, Tom

2018-01-01

Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals) during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.

Dopamine reward prediction errors reflect hidden state inference across time

Science.gov (United States)

Starkweather, Clara Kwon; Babayan, Benedicte M.; Uchida, Naoshige; Gershman, Samuel J.

2017-01-01

Midbrain dopamine neurons signal reward prediction error (RPE), or actual minus expected reward. The temporal difference (TD) learning model has been a cornerstone in understanding how dopamine RPEs could drive associative learning. Classically, TD learning imparts value to features that serially track elapsed time relative to observable stimuli. In the real world, however, sensory stimuli provide ambiguous information about the hidden state of the environment, leading to the proposal that TD learning might instead compute a value signal based on an inferred distribution of hidden states (a ‘belief state’). In this work, we asked whether dopaminergic signaling supports a TD learning framework that operates over hidden states. We found that dopamine signaling exhibited a striking difference between two tasks that differed only with respect to whether reward was delivered deterministically. Our results favor an associative learning rule that combines cached values with hidden state inference. PMID:28263301

When theory and biology differ: The relationship between reward prediction errors and expectancy.

Science.gov (United States)

Williams, Chad C; Hassall, Cameron D; Trska, Robert; Holroyd, Clay B; Krigolson, Olave E

2017-10-01

Comparisons between expectations and outcomes are critical for learning. Termed prediction errors, the violations of expectancy that occur when outcomes differ from expectations are used to modify value and shape behaviour. In the present study, we examined how a wide range of expectancy violations impacted neural signals associated with feedback processing. Participants performed a time estimation task in which they had to guess the duration of one second while their electroencephalogram was recorded. In a key manipulation, we varied task difficulty across the experiment to create a range of different feedback expectancies - reward feedback was either very expected, expected, 50/50, unexpected, or very unexpected. As predicted, the amplitude of the reward positivity, a component of the human event-related brain potential associated with feedback processing, scaled inversely with expectancy (e.g., unexpected feedback yielded a larger reward positivity than expected feedback). Interestingly, the scaling of the reward positivity to outcome expectancy was not linear as would be predicted by some theoretical models. Specifically, we found that the amplitude of the reward positivity was about equivalent for very expected and expected feedback, and for very unexpected and unexpected feedback. As such, our results demonstrate a sigmoidal relationship between reward expectancy and the amplitude of the reward positivity, with interesting implications for theories of reinforcement learning. Copyright © 2017 Elsevier B.V. All rights reserved.

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans.

Science.gov (United States)

Fouragnan, Elsa; Queirazza, Filippo; Retzler, Chris; Mullinger, Karen J; Philiastides, Marios G

2017-07-06

Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.

Altered neural reward and loss processing and prediction error signalling in depression

Science.gov (United States)

Ubl, Bettina; Kuehner, Christine; Kirsch, Peter; Ruttorf, Michaela

2015-01-01

Dysfunctional processing of reward and punishment may play an important role in depression. However, functional magnetic resonance imaging (fMRI) studies have shown heterogeneous results for reward processing in fronto-striatal regions. We examined neural responsivity associated with the processing of reward and loss during anticipation and receipt of incentives and related prediction error (PE) signalling in depressed individuals. Thirty medication-free depressed persons and 28 healthy controls performed an fMRI reward paradigm. Regions of interest analyses focused on neural responses during anticipation and receipt of gains and losses and related PE-signals. Additionally, we assessed the relationship between neural responsivity during gain/loss processing and hedonic capacity. When compared with healthy controls, depressed individuals showed reduced fronto-striatal activity during anticipation of gains and losses. The groups did not significantly differ in response to reward and loss outcomes. In depressed individuals, activity increases in the orbitofrontal cortex and nucleus accumbens during reward anticipation were associated with hedonic capacity. Depressed individuals showed an absence of reward-related PEs but encoded loss-related PEs in the ventral striatum. Depression seems to be linked to blunted responsivity in fronto-striatal regions associated with limited motivational responses for rewards and losses. Alterations in PE encoding might mirror blunted reward- and enhanced loss-related associative learning in depression. PMID:25567763

The Attraction Effect Modulates Reward Prediction Errors and Intertemporal Choices.

Science.gov (United States)

Gluth, Sebastian; Hotaling, Jared M; Rieskamp, Jörg

2017-01-11

Classical economic theory contends that the utility of a choice option should be independent of other options. This view is challenged by the attraction effect, in which the relative preference between two options is altered by the addition of a third, asymmetrically dominated option. Here, we leveraged the attraction effect in the context of intertemporal choices to test whether both decisions and reward prediction errors (RPE) in the absence of choice violate the independence of irrelevant alternatives principle. We first demonstrate that intertemporal decision making is prone to the attraction effect in humans. In an independent group of participants, we then investigated how this affects the neural and behavioral valuation of outcomes using a novel intertemporal lottery task and fMRI. Participants' behavioral responses (i.e., satisfaction ratings) were modulated systematically by the attraction effect and this modulation was correlated across participants with the respective change of the RPE signal in the nucleus accumbens. Furthermore, we show that, because exponential and hyperbolic discounting models are unable to account for the attraction effect, recently proposed sequential sampling models might be more appropriate to describe intertemporal choices. Our findings demonstrate for the first time that the attraction effect modulates subjective valuation even in the absence of choice. The findings also challenge the prospect of using neuroscientific methods to measure utility in a context-free manner and have important implications for theories of reinforcement learning and delay discounting. Many theories of value-based decision making assume that people first assess the attractiveness of each option independently of each other and then pick the option with the highest subjective value. The attraction effect, however, shows that adding a new option to a choice set can change the relative value of the existing options, which is a violation of the independence

A causal link between prediction errors, dopamine neurons and learning.

Science.gov (United States)

Steinberg, Elizabeth E; Keiflin, Ronald; Boivin, Josiah R; Witten, Ilana B; Deisseroth, Karl; Janak, Patricia H

2013-07-01

Situations in which rewards are unexpectedly obtained or withheld represent opportunities for new learning. Often, this learning includes identifying cues that predict reward availability. Unexpected rewards strongly activate midbrain dopamine neurons. This phasic signal is proposed to support learning about antecedent cues by signaling discrepancies between actual and expected outcomes, termed a reward prediction error. However, it is unknown whether dopamine neuron prediction error signaling and cue-reward learning are causally linked. To test this hypothesis, we manipulated dopamine neuron activity in rats in two behavioral procedures, associative blocking and extinction, that illustrate the essential function of prediction errors in learning. We observed that optogenetic activation of dopamine neurons concurrent with reward delivery, mimicking a prediction error, was sufficient to cause long-lasting increases in cue-elicited reward-seeking behavior. Our findings establish a causal role for temporally precise dopamine neuron signaling in cue-reward learning, bridging a critical gap between experimental evidence and influential theoretical frameworks.

Observing others stay or switch - How social prediction errors are integrated into reward reversal learning.

Science.gov (United States)

Ihssen, Niklas; Mussweiler, Thomas; Linden, David E J

2016-08-01

Reward properties of stimuli can undergo sudden changes, and the detection of these 'reversals' is often made difficult by the probabilistic nature of rewards/punishments. Here we tested whether and how humans use social information (someone else's choices) to overcome uncertainty during reversal learning. We show a substantial social influence during reversal learning, which was modulated by the type of observed behavior. Participants frequently followed observed conservative choices (no switches after punishment) made by the (fictitious) other player but ignored impulsive choices (switches), even though the experiment was set up so that both types of response behavior would be similarly beneficial/detrimental (Study 1). Computational modeling showed that participants integrated the observed choices as a 'social prediction error' instead of ignoring or blindly following the other player. Modeling also confirmed higher learning rates for 'conservative' versus 'impulsive' social prediction errors. Importantly, this 'conservative bias' was boosted by interpersonal similarity, which in conjunction with the lack of effects observed in a non-social control experiment (Study 2) confirmed its social nature. A third study suggested that relative weighting of observed impulsive responses increased with increased volatility (frequency of reversals). Finally, simulations showed that in the present paradigm integrating social and reward information was not necessarily more adaptive to maximize earnings than learning from reward alone. Moreover, integrating social information increased accuracy only when conservative and impulsive choices were weighted similarly during learning. These findings suggest that to guide decisions in choice contexts that involve reward reversals humans utilize social cues conforming with their preconceptions more strongly than cues conflicting with them, especially when the other is similar. Copyright © 2016 The Authors. Published by Elsevier B

Phasic dopamine as a prediction error of intrinsic and extrinsic reinforcements driving both action acquisition and reward maximization: a simulated robotic study.

Science.gov (United States)

Mirolli, Marco; Santucci, Vieri G; Baldassarre, Gianluca

2013-03-01

An important issue of recent neuroscientific research is to understand the functional role of the phasic release of dopamine in the striatum, and in particular its relation to reinforcement learning. The literature is split between two alternative hypotheses: one considers phasic dopamine as a reward prediction error similar to the computational TD-error, whose function is to guide an animal to maximize future rewards; the other holds that phasic dopamine is a sensory prediction error signal that lets the animal discover and acquire novel actions. In this paper we propose an original hypothesis that integrates these two contrasting positions: according to our view phasic dopamine represents a TD-like reinforcement prediction error learning signal determined by both unexpected changes in the environment (temporary, intrinsic reinforcements) and biological rewards (permanent, extrinsic reinforcements). Accordingly, dopamine plays the functional role of driving both the discovery and acquisition of novel actions and the maximization of future rewards. To validate our hypothesis we perform a series of experiments with a simulated robotic system that has to learn different skills in order to get rewards. We compare different versions of the system in which we vary the composition of the learning signal. The results show that only the system reinforced by both extrinsic and intrinsic reinforcements is able to reach high performance in sufficiently complex conditions. Copyright © 2013 Elsevier Ltd. All rights reserved.

Moderation of the Relationship Between Reward Expectancy and Prediction Error-Related Ventral Striatal Reactivity by Anhedonia in Unmedicated Major Depressive Disorder: Findings From the EMBARC Study.

Science.gov (United States)

Greenberg, Tsafrir; Chase, Henry W; Almeida, Jorge R; Stiffler, Richelle; Zevallos, Carlos R; Aslam, Haris A; Deckersbach, Thilo; Weyandt, Sarah; Cooper, Crystal; Toups, Marisa; Carmody, Thomas; Kurian, Benji; Peltier, Scott; Adams, Phillip; McInnis, Melvin G; Oquendo, Maria A; McGrath, Patrick J; Fava, Maurizio; Weissman, Myrna; Parsey, Ramin; Trivedi, Madhukar H; Phillips, Mary L

2015-09-01

Anhedonia, disrupted reward processing, is a core symptom of major depressive disorder. Recent findings demonstrate altered reward-related ventral striatal reactivity in depressed individuals, but the extent to which this is specific to anhedonia remains poorly understood. The authors examined the effect of anhedonia on reward expectancy (expected outcome value) and prediction error- (discrepancy between expected and actual outcome) related ventral striatal reactivity, as well as the relationship between these measures. A total of 148 unmedicated individuals with major depressive disorder and 31 healthy comparison individuals recruited for the multisite EMBARC (Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care) study underwent functional MRI during a well-validated reward task. Region of interest and whole-brain data were examined in the first- (N=78) and second- (N=70) recruited cohorts, as well as the total sample, of depressed individuals, and in healthy individuals. Healthy, but not depressed, individuals showed a significant inverse relationship between reward expectancy and prediction error-related right ventral striatal reactivity. Across all participants, and in depressed individuals only, greater anhedonia severity was associated with a reduced reward expectancy-prediction error inverse relationship, even after controlling for other symptoms. The normal reward expectancy and prediction error-related ventral striatal reactivity inverse relationship concords with conditioning models, predicting a shift in ventral striatal responding from reward outcomes to reward cues. This study shows, for the first time, an absence of this relationship in two cohorts of unmedicated depressed individuals and a moderation of this relationship by anhedonia, suggesting reduced reward-contingency learning with greater anhedonia. These findings help elucidate neural mechanisms of anhedonia, as a step toward identifying potential biosignatures

Spared internal but impaired external reward prediction error signals in major depressive disorder during reinforcement learning.

Science.gov (United States)

Bakic, Jasmina; Pourtois, Gilles; Jepma, Marieke; Duprat, Romain; De Raedt, Rudi; Baeken, Chris

2017-01-01

Major depressive disorder (MDD) creates debilitating effects on a wide range of cognitive functions, including reinforcement learning (RL). In this study, we sought to assess whether reward processing as such, or alternatively the complex interplay between motivation and reward might potentially account for the abnormal reward-based learning in MDD. A total of 35 treatment resistant MDD patients and 44 age matched healthy controls (HCs) performed a standard probabilistic learning task. RL was titrated using behavioral, computational modeling and event-related brain potentials (ERPs) data. MDD patients showed comparable learning rate compared to HCs. However, they showed decreased lose-shift responses as well as blunted subjective evaluations of the reinforcers used during the task, relative to HCs. Moreover, MDD patients showed normal internal (at the level of error-related negativity, ERN) but abnormal external (at the level of feedback-related negativity, FRN) reward prediction error (RPE) signals during RL, selectively when additional efforts had to be made to establish learning. Collectively, these results lend support to the assumption that MDD does not impair reward processing per se during RL. Instead, it seems to alter the processing of the emotional value of (external) reinforcers during RL, when additional intrinsic motivational processes have to be engaged. © 2016 Wiley Periodicals, Inc.

Moderation of the Relationship Between Reward Expectancy and Prediction Error-Related Ventral Striatal Reactivity by Anhedonia in Unmedicated Major Depressive Disorder: Findings From the EMBARC Study

Science.gov (United States)

Greenberg, Tsafrir; Chase, Henry W.; Almeida, Jorge R.; Stiffler, Richelle; Zevallos, Carlos R.; Aslam, Haris A.; Deckersbach, Thilo; Weyandt, Sarah; Cooper, Crystal; Toups, Marisa; Carmody, Thomas; Kurian, Benji; Peltier, Scott; Adams, Phillip; McInnis, Melvin G.; Oquendo, Maria A.; McGrath, Patrick J.; Fava, Maurizio; Weissman, Myrna; Parsey, Ramin; Trivedi, Madhukar H.; Phillips, Mary L.

2016-01-01

Objective Anhedonia, disrupted reward processing, is a core symptom of major depressive disorder. Recent findings demonstrate altered reward-related ventral striatal reactivity in depressed individuals, but the extent to which this is specific to anhedonia remains poorly understood. The authors examined the effect of anhedonia on reward expectancy (expected outcome value) and prediction error-(discrepancy between expected and actual outcome) related ventral striatal reactivity, as well as the relationship between these measures. Method A total of 148 unmedicated individuals with major depressive disorder and 31 healthy comparison individuals recruited for the multisite EMBARC (Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care) study underwent functional MRI during a well-validated reward task. Region of interest and whole-brain data were examined in the first- (N=78) and second- (N=70) recruited cohorts, as well as the total sample, of depressed individuals, and in healthy individuals. Results Healthy, but not depressed, individuals showed a significant inverse relationship between reward expectancy and prediction error-related right ventral striatal reactivity. Across all participants, and in depressed individuals only, greater anhedonia severity was associated with a reduced reward expectancy-prediction error inverse relationship, even after controlling for other symptoms. Conclusions The normal reward expectancy and prediction error-related ventral striatal reactivity inverse relationship concords with conditioning models, predicting a shift in ventral striatal responding from reward outcomes to reward cues. This study shows, for the first time, an absence of this relationship in two cohorts of unmedicated depressed individuals and a moderation of this relationship by anhedonia, suggesting reduced reward-contingency learning with greater anhedonia. These findings help elucidate neural mechanisms of anhedonia, as a step toward

Beyond reward prediction errors: the role of dopamine in movement kinematics

Directory of Open Access Journals (Sweden)

Joseph eBarter

2015-05-01

Full Text Available We recorded activity of dopamine (DA neurons in the substantia nigra pars compacta in unrestrained mice while monitoring their movements with video tracking. Our approach allows an unbiased examination of the continuous relationship between single unit activity and behavior. Although DA neurons show characteristic burst firing following cue or reward presentation, as previously reported, their activity can be explained by the representation of actual movement kinematics. Unlike neighboring pars reticulata GABAergic output neurons, which can represent vector components of position, DA neurons represent vector components of velocity or acceleration. We found neurons related to movements in four directions—up, down, left right. For horizontal movements, there is significant lateralization of neurons: the left nigra contains more rightward neurons, whereas the right nigra contains more leftward neurons. The relationship between DA activity and movement kinematics was found on both appetitive trials using sucrose and aversive trials using air puff, showing that these neurons belong to a velocity control circuit that can be used for any number of purposes, whether to seek reward or to avoid harm. In support of this conclusion, mimicry of the phasic activation of DA neurons with selective optogenetic stimulation could also generate movements. Contrary to the popular hypothesis that DA neurons encode reward prediction errors, our results suggest that nigrostriatal DA plays an essential role in controlling the kinematics of voluntary movements. We hypothesize that DA signaling implements gain adjustment for adaptive transition control, and describe a new model of BG in which DA functions to adjust the gain of a transition controller. This model has significant implications for our understanding of movement disorders implicating DA and the BG.

How we learn to make decisions: rapid propagation of reinforcement learning prediction errors in humans.

Science.gov (United States)

Krigolson, Olav E; Hassall, Cameron D; Handy, Todd C

2014-03-01

Our ability to make decisions is predicated upon our knowledge of the outcomes of the actions available to us. Reinforcement learning theory posits that actions followed by a reward or punishment acquire value through the computation of prediction errors-discrepancies between the predicted and the actual reward. A multitude of neuroimaging studies have demonstrated that rewards and punishments evoke neural responses that appear to reflect reinforcement learning prediction errors [e.g., Krigolson, O. E., Pierce, L. J., Holroyd, C. B., & Tanaka, J. W. Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1833-1840, 2009; Bayer, H. M., & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141, 2005; O'Doherty, J. P. Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14, 769-776, 2004; Holroyd, C. B., & Coles, M. G. H. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709, 2002]. Here, we used the brain ERP technique to demonstrate that not only do rewards elicit a neural response akin to a prediction error but also that this signal rapidly diminished and propagated to the time of choice presentation with learning. Specifically, in a simple, learnable gambling task, we show that novel rewards elicited a feedback error-related negativity that rapidly decreased in amplitude with learning. Furthermore, we demonstrate the existence of a reward positivity at choice presentation, a previously unreported ERP component that has a similar timing and topography as the feedback error-related negativity that increased in amplitude with learning. The pattern of results we observed mirrored the output of a computational model that we implemented to compute reward

From prediction error to incentive salience: mesolimbic computation of reward motivation

Science.gov (United States)

Berridge, Kent C.

2011-01-01

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I will discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g., drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously-learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus a consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To comprehend these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience. PMID:22487042

From prediction error to incentive salience: mesolimbic computation of reward motivation.

Science.gov (United States)

Berridge, Kent C

2012-04-01

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience. © 2012 The Author. European Journal of Neuroscience © 2012 Federation of European Neuroscience Societies and Blackwell Publishing Ltd.

Critical evidence for the prediction error theory in associative learning.

Science.gov (United States)

Terao, Kanta; Matsumoto, Yukihisa; Mizunami, Makoto

2015-03-10

In associative learning in mammals, it is widely accepted that the discrepancy, or error, between actual and predicted reward determines whether learning occurs. Complete evidence for the prediction error theory, however, has not been obtained in any learning systems: Prediction error theory stems from the finding of a blocking phenomenon, but blocking can also be accounted for by other theories, such as the attentional theory. We demonstrated blocking in classical conditioning in crickets and obtained evidence to reject the attentional theory. To obtain further evidence supporting the prediction error theory and rejecting alternative theories, we constructed a neural model to match the prediction error theory, by modifying our previous model of learning in crickets, and we tested a prediction from the model: the model predicts that pharmacological intervention of octopaminergic transmission during appetitive conditioning impairs learning but not formation of reward prediction itself, and it thus predicts no learning in subsequent training. We observed such an "auto-blocking", which could be accounted for by the prediction error theory but not by other competitive theories to account for blocking. This study unambiguously demonstrates validity of the prediction error theory in associative learning.

Hemispheric Asymmetries in Striatal Reward Responses Relate to Approach-Avoidance Learning and Encoding of Positive-Negative Prediction Errors in Dopaminergic Midbrain Regions.

Science.gov (United States)

Aberg, Kristoffer Carl; Doell, Kimberly C; Schwartz, Sophie

2015-10-28

Some individuals are better at learning about rewarding situations, whereas others are inclined to avoid punishments (i.e., enhanced approach or avoidance learning, respectively). In reinforcement learning, action values are increased when outcomes are better than predicted (positive prediction errors [PEs]) and decreased for worse than predicted outcomes (negative PEs). Because actions with high and low values are approached and avoided, respectively, individual differences in the neural encoding of PEs may influence the balance between approach-avoidance learning. Recent correlational approaches also indicate that biases in approach-avoidance learning involve hemispheric asymmetries in dopamine function. However, the computational and neural mechanisms underpinning such learning biases remain unknown. Here we assessed hemispheric reward asymmetry in striatal activity in 34 human participants who performed a task involving rewards and punishments. We show that the relative difference in reward response between hemispheres relates to individual biases in approach-avoidance learning. Moreover, using a computational modeling approach, we demonstrate that better encoding of positive (vs negative) PEs in dopaminergic midbrain regions is associated with better approach (vs avoidance) learning, specifically in participants with larger reward responses in the left (vs right) ventral striatum. Thus, individual dispositions or traits may be determined by neural processes acting to constrain learning about specific aspects of the world. Copyright © 2015 the authors 0270-6474/15/3514491-10$15.00/0.

Roles of dopamine neurons in mediating the prediction error in aversive learning in insects.

Science.gov (United States)

Terao, Kanta; Mizunami, Makoto

2017-10-31

In associative learning in mammals, it is widely accepted that the discrepancy, or error, between actual and predicted reward determines whether learning occurs. The prediction error theory has been proposed to account for the finding of a blocking phenomenon, in which pairing of a stimulus X with an unconditioned stimulus (US) could block subsequent association of a second stimulus Y to the US when the two stimuli were paired in compound with the same US. Evidence for this theory, however, has been imperfect since blocking can also be accounted for by competitive theories. We recently reported blocking in classical conditioning of an odor with water reward in crickets. We also reported an "auto-blocking" phenomenon in appetitive learning, which supported the prediction error theory and rejected alternative theories. The presence of auto-blocking also suggested that octopamine neurons mediate reward prediction error signals. Here we show that blocking and auto-blocking occur in aversive learning to associate an odor with salt water (US) in crickets, and our results suggest that dopamine neurons mediate aversive prediction error signals. We conclude that the prediction error theory is applicable to both appetitive learning and aversive learning in insects.

Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model

Directory of Open Access Journals (Sweden)

Paul eChorley

2011-05-01

Full Text Available Dopaminergic neurons in the mammalian substantia nigra displaycharacteristic phasic responses to stimuli which reliably predict thereceipt of primary rewards. These responses have been suggested toencode reward prediction-errors similar to those used in reinforcementlearning. Here, we propose a model of dopaminergic activity in whichprediction error signals are generated by the joint action ofshort-latency excitation and long-latency inhibition, in a networkundergoing dopaminergic neuromodulation of both spike-timing dependentsynaptic plasticity and neuronal excitability. In contrast toprevious models, sensitivity to recent events is maintained by theselective modification of specific striatal synapses, efferent tocortical neurons exhibiting stimulus-specific, temporally extendedactivity patterns. Our model shows, in the presence of significantbackground activity, (i a shift in dopaminergic response from rewardto reward predicting stimuli, (ii preservation of a response tounexpected rewards, and (iii a precisely-timed below-baseline dip inactivity observed when expected rewards are omitted.

No unified reward prediction error in local field potentials from the human nucleus accumbens: evidence from epilepsy patients.

Science.gov (United States)

Stenner, Max-Philipp; Rutledge, Robb B; Zaehle, Tino; Schmitt, Friedhelm C; Kopitzki, Klaus; Kowski, Alexander B; Voges, Jürgen; Heinze, Hans-Jochen; Dolan, Raymond J

2015-08-01

Functional magnetic resonance imaging (fMRI), cyclic voltammetry, and single-unit electrophysiology studies suggest that signals measured in the nucleus accumbens (Nacc) during value-based decision making represent reward prediction errors (RPEs), the difference between actual and predicted rewards. Here, we studied the precise temporal and spectral pattern of reward-related signals in the human Nacc. We recorded local field potentials (LFPs) from the Nacc of six epilepsy patients during an economic decision-making task. On each trial, patients decided whether to accept or reject a gamble with equal probabilities of a monetary gain or loss. The behavior of four patients was consistent with choices being guided by value expectations. Expected value signals before outcome onset were observed in three of those patients, at varying latencies and with nonoverlapping spectral patterns. Signals after outcome onset were correlated with RPE regressors in all subjects. However, further analysis revealed that these signals were better explained as outcome valence rather than RPE signals, with gamble gains and losses differing in the power of beta oscillations and in evoked response amplitudes. Taken together, our results do not support the idea that postsynaptic potentials in the Nacc represent a RPE that unifies outcome magnitude and prior value expectation. We discuss the generalizability of our findings to healthy individuals and the relation of our results to measurements of RPE signals obtained from the Nacc with other methods. Copyright © 2015 the American Physiological Society.

An MEG signature corresponding to an axiomatic model of reward prediction error.

Science.gov (United States)

Talmi, Deborah; Fuentemilla, Lluis; Litvak, Vladimir; Duzel, Emrah; Dolan, Raymond J

2012-01-02

Optimal decision-making is guided by evaluating the outcomes of previous decisions. Prediction errors are theoretical teaching signals which integrate two features of an outcome: its inherent value and prior expectation of its occurrence. To uncover the magnetic signature of prediction errors in the human brain we acquired magnetoencephalographic (MEG) data while participants performed a gambling task. Our primary objective was to use formal criteria, based upon an axiomatic model (Caplin and Dean, 2008a), to determine the presence and timing profile of MEG signals that express prediction errors. We report analyses at the sensor level, implemented in SPM8, time locked to outcome onset. We identified, for the first time, a MEG signature of prediction error, which emerged approximately 320 ms after an outcome and expressed as an interaction between outcome valence and probability. This signal followed earlier, separate signals for outcome valence and probability, which emerged approximately 200 ms after an outcome. Strikingly, the time course of the prediction error signal, as well as the early valence signal, resembled the Feedback-Related Negativity (FRN). In simultaneously acquired EEG data we obtained a robust FRN, but the win and loss signals that comprised this difference wave did not comply with the axiomatic model. Our findings motivate an explicit examination of the critical issue of timing embodied in computational models of prediction errors as seen in human electrophysiological data. Copyright © 2011 Elsevier Inc. All rights reserved.

The role of reward and reward uncertainty in episodic memory

OpenAIRE

Mason, Alice; Farrell, Simon; Howard-Jones, Paul; Ludwig, Casimir

2017-01-01

Declarative memory has been found to be sensitive to reward-related changes in the environment. The reward signal can be broken down into information regarding the expected value of the reward, reward uncertainty and the prediction error. Research has established that high as opposed to low reward values enhance declarative memory. Research in neuroscience suggests that high uncertainty activates the reward system, which could lead to enhanced learning and memory. Here we present the results ...

Optogenetic stimulation in a computational model of the basal ganglia biases action selection and reward prediction error.

Science.gov (United States)

Berthet, Pierre; Lansner, Anders

2014-01-01

Optogenetic stimulation of specific types of medium spiny neurons (MSNs) in the striatum has been shown to bias the selection of mice in a two choices task. This shift is dependent on the localisation and on the intensity of the stimulation but also on the recent reward history. We have implemented a way to simulate this increased activity produced by the optical flash in our computational model of the basal ganglia (BG). This abstract model features the direct and indirect pathways commonly described in biology, and a reward prediction pathway (RP). The framework is similar to Actor-Critic methods and to the ventral/dorsal distinction in the striatum. We thus investigated the impact on the selection caused by an added stimulation in each of the three pathways. We were able to reproduce in our model the bias in action selection observed in mice. Our results also showed that biasing the reward prediction is sufficient to create a modification in the action selection. However, we had to increase the percentage of trials with stimulation relative to that in experiments in order to impact the selection. We found that increasing only the reward prediction had a different effect if the stimulation in RP was action dependent (only for a specific action) or not. We further looked at the evolution of the change in the weights depending on the stage of learning within a block. A bias in RP impacts the plasticity differently depending on that stage but also on the outcome. It remains to experimentally test how the dopaminergic neurons are affected by specific stimulations of neurons in the striatum and to relate data to predictions of our model.

Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia.

Science.gov (United States)

Diuk, Carlos; Tsai, Karin; Wallis, Jonathan; Botvinick, Matthew; Niv, Yael

2013-03-27

Studies suggest that dopaminergic neurons report a unitary, global reward prediction error signal. However, learning in complex real-life tasks, in particular tasks that show hierarchical structure, requires multiple prediction errors that may coincide in time. We used functional neuroimaging to measure prediction error signals in humans performing such a hierarchical task involving simultaneous, uncorrelated prediction errors. Analysis of signals in a priori anatomical regions of interest in the ventral striatum and the ventral tegmental area indeed evidenced two simultaneous, but separable, prediction error signals corresponding to the two levels of hierarchy in the task. This result suggests that suitably designed tasks may reveal a more intricate pattern of firing in dopaminergic neurons. Moreover, the need for downstream separation of these signals implies possible limitations on the number of different task levels that we can learn about simultaneously.

Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

Science.gov (United States)

Kishida, Kenneth T.; Saez, Ignacio; Lohrenz, Terry; Witcher, Mark R.; Laxton, Adrian W.; Tatter, Stephen B.; White, Jason P.; Ellis, Thomas L.; Phillips, Paul E. M.; Montague, P. Read

2016-01-01

In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson’s disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson’s disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons. PMID:26598677

Premotor and Motor Cortices Encode Reward.

Directory of Open Access Journals (Sweden)

Pavan Ramkumar

Full Text Available Rewards associated with actions are critical for motivation and learning about the consequences of one's actions on the world. The motor cortices are involved in planning and executing movements, but it is unclear whether they encode reward over and above limb kinematics and dynamics. Here, we report a categorical reward signal in dorsal premotor (PMd and primary motor (M1 neurons that corresponds to an increase in firing rates when a trial was not rewarded regardless of whether or not a reward was expected. We show that this signal is unrelated to error magnitude, reward prediction error, or other task confounds such as reward consumption, return reach plan, or kinematic differences across rewarded and unrewarded trials. The availability of reward information in motor cortex is crucial for theories of reward-based learning and motivational influences on actions.

Baseline frontostriatal-limbic connectivity predicts reward-based memory formation.

Science.gov (United States)

Hamann, Janne M; Dayan, Eran; Hummel, Friedhelm C; Cohen, Leonardo G

2014-12-01

Reward mediates the acquisition and long-term retention of procedural skills in humans. Yet, learning under rewarded conditions is highly variable across individuals and the mechanisms that determine interindividual variability in rewarded learning are not known. We postulated that baseline functional connectivity in a large-scale frontostriatal-limbic network could predict subsequent interindividual variability in rewarded learning. Resting-state functional MRI was acquired in two groups of subjects (n = 30) who then trained on a visuomotor procedural learning task with or without reward feedback. We then tested whether baseline functional connectivity within the frontostriatal-limbic network predicted memory strength measured immediately, 24 h and 1 month after training in both groups. We found that connectivity in the frontostriatal-limbic network predicted interindividual variability in the rewarded but not in the unrewarded learning group. Prediction was strongest for long-term memory. Similar links between connectivity and reward-based memory were absent in two control networks, a fronto-parieto-temporal language network and the dorsal attention network. The results indicate that baseline functional connectivity within the frontostriatal-limbic network successfully predicts long-term retention of rewarded learning. © 2014 Wiley Periodicals, Inc.

Social learning through prediction error in the brain

Science.gov (United States)

Joiner, Jessica; Piva, Matthew; Turrin, Courtney; Chang, Steve W. C.

2017-06-01

Learning about the world is critical to survival and success. In social animals, learning about others is a necessary component of navigating the social world, ultimately contributing to increasing evolutionary fitness. How humans and nonhuman animals represent the internal states and experiences of others has long been a subject of intense interest in the developmental psychology tradition, and, more recently, in studies of learning and decision making involving self and other. In this review, we explore how psychology conceptualizes the process of representing others, and how neuroscience has uncovered correlates of reinforcement learning signals to explore the neural mechanisms underlying social learning from the perspective of representing reward-related information about self and other. In particular, we discuss self-referenced and other-referenced types of reward prediction errors across multiple brain structures that effectively allow reinforcement learning algorithms to mediate social learning. Prediction-based computational principles in the brain may be strikingly conserved between self-referenced and other-referenced information.

Interaction of Instrumental and Goal-Directed Learning Modulates Prediction Error Representations in the Ventral Striatum.

Science.gov (United States)

Guo, Rong; Böhmer, Wendelin; Hebart, Martin; Chien, Samson; Sommer, Tobias; Obermayer, Klaus; Gläscher, Jan

2016-12-14

Goal-directed and instrumental learning are both important controllers of human behavior. Learning about which stimulus event occurs in the environment and the reward associated with them allows humans to seek out the most valuable stimulus and move through the environment in a goal-directed manner. Stimulus-response associations are characteristic of instrumental learning, whereas response-outcome associations are the hallmark of goal-directed learning. Here we provide behavioral, computational, and neuroimaging results from a novel task in which stimulus-response and response-outcome associations are learned simultaneously but dominate behavior at different stages of the experiment. We found that prediction error representations in the ventral striatum depend on which type of learning dominates. Furthermore, the amygdala tracks the time-dependent weighting of stimulus-response versus response-outcome learning. Our findings suggest that the goal-directed and instrumental controllers dynamically engage the ventral striatum in representing prediction errors whenever one of them is dominating choice behavior. Converging evidence in human neuroimaging studies has shown that the reward prediction errors are correlated with activity in the ventral striatum. Our results demonstrate that this region is simultaneously correlated with a stimulus prediction error. Furthermore, the learning system that is currently dominating behavioral choice dynamically engages the ventral striatum for computing its prediction errors. This demonstrates that the prediction error representations are highly dynamic and influenced by various experimental context. This finding points to a general role of the ventral striatum in detecting expectancy violations and encoding error signals regardless of the specific nature of the reinforcer itself. Copyright © 2016 the authors 0270-6474/16/3612650-11$15.00/0.

Ventromedial Prefrontal Cortex Activation Is Associated with Memory Formation for Predictable Rewards

Science.gov (United States)

Bialleck, Katharina A.; Schaal, Hans-Peter; Kranz, Thorsten A.; Fell, Juergen; Elger, Christian E.; Axmacher, Nikolai

2011-01-01

During reinforcement learning, dopamine release shifts from the moment of reward consumption to the time point when the reward can be predicted. Previous studies provide consistent evidence that reward-predicting cues enhance long-term memory (LTM) formation of these items via dopaminergic projections to the ventral striatum. However, it is less clear whether memory for items that do not precede a reward but are directly associated with reward consumption is also facilitated. Here, we investigated this question in an fMRI paradigm in which LTM for reward-predicting and neutral cues was compared to LTM for items presented during consumption of reliably predictable as compared to less predictable rewards. We observed activation of the ventral striatum and enhanced memory formation during reward anticipation. During processing of less predictable as compared to reliably predictable rewards, the ventral striatum was activated as well, but items associated with less predictable outcomes were remembered worse than items associated with reliably predictable outcomes. Processing of reliably predictable rewards activated the ventromedial prefrontal cortex (vmPFC), and vmPFC BOLD responses were associated with successful memory formation of these items. Taken together, these findings show that consumption of reliably predictable rewards facilitates LTM formation and is associated with activation of the vmPFC. PMID:21326612

Nicotine Withdrawal Induces Neural Deficits in Reward Processing.

Science.gov (United States)

Oliver, Jason A; Evans, David E; Addicott, Merideth A; Potts, Geoffrey F; Brandon, Thomas H; Drobes, David J

2017-06-01

Nicotine withdrawal reduces neurobiological responses to nonsmoking rewards. Insight into these reward deficits could inform the development of targeted interventions. This study examined the effect of withdrawal on neural and behavioral responses during a reward prediction task. Smokers (N = 48) attended two laboratory sessions following overnight abstinence. Withdrawal was manipulated by having participants smoke three regular nicotine (0.6 mg yield; satiation) or very low nicotine (0.05 mg yield; withdrawal) cigarettes. Electrophysiological recordings of neural activity were obtained while participants completed a reward prediction task that involved viewing four combinations of predictive and reward-determining stimuli: (1) Unexpected Reward; (2) Predicted Reward; (3) Predicted Punishment; (4) Unexpected Punishment. The task evokes a medial frontal negativity that mimics the phasic pattern of dopaminergic firing in ventral tegmental regions associated with reward prediction errors. Nicotine withdrawal decreased the amplitude of the medial frontal negativity equally across all trial types (p nicotine dependence (p Nicotine withdrawal had equivocal impact across trial types, suggesting reward processing deficits are unlikely to stem from changes in phasic dopaminergic activity during prediction errors. Effects on tonic activity may be more pronounced. Pharmacological interventions directly targeting the dopamine system and behavioral interventions designed to increase reward motivation and responsiveness (eg, behavioral activation) may aid in mitigating withdrawal symptoms and potentially improving smoking cessation outcomes. Findings from this study indicate nicotine withdrawal impacts reward processing signals that are observable in smokers' neural activity. This may play a role in the subjective aversive experience of nicotine withdrawal and potentially contribute to smoking relapse. Interventions that address abnormal responding to both pleasant and

Human dorsal striatum encodes prediction errors during observational learning of instrumental actions.

Science.gov (United States)

Cooper, Jeffrey C; Dunne, Simon; Furey, Teresa; O'Doherty, John P

2012-01-01

The dorsal striatum plays a key role in the learning and expression of instrumental reward associations that are acquired through direct experience. However, not all learning about instrumental actions require direct experience. Instead, humans and other animals are also capable of acquiring instrumental actions by observing the experiences of others. In this study, we investigated the extent to which human dorsal striatum is involved in observational as well as experiential instrumental reward learning. Human participants were scanned with fMRI while they observed a confederate over a live video performing an instrumental conditioning task to obtain liquid juice rewards. Participants also performed a similar instrumental task for their own rewards. Using a computational model-based analysis, we found reward prediction errors in the dorsal striatum not only during the experiential learning condition but also during observational learning. These results suggest a key role for the dorsal striatum in learning instrumental associations, even when those associations are acquired purely by observing others.

Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning.

Science.gov (United States)

Zhu, Lusha; Mathewson, Kyle E; Hsu, Ming

2012-01-31

Decision-making in the presence of other competitive intelligent agents is fundamental for social and economic behavior. Such decisions require agents to behave strategically, where in addition to learning about the rewards and punishments available in the environment, they also need to anticipate and respond to actions of others competing for the same rewards. However, whereas we know much about strategic learning at both theoretical and behavioral levels, we know relatively little about the underlying neural mechanisms. Here, we show using a multi-strategy competitive learning paradigm that strategic choices can be characterized by extending the reinforcement learning (RL) framework to incorporate agents' beliefs about the actions of their opponents. Furthermore, using this characterization to generate putative internal values, we used model-based functional magnetic resonance imaging to investigate neural computations underlying strategic learning. We found that the distinct notions of prediction errors derived from our computational model are processed in a partially overlapping but distinct set of brain regions. Specifically, we found that the RL prediction error was correlated with activity in the ventral striatum. In contrast, activity in the ventral striatum, as well as the rostral anterior cingulate (rACC), was correlated with a previously uncharacterized belief-based prediction error. Furthermore, activity in rACC reflected individual differences in degree of engagement in belief learning. These results suggest a model of strategic behavior where learning arises from interaction of dissociable reinforcement and belief-based inputs.

Prediction-error of Prediction Error (PPE)-based Reversible Data Hiding

OpenAIRE

Wu, Han-Zhou; Wang, Hong-Xia; Shi, Yun-Qing

2016-01-01

This paper presents a novel reversible data hiding (RDH) algorithm for gray-scaled images, in which the prediction-error of prediction error (PPE) of a pixel is used to carry the secret data. In the proposed method, the pixels to be embedded are firstly predicted with their neighboring pixels to obtain the corresponding prediction errors (PEs). Then, by exploiting the PEs of the neighboring pixels, the prediction of the PEs of the pixels can be determined. And, a sorting technique based on th...

AN EXTENDED REINFORCEMENT LEARNING MODEL OF BASAL GANGLIA TO UNDERSTAND THE CONTRIBUTIONS OF SEROTONIN AND DOPAMINE IN RISK-BASED DECISION MAKING, REWARD PREDICTION, AND PUNISHMENT LEARNING

Directory of Open Access Journals (Sweden)

Pragathi Priyadharsini Balasubramani

2014-04-01

Full Text Available Although empirical and neural studies show that serotonin (5HT plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL-framework. The model depicts the roles of dopamine (DA and serotonin (5HT in Basal Ganglia (BG. In this model, the DA signal is represented by the temporal difference error (δ, while the 5HT signal is represented by a parameter (α that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: 1 Risk-sensitive decision making, where 5HT controls risk assessment, 2 Temporal reward prediction, where 5HT controls time-scale of reward prediction, and 3 Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.

Trait Anticipatory Pleasure Predicts Effort Expenditure for Reward.

Directory of Open Access Journals (Sweden)

Joachim T Geaney

Full Text Available Research in motivation and emotion has been increasingly influenced by the perspective that processes underpinning the motivated approach of rewarding goals are distinct from those underpinning enjoyment during reward consummation. This distinction recently inspired the construction of the Temporal Experience of Pleasure Scale (TEPS, a self-report measure that distinguishes trait anticipatory pleasure (pre-reward feelings of desire from consummatory pleasure (feelings of enjoyment and gratification upon reward attainment. In a university community sample (N = 97, we examined the TEPS subscales as predictors of (1 the willingness to expend effort for monetary rewards, and (2 affective responses to a pleasant mood induction procedure. Results showed that both anticipatory pleasure and a well-known trait measure of reward motivation predicted effort-expenditure for rewards when the probability of being rewarded was relatively low. Against expectations, consummatory pleasure was unrelated to induced pleasant affect. Taken together, our findings provide support for the validity of the TEPS anticipatory pleasure scale, but not the consummatory pleasure scale.

Music-related reward responses predict episodic memory performance.

Science.gov (United States)

Ferreri, Laura; Rodriguez-Fornells, Antoni

2017-12-01

Music represents a special type of reward involving the recruitment of the mesolimbic dopaminergic system. According to recent theories on episodic memory formation, as dopamine strengthens the synaptic potentiation produced by learning, stimuli triggering dopamine release could result in long-term memory improvements. Here, we behaviourally test whether music-related reward responses could modulate episodic memory performance. Thirty participants rated (in terms of arousal, familiarity, emotional valence, and reward) and encoded unfamiliar classical music excerpts. Twenty-four hours later, their episodic memory was tested (old/new recognition and remember/know paradigm). Results revealed an influence of music-related reward responses on memory: excerpts rated as more rewarding were significantly better recognized and remembered. Furthermore, inter-individual differences in the ability to experience musical reward, measured through the Barcelona Music Reward Questionnaire, positively predicted memory performance. Taken together, these findings shed new light on the relationship between music, reward and memory, showing for the first time that music-driven reward responses are directly implicated in higher cognitive functions and can account for individual differences in memory performance.

Differential encoding of factors influencing predicted reward value in monkey rostral anterior cingulate cortex.

Science.gov (United States)

Toda, Koji; Sugase-Miyamoto, Yasuko; Mizuhiki, Takashi; Inaba, Kiyonori; Richmond, Barry J; Shidara, Munetaka

2012-01-01

The value of a predicted reward can be estimated based on the conjunction of both the intrinsic reward value and the length of time to obtain it. The question we addressed is how the two aspects, reward size and proximity to reward, influence the responses of neurons in rostral anterior cingulate cortex (rACC), a brain region thought to play an important role in reward processing. We recorded from single neurons while two monkeys performed a multi-trial reward schedule task. The monkeys performed 1-4 sequential color discrimination trials to obtain a reward of 1-3 liquid drops. There were two task conditions, a valid cue condition, where the number of trials and reward amount were associated with visual cues, and a random cue condition, where the cue was picked from the cue set at random. In the valid cue condition, the neuronal firing is strongly modulated by the predicted reward proximity during the trials. Information about the predicted reward amount is almost absent at those times. In substantial subpopulations, the neuronal responses decreased or increased gradually through schedule progress to the predicted outcome. These two gradually modulating signals could be used to calculate the effect of time on the perception of reward value. In the random cue condition, little information about the reward proximity or reward amount is encoded during the course of the trial before reward delivery, but when the reward is actually delivered the responses reflect both the reward proximity and reward amount. Our results suggest that the rACC neurons encode information about reward proximity and amount in a manner that is dependent on utility of reward information. The manner in which the information is represented could be used in the moment-to-moment calculation of the effect of time and amount on predicted outcome value.

Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain.

Science.gov (United States)

Niv, Yael; Edlund, Jeffrey A; Dayan, Peter; O'Doherty, John P

2012-01-11

Humans and animals are exquisitely, though idiosyncratically, sensitive to risk or variance in the outcomes of their actions. Economic, psychological, and neural aspects of this are well studied when information about risk is provided explicitly. However, we must normally learn about outcomes from experience, through trial and error. Traditional models of such reinforcement learning focus on learning about the mean reward value of cues and ignore higher order moments such as variance. We used fMRI to test whether the neural correlates of human reinforcement learning are sensitive to experienced risk. Our analysis focused on anatomically delineated regions of a priori interest in the nucleus accumbens, where blood oxygenation level-dependent (BOLD) signals have been suggested as correlating with quantities derived from reinforcement learning. We first provide unbiased evidence that the raw BOLD signal in these regions corresponds closely to a reward prediction error. We then derive from this signal the learned values of cues that predict rewards of equal mean but different variance and show that these values are indeed modulated by experienced risk. Moreover, a close neurometric-psychometric coupling exists between the fluctuations of the experience-based evaluations of risky options that we measured neurally and the fluctuations in behavioral risk aversion. This suggests that risk sensitivity is integral to human learning, illuminating economic models of choice, neuroscientific models of affective learning, and the workings of the underlying neural mechanisms.

Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia.

Directory of Open Access Journals (Sweden)

Erin C Dowd

Full Text Available Reward processing abnormalities have been implicated in the pathophysiology of negative symptoms such as anhedonia and avolition in schizophrenia. However, studies examining neural responses to reward anticipation and receipt have largely relied on instrumental tasks, which may confound reward processing abnormalities with deficits in response selection and execution. 25 chronic, medicated outpatients with schizophrenia and 20 healthy controls underwent functional magnetic resonance imaging using a pavlovian reward prediction paradigm with no response requirements. Subjects passively viewed cues that predicted subsequent receipt of monetary reward or non-reward, and blood-oxygen-level-dependent signal was measured at the time of cue presentation and receipt. At the group level, neural responses to both reward anticipation and receipt were largely similar between groups. At the time of cue presentation, striatal anticipatory responses did not differ between patients and controls. Right anterior insula demonstrated greater activation for nonreward than reward cues in controls, and for reward than nonreward cues in patients. At the time of receipt, robust responses to receipt of reward vs. nonreward were seen in striatum, midbrain, and frontal cortex in both groups. Furthermore, both groups demonstrated responses to unexpected versus expected outcomes in cortical areas including bilateral dorsolateral prefrontal cortex. Individual difference analyses in patients revealed an association between physical anhedonia and activity in ventral striatum and ventromedial prefrontal cortex during anticipation of reward, in which greater anhedonia severity was associated with reduced activation to money versus no-money cues. In ventromedial prefrontal cortex, this relationship held among both controls and patients, suggesting a relationship between anticipatory activity and anhedonia irrespective of diagnosis. These findings suggest that in the absence of

The habenula governs the attribution of incentive salience to reward predictive cues

Science.gov (United States)

Danna, Carey L.; Shepard, Paul D.; Elmer, Greg I.

2013-01-01

The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr), the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms. PMID:24368898

The habenula governs the attribution of incentive salience to reward predictive cues.

Directory of Open Access Journals (Sweden)

Carey L. Danna

2013-12-01

Full Text Available The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb, a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr, the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms.

Neurobiological underpinnings of reward anticipation and outcome evaluation in gambling disorder

DEFF Research Database (Denmark)

Linnet, Jakob

2014-01-01

Gambling disorder is characterized by persistent and recurrent maladaptive gambling behavior, which leads to clinically significant impairment or distress. The disorder is associated with dysfunctions in the dopamine system. The dopamine system codes reward anticipation and outcome evaluation....... Reward anticipation refers to dopaminergic activation prior to reward, while outcome evaluation refers to dopaminergic activation after reward. This article reviews evidence of dopaminergic dysfunctions in reward anticipation and outcome evaluation in gambling disorder from two vantage points: a model...... of reward prediction and reward prediction error by Wolfram Schultz et al. and a model of “wanting” and “liking” by Terry E. Robinson and Kent C. Berridge. Both models offer important insights on the study of dopaminergic dysfunctions in addiction, and implications for the study of dopaminergic dysfunctions...

Absorbed in the task : Personality measures predict engagement during task performance as tracked by error negativity and asymmetrical frontal activity

NARCIS (Netherlands)

Tops, Mattie; Boksem, Maarten A. S.

2010-01-01

We hypothesized that interactions between traits and context predict task engagement, as measured by the amplitude of the error-related negativity (ERN), performance, and relative frontal activity asymmetry (RFA). In Study 1, we found that drive for reward, absorption, and constraint independently

Adaptive scaling of reward in episodic memory:a replication study

OpenAIRE

Mason, Alice; Ludwig, Casimir; Farrell, Simon

2017-01-01

Reward is thought to enhance episodic memory formation via dopaminergic consolidation. Bunzeck, Dayan, Dolan, and Duzel [(2010). A common mechanism for adaptive scaling of reward and novelty. Human Brain Mapping, 31, 1380–1394] provided functional magnetic resonance imaging (fMRI) and behavioural evidence that reward and episodic memory systems are sensitive to the contextual value of a reward—whether it is relatively higher or lower—as opposed to absolute value or prediction error. We carrie...

Hedging Your Bets by Learning Reward Correlations in the Human Brain

Science.gov (United States)

Wunderlich, Klaus; Symmonds, Mkael; Bossaerts, Peter; Dolan, Raymond J.

2011-01-01

Summary Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling. PMID:21943609

Dissociated roles of the anterior cingulate cortex in reward and conflict processing as revealed by the feedback error-related negativity and N200.

Science.gov (United States)

Baker, Travis E; Holroyd, Clay B

2011-04-01

The reinforcement learning theory of the error-related negativity (ERN) holds that the impact of reward signals carried by the midbrain dopamine system modulates activity of the anterior cingulate cortex (ACC), alternatively disinhibiting and inhibiting the ACC following unpredicted error and reward events, respectively. According to a recent formulation of the theory, activity that is intrinsic to the ACC produces a component of the event-related brain potential (ERP) called the N200, and following unpredicted rewards, the N200 is suppressed by extrinsically applied positive dopamine reward signals, resulting in an ERP component called the feedback-ERN (fERN). Here we demonstrate that, despite extensive spatial and temporal overlap between the two ERP components, the functional processes indexed by the N200 (conflict) and the fERN (reward) are dissociable. These results point toward avenues for future investigation. Copyright © 2011 Elsevier B.V. All rights reserved.

Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control.

Science.gov (United States)

Manohar, Sanjay G; Chong, Trevor T-J; Apps, Matthew A J; Batla, Amit; Stamelou, Maria; Jarman, Paul R; Bhatia, Kailash P; Husain, Masud

2015-06-29

Speed-accuracy trade-off is an intensively studied law governing almost all behavioral tasks across species. Here we show that motivation by reward breaks this law, by simultaneously invigorating movement and improving response precision. We devised a model to explain this paradoxical effect of reward by considering a new factor: the cost of control. Exerting control to improve response precision might itself come at a cost--a cost to attenuate a proportion of intrinsic neural noise. Applying a noise-reduction cost to optimal motor control predicted that reward can increase both velocity and accuracy. Similarly, application to decision-making predicted that reward reduces reaction times and errors in cognitive control. We used a novel saccadic distraction task to quantify the speed and accuracy of both movements and decisions under varying reward. Both faster speeds and smaller errors were observed with higher incentives, with the results best fitted by a model including a precision cost. Recent theories consider dopamine to be a key neuromodulator in mediating motivational effects of reward. We therefore examined how Parkinson's disease (PD), a condition associated with dopamine depletion, alters the effects of reward. Individuals with PD showed reduced reward sensitivity in their speed and accuracy, consistent in our model with higher noise-control costs. Including a cost of control over noise explains how reward may allow apparent performance limits to be surpassed. On this view, the pattern of reduced reward sensitivity in PD patients can specifically be accounted for by a higher cost for controlling noise. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Ethanol Exposure History and Alcoholic Reward Differentially Alter Dopamine Release in the Nucleus Accumbens to a Reward-Predictive Cue.

Science.gov (United States)

Fiorenza, Amanda M; Shnitko, Tatiana A; Sullivan, Kaitlin M; Vemuru, Sudheer R; Gomez-A, Alexander; Esaki, Julie Y; Boettiger, Charlotte A; Da Cunha, Claudio; Robinson, Donita L

2018-06-01

Conditioned stimuli (CS) that predict reward delivery acquire the ability to induce phasic dopamine release in the nucleus accumbens (NAc). This dopamine release may facilitate conditioned approach behavior, which often manifests as approach to the site of reward delivery (called "goal-tracking") or to the CS itself (called "sign-tracking"). Previous research has linked sign-tracking in particular to impulsivity and drug self-administration, and addictive drugs may promote the expression of sign-tracking. Ethanol (EtOH) acutely promotes phasic release of dopamine in the accumbens, but it is unknown whether an alcoholic reward alters dopamine release to a CS. We hypothesized that Pavlovian conditioning with an alcoholic reward would increase dopamine release triggered by the CS and subsequent sign-tracking behavior. Moreover, we predicted that chronic intermittent EtOH (CIE) exposure would promote sign-tracking while acute administration of naltrexone (NTX) would reduce it. Rats received 14 doses of EtOH (3 to 5 g/kg, intragastric) or water followed by 6 days of Pavlovian conditioning training. Rewards were a chocolate solution with or without 10% (w/v) alcohol. We used fast-scan cyclic voltammetry to measure phasic dopamine release in the NAc core in response to the CS and the rewards. We also determined the effect of NTX (1 mg/kg, subcutaneous) on conditioned approach. Both CIE and alcoholic reward, individually but not together, associated with greater dopamine to the CS than control conditions. However, this increase in dopamine release was not linked to greater sign-tracking, as both CIE and alcoholic reward shifted conditioned approach from sign-tracking behavior to goal-tracking behavior. However, they both also increased sensitivity to NTX, which reduced goal-tracking behavior. While a history of EtOH exposure or alcoholic reward enhanced dopamine release to a CS, they did not promote sign-tracking under the current conditions. These findings are

Neural evidence for description dependent reward processing in the framing effect.

Science.gov (United States)

Yu, Rongjun; Zhang, Ping

2014-01-01

Human decision making can be influenced by emotionally valenced contexts, known as the framing effect. We used event-related brain potentials to investigate how framing influences the encoding of reward. We found that the feedback related negativity (FRN), which indexes the "worse than expected" negative prediction error in the anterior cingulate cortex (ACC), was more negative for the negative frame than for the positive frame in the win domain. Consistent with previous findings that the FRN is not sensitive to "better than expected" positive prediction error, the FRN did not differentiate the positive and negative frame in the loss domain. Our results provide neural evidence that the description invariance principle which states that reward representation and decision making are not influenced by how options are presented is violated in the framing effect.

Neural evidence for description dependent reward processing in the framing effect

Directory of Open Access Journals (Sweden)

Rongjun eYu

2014-03-01

Full Text Available Human decision making can be influenced by emotionally valenced contexts, known as the framing effect. We used event-related brain potentials to investigate how framing influences the encoding of reward. We found that the feedback related negativity (FRN, which indexes the worse than expected negative prediction error in the anterior cingulate cortex, was more negative for the negative frame than for the positive frame in the win domain. Consistent with previous findings that the FRN is not sensitive to better than expected positive prediction error, the FRN did not differentiate the positive and negative frame in the loss domain. Our results provide neural evidence that the description invariance principle which states that reward representation and decision making are not influenced by how options are presented is violated in the framing effect.

BOLD responses in reward regions to hypothetical and imaginary monetary rewards.

Science.gov (United States)

Miyapuram, Krishna P; Tobler, Philippe N; Gregorios-Pippas, Lucy; Schultz, Wolfram

2012-01-16

Monetary rewards are uniquely human. Because money is easy to quantify and present visually, it is the reward of choice for most fMRI studies, even though it cannot be handed over to participants inside the scanner. A typical fMRI study requires hundreds of trials and thus small amounts of monetary rewards per trial (e.g. 5p) if all trials are to be treated equally. However, small payoffs can have detrimental effects on performance due to their limited buying power. Hypothetical monetary rewards can overcome the limitations of smaller monetary rewards but it is less well known whether predictors of hypothetical rewards activate reward regions. In two experiments, visual stimuli were associated with hypothetical monetary rewards. In Experiment 1, we used stimuli predicting either visually presented or imagined hypothetical monetary rewards, together with non-rewarding control pictures. Activations to reward predictive stimuli occurred in reward regions, namely the medial orbitofrontal cortex and midbrain. In Experiment 2, we parametrically varied the amount of visually presented hypothetical monetary reward keeping constant the amount of actually received reward. Graded activation in midbrain was observed to stimuli predicting increasing hypothetical rewards. The results demonstrate the efficacy of using hypothetical monetary rewards in fMRI studies. Copyright © 2011 Elsevier Inc. All rights reserved.

Distinct prediction errors in mesostriatal circuits of the human brain mediate learning about the values of both states and actions: evidence from high-resolution fMRI.

Science.gov (United States)

Colas, Jaron T; Pauli, Wolfgang M; Larsen, Tobias; Tyszka, J Michael; O'Doherty, John P

2017-10-01

Prediction-error signals consistent with formal models of "reinforcement learning" (RL) have repeatedly been found within dopaminergic nuclei of the midbrain and dopaminoceptive areas of the striatum. However, the precise form of the RL algorithms implemented in the human brain is not yet well determined. Here, we created a novel paradigm optimized to dissociate the subtypes of reward-prediction errors that function as the key computational signatures of two distinct classes of RL models-namely, "actor/critic" models and action-value-learning models (e.g., the Q-learning model). The state-value-prediction error (SVPE), which is independent of actions, is a hallmark of the actor/critic architecture, whereas the action-value-prediction error (AVPE) is the distinguishing feature of action-value-learning algorithms. To test for the presence of these prediction-error signals in the brain, we scanned human participants with a high-resolution functional magnetic-resonance imaging (fMRI) protocol optimized to enable measurement of neural activity in the dopaminergic midbrain as well as the striatal areas to which it projects. In keeping with the actor/critic model, the SVPE signal was detected in the substantia nigra. The SVPE was also clearly present in both the ventral striatum and the dorsal striatum. However, alongside these purely state-value-based computations we also found evidence for AVPE signals throughout the striatum. These high-resolution fMRI findings suggest that model-free aspects of reward learning in humans can be explained algorithmically with RL in terms of an actor/critic mechanism operating in parallel with a system for more direct action-value learning.

Ventromedial Frontal Cortex Is Critical for Guiding Attention to Reward-Predictive Visual Features in Humans.

Science.gov (United States)

Vaidya, Avinash R; Fellows, Lesley K

2015-09-16

Adaptively interacting with our environment requires extracting information that will allow us to successfully predict reward. This can be a challenge, particularly when there are many candidate cues, and when rewards are probabilistic. Recent work has demonstrated that visual attention is allocated to stimulus features that have been associated with reward on previous trials. The ventromedial frontal lobe (VMF) has been implicated in learning in dynamic environments of this kind, but the mechanism by which this region influences this process is not clear. Here, we hypothesized that the VMF plays a critical role in guiding attention to reward-predictive stimulus features based on feedback. We tested the effects of VMF damage in human subjects on a visual search task in which subjects were primed to attend to task-irrelevant colors associated with different levels of reward, incidental to the search task. Consistent with previous work, we found that distractors had a greater influence on reaction time when they appeared in colors associated with high reward in the previous trial compared with colors associated with low reward in healthy control subjects and patients with prefrontal damage sparing the VMF. However, this reward modulation of attentional priming was absent in patients with VMF damage. Thus, an intact VMF is necessary for directing attention based on experience with cue-reward associations. We suggest that this region plays a role in selecting reward-predictive cues to facilitate future learning. There has been a swell of interest recently in the ventromedial frontal cortex (VMF), a brain region critical to associative learning. However, the underlying mechanism by which this region guides learning is not well understood. Here, we tested the effects of damage to this region in humans on a task in which rewards were linked incidentally to visual features, resulting in trial-by-trial attentional priming. Controls and subjects with prefrontal damage

Adaptive scaling of reward in episodic memory: a replication study.

Science.gov (United States)

Mason, Alice; Ludwig, Casimir; Farrell, Simon

2017-11-01

Reward is thought to enhance episodic memory formation via dopaminergic consolidation. Bunzeck, Dayan, Dolan, and Duzel [(2010). A common mechanism for adaptive scaling of reward and novelty. Human Brain Mapping, 31, 1380-1394] provided functional magnetic resonance imaging (fMRI) and behavioural evidence that reward and episodic memory systems are sensitive to the contextual value of a reward-whether it is relatively higher or lower-as opposed to absolute value or prediction error. We carried out a direct replication of their behavioural study and did not replicate their finding that memory performance associated with reward follows this pattern of adaptive scaling. An effect of reward outcome was in the opposite direction to that in the original study, with lower reward outcomes leading to better memory than higher outcomes. There was a marginal effect of reward context, suggesting that expected value affected memory performance. We discuss the robustness of the reward memory relationship to variations in reward context, and whether other reward-related factors have a more reliable influence on episodic memory.

The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study.

Science.gov (United States)

Mensen, Armand; Poryazova, Rositsa; Huegli, Gordana; Baumann, Christian R; Schwartz, Sophie; Khatami, Ramin

2015-01-01

The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson's disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160 ms both patient groups had reduced ERP amplitude compared to controls. Later at 250 ms, both patient groups also showed a clear event-related potential (ERP), which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN) potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson's. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.

The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study.

Directory of Open Access Journals (Sweden)

Armand Mensen

Full Text Available The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson's disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160 ms both patient groups had reduced ERP amplitude compared to controls. Later at 250 ms, both patient groups also showed a clear event-related potential (ERP, which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson's. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.

Cardiac Concomitants of Feedback and Prediction Error Processing in Reinforcement Learning

Science.gov (United States)

Kastner, Lucas; Kube, Jana; Villringer, Arno; Neumann, Jane

2017-01-01

Successful learning hinges on the evaluation of positive and negative feedback. We assessed differential learning from reward and punishment in a monetary reinforcement learning paradigm, together with cardiac concomitants of positive and negative feedback processing. On the behavioral level, learning from reward resulted in more advantageous behavior than learning from punishment, suggesting a differential impact of reward and punishment on successful feedback-based learning. On the autonomic level, learning and feedback processing were closely mirrored by phasic cardiac responses on a trial-by-trial basis: (1) Negative feedback was accompanied by faster and prolonged heart rate deceleration compared to positive feedback. (2) Cardiac responses shifted from feedback presentation at the beginning of learning to stimulus presentation later on. (3) Most importantly, the strength of phasic cardiac responses to the presentation of feedback correlated with the strength of prediction error signals that alert the learner to the necessity for behavioral adaptation. Considering participants' weight status and gender revealed obesity-related deficits in learning to avoid negative consequences and less consistent behavioral adaptation in women compared to men. In sum, our results provide strong new evidence for the notion that during learning phasic cardiac responses reflect an internal value and feedback monitoring system that is sensitive to the violation of performance-based expectations. Moreover, inter-individual differences in weight status and gender may affect both behavioral and autonomic responses in reinforcement-based learning. PMID:29163004

Cardiac Concomitants of Feedback and Prediction Error Processing in Reinforcement Learning

Directory of Open Access Journals (Sweden)

Lucas Kastner

2017-10-01

Full Text Available Successful learning hinges on the evaluation of positive and negative feedback. We assessed differential learning from reward and punishment in a monetary reinforcement learning paradigm, together with cardiac concomitants of positive and negative feedback processing. On the behavioral level, learning from reward resulted in more advantageous behavior than learning from punishment, suggesting a differential impact of reward and punishment on successful feedback-based learning. On the autonomic level, learning and feedback processing were closely mirrored by phasic cardiac responses on a trial-by-trial basis: (1 Negative feedback was accompanied by faster and prolonged heart rate deceleration compared to positive feedback. (2 Cardiac responses shifted from feedback presentation at the beginning of learning to stimulus presentation later on. (3 Most importantly, the strength of phasic cardiac responses to the presentation of feedback correlated with the strength of prediction error signals that alert the learner to the necessity for behavioral adaptation. Considering participants' weight status and gender revealed obesity-related deficits in learning to avoid negative consequences and less consistent behavioral adaptation in women compared to men. In sum, our results provide strong new evidence for the notion that during learning phasic cardiac responses reflect an internal value and feedback monitoring system that is sensitive to the violation of performance-based expectations. Moreover, inter-individual differences in weight status and gender may affect both behavioral and autonomic responses in reinforcement-based learning.

The error in total error reduction.

Science.gov (United States)

Witnauer, James E; Urcelay, Gonzalo P; Miller, Ralph R

2014-02-01

Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered. Copyright © 2013 Elsevier Inc. All rights reserved.

Comparison of Prediction-Error-Modelling Criteria

DEFF Research Database (Denmark)

Jørgensen, John Bagterp; Jørgensen, Sten Bay

2007-01-01

Single and multi-step prediction-error-methods based on the maximum likelihood and least squares criteria are compared. The prediction-error methods studied are based on predictions using the Kalman filter and Kalman predictors for a linear discrete-time stochastic state space model, which is a r...

Neural reward and punishment sensitivity in cigarette smokers.

Science.gov (United States)

Potts, Geoffrey F; Bloom, Erika L; Evans, David E; Drobes, David J

2014-11-01

Nicotine addiction remains a major public health problem but the neural substrates of addictive behavior remain unknown. One characteristic of smoking behavior is impulsive choice, selecting the immediate reward of smoking despite the potential long-term negative consequences. This suggests that drug users, including cigarette smokers, may be more sensitive to rewards and less sensitive to punishment. We used event-related potentials (ERPs) to test the hypothesis that smokers are more responsive to reward signals and less responsive to punishment, potentially predisposing them to risky behavior. We conducted two experiments, one using a reward prediction design to elicit a Medial Frontal Negativity (MFN) and one using a reward- and punishment-motivated flanker task to elicit an Error Related Negativity (ERN), ERP components thought to index activity in the cortical projection of the dopaminergic reward system. The smokers had a greater MFN response to unpredicted rewards, and non-smokers, but not smokers, had a larger ERN on punishment motivated trials indicating that smokers are more reward sensitive and less punishment sensitive than nonsmokers, overestimating the appetitive value and underestimating aversive outcomes of stimuli and actions. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Neural evidence for description dependent reward processing in the framing effect

Science.gov (United States)

Yu, Rongjun; Zhang, Ping

2014-01-01

Human decision making can be influenced by emotionally valenced contexts, known as the framing effect. We used event-related brain potentials to investigate how framing influences the encoding of reward. We found that the feedback related negativity (FRN), which indexes the “worse than expected” negative prediction error in the anterior cingulate cortex (ACC), was more negative for the negative frame than for the positive frame in the win domain. Consistent with previous findings that the FRN is not sensitive to “better than expected” positive prediction error, the FRN did not differentiate the positive and negative frame in the loss domain. Our results provide neural evidence that the description invariance principle which states that reward representation and decision making are not influenced by how options are presented is violated in the framing effect. PMID:24733998

Estimation of genetic connectedness diagnostics based on prediction errors without the prediction error variance-covariance matrix.

Science.gov (United States)

Holmes, John B; Dodds, Ken G; Lee, Michael A

2017-03-02

An important issue in genetic evaluation is the comparability of random effects (breeding values), particularly between pairs of animals in different contemporary groups. This is usually referred to as genetic connectedness. While various measures of connectedness have been proposed in the literature, there is general agreement that the most appropriate measure is some function of the prediction error variance-covariance matrix. However, obtaining the prediction error variance-covariance matrix is computationally demanding for large-scale genetic evaluations. Many alternative statistics have been proposed that avoid the computational cost of obtaining the prediction error variance-covariance matrix, such as counts of genetic links between contemporary groups, gene flow matrices, and functions of the variance-covariance matrix of estimated contemporary group fixed effects. In this paper, we show that a correction to the variance-covariance matrix of estimated contemporary group fixed effects will produce the exact prediction error variance-covariance matrix averaged by contemporary group for univariate models in the presence of single or multiple fixed effects and one random effect. We demonstrate the correction for a series of models and show that approximations to the prediction error matrix based solely on the variance-covariance matrix of estimated contemporary group fixed effects are inappropriate in certain circumstances. Our method allows for the calculation of a connectedness measure based on the prediction error variance-covariance matrix by calculating only the variance-covariance matrix of estimated fixed effects. Since the number of fixed effects in genetic evaluation is usually orders of magnitudes smaller than the number of random effect levels, the computational requirements for our method should be reduced.

Dopamine, reward learning, and active inference.

Science.gov (United States)

FitzGerald, Thomas H B; Dolan, Raymond J; Friston, Karl

2015-01-01

Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions.

Science.gov (United States)

Simen, Patrick; Contreras, David; Buck, Cara; Hu, Peter; Holmes, Philip; Cohen, Jonathan D

2009-12-01

The drift-diffusion model (DDM) implements an optimal decision procedure for stationary, 2-alternative forced-choice tasks. The height of a decision threshold applied to accumulating information on each trial determines a speed-accuracy tradeoff (SAT) for the DDM, thereby accounting for a ubiquitous feature of human performance in speeded response tasks. However, little is known about how participants settle on particular tradeoffs. One possibility is that they select SATs that maximize a subjective rate of reward earned for performance. For the DDM, there exist unique, reward-rate-maximizing values for its threshold and starting point parameters in free-response tasks that reward correct responses (R. Bogacz, E. Brown, J. Moehlis, P. Holmes, & J. D. Cohen, 2006). These optimal values vary as a function of response-stimulus interval, prior stimulus probability, and relative reward magnitude for correct responses. We tested the resulting quantitative predictions regarding response time, accuracy, and response bias under these task manipulations and found that grouped data conformed well to the predictions of an optimally parameterized DDM.

From conflict management to reward-based decision making: actors and critics in primate medial frontal cortex.

Science.gov (United States)

Silvetti, Massimo; Alexander, William; Verguts, Tom; Brown, Joshua W

2014-10-01

The role of the medial prefrontal cortex (mPFC) and especially the anterior cingulate cortex has been the subject of intense debate for the last decade. A number of theories have been proposed to account for its function. Broadly speaking, some emphasize cognitive control, whereas others emphasize value processing; specific theories concern reward processing, conflict detection, error monitoring, and volatility detection, among others. Here we survey and evaluate them relative to experimental results from neurophysiological, anatomical, and cognitive studies. We argue for a new conceptualization of mPFC, arising from recent computational modeling work. Based on reinforcement learning theory, these new models propose that mPFC is an Actor-Critic system. This system is aimed to predict future events including rewards, to evaluate errors in those predictions, and finally, to implement optimal skeletal-motor and visceromotor commands to obtain reward. This framework provides a comprehensive account of mPFC function, accounting for and predicting empirical results across different levels of analysis, including monkey neurophysiology, human ERP, human neuroimaging, and human behavior. Copyright © 2013 Elsevier Ltd. All rights reserved.

A balance of activity in brain control and reward systems predicts self-regulatory outcomes

OpenAIRE

Lopez, Richard B.; Chen, Pin-Hao A.; Huckins, Jeremy F.; Hofmann, Wilhelm; Kelley, William M.; Heatherton, Todd F.

2017-01-01

Abstract Previous neuroimaging work has shown that increased reward-related activity following exposure to food cues is predictive of self-control failure. The balance model suggests that self-regulation failures result from an imbalance in reward and executive control mechanisms. However, an open question is whether the relative balance of activity in brain systems associated with executive control (vs reward) supports self-regulatory outcomes when people encounter tempting cues in daily lif...

Cognitive capacity limitations and Need for Cognition differentially predict reward-induced cognitive effort expenditure.

Science.gov (United States)

Sandra, Dasha A; Otto, A Ross

2018-03-01

While psychological, economic, and neuroscientific accounts of behavior broadly maintain that people minimize expenditure of cognitive effort, empirical work reveals how reward incentives can mobilize increased cognitive effort expenditure. Recent theories posit that the decision to expend effort is governed, in part, by a cost-benefit tradeoff whereby the potential benefits of mental effort can offset the perceived costs of effort exertion. Taking an individual differences approach, the present study examined whether one's executive function capacity, as measured by Stroop interference, predicts the extent to which reward incentives reduce switch costs in a task-switching paradigm, which indexes additional expenditure of cognitive effort. In accordance with the predictions of a cost-benefit account of effort, we found that a low executive function capacity-and, relatedly, a low intrinsic motivation to expend effort (measured by Need for Cognition)-predicted larger increase in cognitive effort expenditure in response to monetary reward incentives, while individuals with greater executive function capacity-and greater intrinsic motivation to expend effort-were less responsive to reward incentives. These findings suggest that an individual's cost-benefit tradeoff is constrained by the perceived costs of exerting cognitive effort. Copyright © 2017 Elsevier B.V. All rights reserved.

Dopamine, reward learning, and active inference

Directory of Open Access Journals (Sweden)

Thomas eFitzgerald

2015-11-01

Full Text Available Temporal difference learning models propose phasic dopamine signalling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behaviour. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

Reward sensitivity predicts ice cream-related attentional bias assessed by inattentional blindness.

Science.gov (United States)

Li, Xiaoming; Tao, Qian; Fang, Ya; Cheng, Chen; Hao, Yangyang; Qi, Jianjun; Li, Yu; Zhang, Wei; Wang, Ying; Zhang, Xiaochu

2015-06-01

The cognitive mechanism underlying the association between individual differences in reward sensitivity and food craving is unknown. The present study explored the mechanism by examining the role of reward sensitivity in attentional bias toward ice cream cues. Forty-nine college students who displayed high level of ice cream craving (HICs) and 46 who displayed low level of ice cream craving (LICs) performed an inattentional blindness (IB) task which was used to assess attentional bias for ice cream. In addition, reward sensitivity and coping style were assessed by the Behavior Inhibition System/Behavior Activation System Scales and Simplified Coping Style Questionnaire. Results showed significant higher identification rate of the critical stimulus in the HICs than LICs, suggesting greater attentional bias for ice cream in the HICs. It was indicated that attentional bias for food cues persisted even under inattentional condition. Furthermore, a significant correlation was found between the attentional bias and reward sensitivity after controlling for coping style, and reward sensitivity predicted attentional bias for food cues. The mediation analyses showed that attentional bias mediated the relationship between reward sensitivity and food craving. Those findings suggest that the association between individual differences in reward sensitivity and food craving may be attributed to attentional bias for food-related cues. Copyright © 2015 Elsevier Ltd. All rights reserved.

Withholding a Reward-driven Action: Studies of the Rise and Fall of Motor Activation and the Effect of Cognitive Depletion.

Science.gov (United States)

Freeman, Scott M; Aron, Adam R

2016-02-01

Controlling an inappropriate response tendency in the face of a reward-predicting stimulus likely depends on the strength of the reward-driven activation, the strength of a putative top-down control process, and their relative timing. We developed a rewarded go/no-go paradigm to investigate such dynamics. Participants made rapid responses (on go trials) to high versus low reward-predicting stimuli and sometimes had to withhold responding (on no-go trials) in the face of the same stimuli. Behaviorally, for high versus low reward stimuli, responses were faster on go trials, and there were more errors of commission on no-go trials. We used single-pulse TMS to map out the corticospinal excitability dynamics, especially on no-go trials where control is needed. For successful no-go trials, there was an early rise in motor activation that was then sharply reduced beneath baseline. This activation-reduction pattern was more pronounced for high- versus low-reward trials and in individuals with greater motivational drive for reward. A follow-on experiment showed that, when participants were fatigued by an effortful task, they made more errors on no-go trials for high versus low reward stimuli. Together, these studies show that, when a response is inappropriate, reward-predicting stimuli induce early motor activation, followed by a top-down effortful control process (which we interpret as response suppression) that depends on the strength of the preceding activation. Our findings provide novel information about the activation-suppression dynamics during control over reward-driven actions, and they illustrate how fatigue or depletion leads to control failures in the face of reward.

A balance of activity in brain control and reward systems predicts self-regulatory outcomes.

Science.gov (United States)

Lopez, Richard B; Chen, Pin-Hao A; Huckins, Jeremy F; Hofmann, Wilhelm; Kelley, William M; Heatherton, Todd F

2017-05-01

Previous neuroimaging work has shown that increased reward-related activity following exposure to food cues is predictive of self-control failure. The balance model suggests that self-regulation failures result from an imbalance in reward and executive control mechanisms. However, an open question is whether the relative balance of activity in brain systems associated with executive control (vs reward) supports self-regulatory outcomes when people encounter tempting cues in daily life. Sixty-nine chronic dieters, a population known for frequent lapses in self-control, completed a food cue-reactivity task during an fMRI scanning session, followed by a weeklong sampling of daily eating behaviors via ecological momentary assessment. We related participants' food cue activity in brain systems associated with executive control and reward to real-world eating patterns. Specifically, a balance score representing the amount of activity in brain regions associated with self-regulatory control, relative to automatic reward-related activity, predicted dieters' control over their eating behavior during the following week. This balance measure may reflect individual self-control capacity and be useful for examining self-regulation success in other domains and populations. © The Author (2017). Published by Oxford University Press.

Effects of monetary reward and punishment on information checking behaviour.

Science.gov (United States)

Li, Simon Y W; Cox, Anna L; Or, Calvin; Blandford, Ann

2016-03-01

Two experiments were conducted to examine whether checking one's own work can be motivated by monetary reward and punishment. Participants were randomly assigned to one of three conditions: a flat-rate payment for completing the task (Control); payment increased for error-free performance (Reward); payment decreased for error performance (Punishment). Experiment 1 (N = 90) was conducted with liberal arts students, using a general data-entry task. Experiment 2 (N = 90) replicated Experiment 1 with clinical students and a safety-critical 'cover story' for the task. In both studies, Reward and Punishment resulted in significantly fewer errors, more frequent and longer checking, than Control. No such differences were obtained between the Reward and Punishment conditions. It is concluded that error consequences in terms of monetary reward and punishment can result in more accurate task performance and more rigorous checking behaviour than errors without consequences. However, whether punishment is more effective than reward, or vice versa, remains inconclusive. Copyright © 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.

Abnormal Striatal BOLD Responses to Reward Anticipation and Reward Delivery in ADHD

Science.gov (United States)

Furukawa, Emi; Bado, Patricia; Tripp, Gail; Mattos, Paulo; Wickens, Jeff R.; Bramati, Ivanei E.; Alsop, Brent; Ferreira, Fernanda Meireles; Lima, Debora; Tovar-Moll, Fernanda; Sergeant, Joseph A.; Moll, Jorge

2014-01-01

Altered reward processing has been proposed to contribute to the symptoms of attention deficit hyperactivity disorder (ADHD). The neurobiological mechanism underlying this alteration remains unclear. We hypothesize that the transfer of dopamine release from reward to reward-predicting cues, as normally observed in animal studies, may be deficient in ADHD. Functional magnetic resonance imaging (fMRI) was used to investigate striatal responses to reward-predicting cues and reward delivery in a classical conditioning paradigm. Data from 14 high-functioning and stimulant-naïve young adults with elevated lifetime symptoms of ADHD (8 males, 6 females) and 15 well-matched controls (8 males, 7 females) were included in the analyses. During reward anticipation, increased blood-oxygen-level-dependent (BOLD) responses in the right ventral and left dorsal striatum were observed in controls, but not in the ADHD group. The opposite pattern was observed in response to reward delivery; the ADHD group demonstrated significantly greater BOLD responses in the ventral striatum bilaterally and the left dorsal striatum relative to controls. In the ADHD group, the number of current hyperactivity/impulsivity symptoms was inversely related to ventral striatal responses during reward anticipation and positively associated with responses to reward. The BOLD response patterns observed in the striatum are consistent with impaired predictive dopamine signaling in ADHD, which may explain altered reward-contingent behaviors and symptoms of ADHD. PMID:24586543

Reward-based training of recurrent neural networks for cognitive and value-based tasks.

Science.gov (United States)

Song, H Francis; Yang, Guangyu R; Wang, Xiao-Jing

2017-01-13

Trained neural network models, which exhibit features of neural activity recorded from behaving animals, may provide insights into the circuit mechanisms of cognitive functions through systematic analysis of network activity and connectivity. However, in contrast to the graded error signals commonly used to train networks through supervised learning, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when optimal behavior depends on an animal's internal judgment of confidence or subjective preferences. Here, we implement reward-based training of recurrent neural networks in which a value network guides learning by using the activity of the decision network to predict future reward. We show that such models capture behavioral and electrophysiological findings from well-known experimental paradigms. Our work provides a unified framework for investigating diverse cognitive and value-based computations, and predicts a role for value representation that is essential for learning, but not executing, a task.

Ventral striatum activation to prosocial rewards predicts longitudinal declines in adolescent risk taking.

Science.gov (United States)

Telzer, Eva H; Fuligni, Andrew J; Lieberman, Matthew D; Galván, Adriana

2013-01-01

Adolescence is a period of intensified emotions and an increase in motivated behaviors and passions. Evidence from developmental neuroscience suggests that this heightened emotionality occurs, in part, due to a peak in functional reactivity to rewarding stimuli, which renders adolescents more oriented toward reward-seeking behaviors. Most prior work has focused on how reward sensitivity may create vulnerabilities, leading to increases in risk taking. Here, we test whether heightened reward sensitivity may potentially be an asset for adolescents when engaged in prosocial activities. Thirty-two adolescents were followed over a one-year period to examine whether ventral striatum activation to prosocial rewards predicts decreases in risk taking over a year. Results show that heightened ventral striatum activation to prosocial stimuli relates to longitudinal declines in risk taking. Therefore, the very same neural region that has conferred vulnerability for adolescent risk taking may also be protective against risk taking. Copyright © 2012 Elsevier Ltd. All rights reserved.

Modeling the violation of reward maximization and invariance in reinforcement schedules.

Directory of Open Access Journals (Sweden)

Giancarlo La Camera

2008-08-01

Full Text Available It is often assumed that animals and people adjust their behavior to maximize reward acquisition. In visually cued reinforcement schedules, monkeys make errors in trials that are not immediately rewarded, despite having to repeat error trials. Here we show that error rates are typically smaller in trials equally distant from reward but belonging to longer schedules (referred to as "schedule length effect". This violates the principles of reward maximization and invariance and cannot be predicted by the standard methods of Reinforcement Learning, such as the method of temporal differences. We develop a heuristic model that accounts for all of the properties of the behavior in the reinforcement schedule task but whose predictions are not different from those of the standard temporal difference model in choice tasks. In the modification of temporal difference learning introduced here, the effect of schedule length emerges spontaneously from the sensitivity to the immediately preceding trial. We also introduce a policy for general Markov Decision Processes, where the decision made at each node is conditioned on the motivation to perform an instrumental action, and show that the application of our model to the reinforcement schedule task and the choice task are special cases of this general theoretical framework. Within this framework, Reinforcement Learning can approach contextual learning with the mixture of empirical findings and principled assumptions that seem to coexist in the best descriptions of animal behavior. As examples, we discuss two phenomena observed in humans that often derive from the violation of the principle of invariance: "framing," wherein equivalent options are treated differently depending on the context in which they are presented, and the "sunk cost" effect, the greater tendency to continue an endeavor once an investment in money, effort, or time has been made. The schedule length effect might be a manifestation of these

Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex.

Science.gov (United States)

Klein-Flügge, Miriam Cornelia; Barron, Helen Catharine; Brodersen, Kay Henning; Dolan, Raymond J; Behrens, Timothy Edward John

2013-02-13

A dominant focus in studies of learning and decision-making is the neural coding of scalar reward value. This emphasis ignores the fact that choices are strongly shaped by a rich representation of potential rewards. Here, using fMRI adaptation, we demonstrate that responses in the human orbitofrontal cortex (OFC) encode a representation of the specific type of food reward predicted by a visual cue. By controlling for value across rewards and by linking each reward with two distinct stimuli, we could test for representations of reward-identity that were independent of associative information. Our results show reward-identity representations in a medial-caudal region of OFC, independent of the associated predictive stimulus. This contrasts with a more rostro-lateral OFC region encoding reward-identity representations tied to the predicate stimulus. This demonstration of adaptation in OFC to reward specific representations opens an avenue for investigation of more complex decision mechanisms that are not immediately accessible in standard analyses, which focus on correlates of average activity.

Goal or gold: overlapping reward processes in soccer players upon scoring and winning money.

Directory of Open Access Journals (Sweden)

Alexander Niklas Häusler

Full Text Available Social rewards are important incentives for human behavior. This is especially true in team sports such as the most popular one worldwide: soccer. We investigated reward processing upon scoring a soccer goal in a standard two-versus-one situation and in comparison to winning in a monetary incentive task. The results show a strong overlap in brain activity between the two conditions in established reward regions of the mesolimbic dopaminergic system, including the ventral striatum and ventromedial pre-frontal cortex. The three main components of reward-associated learning, i.e., reward probability (RP, reward reception (RR and reward prediction errors (RPE showed highly similar activation in both con-texts, with only the RR and RPE components displaying overlapping reward activity. Passing and shooting behavior did not correlate with individual egoism scores, but we observe a positive correlation be-tween egoism and activity in the left middle frontal gyrus upon scoring after a pass versus a direct shot. Our findings suggest that rewards in the context of soccer and monetary incentives are based on similar neural processes.

Goal or Gold: Overlapping Reward Processes in Soccer Players upon Scoring and Winning Money

Science.gov (United States)

Häusler, Alexander Niklas; Becker, Benjamin; Bartling, Marcel; Weber, Bernd

2015-01-01

Social rewards are important incentives for human behavior. This is especially true in team sports such as the most popular one worldwide: soccer. We investigated reward processing upon scoring a soccer goal in a standard two-versus-one situation and in comparison to winning in a monetary incentive task. The results show a strong overlap in brain activity between the two conditions in established reward regions of the mesolimbic dopaminergic system, including the ventral striatum and ventromedial pre-frontal cortex. The three main components of reward-associated learning i.e. reward probability (RP), reward reception (RR) and reward prediction errors (RPE) showed highly similar activation in both con-texts, with only the RR and RPE components displaying overlapping reward activity. Passing and shooting behavior did not correlate with individual egoism scores, but we observe a positive correlation be-tween egoism and activity in the left middle frontal gyrus upon scoring after a pass versus a direct shot. Our findings suggest that rewards in the context of soccer and monetary incentives are based on similar neural processes. PMID:25875594

Prediction error, ketamine and psychosis: An updated model.

Science.gov (United States)

Corlett, Philip R; Honey, Garry D; Fletcher, Paul C

2016-11-01

In 2007, we proposed an explanation of delusion formation as aberrant prediction error-driven associative learning. Further, we argued that the NMDA receptor antagonist ketamine provided a good model for this process. Subsequently, we validated the model in patients with psychosis, relating aberrant prediction error signals to delusion severity. During the ensuing period, we have developed these ideas, drawing on the simple principle that brains build a model of the world and refine it by minimising prediction errors, as well as using it to guide perceptual inferences. While previously we focused on the prediction error signal per se, an updated view takes into account its precision, as well as the precision of prior expectations. With this expanded perspective, we see several possible routes to psychotic symptoms - which may explain the heterogeneity of psychotic illness, as well as the fact that other drugs, with different pharmacological actions, can produce psychotomimetic effects. In this article, we review the basic principles of this model and highlight specific ways in which prediction errors can be perturbed, in particular considering the reliability and uncertainty of predictions. The expanded model explains hallucinations as perturbations of the uncertainty mediated balance between expectation and prediction error. Here, expectations dominate and create perceptions by suppressing or ignoring actual inputs. Negative symptoms may arise due to poor reliability of predictions in service of action. By mapping from biology to belief and perception, the account proffers new explanations of psychosis. However, challenges remain. We attempt to address some of these concerns and suggest future directions, incorporating other symptoms into the model, building towards better understanding of psychosis. © The Author(s) 2016.

The role of the dorsal raphé nucleus in reward-seeking behavior

Directory of Open Access Journals (Sweden)

Kae eNakamura

2013-08-01

Full Text Available Pharmacological experiments have shown that the modulation of brain serotonin levels has a strong impact on value-based decision making. Anatomical and physiological evidence also revealed that the dorsal raphé nucleus (DRN, a major source of serotonin, and the dopamine system receive common inputs from brain regions associated with appetitive and aversive information processing. The serotonin and dopamine systems also have reciprocal functional influences on each other. However, the specific mechanism by which serotonin affects value-based decision making is not clear.To understand the information carried by the DRN for reward-seeking behavior, we measured single neuron activity in the primate DRN during the performance of saccade tasks to obtain different amounts of a reward. We found that DRN neuronal activity was characterized by tonic modulation that was altered by the expected and received reward value. Consistent reward-dependent modulation across different task periods suggested that DRN activity kept track of the reward value throughout a trial. The DRN was also characterized by modulation of its activity in the opposite direction by different neuronal subgroups, one firing strongly for the prediction and receipt of large rewards, with the other firing strongly for small rewards. Conversely, putative dopamine neurons showed positive phasic responses to reward-indicating cues and the receipt of an unexpected reward amount, which supports the reward prediction error signal hypothesis of dopamine.I suggest that the tonic reward monitoring signal of the DRN, possibly together with its interaction with the dopamine system, reports a continuous level of motivation throughout the performance of a task. Such a signal may provide reward context information to the targets of DRN projections, where it may be integrated further with incoming motivationally salient information.

Neural correlates of RDoC reward constructs in adolescents with diverse psychiatric symptoms: A Reward Flanker Task pilot study.

Science.gov (United States)

Bradley, Kailyn A L; Case, Julia A C; Freed, Rachel D; Stern, Emily R; Gabbay, Vilma

2017-07-01

There has been growing interest under the Research Domain Criteria initiative to investigate behavioral constructs and their underlying neural circuitry. Abnormalities in reward processes are salient across psychiatric conditions and may precede future psychopathology in youth. However, the neural circuitry underlying such deficits has not been well defined. Therefore, in this pilot, we studied youth with diverse psychiatric symptoms and examined the neural underpinnings of reward anticipation, attainment, and positive prediction error (PPE, unexpected reward gain). Clinically, we focused on anhedonia, known to reflect deficits in reward function. Twenty-two psychotropic medication-free youth, 16 with psychiatric symptoms, exhibiting a full range of anhedonia, were scanned during the Reward Flanker Task. Anhedonia severity was quantified using the Snaith-Hamilton Pleasure Scale. Functional magnetic resonance imaging analyses were false discovery rate corrected for multiple comparisons. Anticipation activated a broad network, including the medial frontal cortex and ventral striatum, while attainment activated memory and emotion-related regions such as the hippocampus and parahippocampal gyrus, but not the ventral striatum. PPE activated a right-dominant fronto-temporo-parietal network. Anhedonia was only correlated with activation of the right angular gyrus during anticipation and the left precuneus during PPE at an uncorrected threshold. Findings are preliminary due to the small sample size. This pilot characterized the neural circuitry underlying different aspects of reward processing in youth with diverse psychiatric symptoms. These results highlight the complexity of the neural circuitry underlying reward anticipation, attainment, and PPE. Furthermore, this study underscores the importance of RDoC research in youth. Copyright © 2016 Elsevier B.V. All rights reserved.

Differential Dopamine Release Dynamics in the Nucleus Accumbens Core and Shell Reveal Complementary Signals for Error Prediction and Incentive Motivation.

Science.gov (United States)

Saddoris, Michael P; Cacciapaglia, Fabio; Wightman, R Mark; Carelli, Regina M

2015-08-19

Mesolimbic dopamine (DA) is phasically released during appetitive behaviors, though there is substantive disagreement about the specific purpose of these DA signals. For example, prediction error (PE) models suggest a role of learning, while incentive salience (IS) models argue that the DA signal imbues stimuli with value and thereby stimulates motivated behavior. However, within the nucleus accumbens (NAc) patterns of DA release can strikingly differ between subregions, and as such, it is possible that these patterns differentially contribute to aspects of PE and IS. To assess this, we measured DA release in subregions of the NAc during a behavioral task that spatiotemporally separated sequential goal-directed stimuli. Electrochemical methods were used to measure subsecond NAc dopamine release in the core and shell during a well learned instrumental chain schedule in which rats were trained to press one lever (seeking; SL) to gain access to a second lever (taking; TL) linked with food delivery, and again during extinction. In the core, phasic DA release was greatest following initial SL presentation, but minimal for the subsequent TL and reward events. In contrast, phasic shell DA showed robust release at all task events. Signaling decreased between the beginning and end of sessions in the shell, but not core. During extinction, peak DA release in the core showed a graded decrease for the SL and pauses in release during omitted expected rewards, whereas shell DA release decreased predominantly during the TL. These release dynamics suggest parallel DA signals capable of supporting distinct theories of appetitive behavior. Dopamine signaling in the brain is important for a variety of cognitive functions, such as learning and motivation. Typically, it is assumed that a single dopamine signal is sufficient to support these cognitive functions, though competing theories disagree on how dopamine contributes to reward-based behaviors. Here, we have found that real

Notes on human error analysis and prediction

International Nuclear Information System (INIS)

Rasmussen, J.

1978-11-01

The notes comprise an introductory discussion of the role of human error analysis and prediction in industrial risk analysis. Following this introduction, different classes of human errors and role in industrial systems are mentioned. Problems related to the prediction of human behaviour in reliability and safety analysis are formulated and ''criteria for analyzability'' which must be met by industrial systems so that a systematic analysis can be performed are suggested. The appendices contain illustrative case stories and a review of human error reports for the task of equipment calibration and testing as found in the US Licensee Event Reports. (author)

Error-related anterior cingulate cortex activity and the prediction of conscious error awareness

Directory of Open Access Journals (Sweden)

Catherine eOrr

2012-06-01

Full Text Available Research examining the neural mechanisms associated with error awareness has consistently identified dorsal anterior cingulate activity (ACC as necessary but not predictive of conscious error detection. Two recent studies (Steinhauser and Yeung, 2010; Wessel et al. 2011 have found a contrary pattern of greater dorsal ACC activity (in the form of the error-related negativity during detected errors, but suggested that the greater activity may instead reflect task influences (e.g., response conflict, error probability and or individual variability (e.g., statistical power. We re-analyzed fMRI BOLD data from 56 healthy participants who had previously been administered the Error Awareness Task, a motor Go/No-go response inhibition task in which subjects make errors of commission of which they are aware (Aware errors, or unaware (Unaware errors. Consistent with previous data, the activity in a number of cortical regions was predictive of error awareness, including bilateral inferior parietal and insula cortices, however in contrast to previous studies, including our own smaller sample studies using the same task, error-related dorsal ACC activity was significantly greater during aware errors when compared to unaware errors. While the significantly faster RT for aware errors (compared to unaware was consistent with the hypothesis of higher response conflict increasing ACC activity, we could find no relationship between dorsal ACC activity and the error RT difference. The data suggests that individual variability in error awareness is associated with error-related dorsal ACC activity, and therefore this region may be important to conscious error detection, but it remains unclear what task and individual factors influence error awareness.

Comparing the neural basis of monetary reward and cognitive feedback during information-integration category learning.

Science.gov (United States)

Daniel, Reka; Pollmann, Stefan

2010-01-06

The dopaminergic system is known to play a central role in reward-based learning (Schultz, 2006), yet it was also observed to be involved when only cognitive feedback is given (Aron et al., 2004). Within the domain of information-integration category learning, in which information from several stimulus dimensions has to be integrated predecisionally (Ashby and Maddox, 2005), the importance of contingent feedback is well established (Maddox et al., 2003). We examined the common neural correlates of reward anticipation and prediction error in this task. Sixteen subjects performed two parallel information-integration tasks within a single event-related functional magnetic resonance imaging session but received a monetary reward only for one of them. Similar functional areas including basal ganglia structures were activated in both task versions. In contrast, a single structure, the nucleus accumbens, showed higher activation during monetary reward anticipation compared with the anticipation of cognitive feedback in information-integration learning. Additionally, this activation was predicted by measures of intrinsic motivation in the cognitive feedback task and by measures of extrinsic motivation in the rewarded task. Our results indicate that, although all other structures implicated in category learning are not significantly affected by altering the type of reward, the nucleus accumbens responds to the positive incentive properties of an expected reward depending on the specific type of the reward.

Reward inference by primate prefrontal and striatal neurons.

Science.gov (United States)

Pan, Xiaochuan; Fan, Hongwei; Sawa, Kosuke; Tsuda, Ichiro; Tsukada, Minoru; Sakagami, Masamichi

2014-01-22

The brain contains multiple yet distinct systems involved in reward prediction. To understand the nature of these processes, we recorded single-unit activity from the lateral prefrontal cortex (LPFC) and the striatum in monkeys performing a reward inference task using an asymmetric reward schedule. We found that neurons both in the LPFC and in the striatum predicted reward values for stimuli that had been previously well experienced with set reward quantities in the asymmetric reward task. Importantly, these LPFC neurons could predict the reward value of a stimulus using transitive inference even when the monkeys had not yet learned the stimulus-reward association directly; whereas these striatal neurons did not show such an ability. Nevertheless, because there were two set amounts of reward (large and small), the selected striatal neurons were able to exclusively infer the reward value (e.g., large) of one novel stimulus from a pair after directly experiencing the alternative stimulus with the other reward value (e.g., small). Our results suggest that although neurons that predict reward value for old stimuli in the LPFC could also do so for new stimuli via transitive inference, those in the striatum could only predict reward for new stimuli via exclusive inference. Moreover, the striatum showed more complex functions than was surmised previously for model-free learning.

Electrophysiological Evidence of Atypical Motivation and Reward Processing in Children with Attention-Deficit Hyperactivity Disorder

Science.gov (United States)

Holroyd, Clay B.; Baker, Travis E.; Kerns, Kimberly A.; Muller, Ulrich

2008-01-01

Behavioral and neurophysiological evidence suggest that attention-deficit hyperactivity disorder (ADHD) is characterized by the impact of abnormal reward prediction error signals carried by the midbrain dopamine system on frontal brain areas that implement cognitive control. To investigate this issue, we recorded the event-related brain potential…

Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making.

Science.gov (United States)

Schönberg, Tom; Daw, Nathaniel D; Joel, Daphna; O'Doherty, John P

2007-11-21

The computational framework of reinforcement learning has been used to forward our understanding of the neural mechanisms underlying reward learning and decision-making behavior. It is known that humans vary widely in their performance in decision-making tasks. Here, we used a simple four-armed bandit task in which subjects are almost evenly split into two groups on the basis of their performance: those who do learn to favor choice of the optimal action and those who do not. Using models of reinforcement learning we sought to determine the neural basis of these intrinsic differences in performance by scanning both groups with functional magnetic resonance imaging. We scanned 29 subjects while they performed the reward-based decision-making task. Our results suggest that these two groups differ markedly in the degree to which reinforcement learning signals in the striatum are engaged during task performance. While the learners showed robust prediction error signals in both the ventral and dorsal striatum during learning, the nonlearner group showed a marked absence of such signals. Moreover, the magnitude of prediction error signals in a region of dorsal striatum correlated significantly with a measure of behavioral performance across all subjects. These findings support a crucial role of prediction error signals, likely originating from dopaminergic midbrain neurons, in enabling learning of action selection preferences on the basis of obtained rewards. Thus, spontaneously observed individual differences in decision making performance demonstrate the suggested dependence of this type of learning on the functional integrity of the dopaminergic striatal system in humans.

A Sensor Dynamic Measurement Error Prediction Model Based on NAPSO-SVM.

Science.gov (United States)

Jiang, Minlan; Jiang, Lan; Jiang, Dingde; Li, Fei; Song, Houbing

2018-01-15

Dynamic measurement error correction is an effective way to improve sensor precision. Dynamic measurement error prediction is an important part of error correction, and support vector machine (SVM) is often used for predicting the dynamic measurement errors of sensors. Traditionally, the SVM parameters were always set manually, which cannot ensure the model's performance. In this paper, a SVM method based on an improved particle swarm optimization (NAPSO) is proposed to predict the dynamic measurement errors of sensors. Natural selection and simulated annealing are added in the PSO to raise the ability to avoid local optima. To verify the performance of NAPSO-SVM, three types of algorithms are selected to optimize the SVM's parameters: the particle swarm optimization algorithm (PSO), the improved PSO optimization algorithm (NAPSO), and the glowworm swarm optimization (GSO). The dynamic measurement error data of two sensors are applied as the test data. The root mean squared error and mean absolute percentage error are employed to evaluate the prediction models' performances. The experimental results show that among the three tested algorithms the NAPSO-SVM method has a better prediction precision and a less prediction errors, and it is an effective method for predicting the dynamic measurement errors of sensors.

Reward guides vision when it's your thing: trait reward-seeking in reward-mediated visual priming.

Directory of Open Access Journals (Sweden)

Clayton Hickey

Full Text Available Reward-related mesolimbic dopamine is thought to play an important role in guiding animal behaviour, biasing approach towards potentially beneficial environmental stimuli and away from objects unlikely to garner positive outcome. This is considered to result in part from an impact on perceptual and attentional processes: dopamine initiates a series of cognitive events that result in the priming of reward-associated perceptual features. We have provided behavioural and electrophysiological evidence that this mechanism guides human vision in search, an effect we refer to as reward priming. We have also demonstrated that there is substantial individual variability in this effect. Here we show that behavioural differences in reward priming are predicted remarkably well by a personality index that captures the degree to which a person's behaviour is driven by reward outcome. Participants with reward-seeking personalities are found to be those who allocate visual resources to objects characterized by reward-associated visual features. These results add to a rapidly developing literature demonstrating the crucial role reward plays in attentional control. They additionally illustrate the striking impact personality traits can have on low-level cognitive processes like perception and selective attention.

Intersection of reward and memory in monkey rhinal cortex.

Science.gov (United States)

Clark, Andrew M; Bouret, Sebastien; Young, Adrienne M; Richmond, Barry J

2012-05-16

In humans and other animals, the vigor with which a reward is pursued depends on its desirability, that is, on the reward's predicted value. Predicted value is generally context-dependent, varying according to the value of rewards obtained in the recent and distant past. Signals related to reward prediction and valuation are believed to be encoded in a circuit centered around midbrain dopamine neurons and their targets in the prefrontal cortex and basal ganglia. Notably absent from this hypothesized reward pathway are dopaminergic targets in the medial temporal lobe. Here we show that a key part of the medial temporal lobe memory system previously reported to be important for sensory mnemonic and perceptual processing, the rhinal cortex (Rh), is required for using memories of previous reward values to predict the value of forthcoming rewards. We tested monkeys with bilateral Rh lesions on a task in which reward size varied across blocks of uncued trials. In this experiment, the only cues for predicting current reward value are the sizes of rewards delivered in previous blocks. Unexpectedly, monkeys with Rh ablations, but not intact controls, were insensitive to differences in predicted reward, responding as if they expected all rewards to be of equal magnitude. Thus, it appears that Rh is critical for using memory of previous rewards to predict the value of forthcoming rewards. These results are in agreement with accumulating evidence that Rh is critical for establishing the relationships between temporally interleaved events, which is a key element of episodic memory.

Human medial frontal cortex activity predicts learning from errors.

Science.gov (United States)

Hester, Robert; Barre, Natalie; Murphy, Kevin; Silk, Tim J; Mattingley, Jason B

2008-08-01

Learning from errors is a critical feature of human cognition. It underlies our ability to adapt to changing environmental demands and to tune behavior for optimal performance. The posterior medial frontal cortex (pMFC) has been implicated in the evaluation of errors to control behavior, although it has not previously been shown that activity in this region predicts learning from errors. Using functional magnetic resonance imaging, we examined activity in the pMFC during an associative learning task in which participants had to recall the spatial locations of 2-digit targets and were provided with immediate feedback regarding accuracy. Activity within the pMFC was significantly greater for errors that were subsequently corrected than for errors that were repeated. Moreover, pMFC activity during recall errors predicted future responses (correct vs. incorrect), despite a sizeable interval (on average 70 s) between an error and the next presentation of the same recall probe. Activity within the hippocampus also predicted future performance and correlated with error-feedback-related pMFC activity. A relationship between performance expectations and pMFC activity, in the absence of differing reinforcement value for errors, is consistent with the idea that error-related pMFC activity reflects the extent to which an outcome is "worse than expected."

Probability differently modulating the effects of reward and punishment on visuomotor adaptation.

Science.gov (United States)

Song, Yanlong; Smiley-Oyen, Ann L

2017-12-01

Recent human motor learning studies revealed that punishment seemingly accelerated motor learning but reward enhanced consolidation of motor memory. It is not evident how intrinsic properties of reward and punishment modulate the potentially dissociable effects of reward and punishment on motor learning and motor memory. It is also not clear what causes the dissociation of the effects of reward and punishment. By manipulating probability of distribution, a critical property of reward and punishment, the present study demonstrated that probability had distinct modulation on the effects of reward and punishment in adapting to a sudden visual rotation and consolidation of the adaptation memory. Specifically, two probabilities of monetary reward and punishment distribution, 50 and 100%, were applied during young adult participants adapting to a sudden visual rotation. Punishment and reward showed distinct effects on motor adaptation and motor memory. The group that received punishments in 100% of the adaptation trials adapted significantly faster than the other three groups, but the group that received rewards in 100% of the adaptation trials showed marked savings in re-adapting to the same rotation. In addition, the group that received punishments in 50% of the adaptation trials that were randomly selected also had savings in re-adapting to the same rotation. Sensitivity to sensory prediction error or difference in explicit process induced by reward and punishment may likely contribute to the distinct effects of reward and punishment.

Error analysis in predictive modelling demonstrated on mould data.

Science.gov (United States)

Baranyi, József; Csernus, Olívia; Beczner, Judit

2014-01-17

The purpose of this paper was to develop a predictive model for the effect of temperature and water activity on the growth rate of Aspergillus niger and to determine the sources of the error when the model is used for prediction. Parallel mould growth curves, derived from the same spore batch, were generated and fitted to determine their growth rate. The variances of replicate ln(growth-rate) estimates were used to quantify the experimental variability, inherent to the method of determining the growth rate. The environmental variability was quantified by the variance of the respective means of replicates. The idea is analogous to the "within group" and "between groups" variability concepts of ANOVA procedures. A (secondary) model, with temperature and water activity as explanatory variables, was fitted to the natural logarithm of the growth rates determined by the primary model. The model error and the experimental and environmental errors were ranked according to their contribution to the total error of prediction. Our method can readily be applied to analysing the error structure of predictive models of bacterial growth models, too. © 2013.

Reward Inference by Primate Prefrontal and Striatal Neurons

OpenAIRE

Pan, Xiaochuan; Fan, Hongwei; Sawa, Kosuke; Tsuda, Ichiro; Tsukada, Minoru; Sakagami, Masamichi

2014-01-01

The brain contains multiple yet distinct systems involved in reward prediction. To understand the nature of these processes, we recorded single-unit activity from the lateral prefrontal cortex (LPFC) and the striatum in monkeys performing a reward inference task using an asymmetric reward schedule. We found that neurons both in the LPFC and in the striatum predicted reward values for stimuli that had been previously well experienced with set reward quantities in the asymmetric reward task. Im...

Estimating Model Prediction Error: Should You Treat Predictions as Fixed or Random?

Science.gov (United States)

Wallach, Daniel; Thorburn, Peter; Asseng, Senthold; Challinor, Andrew J.; Ewert, Frank; Jones, James W.; Rotter, Reimund; Ruane, Alexander

2016-01-01

Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEP fixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEP uncertain( X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEP uncertain (X) can be estimated using a random effects ANOVA. It is argued that MSEP uncertain (X) is the more informative uncertainty criterion, because it is specific to each prediction situation.

Attentional Bias Predicts Increased Reward Salience and Risk Taking in Bipolar Disorder.

Science.gov (United States)

Mason, Liam; Trujillo-Barreto, Nelson J; Bentall, Richard P; El-Deredy, Wael

2016-02-15

There is amassing evidence that risky decision-making in bipolar disorder is related to reward-based differences in frontostriatal regions. However, the roles of early attentional and later cognitive processes remain unclear, limiting theoretical understanding and development of targeted interventions. Twenty euthymic bipolar disorder and 19 matched control participants played a Roulette task in which they won and lost money. Event-related potentials and source analysis were used to quantify predominantly sensory-attentional (N1), motivational salience (feedback-related negativities [FRN]), and cognitive appraisal (P300) stages of processing. We predicted that the bipolar disorder group would show increased N1, consistent with increased attentional orienting, and reduced FRN, consistent with a bias to perceive outcomes more favorably. As predicted, the bipolar disorder group showed increased N1 and reduced FRN but no differences in P300. N1 amplitude was additionally associated with real-life risk taking, and N1 source activity was reduced in visual cortex but increased activity in precuneus, frontopolar, and premotor cortex, compared to those of controls. These findings demonstrate an early attentional bias to reward that potentially drives risk taking by priming approach behavior and elevating reward salience in the frontostriatal pathway. Although later cognitive appraisals of these inputs may be relatively intact in remission, interventions targeting attention orienting may also be effective in long-term reduction of relapse. Copyright © 2016 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

Reward salience and risk aversion underlie differential ACC activity in substance dependence.

Science.gov (United States)

Alexander, William H; Fukunaga, Rena; Finn, Peter; Brown, Joshua W

2015-01-01

The medial prefrontal cortex, especially the dorsal anterior cingulate cortex (ACC), has long been implicated in cognitive control and error processing. Although the association between ACC and behavior has been established, it is less clear how ACC contributes to dysfunctional behavior such as substance dependence. Evidence from neuroimaging studies investigating ACC function in substance users is mixed, with some studies showing disengagement of ACC in substance dependent individuals (SDs), while others show increased ACC activity related to substance use. In this study, we investigate ACC function in SDs and healthy individuals performing a change signal task for monetary rewards. Using a priori predictions derived from a recent computational model of ACC, we find that ACC activity differs between SDs and controls in factors related to reward salience and risk aversion between SDs and healthy individuals. Quantitative fits of a computational model to fMRI data reveal significant differences in best fit parameters for reward salience and risk preferences. Specifically, the ACC in SDs shows greater risk aversion, defined as concavity in the utility function, and greater attention to rewards relative to reward omission. Furthermore, across participants risk aversion and reward salience are positively correlated. The results clarify the role that ACC plays in both the reduced sensitivity to omitted rewards and greater reward valuation in SDs. Clinical implications of applying computational modeling in psychiatry are also discussed.

Cortical Brain Activity Reflecting Attentional Biasing Toward Reward-Predicting Cues Covaries with Economic Decision-Making Performance.

Science.gov (United States)

San Martín, René; Appelbaum, Lawrence G; Huettel, Scott A; Woldorff, Marty G

2016-01-01

Adaptive choice behavior depends critically on identifying and learning from outcome-predicting cues. We hypothesized that attention may be preferentially directed toward certain outcome-predicting cues. We studied this possibility by analyzing event-related potential (ERP) responses in humans during a probabilistic decision-making task. Participants viewed pairs of outcome-predicting visual cues and then chose to wager either a small (i.e., loss-minimizing) or large (i.e., gain-maximizing) amount of money. The cues were bilaterally presented, which allowed us to extract the relative neural responses to each cue by using a contralateral-versus-ipsilateral ERP contrast. We found an early lateralized ERP response, whose features matched the attention-shift-related N2pc component and whose amplitude scaled with the learned reward-predicting value of the cues as predicted by an attention-for-reward model. Consistently, we found a double dissociation involving the N2pc. Across participants, gain-maximization positively correlated with the N2pc amplitude to the most reliable gain-predicting cue, suggesting an attentional bias toward such cues. Conversely, loss-minimization was negatively correlated with the N2pc amplitude to the most reliable loss-predicting cue, suggesting an attentional avoidance toward such stimuli. These results indicate that learned stimulus-reward associations can influence rapid attention allocation, and that differences in this process are associated with individual differences in economic decision-making performance. © The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

Self-stimulating rats combine subjective reward magnitude and subjective reward rate multiplicatively.

Science.gov (United States)

Leon, M I; Gallistel, C R

1998-07-01

For rats that bar pressed for intracranial electrical stimulation in a 2-lever matching paradigm with concurrent variable interval schedules of reward, the authors found that the time allocation ratio is based on a multiplicative combination of the ratio of subjective reward magnitudes and the ratio of the rates of reward. Multiplicative combining was observed in a range covering approximately 2 orders of magnitude in the ratio of the rates of reward from about 1:10 to 10:1) and an order of magnitude change in the size of rewards. After determining the relation between the pulse frequency of stimulation and subjective reward magnitude, the authors were able to predict from knowledge of the subjective magnitudes of the rewards and the obtained relative rates of reward the subject's time allocation ratio over a range in which it varied by more than 3 orders of magnitude.

Error sensitivity analysis in 10-30-day extended range forecasting by using a nonlinear cross-prediction error model

Science.gov (United States)

Xia, Zhiye; Xu, Lisheng; Chen, Hongbin; Wang, Yongqian; Liu, Jinbao; Feng, Wenlan

2017-06-01

Extended range forecasting of 10-30 days, which lies between medium-term and climate prediction in terms of timescale, plays a significant role in decision-making processes for the prevention and mitigation of disastrous meteorological events. The sensitivity of initial error, model parameter error, and random error in a nonlinear crossprediction error (NCPE) model, and their stability in the prediction validity period in 10-30-day extended range forecasting, are analyzed quantitatively. The associated sensitivity of precipitable water, temperature, and geopotential height during cases of heavy rain and hurricane is also discussed. The results are summarized as follows. First, the initial error and random error interact. When the ratio of random error to initial error is small (10-6-10-2), minor variation in random error cannot significantly change the dynamic features of a chaotic system, and therefore random error has minimal effect on the prediction. When the ratio is in the range of 10-1-2 (i.e., random error dominates), attention should be paid to the random error instead of only the initial error. When the ratio is around 10-2-10-1, both influences must be considered. Their mutual effects may bring considerable uncertainty to extended range forecasting, and de-noising is therefore necessary. Second, in terms of model parameter error, the embedding dimension m should be determined by the factual nonlinear time series. The dynamic features of a chaotic system cannot be depicted because of the incomplete structure of the attractor when m is small. When m is large, prediction indicators can vanish because of the scarcity of phase points in phase space. A method for overcoming the cut-off effect ( m > 4) is proposed. Third, for heavy rains, precipitable water is more sensitive to the prediction validity period than temperature or geopotential height; however, for hurricanes, geopotential height is most sensitive, followed by precipitable water.

Reward and behavioral factors contributing to the tonic activity of monkey pedunculopontine tegmental nucleus neurons during saccade tasks

Directory of Open Access Journals (Sweden)

Ken-ichi Okada

2016-11-01

Full Text Available The pedunculopontine tegmental nucleus (PPTg in the brainstem plays a role in controlling reinforcement learning and executing conditioned behavior. We previously examined activity of PPTg neurons in monkeys during a reward-conditioned, visually guided saccade task, and reported that a population of these neurons exhibited tonic responses throughout the task period. These tonic responses might depend on prediction of the upcoming reward, successful execution of the task, or both. Here, we sought to further distinguish these factors and to investigate how each contributes to the tonic neuronal activity of the PPTg. In our normal visually guided saccade task, the monkey initially fixated on the central fixation target, then made saccades to the peripheral saccade target, and received a juice reward after the saccade target disappeared. Most of the tonic activity terminated shortly after the reward delivery, when the monkey broke fixation. To distinguish between reward and behavioral epochs, we then changed the task sequence for a block of trials, such that the saccade target remained visible after the reward delivery. Under these visible conditions, the monkeys tended to continue fixating on the saccade target even after the reward delivery. Therefore, the prediction of the upcoming reward and the end of an individual trial were separated in time. Regardless of the task conditions, half of the tonically active PPTg neurons terminated their activity around the time of the reward delivery, consistent with the view that PPTg neurons might send reward prediction signals until the time of reward delivery, which is essential for computing reward prediction error in reinforcement learning. On the other hand, the other half of the tonically active PPTg neurons changed their activity dependent on the task condition. In the normal condition, the tonic responses terminated around the time of the reward delivery, while in the visible condition, the activity

Reward and Behavioral Factors Contributing to the Tonic Activity of Monkey Pedunculopontine Tegmental Nucleus Neurons during Saccade Tasks.

Science.gov (United States)

Okada, Ken-Ichi; Kobayashi, Yasushi

2016-01-01

The pedunculopontine tegmental nucleus (PPTg) in the brainstem plays a role in controlling reinforcement learning and executing conditioned behavior. We previously examined the activity of PPTg neurons in monkeys during a reward-conditioned, visually guided saccade task, and reported that a population of these neurons exhibited tonic responses throughout the task period. These tonic responses might depend on prediction of the upcoming reward, successful execution of the task, or both. Here, we sought to further distinguish these factors and to investigate how each contributes to the tonic neuronal activity of the PPTg. In our normal visually guided saccade task, the monkey initially fixated on the central fixation target (FT), then made saccades to the peripheral saccade target and received a juice reward after the saccade target disappeared. Most of the tonic activity terminated shortly after the reward delivery, when the monkey broke fixation. To distinguish between reward and behavioral epochs, we then changed the task sequence for a block of trials, such that the saccade target remained visible after the reward delivery. Under these visible conditions, the monkeys tended to continue fixating on the saccade target even after the reward delivery. Therefore, the prediction of the upcoming reward and the end of an individual trial were separated in time. Regardless of the task conditions, half of the tonically active PPTg neurons terminated their activity around the time of the reward delivery, consistent with the view that PPTg neurons might send reward prediction signals until the time of reward delivery, which is essential for computing reward prediction error in reinforcement learning. On the other hand, the other half of the tonically active PPTg neurons changed their activity dependent on the task condition. In the normal condition, the tonic responses terminated around the time of the reward delivery, while in the visible condition, the activity continued

Dopaminergic control of motivation and reinforcement learning: a closed-circuit account for reward-oriented behavior.

Science.gov (United States)

Morita, Kenji; Morishima, Mieko; Sakai, Katsuyuki; Kawaguchi, Yasuo

2013-05-15

Humans and animals take actions quickly when they expect that the actions lead to reward, reflecting their motivation. Injection of dopamine receptor antagonists into the striatum has been shown to slow such reward-seeking behavior, suggesting that dopamine is involved in the control of motivational processes. Meanwhile, neurophysiological studies have revealed that phasic response of dopamine neurons appears to represent reward prediction error, indicating that dopamine plays central roles in reinforcement learning. However, previous attempts to elucidate the mechanisms of these dopaminergic controls have not fully explained how the motivational and learning aspects are related and whether they can be understood by the way the activity of dopamine neurons itself is controlled by their upstream circuitries. To address this issue, we constructed a closed-circuit model of the corticobasal ganglia system based on recent findings regarding intracortical and corticostriatal circuit architectures. Simulations show that the model could reproduce the observed distinct motivational effects of D1- and D2-type dopamine receptor antagonists. Simultaneously, our model successfully explains the dopaminergic representation of reward prediction error as observed in behaving animals during learning tasks and could also explain distinct choice biases induced by optogenetic stimulation of the D1 and D2 receptor-expressing striatal neurons. These results indicate that the suggested roles of dopamine in motivational control and reinforcement learning can be understood in a unified manner through a notion that the indirect pathway of the basal ganglia represents the value of states/actions at a previous time point, an empirically driven key assumption of our model.

Blunted striatal response to monetary reward anticipation during smoking abstinence predicts lapse during a contingency-managed quit attempt.

Science.gov (United States)

Sweitzer, Maggie M; Geier, Charles F; Denlinger, Rachel; Forbes, Erika E; Raiff, Bethany R; Dallery, Jesse; McClernon, F J; Donny, Eric C

2016-03-01

Tobacco smoking is associated with dysregulated reward processing within the striatum, characterized by hypersensitivity to smoking rewards and hyposensitivity to non-smoking rewards. This bias toward smoking reward at the expense of alternative rewards is further exacerbated by deprivation from smoking, which may contribute to difficulty maintaining abstinence during a quit attempt. We examined whether abstinence-induced changes in striatal processing of rewards predicted lapse likelihood during a quit attempt supported by contingency management (CM), in which abstinence from smoking was reinforced with money. Thirty-six non-treatment-seeking smokers participated in two functional MRI (fMRI) sessions, one following 24-h abstinence and one following smoking as usual. During each scan, participants completed a rewarded guessing task designed to elicit striatal activation in which they could earn smoking and monetary rewards delivered after the scan. Participants then engaged in a 3-week CM-supported quit attempt. As previously reported, 24-h abstinence was associated with increased striatal activation in anticipation of smoking reward and decreased activation in anticipation of monetary reward. Individuals exhibiting greater decrements in right striatal activation to monetary reward during abstinence (controlling for activation during non-abstinence) were more likely to lapse during CM (p reward. These results are consistent with a growing number of studies indicating the specific importance of disrupted striatal processing of non-drug reward in nicotine dependence and highlight the importance of individual differences in abstinence-induced deficits in striatal function for smoking cessation.

Artificial neural network implementation of a near-ideal error prediction controller

Science.gov (United States)

Mcvey, Eugene S.; Taylor, Lynore Denise

1992-01-01

A theory has been developed at the University of Virginia which explains the effects of including an ideal predictor in the forward loop of a linear error-sampled system. It has been shown that the presence of this ideal predictor tends to stabilize the class of systems considered. A prediction controller is merely a system which anticipates a signal or part of a signal before it actually occurs. It is understood that an exact prediction controller is physically unrealizable. However, in systems where the input tends to be repetitive or limited, (i.e., not random) near ideal prediction is possible. In order for the controller to act as a stability compensator, the predictor must be designed in a way that allows it to learn the expected error response of the system. In this way, an unstable system will become stable by including the predicted error in the system transfer function. Previous and current prediction controller include pattern recognition developments and fast-time simulation which are applicable to the analysis of linear sampled data type systems. The use of pattern recognition techniques, along with a template matching scheme, has been proposed as one realizable type of near-ideal prediction. Since many, if not most, systems are repeatedly subjected to similar inputs, it was proposed that an adaptive mechanism be used to 'learn' the correct predicted error response. Once the system has learned the response of all the expected inputs, it is necessary only to recognize the type of input with a template matching mechanism and then to use the correct predicted error to drive the system. Suggested here is an alternate approach to the realization of a near-ideal error prediction controller, one designed using Neural Networks. Neural Networks are good at recognizing patterns such as system responses, and the back-propagation architecture makes use of a template matching scheme. In using this type of error prediction, it is assumed that the system error

Expected reward value and reward uncertainty have temporally dissociable effects on memory formation

OpenAIRE

Adcock, R; Clement, Nathaniel; Chiew, Kimberly; Dickerson, Kathryn; Stanek, Jessica

2018-01-01

Anticipating rewards has been shown to enhance memory formation. While substantial evidence implicates dopamine in this behavioral effect, the precise mechanisms remain ambiguous. Because dopamine nuclei show two distinct physiological signatures of reward prediction, we hypothesized two dissociable effects on memory formation. These two signatures are a phasic dopamine response immediately following a reward cue that encodes its expected value, and a sustained, ramping dopamine response that...

Reward contingencies and the recalibration of task monitoring and reward systems: a high-density electrical mapping study.

Science.gov (United States)

Morie, K P; De Sanctis, P; Foxe, J J

2014-07-25

Task execution almost always occurs in the context of reward-seeking or punishment-avoiding behavior. As such, ongoing task-monitoring systems are influenced by reward anticipation systems. In turn, when a task has been executed either successfully or unsuccessfully, future iterations of that task will be re-titrated on the basis of the task outcome. Here, we examined the neural underpinnings of the task-monitoring and reward-evaluation systems to better understand how they govern reward-seeking behavior. Twenty-three healthy adult participants performed a task where they accrued points that equated to real world value (gift cards) by responding as rapidly as possible within an allotted timeframe, while success rate was titrated online by changing the duration of the timeframe dependent on participant performance. Informative cues initiated each trial, indicating the probability of potential reward or loss (four levels from very low to very high). We manipulated feedback by first informing participants of task success/failure, after which a second feedback signal indicated actual magnitude of reward/loss. High-density electroencephalography (EEG) recordings allowed for examination of event-related potentials (ERPs) to the informative cues and in turn, to both feedback signals. Distinct ERP components associated with reward cues, task-preparatory and task-monitoring processes, and reward feedback processes were identified. Unsurprisingly, participants displayed increased ERP amplitudes associated with task-preparatory processes following cues that predicted higher chances of reward. They also rapidly updated reward and loss prediction information dependent on task performance after the first feedback signal. Finally, upon reward receipt, initial reward probability was no longer taken into account. Rather, ERP measures suggested that only the magnitude of actual reward or loss was now processed. Reward and task-monitoring processes are clearly dissociable, but

Dopamine and extinction: A convergence of theory with fear and reward circuitry

OpenAIRE

Abraham, Antony D.; Neve, Kim A.; Lattal, K. Matthew

2013-01-01

Research on dopamine lies at the intersection of sophisticated theoretical and neurobiological approaches to learning and memory. Dopamine has been shown to be critical for many processes that drive learning and memory, including motivation, prediction error, incentive salience, memory consolidation, and response output. Theories of dopamine’s function in these processes have, for the most part, been developed from behavioral approaches that examine learning mechanisms in reward-related tasks...

Reward and relief dimensions of temptation to drink: construct validity and role in predicting differential benefit from acamprosate and naltrexone.

Science.gov (United States)

Roos, Corey R; Mann, Karl; Witkiewitz, Katie

2017-11-01

Researchers have sought to distinguish between individuals whose alcohol use disorder (AUD) is maintained by drinking to relieve negative affect ('relief drinkers') and those whose AUD is maintained by the rewarding effects of alcohol ('reward drinkers'). As an opioid receptor antagonist, naltrexone may be particularly effective for reward drinkers. Acamprosate, which has been shown to down-regulate the glutamatergic system, may be particularly effective for relief drinkers. This study sought to replicate and extend prior work (PREDICT study; Glöckner-Rist et al. ) by examining dimensions of reward and relief temptation to drink and subtypes of individuals with distinct patterns of reward/relief temptation. We utilized data from two randomized clinical trials for AUD (Project MATCH, n = 1726 and COMBINE study, n = 1383). We also tested whether classes of reward/relief temptation would predict differential response to naltrexone and acamprosate in COMBINE. Results replicated prior work by identifying reward and relief temptation factors, which had excellent reliability and construct validity. Using factor mixture modeling, we identified five distinct classes of reward/relief temptation that replicated across studies. In COMBINE, we found a significant class-by-acamprosate interaction effect. Among those most likely classified in the high relief/moderate reward temptation class, individuals had better drinking outcomes if assigned to acamprosate versus placebo. We did not find a significant class-by-naltrexone interaction effect. Our study questions the orthogonal classification of drinkers into only two types (reward or relief drinkers) and adds to the body of research on moderators of acamprosate, which may inform clinical decision making in the treatment of AUD. © 2016 Society for the Study of Addiction.

Prediction-error variance in Bayesian model updating: a comparative study

Science.gov (United States)

Asadollahi, Parisa; Li, Jian; Huang, Yong

2017-04-01

In Bayesian model updating, the likelihood function is commonly formulated by stochastic embedding in which the maximum information entropy probability model of prediction error variances plays an important role and it is Gaussian distribution subject to the first two moments as constraints. The selection of prediction error variances can be formulated as a model class selection problem, which automatically involves a trade-off between the average data-fit of the model class and the information it extracts from the data. Therefore, it is critical for the robustness in the updating of the structural model especially in the presence of modeling errors. To date, three ways of considering prediction error variances have been seem in the literature: 1) setting constant values empirically, 2) estimating them based on the goodness-of-fit of the measured data, and 3) updating them as uncertain parameters by applying Bayes' Theorem at the model class level. In this paper, the effect of different strategies to deal with the prediction error variances on the model updating performance is investigated explicitly. A six-story shear building model with six uncertain stiffness parameters is employed as an illustrative example. Transitional Markov Chain Monte Carlo is used to draw samples of the posterior probability density function of the structure model parameters as well as the uncertain prediction variances. The different levels of modeling uncertainty and complexity are modeled through three FE models, including a true model, a model with more complexity, and a model with modeling error. Bayesian updating is performed for the three FE models considering the three aforementioned treatments of the prediction error variances. The effect of number of measurements on the model updating performance is also examined in the study. The results are compared based on model class assessment and indicate that updating the prediction error variances as uncertain parameters at the model

Multi-layer network utilizing rewarded spike time dependent plasticity to learn a foraging task.

Directory of Open Access Journals (Sweden)

Pavel Sanda

2017-09-01

Full Text Available Neural networks with a single plastic layer employing reward modulated spike time dependent plasticity (STDP are capable of learning simple foraging tasks. Here we demonstrate advanced pattern discrimination and continuous learning in a network of spiking neurons with multiple plastic layers. The network utilized both reward modulated and non-reward modulated STDP and implemented multiple mechanisms for homeostatic regulation of synaptic efficacy, including heterosynaptic plasticity, gain control, output balancing, activity normalization of rewarded STDP and hard limits on synaptic strength. We found that addition of a hidden layer of neurons employing non-rewarded STDP created neurons that responded to the specific combinations of inputs and thus performed basic classification of the input patterns. When combined with a following layer of neurons implementing rewarded STDP, the network was able to learn, despite the absence of labeled training data, discrimination between rewarding patterns and the patterns designated as punishing. Synaptic noise allowed for trial-and-error learning that helped to identify the goal-oriented strategies which were effective in task solving. The study predicts a critical set of properties of the spiking neuronal network with STDP that was sufficient to solve a complex foraging task involving pattern classification and decision making.

Neural evidence for description dependent reward processing in the framing effect

OpenAIRE

Yu, Rongjun; Zhang, Ping

2014-01-01

Human decision making can be influenced by emotionally valenced contexts, known as the framing effect. We used event-related brain potentials to investigate how framing influences the encoding of reward. We found that the feedback related negativity (FRN), which indexes the “worse than expected” negative prediction error in the anterior cingulate cortex (ACC), was more negative for the negative frame than for the positive frame in the win domain. Consistent with previous findings that the FRN...

The amygdala, reward and emotion.

Science.gov (United States)

Murray, Elisabeth A

2007-11-01

Recent research provides new insights into amygdala contributions to positive emotion and reward. Studies of neuronal activity in the monkey amygdala and of autonomic responses mediated by the monkey amygdala show that, contrary to a widely held view, the amygdala is just as important for processing positive reward and reinforcement as it is for negative. In addition, neuropsychological studies reveal that the amygdala is essential for only a fraction of what might be considered 'stimulus-reward processing', and that the neural substrates for emotion and reward are partially nonoverlapping. Finally, evidence suggests that two systems within the amygdala, operating in parallel, enable reward-predicting cues to influence behavior; one mediates a general, arousing effect of reward and the other links the sensory properties of reward to emotion.

Prospective detection of large prediction errors: a hypothesis testing approach

International Nuclear Information System (INIS)

Ruan, Dan

2010-01-01

Real-time motion management is important in radiotherapy. In addition to effective monitoring schemes, prediction is required to compensate for system latency, so that treatment can be synchronized with tumor motion. However, it is difficult to predict tumor motion at all times, and it is critical to determine when large prediction errors may occur. Such information can be used to pause the treatment beam or adjust monitoring/prediction schemes. In this study, we propose a hypothesis testing approach for detecting instants corresponding to potentially large prediction errors in real time. We treat the future tumor location as a random variable, and obtain its empirical probability distribution with the kernel density estimation-based method. Under the null hypothesis, the model probability is assumed to be a concentrated Gaussian centered at the prediction output. Under the alternative hypothesis, the model distribution is assumed to be non-informative uniform, which reflects the situation that the future position cannot be inferred reliably. We derive the likelihood ratio test (LRT) for this hypothesis testing problem and show that with the method of moments for estimating the null hypothesis Gaussian parameters, the LRT reduces to a simple test on the empirical variance of the predictive random variable. This conforms to the intuition to expect a (potentially) large prediction error when the estimate is associated with high uncertainty, and to expect an accurate prediction when the uncertainty level is low. We tested the proposed method on patient-derived respiratory traces. The 'ground-truth' prediction error was evaluated by comparing the prediction values with retrospective observations, and the large prediction regions were subsequently delineated by thresholding the prediction errors. The receiver operating characteristic curve was used to describe the performance of the proposed hypothesis testing method. Clinical implication was represented by miss

Initial uncertainty in Pavlovian reward prediction persistently elevates incentive salience and extends sign-tracking to normally unattractive cues.

Science.gov (United States)

Robinson, Mike J F; Anselme, Patrick; Fischer, Adam M; Berridge, Kent C

2014-06-01

Uncertainty is a component of many gambling games and may play a role in incentive motivation and cue attraction. Uncertainty can increase the attractiveness for predictors of reward in the Pavlovian procedure of autoshaping, visible as enhanced sign-tracking (or approach and nibbles) by rats of a metal lever whose sudden appearance acts as a conditioned stimulus (CS+) to predict sucrose pellets as an unconditioned stimulus (UCS). Here we examined how reward uncertainty might enhance incentive salience as sign-tracking both in intensity and by broadening the range of attractive CS+s. We also examined whether initially induced uncertainty enhancements of CS+ attraction can endure beyond uncertainty itself, and persist even when Pavlovian prediction becomes 100% certain. Our results show that uncertainty can broaden incentive salience attribution to make CS cues attractive that would otherwise not be (either because they are too distal from reward or too risky to normally attract sign-tracking). In addition, uncertainty enhancement of CS+ incentive salience, once induced by initial exposure, persisted even when Pavlovian CS-UCS correlations later rose toward 100% certainty in prediction. Persistence suggests an enduring incentive motivation enhancement potentially relevant to gambling, which in some ways resembles incentive-sensitization. Higher motivation to uncertain CS+s leads to more potent attraction to these cues when they predict the delivery of uncertain rewards. In humans, those cues might possibly include the sights and sounds associated with gambling, which contribute a major component of the play immersion experienced by problematic gamblers. Copyright © 2014 Elsevier B.V. All rights reserved.

Climbing fibers predict movement kinematics and performance errors.

Science.gov (United States)

Streng, Martha L; Popa, Laurentiu S; Ebner, Timothy J

2017-09-01

Requisite for understanding cerebellar function is a complete characterization of the signals provided by complex spike (CS) discharge of Purkinje cells, the output neurons of the cerebellar cortex. Numerous studies have provided insights into CS function, with the most predominant view being that they are evoked by error events. However, several reports suggest that CSs encode other aspects of movements and do not always respond to errors or unexpected perturbations. Here, we evaluated CS firing during a pseudo-random manual tracking task in the monkey ( Macaca mulatta ). This task provides extensive coverage of the work space and relative independence of movement parameters, delivering a robust data set to assess the signals that activate climbing fibers. Using reverse correlation, we determined feedforward and feedback CSs firing probability maps with position, velocity, and acceleration, as well as position error, a measure of tracking performance. The direction and magnitude of the CS modulation were quantified using linear regression analysis. The major findings are that CSs significantly encode all three kinematic parameters and position error, with acceleration modulation particularly common. The modulation is not related to "events," either for position error or kinematics. Instead, CSs are spatially tuned and provide a linear representation of each parameter evaluated. The CS modulation is largely predictive. Similar analyses show that the simple spike firing is modulated by the same parameters as the CSs. Therefore, CSs carry a broader array of signals than previously described and argue for climbing fiber input having a prominent role in online motor control. NEW & NOTEWORTHY This article demonstrates that complex spike (CS) discharge of cerebellar Purkinje cells encodes multiple parameters of movement, including motor errors and kinematics. The CS firing is not driven by error or kinematic events; instead it provides a linear representation of each

Validation and extension of the reward-mountain model.

Science.gov (United States)

Breton, Yannick-André; Mullett, Ada; Conover, Kent; Shizgal, Peter

2013-01-01

The reward-mountain model relates the vigor of reward seeking to the strength and cost of reward. Application of this model provides information about the stage of processing at which manipulations such as drug administration, lesions, deprivation states, and optogenetic interventions act to alter reward seeking. The model has been updated by incorporation of new information about frequency following in the directly stimulated neurons responsible for brain stimulation reward and about the function that maps objective opportunity costs into subjective ones. The behavioral methods for applying the model have been updated and improved as well. To assess the impact of these changes, two related predictions of the model that were supported by earlier work have been retested: (1) altering the duration of rewarding brain stimulation should change the pulse frequency required to produce a reward of half-maximal intensity, and (2) this manipulation should not change the opportunity cost at which half-maximal performance is directed at earning a maximally intense reward. Prediction 1 was supported in all six subjects, but prediction 2 was supported in only three. The latter finding is interpreted to reflect recruitment, at some stimulation sites, of a heterogeneous reward substrate comprising dual, parallel circuits that integrate the stimulation-induced neural signals.

SHERPA: A systematic human error reduction and prediction approach

International Nuclear Information System (INIS)

Embrey, D.E.

1986-01-01

This paper describes a Systematic Human Error Reduction and Prediction Approach (SHERPA) which is intended to provide guidelines for human error reduction and quantification in a wide range of human-machine systems. The approach utilizes as its basic current cognitive models of human performance. The first module in SHERPA performs task and human error analyses, which identify likely error modes, together with guidelines for the reduction of these errors by training, procedures and equipment redesign. The second module uses a SARAH approach to quantify the probability of occurrence of the errors identified earlier, and provides cost benefit analyses to assist in choosing the appropriate error reduction approaches in the third module

“Liking” and “Wanting” Linked to Reward Deficiency Syndrome (RDS): Hypothesizing Differential Responsivity in Brain Reward Circuitry

OpenAIRE

Blum, Kenneth; Gardner, Eliot; Oscar-Berman, Marlene; Gold, Mark

2012-01-01

In an attempt to resolve controversy regarding the causal contributions of mesolimbic dopamine (DA) systems to reward, we evaluate the three main competing explanatory categories: “liking,” “learning,” and “wanting” [1]. That is, DA may mediate (a) the hedonic impact of reward (liking), (b) learned predictions about rewarding effects (learning), or (c) the pursuit of rewards by attributing incentive salience to reward-related stimuli (wanting). We evaluate these hypotheses, especially as they...

Does the sensorimotor system minimize prediction error or select the most likely prediction during object lifting?

Science.gov (United States)

McGregor, Heather R.; Pun, Henry C. H.; Buckingham, Gavin; Gribble, Paul L.

2016-01-01

The human sensorimotor system is routinely capable of making accurate predictions about an object's weight, which allows for energetically efficient lifts and prevents objects from being dropped. Often, however, poor predictions arise when the weight of an object can vary and sensory cues about object weight are sparse (e.g., picking up an opaque water bottle). The question arises, what strategies does the sensorimotor system use to make weight predictions when one is dealing with an object whose weight may vary? For example, does the sensorimotor system use a strategy that minimizes prediction error (minimal squared error) or one that selects the weight that is most likely to be correct (maximum a posteriori)? In this study we dissociated the predictions of these two strategies by having participants lift an object whose weight varied according to a skewed probability distribution. We found, using a small range of weight uncertainty, that four indexes of sensorimotor prediction (grip force rate, grip force, load force rate, and load force) were consistent with a feedforward strategy that minimizes the square of prediction errors. These findings match research in the visuomotor system, suggesting parallels in underlying processes. We interpret our findings within a Bayesian framework and discuss the potential benefits of using a minimal squared error strategy. NEW & NOTEWORTHY Using a novel experimental model of object lifting, we tested whether the sensorimotor system models the weight of objects by minimizing lifting errors or by selecting the statistically most likely weight. We found that the sensorimotor system minimizes the square of prediction errors for object lifting. This parallels the results of studies that investigated visually guided reaching, suggesting an overlap in the underlying mechanisms between tasks that involve different sensory systems. PMID:27760821

Impaired cross-talk between mesolimbic food reward processing and metabolic signaling predicts body mass index

Directory of Open Access Journals (Sweden)

Joe J Simon

2014-10-01

Full Text Available The anticipation of the pleasure derived from food intake drives the motivation to eat, and hence facilitate overconsumption of food which ultimately results in obesity. Brain imaging studies provide evidence that mesolimbic brain regions underlie both general as well as food related anticipatory reward processing. In light of this knowledge, the present study examined the neural responsiveness of the ventral striatum in participants with a broad BMI spectrum. The study differentiated between general (i.e. monetary and food related anticipatory reward processing. We recruited a sample of volunteers with greatly varying body weights, ranging from a low BMI (below 20 kg/m² over a normal (20 to 25 kg/m² and overweight (25 to 30 kg/m² BMI, to class I (30 to 35 kg/m² and class II (35 to 40 kg/m² obesity. A total of 24 participants underwent functional magnetic resonance imaging whilst performing both a food and monetary incentive delay task, which allows to measure neural activation during the anticipation of rewards. After the presentation of a cue indicating the amount of food or money to be won, participants had to react correctly in order to earn snack points or money coins which could then be exchanged for real food or money, respectively, at the end of the experiment. During the anticipation of both types of rewards, participants displayed activity in the ventral striatum, a region that plays a pivotal role in the anticipation of rewards. Additionally, we observed that specifically anticipatory food reward processing predicted the individual BMI (current and maximum lifetime. This relation was found to be mediated by impaired hormonal satiety signaling, i.e. increased leptin levels and insulin resistance. These findings suggest that heightened food reward motivation contributes to obesity through impaired metabolic signaling.

Low social rhythm regularity predicts first onset of bipolar spectrum disorders among at-risk individuals with reward hypersensitivity.

Science.gov (United States)

Alloy, Lauren B; Boland, Elaine M; Ng, Tommy H; Whitehouse, Wayne G; Abramson, Lyn Y

2015-11-01

The social zeitgeber model (Ehlers, Frank, & Kupfer, 1988) suggests that irregular daily schedules or social rhythms provide vulnerability to bipolar spectrum disorders. This study tested whether social rhythm regularity prospectively predicted first lifetime onset of bipolar spectrum disorders in adolescents already at risk for bipolar disorder based on exhibiting reward hypersensitivity. Adolescents (ages 14-19 years) previously screened to have high (n = 138) or moderate (n = 95) reward sensitivity, but no lifetime history of bipolar spectrum disorder, completed measures of depressive and manic symptoms, family history of bipolar disorder, and the Social Rhythm Metric. They were followed prospectively with semistructured diagnostic interviews every 6 months for an average of 31.7 (SD = 20.1) months. Hierarchical logistic regression indicated that low social rhythm regularity at baseline predicted greater likelihood of first onset of bipolar spectrum disorder over follow-up among high-reward-sensitivity adolescents but not moderate-reward-sensitivity adolescents, controlling for follow-up time, gender, age, family history of bipolar disorder, and initial manic and depressive symptoms (β = -.150, Wald = 4.365, p = .037, odds ratio = .861, 95% confidence interval [.748, .991]). Consistent with the social zeitgeber theory, low social rhythm regularity provides vulnerability to first onset of bipolar spectrum disorder among at-risk adolescents. It may be possible to identify adolescents at risk for developing a bipolar spectrum disorder based on exhibiting both reward hypersensitivity and social rhythm irregularity before onset occurs. (c) 2015 APA, all rights reserved).

The Pupillary Orienting Response Predicts Adaptive Behavioral Adjustment after Errors.

Directory of Open Access Journals (Sweden)

Peter R Murphy

Full Text Available Reaction time (RT is commonly observed to slow down after an error. This post-error slowing (PES has been thought to arise from the strategic adoption of a more cautious response mode following deployment of cognitive control. Recently, an alternative account has suggested that PES results from interference due to an error-evoked orienting response. We investigated whether error-related orienting may in fact be a pre-cursor to adaptive post-error behavioral adjustment when the orienting response resolves before subsequent trial onset. We measured pupil dilation, a prototypical measure of autonomic orienting, during performance of a choice RT task with long inter-stimulus intervals, and found that the trial-by-trial magnitude of the error-evoked pupil response positively predicted both PES magnitude and the likelihood that the following response would be correct. These combined findings suggest that the magnitude of the error-related orienting response predicts an adaptive change of response strategy following errors, and thereby promote a reconciliation of the orienting and adaptive control accounts of PES.

Using lexical variables to predict picture-naming errors in jargon aphasia

Directory of Open Access Journals (Sweden)

Catherine Godbold

2015-04-01

Full Text Available Introduction Individuals with jargon aphasia produce fluent output which often comprises high proportions of non-word errors (e.g., maf for dog. Research has been devoted to identifying the underlying mechanisms behind such output. Some accounts posit a reduced flow of spreading activation between levels in the lexical network (e.g., Robson et al., 2003. If activation level differences across the lexical network are a cause of non-word outputs, we would predict improved performance when target items reflect an increased flow of activation between levels (e.g. more frequently-used words are often represented by higher resting levels of activation. This research investigates the effect of lexical properties of targets (e.g., frequency, imageability on accuracy, error type (real word vs. non-word and target-error overlap of non-word errors in a picture naming task by individuals with jargon aphasia. Method Participants were 17 individuals with Wernicke’s aphasia, who produced a high proportion of non-word errors (>20% of errors on the Philadelphia Naming Test (PNT; Roach et al., 1996. The data were retrieved from the Moss Aphasic Psycholinguistic Database Project (MAPPD, Mirman et al., 2010. We used a series of mixed models to test whether lexical variables predicted accuracy, error type (real word vs. non-word and target-error overlap for the PNT data. As lexical variables tend to be highly correlated, we performed a principal components analysis to reduce the variables into five components representing variables associated with phonology (length, phonotactic probability, neighbourhood density and neighbourhood frequency, semantics (imageability and concreteness, usage (frequency and age-of-acquisition, name agreement and visual complexity. Results and Discussion Table 1 shows the components that made a significant contribution to each model. Individuals with jargon aphasia produced more correct responses and fewer non-word errors relative to

The conditions that promote fear learning: prediction error and Pavlovian fear conditioning.

Science.gov (United States)

Li, Susan Shi Yuan; McNally, Gavan P

2014-02-01

A key insight of associative learning theory is that learning depends on the actions of prediction error: a discrepancy between the actual and expected outcomes of a conditioning trial. When positive, such error causes increments in associative strength and, when negative, such error causes decrements in associative strength. Prediction error can act directly on fear learning by determining the effectiveness of the aversive unconditioned stimulus or indirectly by determining the effectiveness, or associability, of the conditioned stimulus. Evidence from a variety of experimental preparations in human and non-human animals suggest that discrete neural circuits code for these actions of prediction error during fear learning. Here we review the circuits and brain regions contributing to the neural coding of prediction error during fear learning and highlight areas of research (safety learning, extinction, and reconsolidation) that may profit from this approach to understanding learning. Crown Copyright © 2013. Published by Elsevier Inc. All rights reserved.

Reward, Context, and Human Behaviour

Directory of Open Access Journals (Sweden)

Clare L. Blaukopf

2007-01-01

Full Text Available Animal models of reward processing have revealed an extensive network of brain areas that process different aspects of reward, from expectation and prediction to calculation of relative value. These results have been confirmed and extended in human neuroimaging to encompass secondary rewards more unique to humans, such as money. The majority of the extant literature covers the brain areas associated with rewards whilst neglecting analysis of the actual behaviours that these rewards generate. This review strives to redress this imbalance by illustrating the importance of looking at the behavioural outcome of rewards and the context in which they are produced. Following a brief review of the literature of reward-related activity in the brain, we examine the effect of reward context on actions. These studies reveal how the presence of reward vs. reward and punishment, or being conscious vs. unconscious of reward-related actions, differentially influence behaviour. The latter finding is of particular importance given the extent to which animal models are used in understanding the reward systems of the human mind. It is clear that further studies are needed to learn about the human reaction to reward in its entirety, including any distinctions between conscious and unconscious behaviours. We propose that studies of reward entail a measure of the animal's (human or nonhuman knowledge of the reward and knowledge of its own behavioural outcome to achieve that reward.

Learning about Expectation Violation from Prediction Error Paradigms – A Meta-Analysis on Brain Processes Following a Prediction Error

Directory of Open Access Journals (Sweden)

Lisa D’Astolfo

2017-07-01

Full Text Available Modifying patients’ expectations by exposing them to expectation violation situations (thus maximizing the difference between the expected and the actual situational outcome is proposed to be a crucial mechanism for therapeutic success for a variety of different mental disorders. However, clinical observations suggest that patients often maintain their expectations regardless of experiences contradicting their expectations. It remains unclear which information processing mechanisms lead to modification or persistence of patients’ expectations. Insight in the processing could be provided by Neuroimaging studies investigating prediction error (PE, i.e., neuronal reactions to non-expected stimuli. Two methods are often used to investigate the PE: (1 paradigms, in which participants passively observe PEs (”passive” paradigms and (2 paradigms, which encourage a behavioral adaptation following a PE (“active” paradigms. These paradigms are similar to the methods used to induce expectation violations in clinical settings: (1 the confrontation with an expectation violation situation and (2 an enhanced confrontation in which the patient actively challenges his expectation. We used this similarity to gain insight in the different neuronal processing of the two PE paradigms. We performed a meta-analysis contrasting neuronal activity of PE paradigms encouraging a behavioral adaptation following a PE and paradigms enforcing passiveness following a PE. We found more neuronal activity in the striatum, the insula and the fusiform gyrus in studies encouraging behavioral adaptation following a PE. Due to the involvement of reward assessment and avoidance learning associated with the striatum and the insula we propose that the deliberate execution of action alternatives following a PE is associated with the integration of new information into previously existing expectations, therefore leading to an expectation change. While further research is needed

Predicting Motivation: Computational Models of PFC Can Explain Neural Coding of Motivation and Effort-based Decision-making in Health and Disease.

Science.gov (United States)

Vassena, Eliana; Deraeve, James; Alexander, William H

2017-10-01

Human behavior is strongly driven by the pursuit of rewards. In daily life, however, benefits mostly come at a cost, often requiring that effort be exerted to obtain potential benefits. Medial PFC (MPFC) and dorsolateral PFC (DLPFC) are frequently implicated in the expectation of effortful control, showing increased activity as a function of predicted task difficulty. Such activity partially overlaps with expectation of reward and has been observed both during decision-making and during task preparation. Recently, novel computational frameworks have been developed to explain activity in these regions during cognitive control, based on the principle of prediction and prediction error (predicted response-outcome [PRO] model [Alexander, W. H., & Brown, J. W. Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14, 1338-1344, 2011], hierarchical error representation [HER] model [Alexander, W. H., & Brown, J. W. Hierarchical error representation: A computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Computation, 27, 2354-2410, 2015]). Despite the broad explanatory power of these models, it is not clear whether they can also accommodate effects related to the expectation of effort observed in MPFC and DLPFC. Here, we propose a translation of these computational frameworks to the domain of effort-based behavior. First, we discuss how the PRO model, based on prediction error, can explain effort-related activity in MPFC, by reframing effort-based behavior in a predictive context. We propose that MPFC activity reflects monitoring of motivationally relevant variables (such as effort and reward), by coding expectations and discrepancies from such expectations. Moreover, we derive behavioral and neural model-based predictions for healthy controls and clinical populations with impairments of motivation. Second, we illustrate the possible translation to effort-based behavior of the HER model, an extended version of PRO

Inhibiting food reward: delay discounting, food reward sensitivity, and palatable food intake in overweight and obese women.

Science.gov (United States)

Appelhans, Bradley M; Woolf, Kathleen; Pagoto, Sherry L; Schneider, Kristin L; Whited, Matthew C; Liebman, Rebecca

2011-11-01

Overeating is believed to result when the appetitive motivation to consume palatable food exceeds an individual's capacity for inhibitory control of eating. This hypothesis was supported in recent studies involving predominantly normal weight women, but has not been tested in obese populations. The current study tested the interaction between food reward sensitivity and inhibitory control in predicting palatable food intake among energy-replete overweight and obese women (N = 62). Sensitivity to palatable food reward was measured with the Power of Food Scale. Inhibitory control was assessed with a computerized choice task that captures the tendency to discount large delayed rewards relative to smaller immediate rewards. Participants completed an eating in the absence of hunger protocol in which homeostatic energy needs were eliminated with a bland preload of plain oatmeal, followed by a bogus laboratory taste test of palatable and bland snacks. The interaction between food reward sensitivity and inhibitory control was a significant predictor of palatable food intake in regression analyses controlling for BMI and the amount of preload consumed. Probing this interaction indicated that higher food reward sensitivity predicted greater palatable food intake at low levels of inhibitory control, but was not associated with intake at high levels of inhibitory control. As expected, no associations were found in a similar regression analysis predicting intake of bland foods. Findings support a neurobehavioral model of eating behavior in which sensitivity to palatable food reward drives overeating only when accompanied by insufficient inhibitory control. Strengthening inhibitory control could enhance weight management programs.

Seismic attenuation relationship with homogeneous and heterogeneous prediction-error variance models

Science.gov (United States)

Mu, He-Qing; Xu, Rong-Rong; Yuen, Ka-Veng

2014-03-01

Peak ground acceleration (PGA) estimation is an important task in earthquake engineering practice. One of the most well-known models is the Boore-Joyner-Fumal formula, which estimates the PGA using the moment magnitude, the site-to-fault distance and the site foundation properties. In the present study, the complexity for this formula and the homogeneity assumption for the prediction-error variance are investigated and an efficiency-robustness balanced formula is proposed. For this purpose, a reduced-order Monte Carlo simulation algorithm for Bayesian model class selection is presented to obtain the most suitable predictive formula and prediction-error model for the seismic attenuation relationship. In this approach, each model class (a predictive formula with a prediction-error model) is evaluated according to its plausibility given the data. The one with the highest plausibility is robust since it possesses the optimal balance between the data fitting capability and the sensitivity to noise. A database of strong ground motion records in the Tangshan region of China is obtained from the China Earthquake Data Center for the analysis. The optimal predictive formula is proposed based on this database. It is shown that the proposed formula with heterogeneous prediction-error variance is much simpler than the attenuation model suggested by Boore, Joyner and Fumal (1993).

Multiple reward-cue contingencies favor expectancy over uncertainty in shaping the reward-cue attentional salience.

Science.gov (United States)

De Tommaso, Matteo; Mastropasqua, Tommaso; Turatto, Massimo

2018-01-25

Reward-predicting cues attract attention because of their motivational value. A debated question regards the conditions under which the cue's attentional salience is governed more by reward expectancy rather than by reward uncertainty. To help shedding light on this relevant issue, here, we manipulated expectancy and uncertainty using three levels of reward-cue contingency, so that, for example, a high level of reward expectancy (p = .8) was compared with the highest level of reward uncertainty (p = .5). In Experiment 1, the best reward-cue during conditioning was preferentially attended in a subsequent visual search task. This result was replicated in Experiment 2, in which the cues were matched in terms of response history. In Experiment 3, we implemented a hybrid procedure consisting of two phases: an omission contingency procedure during conditioning, followed by a visual search task as in the previous experiments. Crucially, during both phases, the reward-cues were never task relevant. Results confirmed that, when multiple reward-cue contingencies are explored by a human observer, expectancy is the major factor controlling both the attentional and the oculomotor salience of the reward-cue.

A second study of the prediction of cognitive errors using the 'CREAM' technique

International Nuclear Information System (INIS)

Collier, Steve; Andresen, Gisle

2000-03-01

Some human errors, such as errors of commission and knowledge-based errors, are not adequately modelled in probabilistic safety assessments. Even qualitative methods for handling these sorts of errors are comparatively underdeveloped. The 'Cognitive Reliability and Error Analysis Method' (CREAM) was recently developed for prediction of cognitive error modes. It has not yet been comprehensively established how reliable, valid and generally useful it could be to researchers and practitioners. A previous study of CREAM at Halden was promising, showing a relationship between errors predicted in advance and those that actually occurred in simulated fault scenarios. The present study continues this work. CREAM was used to make predictions of cognitive error modes throughout two rather difficult fault scenarios. Predictions were made of the most likely cognitive error mode, were one to occur at all, at several points throughout the expected scenarios, based upon the scenario design and description. Each scenario was then run 15 times with different operators. Error modes occurring during simulations were later scored using the task description for the scenario, videotapes of operator actions, eye-track recording, operators' verbal protocols and an expert's concurrent commentary. The scoring team had no previous substantive knowledge of the experiment or the techniques used, so as to provide a more stringent test of the data and knowledge needed for scoring. The scored error modes were then compared with the CREAM predictions to assess the degree of agreement. Some cognitive error modes were predicted successfully, but the results were generally not so encouraging as the previous study. Several problems were found with both the CREAM technique and the data needed to complete the analysis. It was felt that further development was needed before this kind of analysis can be reliable and valid, either in a research setting or as a practitioner's tool in a safety assessment

Sensitivity to Temporal Reward Structure in Amygdala Neurons

OpenAIRE

Bermudez, Maria A.; Göbel, Carl; Schultz, Wolfram

2012-01-01

Summary The time of reward and the temporal structure of reward occurrence fundamentally influence behavioral reinforcement and decision processes [1–11]. However, despite knowledge about timing in sensory and motor systems [12–17], we know little about temporal mechanisms of neuronal reward processing. In this experiment, visual stimuli predicted different instantaneous probabilities of reward occurrence that resulted in specific temporal reward structures. Licking behavior demonstrated that...

Temporal Prediction Errors Affect Short-Term Memory Scanning Response Time.

Science.gov (United States)

Limongi, Roberto; Silva, Angélica M

2016-11-01

The Sternberg short-term memory scanning task has been used to unveil cognitive operations involved in time perception. Participants produce time intervals during the task, and the researcher explores how task performance affects interval production - where time estimation error is the dependent variable of interest. The perspective of predictive behavior regards time estimation error as a temporal prediction error (PE), an independent variable that controls cognition, behavior, and learning. Based on this perspective, we investigated whether temporal PEs affect short-term memory scanning. Participants performed temporal predictions while they maintained information in memory. Model inference revealed that PEs affected memory scanning response time independently of the memory-set size effect. We discuss the results within the context of formal and mechanistic models of short-term memory scanning and predictive coding, a Bayes-based theory of brain function. We state the hypothesis that our finding could be associated with weak frontostriatal connections and weak striatal activity.

Excessive body fat linked to blunted somatosensory cortex response to general reward in adolescents.

Science.gov (United States)

Navas, J F; Barrós-Loscertales, A; Costumero-Ramos, V; Verdejo-Román, J; Vilar-López, R; Verdejo-García, A

2018-01-01

The brain reward system is key to understanding adolescent obesity in the current obesogenic environment, rich in highly appetising stimuli, to which adolescents are particularly sensitive. We aimed to examine the association between body fat levels and brain reward system responsivity to general (monetary) rewards in male and female adolescents. Sixty-eight adolescents (34 females; mean age (s.d.)= 16.56 (1.35)) were measured for body fat levels with bioelectric impedance, and underwent a functional magnetic resonance imaging (fMRI) scan during the Monetary Incentive Delay (MID) task. The MID task reliably elicits brain activations associated with two fundamental aspects of reward processing: anticipation and feedback. We conducted regression analyses to examine the association between body fat and brain reward system responsivity during reward anticipation and feedback, while controlling for sex, age and socioeconomic status. We also analysed the moderating impact of sex on the relationship between fat levels and brain responsivity measures. Brain imaging analyses were corrected for multiple comparisons, with a cluster-defining threshold of Preward feedback after controlling for key sociodemographic variables. Although we did not find significant associations between body fat and brain activations during reward anticipation, S1/supramarginal gyrus activation during feedback was linked to increased negative prediction error, that is, less reward than expected, in illustrative post hoc analyses. Sex did not significantly moderate the association between body fat and brain activation in the MID task. In adolescents, higher adiposity is linked to hypo-responsivity of somatosensory regions during general (monetary) reward feedback. Findings suggest that adolescents with excess weight have blunted activation in somatosensory regions involved in reward feedback learning.

Social reward shapes attentional biases.

Science.gov (United States)

Anderson, Brian A

2016-01-01

Paying attention to stimuli that predict a reward outcome is important for an organism to survive and thrive. When visual stimuli are associated with tangible, extrinsic rewards such as money or food, these stimuli acquire high attentional priority and come to automatically capture attention. In humans and other primates, however, many behaviors are not motivated directly by such extrinsic rewards, but rather by the social feedback that results from performing those behaviors. In the present study, I examine whether positive social feedback can similarly influence attentional bias. The results show that stimuli previously associated with a high probability of positive social feedback elicit value-driven attentional capture, much like stimuli associated with extrinsic rewards. Unlike with extrinsic rewards, however, such stimuli also influence task-specific motivation. My findings offer a potential mechanism by which social reward shapes the information that we prioritize when perceiving the world around us.

Consolidation power of extrinsic rewards: reward cues enhance long-term memory for irrelevant past events.

Science.gov (United States)

Murayama, Kou; Kitagami, Shinji

2014-02-01

Recent research suggests that extrinsic rewards promote memory consolidation through dopaminergic modulation processes. However, no conclusive behavioral evidence exists given that the influence of extrinsic reward on attention and motivation during encoding and consolidation processes are inherently confounded. The present study provides behavioral evidence that extrinsic rewards (i.e., monetary incentives) enhance human memory consolidation independently of attention and motivation. Participants saw neutral pictures, followed by a reward or control cue in an unrelated context. Our results (and a direct replication study) demonstrated that the reward cue predicted a retrograde enhancement of memory for the preceding neutral pictures. This retrograde effect was observed only after a delay, not immediately upon testing. An additional experiment showed that emotional arousal or unconscious resource mobilization cannot explain the retrograde enhancement effect. These results provide support for the notion that the dopaminergic memory consolidation effect can result from extrinsic reward.

An Update on the Role of Serotonin and its Interplay with Dopamine for Reward.

Science.gov (United States)

Fischer, Adrian G; Ullsperger, Markus

2017-01-01

The specific role of serotonin and its interplay with dopamine (DA) in adaptive, reward guided behavior as well as drug dependance, still remains elusive. Recently, novel methods allowed cell type specific anatomical, functional and interventional analyses of serotonergic and dopaminergic circuits, promising significant advancement in understanding their functional roles. Furthermore, it is increasingly recognized that co-release of neurotransmitters is functionally relevant, understanding of which is required in order to interpret results of pharmacological studies and their relationship to neural recordings. Here, we review recent animal studies employing such techniques with the aim to connect their results to effects observed in human pharmacological studies and subjective effects of drugs. It appears that the additive effect of serotonin and DA conveys significant reward related information and is subjectively highly euphorizing. Neither DA nor serotonin alone have such an effect. This coincides with optogenetically targeted recordings in mice, where the dopaminergic system codes reward prediction errors (PE), and the serotonergic system mainly unsigned PE. Overall, this pattern of results indicates that joint activity between both systems carries essential reward information and invites parallel investigation of both neurotransmitter systems.

An Update on the Role of Serotonin and its Interplay with Dopamine for Reward

Directory of Open Access Journals (Sweden)

Adrian G. Fischer

2017-10-01

Full Text Available The specific role of serotonin and its interplay with dopamine (DA in adaptive, reward guided behavior as well as drug dependance, still remains elusive. Recently, novel methods allowed cell type specific anatomical, functional and interventional analyses of serotonergic and dopaminergic circuits, promising significant advancement in understanding their functional roles. Furthermore, it is increasingly recognized that co-release of neurotransmitters is functionally relevant, understanding of which is required in order to interpret results of pharmacological studies and their relationship to neural recordings. Here, we review recent animal studies employing such techniques with the aim to connect their results to effects observed in human pharmacological studies and subjective effects of drugs. It appears that the additive effect of serotonin and DA conveys significant reward related information and is subjectively highly euphorizing. Neither DA nor serotonin alone have such an effect. This coincides with optogenetically targeted recordings in mice, where the dopaminergic system codes reward prediction errors (PE, and the serotonergic system mainly unsigned PE. Overall, this pattern of results indicates that joint activity between both systems carries essential reward information and invites parallel investigation of both neurotransmitter systems.

Development of a prototype system for prediction of the group error at the maintenance work

International Nuclear Information System (INIS)

Yoshino, Kenji; Hirotsu, Yuuko

2001-01-01

This paper described on development and performance evaluation of a prototype system for prediction of the group error at the maintenance work. The results so far are as follows. (1) When a user inputs the existence and the grade of the feature factor of the maintenance work as a prediction object, an organization and an organization factor and a group PSF put into the system. The maintenance group error to target can be predicted through the prediction model which consists of a class of seven stages. (2) This system by utilizing the information on a prediction result database, it can be use not only for prediction of a maintenance group but for various safe Activity, such as KYT(Kiken Yochi Training) and TBM(Tool Box Meeting). (3) This system predicts a cooperation error at highest rate, and subsequently. Predicts the detection error at a high rate. and to the decision-making. Error, the transfer error and the state cognitive error, and state error, it has the characteristics predicted at almost same rate. (4) if it has full knowledge even if the feature, such as the enforcement conditions of maintenance work, and organization, even if the user has neither the knowledge about a human factor, users experience, anyone of this system is slight about the extent, generating of a maintenance group error made difficult from the former logically and systematically, it can predict with business time for about 15 minutes. (author)

Brain Circuits Encoding Reward from Pain Relief.

Science.gov (United States)

Navratilova, Edita; Atcherley, Christopher W; Porreca, Frank

2015-11-01

Relief from pain in humans is rewarding and pleasurable. Primary rewards, or reward-predictive cues, are encoded in brain reward/motivational circuits. While considerable advances have been made in our understanding of reward circuits underlying positive reinforcement, less is known about the circuits underlying the hedonic and reinforcing actions of pain relief. We review findings from electrophysiological, neuroimaging, and behavioral studies supporting the concept that the rewarding effect of pain relief requires opioid signaling in the anterior cingulate cortex (ACC), activation of midbrain dopamine neurons, and the release of dopamine in the nucleus accumbens (NAc). Understanding of circuits that govern the reward of pain relief may allow the discovery of more effective and satisfying therapies for patients with acute or chronic pain.

An imperfect dopaminergic error signal can drive temporal-difference learning.

Directory of Open Access Journals (Sweden)

Wiebke Potjans

2011-05-01

Full Text Available An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.

Accounting for Dynamic Fluctuations across Time when Examining fMRI Test-Retest Reliability: Analysis of a Reward Paradigm in the EMBARC Study.

Directory of Open Access Journals (Sweden)

Henry W Chase

Full Text Available Longitudinal investigation of the neural correlates of reward processing in depression may represent an important step in defining effective biomarkers for antidepressant treatment outcome prediction, but the reliability of reward-related activation is not well understood. Thirty-seven healthy control participants were scanned using fMRI while performing a reward-related guessing task on two occasions, approximately one week apart. Two main contrasts were examined: right ventral striatum (VS activation fMRI BOLD signal related to signed prediction errors (PE and reward expectancy (RE. We also examined bilateral visual cortex activation coupled to outcome anticipation. Significant VS PE-related activity was observed at the first testing session, but at the second testing session, VS PE-related activation was significantly reduced. Conversely, significant VS RE-related activity was observed at time 2 but not time 1. Increases in VS RE-related activity from time 1 to time 2 were significantly associated with decreases in VS PE-related activity from time 1 to time 2 across participants. Intraclass correlations (ICCs in VS were very low. By contrast, visual cortex activation had much larger ICCs, particularly in individuals with high quality data. Dynamic changes in brain activation are widely predicted, and failure to account for these changes could lead to inaccurate evaluations of the reliability of functional MRI signals. Conventional measures of reliability cannot distinguish between changes specified by algorithmic models of neural function and noisy signal. Here, we provide evidence for the former possibility: reward-related VS activations follow the pattern predicted by temporal difference models of reward learning but have low ICCs.

How to Avoid Errors in Error Propagation: Prediction Intervals and Confidence Intervals in Forest Biomass

Science.gov (United States)

Lilly, P.; Yanai, R. D.; Buckley, H. L.; Case, B. S.; Woollons, R. C.; Holdaway, R. J.; Johnson, J.

2016-12-01

Calculations of forest biomass and elemental content require many measurements and models, each contributing uncertainty to the final estimates. While sampling error is commonly reported, based on replicate plots, error due to uncertainty in the regression used to estimate biomass from tree diameter is usually not quantified. Some published estimates of uncertainty due to the regression models have used the uncertainty in the prediction of individuals, ignoring uncertainty in the mean, while others have propagated uncertainty in the mean while ignoring individual variation. Using the simple case of the calcium concentration of sugar maple leaves, we compare the variation among individuals (the standard deviation) to the uncertainty in the mean (the standard error) and illustrate the declining importance in the prediction of individual concentrations as the number of individuals increases. For allometric models, the analogous statistics are the prediction interval (or the residual variation in the model fit) and the confidence interval (describing the uncertainty in the best fit model). The effect of propagating these two sources of error is illustrated using the mass of sugar maple foliage. The uncertainty in individual tree predictions was large for plots with few trees; for plots with 30 trees or more, the uncertainty in individuals was less important than the uncertainty in the mean. Authors of previously published analyses have reanalyzed their data to show the magnitude of these two sources of uncertainty in scales ranging from experimental plots to entire countries. The most correct analysis will take both sources of uncertainty into account, but for practical purposes, country-level reports of uncertainty in carbon stocks, as required by the IPCC, can ignore the uncertainty in individuals. Ignoring the uncertainty in the mean will lead to exaggerated estimates of confidence in estimates of forest biomass and carbon and nutrient contents.

Individual differences in regulatory focus predict neural response to reward.

Science.gov (United States)

Scult, Matthew A; Knodt, Annchen R; Hanson, Jamie L; Ryoo, Minyoung; Adcock, R Alison; Hariri, Ahmad R; Strauman, Timothy J

2017-08-01

Although goal pursuit is related to both functioning of the brain's reward circuits and psychological factors, the literatures surrounding these concepts have often been separate. Here, we use the psychological construct of regulatory focus to investigate individual differences in neural response to reward. Regulatory focus theory proposes two motivational orientations for personal goal pursuit: (1) promotion, associated with sensitivity to potential gain, and (2) prevention, associated with sensitivity to potential loss. The monetary incentive delay task was used to manipulate reward circuit function, along with instructional framing corresponding to promotion and prevention in a within-subject design. We observed that the more promotion oriented an individual was, the lower their ventral striatum response to gain cues. Follow-up analyses revealed that greater promotion orientation was associated with decreased ventral striatum response even to no-value cues, suggesting that promotion orientation may be associated with relatively hypoactive reward system function. The findings are also likely to represent an interaction between the cognitive and motivational characteristics of the promotion system with the task demands. Prevention orientation did not correlate with ventral striatum response to gain cues, supporting the discriminant validity of regulatory focus theory. The results highlight a dynamic association between individual differences in self-regulation and reward system function.

Genetic influences on functional connectivity associated with feedback processing and prediction error: Phase coupling of theta-band oscillations in twins.

Science.gov (United States)

Demiral, Şükrü Barış; Golosheykin, Simon; Anokhin, Andrey P

2017-05-01

Detection and evaluation of the mismatch between the intended and actually obtained result of an action (reward prediction error) is an integral component of adaptive self-regulation of behavior. Extensive human and animal research has shown that evaluation of action outcome is supported by a distributed network of brain regions in which the anterior cingulate cortex (ACC) plays a central role, and the integration of distant brain regions into a unified feedback-processing network is enabled by long-range phase synchronization of cortical oscillations in the theta band. Neural correlates of feedback processing are associated with individual differences in normal and abnormal behavior, however, little is known about the role of genetic factors in the cerebral mechanisms of feedback processing. Here we examined genetic influences on functional cortical connectivity related to prediction error in young adult twins (age 18, n=399) using event-related EEG phase coherence analysis in a monetary gambling task. To identify prediction error-specific connectivity pattern, we compared responses to loss and gain feedback. Monetary loss produced a significant increase of theta-band synchronization between the frontal midline region and widespread areas of the scalp, particularly parietal areas, whereas gain resulted in increased synchrony primarily within the posterior regions. Genetic analyses showed significant heritability of frontoparietal theta phase synchronization (24 to 46%), suggesting that individual differences in large-scale network dynamics are under substantial genetic control. We conclude that theta-band synchronization of brain oscillations related to negative feedback reflects genetically transmitted differences in the neural mechanisms of feedback processing. To our knowledge, this is the first evidence for genetic influences on task-related functional brain connectivity assessed using direct real-time measures of neuronal synchronization. Copyright © 2016

Hippocampus, delay discounting, and vicarious trial-and-error.

Science.gov (United States)

Bett, David; Murdoch, Lauren H; Wood, Emma R; Dudchenko, Paul A

2015-05-01

In decision-making, an immediate reward is usually preferred to a delayed reward, even if the latter is larger. We tested whether the hippocampus is necessary for this form of temporal discounting, and for vicarious trial-and-error at the decision point. Rats were trained on a recently developed, adjustable delay-discounting task (Papale et al. (2012) Cogn Affect Behav Neurosci 12:513-526), which featured a choice between a small, nearly immediate reward, and a larger, delayed reward. Rats then received either hippocampus or sham lesions. Animals with hippocampus lesions adjusted the delay for the larger reward to a level similar to that of sham-lesioned animals, suggesting a similar valuation capacity. However, the hippocampus lesion group spent significantly longer investigating the small and large rewards in the first part of the sessions, and were less sensitive to changes in the amount of reward in the large reward maze arm. Both sham- and hippocampus-lesioned rats showed a greater amount of vicarious trial-and-error on trials in which the delay was adjusted. In a nonadjusting version of the delay discounting task, animals with hippocampus lesions showed more variability in their preference for a larger reward that was delayed by 10 s compared with sham-lesioned animals. To verify the lesion behaviorally, rat were subsequently trained on a water maze task, and rats with hippocampus lesions were significantly impaired compared with sham-lesioned animals. The findings on the delay discounting tasks suggest that damage to the hippocampus may impair the detection of reward magnitude. © 2014 Wiley Periodicals, Inc.

Estimation of Separation Buffers for Wind-Prediction Error in an Airborne Separation Assistance System

Science.gov (United States)

Consiglio, Maria C.; Hoadley, Sherwood T.; Allen, B. Danette

2009-01-01

Wind prediction errors are known to affect the performance of automated air traffic management tools that rely on aircraft trajectory predictions. In particular, automated separation assurance tools, planned as part of the NextGen concept of operations, must be designed to account and compensate for the impact of wind prediction errors and other system uncertainties. In this paper we describe a high fidelity batch simulation study designed to estimate the separation distance required to compensate for the effects of wind-prediction errors throughout increasing traffic density on an airborne separation assistance system. These experimental runs are part of the Safety Performance of Airborne Separation experiment suite that examines the safety implications of prediction errors and system uncertainties on airborne separation assurance systems. In this experiment, wind-prediction errors were varied between zero and forty knots while traffic density was increased several times current traffic levels. In order to accurately measure the full unmitigated impact of wind-prediction errors, no uncertainty buffers were added to the separation minima. The goal of the study was to measure the impact of wind-prediction errors in order to estimate the additional separation buffers necessary to preserve separation and to provide a baseline for future analyses. Buffer estimations from this study will be used and verified in upcoming safety evaluation experiments under similar simulation conditions. Results suggest that the strategic airborne separation functions exercised in this experiment can sustain wind prediction errors up to 40kts at current day air traffic density with no additional separation distance buffer and at eight times the current day with no more than a 60% increase in separation distance buffer.

Prediction Error During Functional and Non-Functional Action Sequences

DEFF Research Database (Denmark)

Nielbo, Kristoffer Laigaard; Sørensen, Jesper

2013-01-01

recurrent networks were made and the results are presented in this article. The simulations show that non-functional action sequences do indeed increase prediction error, but that context representations, such as abstract goal information, can modulate the error signal considerably. It is also shown...... that the networks are sensitive to boundaries between sequences in both functional and non-functional actions....

Seasonal prediction of Indian summer monsoon rainfall in NCEP CFSv2: forecast and predictability error

Science.gov (United States)

Pokhrel, Samir; Saha, Subodh Kumar; Dhakate, Ashish; Rahman, Hasibur; Chaudhari, Hemantkumar S.; Salunke, Kiran; Hazra, Anupam; Sujith, K.; Sikka, D. R.

2016-04-01

A detailed analysis of sensitivity to the initial condition for the simulation of the Indian summer monsoon using retrospective forecast by the latest version of the Climate Forecast System version-2 (CFSv2) is carried out. This study primarily focuses on the tropical region of Indian and Pacific Ocean basin, with special emphasis on the Indian land region. The simulated seasonal mean and the inter-annual standard deviations of rainfall, upper and lower level atmospheric circulations and Sea Surface Temperature (SST) tend to be more skillful as the lead forecast time decreases (5 month lead to 0 month lead time i.e. L5-L0). In general spatial correlation (bias) increases (decreases) as forecast lead time decreases. This is further substantiated by their averaged value over the selected study regions over the Indian and Pacific Ocean basins. The tendency of increase (decrease) of model bias with increasing (decreasing) forecast lead time also indicates the dynamical drift of the model. Large scale lower level circulation (850 hPa) shows enhancement of anomalous westerlies (easterlies) over the tropical region of the Indian Ocean (Western Pacific Ocean), which indicates the enhancement of model error with the decrease in lead time. At the upper level circulation (200 hPa) biases in both tropical easterly jet and subtropical westerlies jet tend to decrease as the lead time decreases. Despite enhancement of the prediction skill, mean SST bias seems to be insensitive to the initialization. All these biases are significant and together they make CFSv2 vulnerable to seasonal uncertainties in all the lead times. Overall the zeroth lead (L0) seems to have the best skill, however, in case of Indian summer monsoon rainfall (ISMR), the 3 month lead forecast time (L3) has the maximum ISMR prediction skill. This is valid using different independent datasets, wherein these maximum skill scores are 0.64, 0.42 and 0.57 with respect to the Global Precipitation Climatology Project

Emphasizing the 'positive' in positive reinforcement: Using non-binary rewarding for training monkeys on cognitive tasks.

Science.gov (United States)

Fischer, Benjamin; Wegener, Detlef

2018-04-04

Non-human primates constitute an indispensable model system for studying higher brain func-tions at the neurophysiological level. Studies involving these animals elucidated the neuronal mechanisms of various cognitive and executive functions, such as visual attention, working memory, and decision-making. Positive Reinforcement Training (PRT) constitutes the gold standard for training animals on the cognitive tasks employed in these studies. In the laboratory, PRT is usually based on applying a liquid reward as the reinforcer to strengthen the desired be-havior, and absence of the reward if the animal's response was wrong. By trial-and-error, the monkey may adopt its behavior and successfully reduces the number of error trials, and eventu-ally learns even very sophisticated tasks. However, progress and success of the training strongly depend on reasonable error rates. If errors get too frequent, they may cause a drop in the ani-mal's motivation to cooperate, or its adaptation to high error rates and poor overall perfor-mance. We here introduce an alternative training regime to minimize errors and base the critical information for learning on graded rewarding. For every new task rule, the feedback to the ani-mal is provided by different amounts of reward to distinguish the desired, optimal behavior from less optimally behavior. We applied this regime in different situations during training of visual attention tasks, and analyzed behavioral performance and reaction times to evaluate its effec-tiveness. For both simple and complex behaviors, graded rewarding was found to constitute a powerful technique allowing for effective training without trade-off in accessible task difficulty or task performance.

Surprised at all the entropy: hippocampal, caudate and midbrain contributions to learning from prediction errors.

Directory of Open Access Journals (Sweden)

Anne-Marike Schiffer

Full Text Available Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.

Surprised at all the entropy: hippocampal, caudate and midbrain contributions to learning from prediction errors.

Science.gov (United States)

Schiffer, Anne-Marike; Ahlheim, Christiane; Wurm, Moritz F; Schubotz, Ricarda I

2012-01-01

Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.

The impact of a total reward system of work engagement

Directory of Open Access Journals (Sweden)

Crystal Hoole

2016-11-01

Research purpose: The overall purpose of this study was to explore the relationship between total rewards and work engagement in a South African context and to determine which reward categories predict work engagement. The study further endeavoured to determine whether gender and age had a moderating effect on the relationship between total rewards and engagement. Motivation for the study: Statistics report that less than 30% of all working people are optimally engaged in their work. Considering that individuals spend more than a third of their lives at work committing themselves emotionally, physically and psychologically – research indicates that employees are no longer satisfied with traditional reward systems and want to feel valued and appreciated. Research approach, design and method: In this quantitative, cross-sectional research design using a non-probability convenience and purposive sampling strategy, 318 questionnaires were collected and analysed from financial institutions in Gauteng in which opinions were sought on the importance of different types of rewards structures and preferences, and how engaged they are in their workplace. The 17-item UWES and Nienaber total reward preference model were the chosen measuring instruments. Main findings: A small statistically significant correlation (r = 0.25; p < 0.05; small effect was found between total rewards and work engagement, and 12% of the variance of work engagement was explained. Only performance and career management significantly predicted work engagement. Practical/Managerial implications: Although small, the significant correlation between total rewards and work engagement implies that total rewards are important motivators for employees in the workplace. Of the total rewards scales tested, only performance and career management significantly predicted work engagement, suggesting that more research is needed. Organisations seeking to implement total reward strategies should pay specific

Threat and error management for anesthesiologists: a predictive risk taxonomy

Science.gov (United States)

Ruskin, Keith J.; Stiegler, Marjorie P.; Park, Kellie; Guffey, Patrick; Kurup, Viji; Chidester, Thomas

2015-01-01

Purpose of review Patient care in the operating room is a dynamic interaction that requires cooperation among team members and reliance upon sophisticated technology. Most human factors research in medicine has been focused on analyzing errors and implementing system-wide changes to prevent them from recurring. We describe a set of techniques that has been used successfully by the aviation industry to analyze errors and adverse events and explain how these techniques can be applied to patient care. Recent findings Threat and error management (TEM) describes adverse events in terms of risks or challenges that are present in an operational environment (threats) and the actions of specific personnel that potentiate or exacerbate those threats (errors). TEM is a technique widely used in aviation, and can be adapted for the use in a medical setting to predict high-risk situations and prevent errors in the perioperative period. A threat taxonomy is a novel way of classifying and predicting the hazards that can occur in the operating room. TEM can be used to identify error-producing situations, analyze adverse events, and design training scenarios. Summary TEM offers a multifaceted strategy for identifying hazards, reducing errors, and training physicians. A threat taxonomy may improve analysis of critical events with subsequent development of specific interventions, and may also serve as a framework for training programs in risk mitigation. PMID:24113268

Cognitive emotion regulation enhances aversive prediction error activity while reducing emotional responses.

Science.gov (United States)

Mulej Bratec, Satja; Xie, Xiyao; Schmid, Gabriele; Doll, Anselm; Schilbach, Leonhard; Zimmer, Claus; Wohlschläger, Afra; Riedl, Valentin; Sorg, Christian

2015-12-01

Cognitive emotion regulation is a powerful way of modulating emotional responses. However, despite the vital role of emotions in learning, it is unknown whether the effect of cognitive emotion regulation also extends to the modulation of learning. Computational models indicate prediction error activity, typically observed in the striatum and ventral tegmental area, as a critical neural mechanism involved in associative learning. We used model-based fMRI during aversive conditioning with and without cognitive emotion regulation to test the hypothesis that emotion regulation would affect prediction error-related neural activity in the striatum and ventral tegmental area, reflecting an emotion regulation-related modulation of learning. Our results show that cognitive emotion regulation reduced emotion-related brain activity, but increased prediction error-related activity in a network involving ventral tegmental area, hippocampus, insula and ventral striatum. While the reduction of response activity was related to behavioral measures of emotion regulation success, the enhancement of prediction error-related neural activity was related to learning performance. Furthermore, functional connectivity between the ventral tegmental area and ventrolateral prefrontal cortex, an area involved in regulation, was specifically increased during emotion regulation and likewise related to learning performance. Our data, therefore, provide first-time evidence that beyond reducing emotional responses, cognitive emotion regulation affects learning by enhancing prediction error-related activity, potentially via tegmental dopaminergic pathways. Copyright © 2015 Elsevier Inc. All rights reserved.

BANKRUPTCY PREDICTION MODEL WITH ZETAc OPTIMAL CUT-OFF SCORE TO CORRECT TYPE I ERRORS

Directory of Open Access Journals (Sweden)

Mohamad Iwan

2005-06-01

This research has successfully attained the following results: (1 type I error is in fact 59,83 times more costly compared to type II error, (2 22 ratios distinguish between bankrupt and non-bankrupt groups, (3 2 financial ratios proved to be effective in predicting bankruptcy, (4 prediction using ZETAc optimal cut-off score predicts more companies filing for bankruptcy within one year compared to prediction using Hair et al. optimum cutting score, (5 Although prediction using Hair et al. optimum cutting score is more accurate, prediction using ZETAc optimal cut-off score proved to be able to minimize cost incurred from classification errors.

Distinct Reward Properties are Encoded via Corticostriatal Interactions.

Science.gov (United States)

Smith, David V; Rigney, Anastasia E; Delgado, Mauricio R

2016-02-02

The striatum serves as a critical brain region for reward processing. Yet, understanding the link between striatum and reward presents a challenge because rewards are composed of multiple properties. Notably, affective properties modulate emotion while informative properties help obtain future rewards. We approached this problem by emphasizing affective and informative reward properties within two independent guessing games. We found that both reward properties evoked activation within the nucleus accumbens, a subregion of the striatum. Striatal responses to informative, but not affective, reward properties predicted subsequent utilization of information for obtaining monetary reward. We hypothesized that activation of the striatum may be necessary but not sufficient to encode distinct reward properties. To investigate this possibility, we examined whether affective and informative reward properties were differentially encoded in corticostriatal interactions. Strikingly, we found that the striatum exhibited dissociable connectivity patterns with the ventrolateral prefrontal cortex, with increasing connectivity for affective reward properties and decreasing connectivity for informative reward properties. Our results demonstrate that affective and informative reward properties are encoded via corticostriatal interactions. These findings highlight how corticostriatal systems contribute to reward processing, potentially advancing models linking striatal activation to behavior.

'Proactive' use of cue-context congruence for building reinforcement learning's reward function.

Science.gov (United States)

Zsuga, Judit; Biro, Klara; Tajti, Gabor; Szilasi, Magdolna Emma; Papp, Csaba; Juhasz, Bela; Gesztelyi, Rudolf

2016-10-28

Reinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent's knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent's control either using, or not using a model. In the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively. Based on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.

Potential effects of reward and loss avoidance in overweight adolescents.

Science.gov (United States)

Reyes, Sussanne; Peirano, Patricio; Luna, Beatriz; Lozoff, Betsy; Algarín, Cecilia

2015-08-01

Reward system and inhibitory control are brain functions that exert an influence on eating behavior regulation. We studied the differences in inhibitory control and sensitivity to reward and loss avoidance between overweight/obese and normal-weight adolescents. We assessed 51 overweight/obese and 52 normal-weight 15-y-old Chilean adolescents. The groups were similar regarding sex and intelligence quotient. Using Antisaccade and Incentive tasks, we evaluated inhibitory control and the effect of incentive trials (neutral, loss avoidance, and reward) on generating correct and incorrect responses (latency and error rate). Compared to normal-weight group participants, overweight/obese adolescents showed shorter latency for incorrect antisaccade responses (186.0 (95% CI: 176.8-195.2) vs. 201.3 ms (95% CI: 191.2-211.5), P reward (41.0 (95% CI: 34.5-47.5) vs. 49.8% (95% CI: 43.0-55.1), P reward and loss avoidance trials. These findings could suggest that an imbalance of inhibition and reward systems influence their eating behavior.

Bio-robots automatic navigation with electrical reward stimulation.

Science.gov (United States)

Sun, Chao; Zhang, Xinlu; Zheng, Nenggan; Chen, Weidong; Zheng, Xiaoxiang

2012-01-01

Bio-robots that controlled by outer stimulation through brain computer interface (BCI) suffer from the dependence on realtime guidance of human operators. Current automatic navigation methods for bio-robots focus on the controlling rules to force animals to obey man-made commands, with animals' intelligence ignored. This paper proposes a new method to realize the automatic navigation for bio-robots with electrical micro-stimulation as real-time rewards. Due to the reward-seeking instinct and trial-and-error capability, bio-robot can be steered to keep walking along the right route with rewards and correct its direction spontaneously when rewards are deprived. In navigation experiments, rat-robots learn the controlling methods in short time. The results show that our method simplifies the controlling logic and realizes the automatic navigation for rat-robots successfully. Our work might have significant implication for the further development of bio-robots with hybrid intelligence.

Measurement error and timing of predictor values for multivariable risk prediction models are poorly reported.

Science.gov (United States)

Whittle, Rebecca; Peat, George; Belcher, John; Collins, Gary S; Riley, Richard D

2018-05-18

Measurement error in predictor variables may threaten the validity of clinical prediction models. We sought to evaluate the possible extent of the problem. A secondary objective was to examine whether predictors are measured at the intended moment of model use. A systematic search of Medline was used to identify a sample of articles reporting the development of a clinical prediction model published in 2015. After screening according to a predefined inclusion criteria, information on predictors, strategies to control for measurement error and intended moment of model use were extracted. Susceptibility to measurement error for each predictor was classified into low and high risk. Thirty-three studies were reviewed, including 151 different predictors in the final prediction models. Fifty-one (33.7%) predictors were categorised as high risk of error, however this was not accounted for in the model development. Only 8 (24.2%) studies explicitly stated the intended moment of model use and when the predictors were measured. Reporting of measurement error and intended moment of model use is poor in prediction model studies. There is a need to identify circumstances where ignoring measurement error in prediction models is consequential and whether accounting for the error will improve the predictions. Copyright © 2018. Published by Elsevier Inc.

"Liking" and "wanting" linked to Reward Deficiency Syndrome (RDS): hypothesizing differential responsivity in brain reward circuitry.

Science.gov (United States)

Blum, Kenneth; Gardner, Eliot; Oscar-Berman, Marlene; Gold, Mark

2012-01-01

In an attempt to resolve controversy regarding the causal contributions of mesolimbic dopamine (DA) systems to reward, we evaluate the three main competing explanatory categories: "liking,"learning," and "wanting" [1]. That is, DA may mediate (a) the hedonic impact of reward (liking), (b) learned predictions about rewarding effects (learning), or (c) the pursuit of rewards by attributing incentive salience to reward-related stimuli (wanting). We evaluate these hypotheses, especially as they relate to the Reward Deficiency Syndrome (RDS), and we find that the incentive salience or "wanting" hypothesis of DA function is supported by a majority of the evidence. Neuroimaging studies have shown that drugs of abuse, palatable foods, and anticipated behaviors such as sex and gaming affect brain regions involving reward circuitry, and may not be unidirectional. Drugs of abuse enhance DA signaling and sensitize mesolimbic mechanisms that evolved to attribute incentive salience to rewards. Addictive drugs have in common that they are voluntarily selfadministered, they enhance (directly or indirectly) dopaminergic synaptic function in the nucleus accumbens (NAC), and they stimulate the functioning of brain reward circuitry (producing the "high" that drug users seek). Although originally believed simply to encode the set point of hedonic tone, these circuits now are believed to be functionally more complex, also encoding attention, reward expectancy, disconfirmation of reward expectancy, and incentive motivation. Elevated stress levels, together with polymorphisms of dopaminergic genes and other neurotransmitter genetic variants, may have a cumulative effect on vulnerability to addiction. The RDS model of etiology holds very well for a variety of chemical and behavioral addictions.

Neural responses to threat and reward interact to predict stress-related problem drinking: A novel protective role of the amygdala

Science.gov (United States)

2012-01-01

Background Research into neural mechanisms of drug abuse risk has focused on the role of dysfunction in neural circuits for reward. In contrast, few studies have examined the role of dysfunction in neural circuits of threat in mediating drug abuse risk. Although typically regarded as a risk factor for mood and anxiety disorders, threat-related amygdala reactivity may serve as a protective factor against substance use disorders, particularly in individuals with exaggerated responsiveness to reward. Findings We used well-established neuroimaging paradigms to probe threat-related amygdala and reward-related ventral striatum reactivity in a sample of 200 young adult students from the ongoing Duke Neurogenetics Study. Recent life stress and problem drinking were assessed using self-report. We found a significant three-way interaction between threat-related amygdala reactivity, reward-related ventral striatum reactivity, and recent stress, wherein individuals with higher reward-related ventral striatum reactivity exhibit higher levels of problem drinking in the context of stress, but only if they also have lower threat-related amygdala reactivity. This three-way interaction predicted both contemporaneous problem drinking and problem drinking reported three-months later in a subset of participants. Conclusions These findings suggest complex interactions between stress and neural responsiveness to both threat and reward mediate problem drinking. Furthermore, they highlight a novel protective role for threat-related amygdala reactivity against drug use in individuals with high neural reactivity to reward. PMID:23151390

Measurement Error Correction for Predicted Spatiotemporal Air Pollution Exposures.

Science.gov (United States)

Keller, Joshua P; Chang, Howard H; Strickland, Matthew J; Szpiro, Adam A

2017-05-01

Air pollution cohort studies are frequently analyzed in two stages, first modeling exposure then using predicted exposures to estimate health effects in a second regression model. The difference between predicted and unobserved true exposures introduces a form of measurement error in the second stage health model. Recent methods for spatial data correct for measurement error with a bootstrap and by requiring the study design ensure spatial compatibility, that is, monitor and subject locations are drawn from the same spatial distribution. These methods have not previously been applied to spatiotemporal exposure data. We analyzed the association between fine particulate matter (PM2.5) and birth weight in the US state of Georgia using records with estimated date of conception during 2002-2005 (n = 403,881). We predicted trimester-specific PM2.5 exposure using a complex spatiotemporal exposure model. To improve spatial compatibility, we restricted to mothers residing in counties with a PM2.5 monitor (n = 180,440). We accounted for additional measurement error via a nonparametric bootstrap. Third trimester PM2.5 exposure was associated with lower birth weight in the uncorrected (-2.4 g per 1 μg/m difference in exposure; 95% confidence interval [CI]: -3.9, -0.8) and bootstrap-corrected (-2.5 g, 95% CI: -4.2, -0.8) analyses. Results for the unrestricted analysis were attenuated (-0.66 g, 95% CI: -1.7, 0.35). This study presents a novel application of measurement error correction for spatiotemporal air pollution exposures. Our results demonstrate the importance of spatial compatibility between monitor and subject locations and provide evidence of the association between air pollution exposure and birth weight.

Reward System Activation in Response to Alcohol Advertisements Predicts College Drinking.

Science.gov (United States)

Courtney, Andrea L; Rapuano, Kristina M; Sargent, James D; Heatherton, Todd F; Kelley, William M

2018-01-01

In this study, we assess whether activation of the brain's reward system in response to alcohol advertisements is associated with college drinking. Previous research has established a relationship between exposure to alcohol marketing and underage drinking. Within other appetitive domains, the relationship between cue exposure and behavioral enactment is known to rely on activation of the brain's reward system. However, the relationship between neural activation to alcohol advertisements and alcohol consumption has not been studied in a nondisordered population. In this cross-sectional study, 53 college students (32 women) completed a functional magnetic resonance imaging scan while viewing alcohol, food, and control (car and technology) advertisements. Afterward, they completed a survey about their alcohol consumption (including frequency of drinking, typical number of drinks consumed, and frequency of binge drinking) over the previous month. In 43 participants (24 women) meeting inclusion criteria, viewing alcohol advertisements elicited activation in the left orbitofrontal cortex and bilateral ventral striatum-regions of the reward system that typically activate to other appetitive rewards and relate to consumption behaviors. Moreover, the level of self-reported drinking correlated with the magnitude of activation in the left orbitofrontal cortex. Results suggest that alcohol cues are processed within the reward system in a way that may motivate drinking behavior.

Sensitivity for cues predicting reward and punishment in young women with eating disorders

NARCIS (Netherlands)

Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline

Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely

Sensitivity for cues predicting reward and punishment in young women with eating disorders

NARCIS (Netherlands)

Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline

2017-01-01

Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely

Reward and Cognition: Integrating Reinforcement Sensitivity Theory and Social Cognitive Theory to Predict Drinking Behavior.

Science.gov (United States)

Hasking, Penelope; Boyes, Mark; Mullan, Barbara

2015-01-01

Both Reinforcement Sensitivity Theory and Social Cognitive Theory have been applied to understanding drinking behavior. We propose that theoretical relationships between these models support an integrated approach to understanding alcohol use and misuse. We aimed to test an integrated model in which the relationships between reward sensitivity and drinking behavior (alcohol consumption, alcohol-related problems, and symptoms of dependence) were mediated by alcohol expectancies and drinking refusal self-efficacy. Online questionnaires assessing the constructs of interest were completed by 443 Australian adults (M age = 26.40, sd = 1.83) in 2013 and 2014. Path analysis revealed both direct and indirect effects and implicated two pathways to drinking behavior with differential outcomes. Drinking refusal self-efficacy both in social situations and for emotional relief was related to alcohol consumption. Sensitivity to reward was associated with alcohol-related problems, but operated through expectations of increased confidence and personal belief in the ability to limit drinking in social situations. Conversely, sensitivity to punishment operated through negative expectancies and drinking refusal self-efficacy for emotional relief to predict symptoms of dependence. Two pathways relating reward sensitivity, alcohol expectancies, and drinking refusal self-efficacy may underlie social and dependent drinking, which has implications for development of intervention to limit harmful drinking.

Error analysis of short term wind power prediction models

International Nuclear Information System (INIS)

De Giorgi, Maria Grazia; Ficarella, Antonio; Tarantino, Marco

2011-01-01

The integration of wind farms in power networks has become an important problem. This is because the electricity produced cannot be preserved because of the high cost of storage and electricity production must follow market demand. Short-long-range wind forecasting over different lengths/periods of time is becoming an important process for the management of wind farms. Time series modelling of wind speeds is based upon the valid assumption that all the causative factors are implicitly accounted for in the sequence of occurrence of the process itself. Hence time series modelling is equivalent to physical modelling. Auto Regressive Moving Average (ARMA) models, which perform a linear mapping between inputs and outputs, and Artificial Neural Networks (ANNs) and Adaptive Neuro-Fuzzy Inference Systems (ANFIS), which perform a non-linear mapping, provide a robust approach to wind power prediction. In this work, these models are developed in order to forecast power production of a wind farm with three wind turbines, using real load data and comparing different time prediction periods. This comparative analysis takes in the first time, various forecasting methods, time horizons and a deep performance analysis focused upon the normalised mean error and the statistical distribution hereof in order to evaluate error distribution within a narrower curve and therefore forecasting methods whereby it is more improbable to make errors in prediction. (author)

Error analysis of short term wind power prediction models

Energy Technology Data Exchange (ETDEWEB)

De Giorgi, Maria Grazia; Ficarella, Antonio; Tarantino, Marco [Dipartimento di Ingegneria dell' Innovazione, Universita del Salento, Via per Monteroni, 73100 Lecce (Italy)

2011-04-15

The integration of wind farms in power networks has become an important problem. This is because the electricity produced cannot be preserved because of the high cost of storage and electricity production must follow market demand. Short-long-range wind forecasting over different lengths/periods of time is becoming an important process for the management of wind farms. Time series modelling of wind speeds is based upon the valid assumption that all the causative factors are implicitly accounted for in the sequence of occurrence of the process itself. Hence time series modelling is equivalent to physical modelling. Auto Regressive Moving Average (ARMA) models, which perform a linear mapping between inputs and outputs, and Artificial Neural Networks (ANNs) and Adaptive Neuro-Fuzzy Inference Systems (ANFIS), which perform a non-linear mapping, provide a robust approach to wind power prediction. In this work, these models are developed in order to forecast power production of a wind farm with three wind turbines, using real load data and comparing different time prediction periods. This comparative analysis takes in the first time, various forecasting methods, time horizons and a deep performance analysis focused upon the normalised mean error and the statistical distribution hereof in order to evaluate error distribution within a narrower curve and therefore forecasting methods whereby it is more improbable to make errors in prediction. (author)

Mean Bias in Seasonal Forecast Model and ENSO Prediction Error.

Science.gov (United States)

Kim, Seon Tae; Jeong, Hye-In; Jin, Fei-Fei

2017-07-20

This study uses retrospective forecasts made using an APEC Climate Center seasonal forecast model to investigate the cause of errors in predicting the amplitude of El Niño Southern Oscillation (ENSO)-driven sea surface temperature variability. When utilizing Bjerknes coupled stability (BJ) index analysis, enhanced errors in ENSO amplitude with forecast lead times are found to be well represented by those in the growth rate estimated by the BJ index. ENSO amplitude forecast errors are most strongly associated with the errors in both the thermocline slope response and surface wind response to forcing over the tropical Pacific, leading to errors in thermocline feedback. This study concludes that upper ocean temperature bias in the equatorial Pacific, which becomes more intense with increasing lead times, is a possible cause of forecast errors in the thermocline feedback and thus in ENSO amplitude.

It's about time: Earlier rewards increase intrinsic motivation.

Science.gov (United States)

Woolley, Kaitlin; Fishbach, Ayelet

2018-06-01

Can immediate (vs. delayed) rewards increase intrinsic motivation? Prior research compared the presence versus absence of rewards. By contrast, this research compared immediate versus delayed rewards, predicting that more immediate rewards increase intrinsic motivation by creating a perceptual fusion between the activity and its goal (i.e., the reward). In support of the hypothesis, framing a reward from watching a news program as more immediate (vs. delayed) increased intrinsic motivation to watch the program (Study 1), and receiving more immediate bonus (vs. delayed, Study 2; and vs. delayed and no bonus, Study 3) increased intrinsic motivation in an experimental task. The effect of reward timing was mediated by the strength of the association between an activity and a reward, and was specific to intrinsic (vs. extrinsic) motivation-immediacy influenced the positive experience of an activity, but not perceived outcome importance (Study 4). In addition, the effect of the timing of rewards was independent of the effect of the magnitude of the rewards (Study 5). (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Post-learning hippocampal dynamics promote preferential retention of rewarding events

Science.gov (United States)

Gruber, Matthias J.; Ritchey, Maureen; Wang, Shao-Fang; Doss, Manoj K.; Ranganath, Charan

2016-01-01

Reward motivation is known to modulate memory encoding, and this effect depends on interactions between the substantia nigra/ ventral tegmental area complex (SN/VTA) and the hippocampus. It is unknown, however, whether these interactions influence offline neural activity in the human brain that is thought to promote memory consolidation. Here, we used functional magnetic resonance imaging (fMRI) to test the effect of reward motivation on post-learning neural dynamics and subsequent memory for objects that were learned in high- or low-reward motivation contexts. We found that post-learning increases in resting-state functional connectivity between the SN/VTA and hippocampus predicted preferential retention of objects that were learned in high-reward contexts. In addition, multivariate pattern classification revealed that hippocampal representations of high-reward contexts were preferentially reactivated during post-learning rest, and the number of hippocampal reactivations was predictive of preferential retention of items learned in high-reward contexts. These findings indicate that reward motivation alters offline post-learning dynamics between the SN/VTA and hippocampus, providing novel evidence for a potential mechanism by which reward could influence memory consolidation. PMID:26875624

Frontal-striatum dysfunction during reward processing: Relationships to amotivation in schizophrenia.

Science.gov (United States)

Chung, Yu Sun; Barch, Deanna M

2016-04-01

Schizophrenia is characterized by deficits of context processing, thought to be related to dorsolateral prefrontal cortex (DLPFC) impairment. Despite emerging evidence suggesting a crucial role of the DLPFC in integrating reward and goal information, we do not know whether individuals with schizophrenia can represent and integrate reward-related context information to modulate cognitive control. To address this question, 36 individuals with schizophrenia (n = 29) or schizoaffective disorder (n = 7) and 27 healthy controls performed a variant of a response conflict task (Padmala & Pessoa, 2011) during fMRI scanning, in both baseline and reward conditions, with monetary incentives on some reward trials. We used a mixed state-item design that allowed us to examine both sustained and transient reward effects on cognitive control. Different from predictions about impaired DLPFC function in schizophrenia, we found an intact pattern of increased sustained DLPFC activity during reward versus baseline blocks in individuals with schizophrenia at a group level but blunted sustained activations in the putamen. Contrary to our predictions, individuals with schizophrenia showed blunted cue-related activations in several regions of the basal ganglia responding to reward-predicting cues. Importantly, as predicted, individual differences in anhedonia/amotivation symptoms severity were significantly associated with reduced sustained DLPFC activation in the same region that showed overall increased activity as a function of reward. These results suggest that individual differences in motivational impairments in schizophrenia may be related to dysfunction of the DLPFC and striatum in motivationally salient situations. (c) 2016 APA, all rights reserved).

Testing the prediction error difference between two predictors

NARCIS (Netherlands)

van de Wiel, M.A.; Berkhof, J.; van Wieringen, W.N.

2009-01-01

We develop an inference framework for the difference in errors between 2 prediction procedures. The 2 procedures may differ in any aspect and possibly utilize different sets of covariates. We apply training and testing on the same data set, which is accommodated by sample splitting. For each split,

Model parameter-related optimal perturbations and their contributions to El Niño prediction errors

Science.gov (United States)

Tao, Ling-Jiang; Gao, Chuan; Zhang, Rong-Hua

2018-04-01

Errors in initial conditions and model parameters (MPs) are the main sources that limit the accuracy of ENSO predictions. In addition to exploring the initial error-induced prediction errors, model errors are equally important in determining prediction performance. In this paper, the MP-related optimal errors that can cause prominent error growth in ENSO predictions are investigated using an intermediate coupled model (ICM) and a conditional nonlinear optimal perturbation (CNOP) approach. Two MPs related to the Bjerknes feedback are considered in the CNOP analysis: one involves the SST-surface wind coupling ({α _τ } ), and the other involves the thermocline effect on the SST ({α _{Te}} ). The MP-related optimal perturbations (denoted as CNOP-P) are found uniformly positive and restrained in a small region: the {α _τ } component is mainly concentrated in the central equatorial Pacific, and the {α _{Te}} component is mainly located in the eastern cold tongue region. This kind of CNOP-P enhances the strength of the Bjerknes feedback and induces an El Niño- or La Niña-like error evolution, resulting in an El Niño-like systematic bias in this model. The CNOP-P is also found to play a role in the spring predictability barrier (SPB) for ENSO predictions. Evidently, such error growth is primarily attributed to MP errors in small areas based on the localized distribution of CNOP-P. Further sensitivity experiments firmly indicate that ENSO simulations are sensitive to the representation of SST-surface wind coupling in the central Pacific and to the thermocline effect in the eastern Pacific in the ICM. These results provide guidance and theoretical support for the future improvement in numerical models to reduce the systematic bias and SPB phenomenon in ENSO predictions.

Reward-dependent learning in neuronal networks for planning and decision making.

Science.gov (United States)

Dehaene, S; Changeux, J P

2000-01-01

gives access to enhanced rates of learning via an elementary process of internal or covert mental simulation. We have recently applied these ideas to a new model, developed with M. Kerszberg, which hypothesizes that prefrontal cortex and its reward-related connections contribute crucially to conscious effortful tasks. This model distinguishes two main computational spaces within the human brain: a unique global workspace composed of distributed and heavily interconnected neurons with long-range axons, and a set of specialized and modular perceptual, motor, memory, evaluative and attentional processors. We postulate that workspace neurons are mobilized in effortful tasks for which the specialized processors do not suffice; they selectively mobilize or suppress, through descending connections, the contribution of specific processor neurons. In the course of task performance, workspace neurons become spontaneously co-activated, forming discrete though variable spatio-temporal patterns subject to modulation by vigilance signals and to selection by reward signals. A computer simulation of the Stroop task shows workspace activation to increase during acquisition of a novel task, effortful execution, and after errors. This model makes predictions concerning the spatio-temporal activation patterns during brain imaging of cognitive tasks, particularly concerning the conditions of activation of dorsolateral prefrontal cortex and anterior cingulate, their relation to reward mechanisms, and their specific reaction during error processing.

Motivational state controls the prediction error in Pavlovian appetitive-aversive interactions.

Science.gov (United States)

Laurent, Vincent; Balleine, Bernard W; Westbrook, R Frederick

2018-01-01

Contemporary theories of learning emphasize the role of a prediction error signal in driving learning, but the nature of this signal remains hotly debated. Here, we used Pavlovian conditioning in rats to investigate whether primary motivational and emotional states interact to control prediction error. We initially generated cues that positively or negatively predicted an appetitive food outcome. We then assessed how these cues modulated aversive conditioning when a novel cue was paired with a foot shock. We found that a positive predictor of food enhances, whereas a negative predictor of that same food impairs, aversive conditioning. Critically, we also showed that the enhancement produced by the positive predictor is removed by reducing the value of its associated food. In contrast, the impairment triggered by the negative predictor remains insensitive to devaluation of its associated food. These findings provide compelling evidence that the motivational value attributed to a predicted food outcome can directly control appetitive-aversive interactions and, therefore, that motivational processes can modulate emotional processes to generate the final error term on which subsequent learning is based. Copyright © 2017 Elsevier Inc. All rights reserved.

Advanced error-prediction LDPC with temperature compensation for highly reliable SSDs

Science.gov (United States)

Tokutomi, Tsukasa; Tanakamaru, Shuhei; Iwasaki, Tomoko Ogura; Takeuchi, Ken

2015-09-01

To improve the reliability of NAND Flash memory based solid-state drives (SSDs), error-prediction LDPC (EP-LDPC) has been proposed for multi-level-cell (MLC) NAND Flash memory (Tanakamaru et al., 2012, 2013), which is effective for long retention times. However, EP-LDPC is not as effective for triple-level cell (TLC) NAND Flash memory, because TLC NAND Flash has higher error rates and is more sensitive to program-disturb error. Therefore, advanced error-prediction LDPC (AEP-LDPC) has been proposed for TLC NAND Flash memory (Tokutomi et al., 2014). AEP-LDPC can correct errors more accurately by precisely describing the error phenomena. In this paper, the effects of AEP-LDPC are investigated in a 2×nm TLC NAND Flash memory with temperature characterization. Compared with LDPC-with-BER-only, the SSD's data-retention time is increased by 3.4× and 9.5× at room-temperature (RT) and 85 °C, respectively. Similarly, the acceptable BER is increased by 1.8× and 2.3×, respectively. Moreover, AEP-LDPC can correct errors with pre-determined tables made at higher temperatures to shorten the measurement time before shipping. Furthermore, it is found that one table can cover behavior over a range of temperatures in AEP-LDPC. As a result, the total table size can be reduced to 777 kBytes, which makes this approach more practical.

Learning Reward Uncertainty in the Basal Ganglia.

Directory of Open Access Journals (Sweden)

John G Mikhael

2016-09-01

Full Text Available Learning the reliability of different sources of rewards is critical for making optimal choices. However, despite the existence of detailed theory describing how the expected reward is learned in the basal ganglia, it is not known how reward uncertainty is estimated in these circuits. This paper presents a class of models that encode both the mean reward and the spread of the rewards, the former in the difference between the synaptic weights of D1 and D2 neurons, and the latter in their sum. In the models, the tendency to seek (or avoid options with variable reward can be controlled by increasing (or decreasing the tonic level of dopamine. The models are consistent with the physiology of and synaptic plasticity in the basal ganglia, they explain the effects of dopaminergic manipulations on choices involving risks, and they make multiple experimental predictions.

Differing Air Traffic Controller Responses to Similar Trajectory Prediction Errors

Science.gov (United States)

Mercer, Joey; Hunt-Espinosa, Sarah; Bienert, Nancy; Laraway, Sean

2016-01-01

A Human-In-The-Loop simulation was conducted in January of 2013 in the Airspace Operations Laboratory at NASA's Ames Research Center. The simulation airspace included two en route sectors feeding the northwest corner of Atlanta's Terminal Radar Approach Control. The focus of this paper is on how uncertainties in the study's trajectory predictions impacted the controllers ability to perform their duties. Of particular interest is how the controllers interacted with the delay information displayed in the meter list and data block while managing the arrival flows. Due to wind forecasts with 30-knot over-predictions and 30-knot under-predictions, delay value computations included errors of similar magnitude, albeit in opposite directions. However, when performing their duties in the presence of these errors, did the controllers issue clearances of similar magnitude, albeit in opposite directions?

Impulsive responding in threat and reward contexts as a function of PTSD symptoms and trait disinhibition.

Science.gov (United States)

Sadeh, Naomi; Spielberg, Jeffrey M; Hayes, Jasmeet P

2018-01-01

We examined current posttraumatic stress disorder (PTSD) symptoms, trait disinhibition, and affective context as contributors to impulsive and self-destructive behavior in 94 trauma-exposed Veterans. Participants completed an affective Go/No-Go task (GNG) with different emotional contexts (threat, reward, and a multidimensional threat/reward condition) and current PTSD, trait disinhibition, and risky/self-destructive behavior measures. PTSD interacted with trait disinhibition to explain recent engagement in risky/self-destructive behavior, with Veterans scoring high on trait disinhibition and current PTSD symptoms reporting the highest levels of these behaviors. On the GNG task, commission errors were also associated with the interaction of PTSD symptoms and trait disinhibition. Specifically, PTSD symptoms were associated with greater commission errors in threat vs. reward contexts for individuals who were low on trait disinhibition. In contrast, veterans high on PTSD and trait disinhibition exhibited the greatest number of commission errors in the multidimensional affective context that involved both threat and reward processing. Results highlight the interactive effects of PTSD and disinhibited personality traits, as well as threat and reward systems, as risk factors for impulsive and self-destructive behavior in trauma-exposed groups. Findings have clinical implications for understanding heterogeneity in the expression of PTSD and its association with disinhibited behavior. Copyright © 2017 Elsevier Ltd. All rights reserved.

Authoritarian parenting predicts reduced electrocortical response to observed adolescent offspring rewards

Science.gov (United States)

Speed, Brittany C.; Nelson, Brady; Bress, Jennifer N.; Hajcak, Greg

2017-01-01

Abstract Parenting styles are robust predictors of offspring outcomes, yet little is known about their neural underpinnings. In this study, 44 parent-adolescent dyads (Mage of adolescent = 12.9) completed a laboratory guessing task while EEG was continuously recorded. In the task, each pair member received feedback about their own monetary wins and losses and also observed the monetary wins and losses of the other member of the pair. We examined the association between self-reported parenting style and parents’ electrophysiological responses to watching their adolescent winning and losing money, dubbed the observational Reward Positivity (RewP) and observational feedback negativity (FN), respectively. Self-reported authoritarian parenting predicted reductions in parents’ observational RewP but not FN. This predictive relationship remained after adjusting for sex of both participants, parents’ responsiveness to their own wins, and parental psychopathology. ‘Exploratory analyses found that permissive parenting was associated with a blunting of the adolescents’ response to their parents’ losses’. These findings suggest that parents’ rapid neural responses to their child’s successes may relate to the harsh parenting behaviors associated with authoritarian parenting. PMID:27613780

Reward-Enhanced Memory in Younger and Older Adults

OpenAIRE

Julia Spaniol; Cécile Schain; Holly J. Bowen

2014-01-01

Objectives. We investigated how the anticipation of remote monetary reward modulates intentional episodic memory formation in younger and older adults. On the basis of prior findings of preserved reward–cognition interactions in aging, we predicted that reward anticipation would be associated with enhanced memory in both younger and older adults. On the basis of previous demonstrations of a time-dependent effect of reward anticipation on memory, we expected the memory enhancement to increase ...

Do Substantia Nigra Dopaminergic Neurons Differentiate Between Reward and Punishment?

Institute of Scientific and Technical Information of China (English)

Michael J. Frank; D. James Surmeier

2009-01-01

The activity of dopaminergic neurons are thought to be increased by stimuli that predict reward and decreased by stimuli that predict aversive outcomes. Recent work by Matsumoto and Hikosaka challenges this model by asserting that stimuli associated with either rewarding or aversive outcomes increase the activity of dopaminergic neurons in the substantia nigra pars compacta.

Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error.

Science.gov (United States)

Faber, Felix A; Hutchison, Luke; Huang, Bing; Gilmer, Justin; Schoenholz, Samuel S; Dahl, George E; Vinyals, Oriol; Kearnes, Steven; Riley, Patrick F; von Lilienfeld, O Anatole

2017-11-14

evidence that ML model predictions deviate from DFT (B3LYP) less than DFT (B3LYP) deviates from experiment for all properties. Furthermore, out-of-sample prediction errors with respect to hybrid DFT reference are on par with, or close to, chemical accuracy. The results suggest that ML models could be more accurate than hybrid DFT if explicitly electron correlated quantum (or experimental) data were available.

Development and performance evaluation of a prototype system for prediction of the group error at the maintenance work

International Nuclear Information System (INIS)

Yoshino, Kenji; Hirotsu, Yuko

2000-01-01

In order to attain zero-izing of much more error rather than it can set to a nuclear power plant, Authors development and its system-izing of the error prediction causal model which predicts group error action at the time of maintenance work were performed. This prototype system has the following feature. (1) When a user inputs the existence and the grade of the existence of the 'feature factor of the maintenance work' as a prediction object, 'an organization and an organization factor', and a 'group PSF (Performance Shaping Factor) factor' into this system. The maintenance group error to target can be predicted through the prediction model which consists of a class of seven stages. (2) This system by utilizing the information on a prediction result database, it can use not only for prediction of a maintenance group error but for various safe activity, such as KYT (dangerous forecast training) and TBM (Tool Box Meeting). (3) This system predicts a cooperation error' at highest rate, and, subsequently predicts the detection error' at a high rate. And to the 'decision-making error', the transfer error' and the 'state cognitive error', it has the characteristic predicted at almost same rate. (4) If it has full knowledge even of the features, such as the enforcement conditions of maintenance work, and organization, even if the user has neither the knowledge about a human factor, nor experience, anyone of this system is slight about the existence, its extent, etc. of generating of a maintenance group error made difficult from the former logically and systematically easily, it can predict in business time for about 15 minutes. (author)

Period, epoch, and prediction errors of ephemerides from continuous sets of timing measurements

Science.gov (United States)

Deeg, H. J.

2015-06-01

Space missions such as Kepler and CoRoT have led to large numbers of eclipse or transit measurements in nearly continuous time series. This paper shows how to obtain the period error in such measurements from a basic linear least-squares fit, and how to correctly derive the timing error in the prediction of future transit or eclipse events. Assuming strict periodicity, a formula for the period error of these time series is derived, σP = σT (12 / (N3-N))1 / 2, where σP is the period error, σT the timing error of a single measurement, and N the number of measurements. Compared to the iterative method for period error estimation by Mighell & Plavchan (2013), this much simpler formula leads to smaller period errors, whose correctness has been verified through simulations. For the prediction of times of future periodic events, usual linear ephemeris were epoch errors are quoted for the first time measurement, are prone to an overestimation of the error of that prediction. This may be avoided by a correction for the duration of the time series. An alternative is the derivation of ephemerides whose reference epoch and epoch error are given for the centre of the time series. For long continuous or near-continuous time series whose acquisition is completed, such central epochs should be the preferred way for the quotation of linear ephemerides. While this work was motivated from the analysis of eclipse timing measures in space-based light curves, it should be applicable to any other problem with an uninterrupted sequence of discrete timings for which the determination of a zero point, of a constant period and of the associated errors is needed.

A test of the reward-value hypothesis.

Science.gov (United States)

Smith, Alexandra E; Dalecki, Stefan J; Crystal, Jonathon D

2017-03-01

Rats retain source memory (memory for the origin of information) over a retention interval of at least 1 week, whereas their spatial working memory (radial maze locations) decays within approximately 1 day. We have argued that different forgetting functions dissociate memory systems. However, the two tasks, in our previous work, used different reward values. The source memory task used multiple pellets of a preferred food flavor (chocolate), whereas the spatial working memory task provided access to a single pellet of standard chow-flavored food at each location. Thus, according to the reward-value hypothesis, enhanced performance in the source memory task stems from enhanced encoding/memory of a preferred reward. We tested the reward-value hypothesis by using a standard 8-arm radial maze task to compare spatial working memory accuracy of rats rewarded with either multiple chocolate or chow pellets at each location using a between-subjects design. The reward-value hypothesis predicts superior accuracy for high-valued rewards. We documented equivalent spatial memory accuracy for high- and low-value rewards. Importantly, a 24-h retention interval produced equivalent spatial working memory accuracy for both flavors. These data are inconsistent with the reward-value hypothesis and suggest that reward value does not explain our earlier findings that source memory survives unusually long retention intervals.

Prediction-error identification of LPV systems : present and beyond

NARCIS (Netherlands)

Toth, R.; Heuberger, P.S.C.; Hof, Van den P.M.J.; Mohammadpour, J.; Scherer, C. W.

2012-01-01

The proposed chapter aims at presenting a unified framework of prediction-error based identification of LPV systems using freshly developed theoretical results. Recently, these methods have got a considerable attention as they have certain advantages in terms of computational complexity, optimality

Predictive error detection in pianists: A combined ERP and motion capture study

Directory of Open Access Journals (Sweden)

Clemens eMaidhof

2013-09-01

Full Text Available Performing a piece of music involves the interplay of several cognitive and motor processes and requires extensive training to achieve a high skill level. However, even professional musicians commit errors occasionally. Previous event-related potential (ERP studies have investigated the neurophysiological correlates of pitch errors during piano performance, and reported pre-error negativity already occurring approximately 70-100 ms before the error had been committed and audible. It was assumed that this pre-error negativity reflects predictive control processes that compare predicted consequences with actual consequences of one’s own actions. However, in previous investigations, correct and incorrect pitch events were confounded by their different tempi. In addition, no data about the underlying movements were available. In the present study, we exploratively recorded the ERPs and 3D movement data of pianists’ fingers simultaneously while they performed fingering exercises from memory. Results showed a pre-error negativity for incorrect keystrokes when both correct and incorrect keystrokes were performed with comparable tempi. Interestingly, even correct notes immediately preceding erroneous keystrokes elicited a very similar negativity. In addition, we explored the possibility of computing ERPs time-locked to a kinematic landmark in the finger motion trajectories defined by when a finger makes initial contact with the key surface, that is, at the onset of tactile feedback. Results suggest that incorrect notes elicited a small difference after the onset of tactile feedback, whereas correct notes preceding incorrect ones elicited negativity before the onset of tactile feedback. The results tentatively suggest that tactile feedback plays an important role in error-monitoring during piano performance, because the comparison between predicted and actual sensory (tactile feedback may provide the information necessary for the detection of an

Abnormal reward functioning across substance use disorders and major depressive disorder: Considering reward as a transdiagnostic mechanism.

Science.gov (United States)

Baskin-Sommers, Arielle R; Foti, Dan

2015-11-01

A common criticism of the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2013) is that its criteria are based more on behavioral descriptions than on underlying biological mechanisms. Increasingly, calls have intensified for a more biologically-based approach to conceptualizing, studying, and treating psychological disorders, as exemplified by the Research Domain Criteria Project (RDoC). Among the most well-studied neurobiological mechanisms is reward processing. Moreover, individual differences in reward sensitivity are related to risk for substance abuse and depression. The current review synthesizes the available preclinical, electrophysiological, and neuroimaging literature on reward processing from a transdiagnostic, multidimensional perspective. Findings are organized with respect to key reward constructs within the Positive Valence Systems domain of the RDoC matrix, including initial responsiveness to reward (physiological 'liking'), approach motivation (physiological 'wanting'), and reward learning/habit formation. In the current review, we (a) describe the neural basis of reward, (b) elucidate differences in reward activity in substance abuse and depression, and (c) suggest a framework for integrating these disparate literatures and discuss the utility of shifting focus from diagnosis to process for understanding liability and co-morbidity. Ultimately, we believe that an integrative focus on abnormal reward functioning across the full continuum of clinically heterogeneous samples, rather than within circumscribed diagnostic categories, might actually help to refine the phenotypes and improve the prediction of onset and recovery of these disorders. Copyright © 2015 Elsevier B.V. All rights reserved.

Prediction and error of baldcypress stem volume from stump diameter

Science.gov (United States)

Bernard R. Parresol

1998-01-01

The need to estimate the volume of removals occurs for many reasons, such as in trespass cases, severance tax reports, and post-harvest assessments. A logarithmic model is presented for prediction of baldcypress total stem cubic foot volume using stump diameter as the independent variable. Because the error of prediction is as important as the volume estimate, the...

Practical guidance on representing the heteroscedasticity of residual errors of hydrological predictions

Science.gov (United States)

McInerney, David; Thyer, Mark; Kavetski, Dmitri; Kuczera, George

2016-04-01

Appropriate representation of residual errors in hydrological modelling is essential for accurate and reliable probabilistic streamflow predictions. In particular, residual errors of hydrological predictions are often heteroscedastic, with large errors associated with high runoff events. Although multiple approaches exist for representing this heteroscedasticity, few if any studies have undertaken a comprehensive evaluation and comparison of these approaches. This study fills this research gap by evaluating a range of approaches for representing heteroscedasticity in residual errors. These approaches include the 'direct' weighted least squares approach and 'transformational' approaches, such as logarithmic, Box-Cox (with and without fitting the transformation parameter), logsinh and the inverse transformation. The study reports (1) theoretical comparison of heteroscedasticity approaches, (2) empirical evaluation of heteroscedasticity approaches using a range of multiple catchments / hydrological models / performance metrics and (3) interpretation of empirical results using theory to provide practical guidance on the selection of heteroscedasticity approaches. Importantly, for hydrological practitioners, the results will simplify the choice of approaches to represent heteroscedasticity. This will enhance their ability to provide hydrological probabilistic predictions with the best reliability and precision for different catchment types (e.g. high/low degree of ephemerality).

Uncertainties of predictions from parton distributions 1, experimental errors

CERN Document Server

Martin, A D; Stirling, William James; Thorne, R S; CERN. Geneva

2003-01-01

We determine the uncertainties on observables arising from the errors on the experimental data that are fitted in the global MRST2001 parton analysis. By diagonalizing the error matrix we produce sets of partons suitable for use within the framework of linear propagation of errors, which is the most convenient method for calculating the uncertainties. Despite the potential limitations of this approach we find that it can be made to work well in practice. This is confirmed by our alternative approach of using the more rigorous Lagrange multiplier method to determine the errors on physical quantities directly. As particular examples we determine the uncertainties on the predictions of the charged-current deep-inelastic structure functions, on the cross-sections for W production and for Higgs boson production via gluon--gluon fusion at the Tevatron and the LHC, on the ratio of W-minus to W-plus production at the LHC and on the moments of the non-singlet quark distributions. We discuss the corresponding uncertain...

The globus pallidus sends reward-related signals to the lateral habenula.

Science.gov (United States)

Hong, Simon; Hikosaka, Okihide

2008-11-26

As a major output station of the basal ganglia, the globus pallidus internal segment (GPi) projects to the thalamus and brainstem nuclei thereby controlling motor behavior. A less well known fact is that the GPi also projects to the lateral habenula (LHb) which is often associated with the limbic system. Using the monkey performing a saccade task with positionally biased reward outcomes, we found that antidromically identified LHb-projecting neurons were distributed mainly in the dorsal and ventral borders of the GPi and that their activity was strongly modulated by expected reward outcomes. A majority of them were excited by the no-reward-predicting target and inhibited by the reward-predicting target. These reward-dependent modulations were similar to those in LHb neurons but started earlier than those in LHb neurons. These results suggest that GPi may initiate reward-related signals through its effects on the LHb, which then influences the dopaminergic and serotonergic systems.

Regulating task-monitoring systems in response to variable reward contingencies and outcomes in cocaine addicts.

Science.gov (United States)

Morie, Kristen P; De Sanctis, Pierfilippo; Garavan, Hugh; Foxe, John J

2016-03-01

We investigated anticipatory and consummatory reward processing in cocaine addiction. In addition, we set out to assess whether task-monitoring systems were appropriately recalibrated in light of variable reward schedules. We also examined neural measures of task-monitoring and reward processing as a function of hedonic tone, since anhedonia is a vulnerability marker for addiction that is obviously germane in the context of reward processing. High-density event-related potentials were recorded while participants performed a speeded response task that systematically varied anticipated probabilities of reward receipt. The paradigm dissociated feedback regarding task success (or failure) from feedback regarding the value of reward (or loss), so that task-monitoring and reward processing could be examined in partial isolation. Twenty-three active cocaine abusers and 23 age-matched healthy controls participated. Cocaine abusers showed amplified anticipatory responses to reward predictive cues, but crucially, these responses were not as strongly modulated by reward probability as in controls. Cocaine users also showed blunted responses to feedback about task success or failure and did not use this information to update predictions about reward. In turn, they showed clearly blunted responses to reward feedback. In controls and users, measures of anhedonia were associated with reward motivation. In cocaine users, anhedonia was also associated with diminished monitoring and reward feedback responses. Findings imply that reward anticipation and monitoring deficiencies in addiction are associated with increased responsiveness to reward cues but impaired ability to predict reward in light of task contingencies, compounded by deficits in responding to actual reward outcomes.

DISRUPTION OF CONDITIONED REWARD ASSOCIATION BY TYPICAL AND ATYPICAL ANTIPSYCHOTICS

Science.gov (United States)

Danna, C.L.; Elmer, G.I.

2013-01-01

Antipsychotic drugs are broadly classified into typical and atypical compounds; they vary in their pharmacological profile however a common component is their antagonist effects at the D2 dopamine receptors (DRD2). Unfortunately, diminished DRD2 activation is generally thought to be associated with the severity of neuroleptic-induced anhedonia. The purpose of this study was to determine the effect of the atypical antipsychotic olanzapine and typical antipsychotic haloperidol in a paradigm that reflects the learned transfer of incentive motivational properties to previously neutral stimuli, namely autoshaping. In order to provide a dosing comparison to a therapeutically relevant endpoint, both drugs were tested against amphetamine-induced disruption of prepulse inhibition as well. In the autoshaping task, rats were exposed to repeated pairings of stimuli that were differentially predictive of reward delivery. Conditioned approach to the reward predictive cue (sign-tracking) and to the reward (goal-tracking) increased during repeated pairings in the vehicle treated rats. Haloperidol and olanzapine completely abolished this behavior at relatively low doses (100 μg/kg). This same dose was the threshold dose for each drug to antagonize the sensorimotor gating deficits produced by amphetamine. At lower doses (3–30 μg/kg) both drugs produced a dose-dependent decrease in conditioned approach to the reward predictive cue. There was no difference between drugs at this dose range which indicates that olanzapine disrupts autoshaping at a significantly lower proposed DRD2 receptor occupancy. Interestingly, neither drug disrupted conditioned approach to the reward at the same dose range that disrupted conditioned approach to the reward predictive cue. Thus, haloperidol and olanzapine, at doses well below what is considered therapeutically relevant, disrupts the attribution of incentive motivational value to previously neutral cues. Drug effects on this dimension of reward

Reversible Watermarking Using Prediction-Error Expansion and Extreme Learning Machine

Directory of Open Access Journals (Sweden)

Guangyong Gao

2015-01-01

Full Text Available Currently, the research for reversible watermarking focuses on the decreasing of image distortion. Aiming at this issue, this paper presents an improvement method to lower the embedding distortion based on the prediction-error expansion (PE technique. Firstly, the extreme learning machine (ELM with good generalization ability is utilized to enhance the prediction accuracy for image pixel value during the watermarking embedding, and the lower prediction error results in the reduction of image distortion. Moreover, an optimization operation for strengthening the performance of ELM is taken to further lessen the embedding distortion. With two popular predictors, that is, median edge detector (MED predictor and gradient-adjusted predictor (GAP, the experimental results for the classical images and Kodak image set indicate that the proposed scheme achieves improvement for the lowering of image distortion compared with the classical PE scheme proposed by Thodi et al. and outperforms the improvement method presented by Coltuc and other existing approaches.

CREME96 and Related Error Rate Prediction Methods

Science.gov (United States)

Adams, James H., Jr.

2012-01-01

Predicting the rate of occurrence of single event effects (SEEs) in space requires knowledge of the radiation environment and the response of electronic devices to that environment. Several analytical models have been developed over the past 36 years to predict SEE rates. The first error rate calculations were performed by Binder, Smith and Holman. Bradford and Pickel and Blandford, in their CRIER (Cosmic-Ray-Induced-Error-Rate) analysis code introduced the basic Rectangular ParallelePiped (RPP) method for error rate calculations. For the radiation environment at the part, both made use of the Cosmic Ray LET (Linear Energy Transfer) spectra calculated by Heinrich for various absorber Depths. A more detailed model for the space radiation environment within spacecraft was developed by Adams and co-workers. This model, together with a reformulation of the RPP method published by Pickel and Blandford, was used to create the CR ME (Cosmic Ray Effects on Micro-Electronics) code. About the same time Shapiro wrote the CRUP (Cosmic Ray Upset Program) based on the RPP method published by Bradford. It was the first code to specifically take into account charge collection from outside the depletion region due to deformation of the electric field caused by the incident cosmic ray. Other early rate prediction methods and codes include the Single Event Figure of Merit, NOVICE, the Space Radiation code and the effective flux method of Binder which is the basis of the SEFA (Scott Effective Flux Approximation) model. By the early 1990s it was becoming clear that CREME and the other early models needed Revision. This revision, CREME96, was completed and released as a WWW-based tool, one of the first of its kind. The revisions in CREME96 included improved environmental models and improved models for calculating single event effects. The need for a revision of CREME also stimulated the development of the CHIME (CRRES/SPACERAD Heavy Ion Model of the Environment) and MACREE (Modeling and

Analysts forecast error : A robust prediction model and its short term trading

NARCIS (Netherlands)

Boudt, Kris; de Goeij, Peter; Thewissen, James; Van Campenhout, Geert

We examine the profitability of implementing a short term trading strategy based on predicting the error in analysts' earnings per share forecasts using publicly available information. Since large earnings surprises may lead to extreme values in the forecast error series that disrupt their smooth

Amygdala mu-opioid receptors mediate the motivating influence of cue-triggered reward expectations.

Science.gov (United States)

Lichtenberg, Nina T; Wassum, Kate M

2017-02-01

Environmental reward-predictive stimuli can retrieve from memory a specific reward expectation that allows them to motivate action and guide choice. This process requires the basolateral amygdala (BLA), but little is known about the signaling systems necessary within this structure. Here we examined the role of the neuromodulatory opioid receptor system in the BLA in such cue-directed action using the outcome-specific Pavlovian-to-instrumental transfer (PIT) test in rats. Inactivation of BLA mu-, but not delta-opioid receptors was found to dose-dependently attenuate the ability of a reward-predictive cue to selectively invigorate the performance of actions directed at the same unique predicted reward (i.e. to express outcome-specific PIT). BLA mu-opioid receptor inactivation did not affect the ability of a reward itself to similarly motivate action (outcome-specific reinstatement), suggesting a more selective role for the BLA mu-opioid receptor in the motivating influence of currently unobservable rewarding events. These data reveal a new role for BLA mu-opioid receptor activation in the cued recall of precise reward memories and the use of this information to motivate specific action plans. © 2016 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.

Estimation of Mechanical Signals in Induction Motors using the Recursive Prediction Error Method

DEFF Research Database (Denmark)

Børsting, H.; Knudsen, Morten; Rasmussen, Henrik

1993-01-01

Sensor feedback of mechanical quantities for control applications in induction motors is troublesome and relative expensive. In this paper a recursive prediction error (RPE) method has successfully been used to estimate the angular rotor speed ........Sensor feedback of mechanical quantities for control applications in induction motors is troublesome and relative expensive. In this paper a recursive prediction error (RPE) method has successfully been used to estimate the angular rotor speed .....

Authoritarian parenting predicts reduced electrocortical response to observed adolescent offspring rewards.

Science.gov (United States)

Levinson, Amanda R; Speed, Brittany C; Nelson, Brady; Bress, Jennifer N; Hajcak, Greg

2017-03-01

Parenting styles are robust predictors of offspring outcomes, yet little is known about their neural underpinnings. In this study, 44 parent-adolescent dyads (Mage of adolescent = 12.9) completed a laboratory guessing task while EEG was continuously recorded. In the task, each pair member received feedback about their own monetary wins and losses and also observed the monetary wins and losses of the other member of the pair. We examined the association between self-reported parenting style and parents' electrophysiological responses to watching their adolescent winning and losing money, dubbed the observational Reward Positivity (RewP) and observational feedback negativity (FN), respectively. Self-reported authoritarian parenting predicted reductions in parents' observational RewP but not FN. This predictive relationship remained after adjusting for sex of both participants, parents' responsiveness to their own wins, and parental psychopathology. 'Exploratory analyses found that permissive parenting was associated with a blunting of the adolescents' response to their parents' losses'. These findings suggest that parents' rapid neural responses to their child's successes may relate to the harsh parenting behaviors associated with authoritarian parenting. © The Author (2016). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

The Human Bathtub: Safety and Risk Predictions Including the Dynamic Probability of Operator Errors

International Nuclear Information System (INIS)

Duffey, Romney B.; Saull, John W.

2006-01-01

Reactor safety and risk are dominated by the potential and major contribution for human error in the design, operation, control, management, regulation and maintenance of the plant, and hence to all accidents. Given the possibility of accidents and errors, now we need to determine the outcome (error) probability, or the chance of failure. Conventionally, reliability engineering is associated with the failure rate of components, or systems, or mechanisms, not of human beings in and interacting with a technological system. The probability of failure requires a prior knowledge of the total number of outcomes, which for any predictive purposes we do not know or have. Analysis of failure rates due to human error and the rate of learning allow a new determination of the dynamic human error rate in technological systems, consistent with and derived from the available world data. The basis for the analysis is the 'learning hypothesis' that humans learn from experience, and consequently the accumulated experience defines the failure rate. A new 'best' equation has been derived for the human error, outcome or failure rate, which allows for calculation and prediction of the probability of human error. We also provide comparisons to the empirical Weibull parameter fitting used in and by conventional reliability engineering and probabilistic safety analysis methods. These new analyses show that arbitrary Weibull fitting parameters and typical empirical hazard function techniques cannot be used to predict the dynamics of human errors and outcomes in the presence of learning. Comparisons of these new insights show agreement with human error data from the world's commercial airlines, the two shuttle failures, and from nuclear plant operator actions and transient control behavior observed in transients in both plants and simulators. The results demonstrate that the human error probability (HEP) is dynamic, and that it may be predicted using the learning hypothesis and the minimum

Influence of precision of emission characteristic parameters on model prediction error of VOCs/formaldehyde from dry building material.

Directory of Open Access Journals (Sweden)

Wenjuan Wei

Full Text Available Mass transfer models are useful in predicting the emissions of volatile organic compounds (VOCs and formaldehyde from building materials in indoor environments. They are also useful for human exposure evaluation and in sustainable building design. The measurement errors in the emission characteristic parameters in these mass transfer models, i.e., the initial emittable concentration (C 0, the diffusion coefficient (D, and the partition coefficient (K, can result in errors in predicting indoor VOC and formaldehyde concentrations. These errors have not yet been quantitatively well analyzed in the literature. This paper addresses this by using modelling to assess these errors for some typical building conditions. The error in C 0, as measured in environmental chambers and applied to a reference living room in Beijing, has the largest influence on the model prediction error in indoor VOC and formaldehyde concentration, while the error in K has the least effect. A correlation between the errors in D, K, and C 0 and the error in the indoor VOC and formaldehyde concentration prediction is then derived for engineering applications. In addition, the influence of temperature on the model prediction of emissions is investigated. It shows the impact of temperature fluctuations on the prediction errors in indoor VOC and formaldehyde concentrations to be less than 7% at 23±0.5°C and less than 30% at 23±2°C.

Altered social reward and attention in anorexia nervosa

Directory of Open Access Journals (Sweden)

Karli K Watson

2010-09-01

Full Text Available Dysfunctional social reward and social orienting attend a variety of neuropsychiatric disorders including autism, schizophrenia, social anxiety, and psychopathy. Here we show that similar social reward and attention dysfunction attend anorexia nervosa, a disorder defined by avoidance of food and extreme weight loss. We measured the implicit reward value of social stimuli for female participants with (n=11 and without (n=11 anorexia nervosa using an econometric choice task and also tracked gaze patterns during free viewing of images of female faces and bodies. As predicted, the reward value of viewing bodies varied inversely with observed body weight for women with anorexia but not neurotypical women, in contrast with their explicit ratings of attractiveness. Surprisingly, women with anorexia nervosa, unlike neurotypical women, did not find female faces rewarding and avoided looking at both the face and eyes—independent of observed body weight. These findings demonstrate comorbid dysfunction in the neural circuits mediating gustatory and social reward in anorexia nervosa.

Individual differences in the habitual use of cognitive reappraisal predict the reward-related processing.

Science.gov (United States)

Sai, Liyang; Wang, Sisi; Ward, Anne; Ku, Yixuan; Sang, Biao

2015-01-01

Recent studies have shown that instructed cognitive reappraisal can regulate the neural processing of reward. However, it is still unclear whether the habitual use of cognitive reappraisal in everyday life is related to brain activity involved in reward processing. In the present study, participants' neural responses to reward were measured using electroencephalography (EEG) recorded during a gambling task and their tendency to use cognitive reappraisal was assessed using the Emotion Regulation Questionnaire (ERQ). Event-related potential (ERP) results indicated that losses on the gambling task elicited greater negative reward-related feedback negativity (FN) than gains. The differential FN between losses and gains was significantly correlated with cognitive reappraisal scores across participants such that individuals with a higher tendency to use cognitive reappraisal showed stronger reward processing (i.e., amplified FN difference between losses and gains). This correlation remained significant after controlling for expressive suppression scores. However, expressive suppression per se was not correlated with FN differences. Taken together, these results suggest that the habitual use of cognitive reappraisal is associated with increased neural processing of reward.

Neural correlates of water reward in thirsty Drosophila

OpenAIRE

Lin, Suewei; Owald, David; Chandra, Vikram; Talbot, Clifford; Huetteroth, Wolf; Waddell, Scott

2014-01-01

Drinking water is innately rewarding to thirsty animals. In addition, the consumed value can be assigned to behavioral actions and predictive sensory cues by associative learning. Here we show that thirst converts water avoidance into water-seeking in naive Drosophila melanogaster. Thirst also permitted flies to learn olfactory cues paired with water reward. Water learning required water taste and

Model Checking Markov Reward Models with Impulse Rewards

NARCIS (Netherlands)

Cloth, Lucia; Katoen, Joost-Pieter; Khattri, Maneesh; Pulungan, Reza; Bondavalli, Andrea; Haverkort, Boudewijn; Tang, Dong

This paper considers model checking of Markov reward models (MRMs), continuous-time Markov chains with state rewards as well as impulse rewards. The reward extension of the logic CSL (Continuous Stochastic Logic) is interpreted over such MRMs, and two numerical algorithms are provided to check the

Learning with three factors: modulating Hebbian plasticity with errors.

Science.gov (United States)

Kuśmierz, Łukasz; Isomura, Takuya; Toyoizumi, Taro

2017-10-01

Synaptic plasticity is a central theme in neuroscience. A framework of three-factor learning rules provides a powerful abstraction, helping to navigate through the abundance of models of synaptic plasticity. It is well-known that the dopamine modulation of learning is related to reward, but theoretical models predict other functional roles of the modulatory third factor; it may encode errors for supervised learning, summary statistics of the population activity for unsupervised learning or attentional feedback. Specialized structures may be needed in order to generate and propagate third factors in the neural network. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.

Memory Consolidation and Neural Substrate of Reward

Directory of Open Access Journals (Sweden)

Redolar-Ripoll, Diego

2012-08-01

Full Text Available The aim of this report is to analyze the relationships between reward and learning and memory processes. Different studies have described how information about rewards influences behavior and how the brain uses this reward information to control learning and memory processes. Reward nature seems to be processed in different ways by neurons in different brain structures, ranging from the detection and perception of rewards to the use of information about predicted rewards for the control of goal-directed behavior. The neural substrate underling this processing of reward information is a reliable way of improving learning and memory processes. Evidence from several studies indicates that this neural system can facilitate memory consolidation in a wide variety of learning tasks. From a molecular perspective, certain cardinal features of reward have been described as forms of memory. Studies of human addicts and studies in animal models of addiction show that chronic drug exposure produces stable changes in the brain at the cellular and molecular levels that underlie the long-lasting behavioral plasticity associated with addiction. These molecular and cellular adaptations involved in addiction are also implicated in learning and memory processes. Dopamine seems to be a critical common signal to activate different genetic mechanisms that ultimately remodel synapses and circuits. Despite memory is an active and complex process mediated by different brain areas, the neural substrate of reward is able to improve memory consolidation in a several paradigms. We believe that there are many equivalent traits between reward and learning and memory processes.

RM-SORN: a reward-modulated self-organizing recurrent neural network.

Science.gov (United States)

Aswolinskiy, Witali; Pipa, Gordon

2015-01-01

Neural plasticity plays an important role in learning and memory. Reward-modulation of plasticity offers an explanation for the ability of the brain to adapt its neural activity to achieve a rewarded goal. Here, we define a neural network model that learns through the interaction of Intrinsic Plasticity (IP) and reward-modulated Spike-Timing-Dependent Plasticity (STDP). IP enables the network to explore possible output sequences and STDP, modulated by reward, reinforces the creation of the rewarded output sequences. The model is tested on tasks for prediction, recall, non-linear computation, pattern recognition, and sequence generation. It achieves performance comparable to networks trained with supervised learning, while using simple, biologically motivated plasticity rules, and rewarding strategies. The results confirm the importance of investigating the interaction of several plasticity rules in the context of reward-modulated learning and whether reward-modulated self-organization can explain the amazing capabilities of the brain.

Haptic Data Processing for Teleoperation Systems: Prediction, Compression and Error Correction

OpenAIRE

Lee, Jae-young

2013-01-01

This thesis explores haptic data processing methods for teleoperation systems, including prediction, compression, and error correction. In the proposed haptic data prediction method, unreliable network conditions, such as time-varying delay and packet loss, are detected by a transport layer protocol. Given the information from the transport layer, a Bayesian approach is introduced to predict position and force data in haptic teleoperation systems. Stability of the proposed method within stoch...

Reward system and temporal pole contributions to affective evaluation during a first person shooter video game

Directory of Open Access Journals (Sweden)

Weber René

2011-07-01

Full Text Available Abstract Background Violent content in video games evokes many concerns but there is little research concerning its rewarding aspects. It was demonstrated that playing a video game leads to striatal dopamine release. It is unclear, however, which aspects of the game cause this reward system activation and if violent content contributes to it. We combined functional Magnetic Resonance Imaging (fMRI with individual affect measures to address the neuronal correlates of violence in a video game. Results Thirteen male German volunteers played a first-person shooter game (Tactical Ops: Assault on Terror during fMRI measurement. We defined success as eliminating opponents, and failure as being eliminated themselves. Affect was measured directly before and after game play using the Positive and Negative Affect Schedule (PANAS. Failure and success events evoked increased activity in visual cortex but only failure decreased activity in orbitofrontal cortex and caudate nucleus. A negative correlation between negative affect and responses to failure was evident in the right temporal pole (rTP. Conclusions The deactivation of the caudate nucleus during failure is in accordance with its role in reward-prediction error: it occurred whenever subject missed an expected reward (being eliminated rather than eliminating the opponent. We found no indication that violence events were directly rewarding for the players. We addressed subjective evaluations of affect change due to gameplay to study the reward system. Subjects reporting greater negative affect after playing the game had less rTP activity associated with failure. The rTP may therefore be involved in evaluating the failure events in a social context, to regulate the players' mood.

Diminished neural responses predict enhanced intrinsic motivation and sensitivity to external incentive.

Science.gov (United States)

Marsden, Karen E; Ma, Wei Ji; Deci, Edward L; Ryan, Richard M; Chiu, Pearl H

2015-06-01

The duration and quality of human performance depend on both intrinsic motivation and external incentives. However, little is known about the neuroscientific basis of this interplay between internal and external motivators. Here, we used functional magnetic resonance imaging to examine the neural substrates of intrinsic motivation, operationalized as the free-choice time spent on a task when this was not required, and tested the neural and behavioral effects of external reward on intrinsic motivation. We found that increased duration of free-choice time was predicted by generally diminished neural responses in regions associated with cognitive and affective regulation. By comparison, the possibility of additional reward improved task accuracy, and specifically increased neural and behavioral responses following errors. Those individuals with the smallest neural responses associated with intrinsic motivation exhibited the greatest error-related neural enhancement under the external contingency of possible reward. Together, these data suggest that human performance is guided by a "tonic" and "phasic" relationship between the neural substrates of intrinsic motivation (tonic) and the impact of external incentives (phasic).

Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology.

Science.gov (United States)

Schultz, Wolfram

2004-04-01

Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.

Functional requirements for reward-modulated spike-timing-dependent plasticity.

Science.gov (United States)

Frémaux, Nicolas; Sprekeler, Henning; Gerstner, Wulfram

2010-10-06

Recent experiments have shown that spike-timing-dependent plasticity is influenced by neuromodulation. We derive theoretical conditions for successful learning of reward-related behavior for a large class of learning rules where Hebbian synaptic plasticity is conditioned on a global modulatory factor signaling reward. We show that all learning rules in this class can be separated into a term that captures the covariance of neuronal firing and reward and a second term that presents the influence of unsupervised learning. The unsupervised term, which is, in general, detrimental for reward-based learning, can be suppressed if the neuromodulatory signal encodes the difference between the reward and the expected reward-but only if the expected reward is calculated for each task and stimulus separately. If several tasks are to be learned simultaneously, the nervous system needs an internal critic that is able to predict the expected reward for arbitrary stimuli. We show that, with a critic, reward-modulated spike-timing-dependent plasticity is capable of learning motor trajectories with a temporal resolution of tens of milliseconds. The relation to temporal difference learning, the relevance of block-based learning paradigms, and the limitations of learning with a critic are discussed.

A test of the reward-contrast hypothesis.

Science.gov (United States)

Dalecki, Stefan J; Panoz-Brown, Danielle E; Crystal, Jonathon D

2017-12-01

Source memory, a facet of episodic memory, is the memory of the origin of information. Whereas source memory in rats is sustained for at least a week, spatial memory degraded after approximately a day. Different forgetting functions may suggest that two memory systems (source memory and spatial memory) are dissociated. However, in previous work, the two tasks used baiting conditions consisting of chocolate and chow flavors; notably, the source memory task used the relatively better flavor. Thus, according to the reward-contrast hypothesis, when chocolate and chow were presented within the same context (i.e., within a single radial maze trial), the chocolate location was more memorable than the chow location because of contrast. We tested the reward-contrast hypothesis using baiting configurations designed to produce reward-contrast. The reward-contrast hypothesis predicts that under these conditions, spatial memory will survive a 24-h retention interval. We documented elimination of spatial memory performance after a 24-h retention interval using a reward-contrast baiting pattern. These data suggest that reward contrast does not explain our earlier findings that source memory survives unusually long retention intervals. Copyright © 2017 Elsevier B.V. All rights reserved.

Striatal dopamine D2 receptor availability predicts the thalamic and medial prefrontal responses to reward in cocaine abusers three years later

International Nuclear Information System (INIS)

Asensio, S.; Goldstein, R.; Romero, M.J.; Romero, F.J.; Wong, C.T.; Alia-Klein, N.; Tomasi, D.; Wang, G.-J.; Telang, F.; Volkow, N.D.; Goldstein, R.Z.

2010-01-01

Low levels of dopamine (DA) D2 receptor availability at a resting baseline have been previously reported in drug addicted individuals and have been associated with reduced ventral and dorsal prefrontal metabolism. The reduction in DA D2 receptor availability along with the reduced ventral frontal metabolism is thought to underlie compromised sensitivity to nondrug reward, a core characteristic of drug addiction. We therefore hypothesized that variability in DA D2 receptor availability at baseline will covary with dynamic responses to monetary reward in addicted individuals. Striatal DA D2 receptor availability was measured with ( 11 C)raclopride and positron emission tomography and response to monetary reward was measured (an average of three years later) with functional magnetic resonance imaging in seven cocaine-addicted individuals. Results show that low DA D2 receptor availability in the dorsal striatum was associated with decreased thalamic response to monetary reward; while low availability in ventral striatum was associated with increased medial prefrontal (Brodmann Area 6/8/32) response to monetary reward. These preliminary results, that need to be replicated in larger sample sizes and validated with healthy controls, suggest that resting striatal DA D2 receptor availability predicts variability in functional responses to a nondrug reinforcer (money) in prefrontal cortex, implicated in behavioral monitoring, and in thalamus, implicated in conditioned responses and expectation, in cocaine-addicted individuals.

Striatal dopamine D2 receptor availability predicts the thalamic and medial prefrontal responses to reward in cocaine abusers three years later

Energy Technology Data Exchange (ETDEWEB)

Asensio, S.; Goldstein, R.; Asensio, S.; Romero, M.J.; Romero, F.J.; Wong, C.T.; Alia-Klein, N.; Tomasi, D.; Wang, G.-J.; Telang, F..; Volkow, N.D.; Goldstein, R.Z.

2010-05-01

Low levels of dopamine (DA) D2 receptor availability at a resting baseline have been previously reported in drug addicted individuals and have been associated with reduced ventral and dorsal prefrontal metabolism. The reduction in DA D2 receptor availability along with the reduced ventral frontal metabolism is thought to underlie compromised sensitivity to nondrug reward, a core characteristic of drug addiction. We therefore hypothesized that variability in DA D2 receptor availability at baseline will covary with dynamic responses to monetary reward in addicted individuals. Striatal DA D2 receptor availability was measured with [{sup 11}C]raclopride and positron emission tomography and response to monetary reward was measured (an average of three years later) with functional magnetic resonance imaging in seven cocaine-addicted individuals. Results show that low DA D2 receptor availability in the dorsal striatum was associated with decreased thalamic response to monetary reward; while low availability in ventral striatum was associated with increased medial prefrontal (Brodmann Area 6/8/32) response to monetary reward. These preliminary results, that need to be replicated in larger sample sizes and validated with healthy controls, suggest that resting striatal DA D2 receptor availability predicts variability in functional responses to a nondrug reinforcer (money) in prefrontal cortex, implicated in behavioral monitoring, and in thalamus, implicated in conditioned responses and expectation, in cocaine-addicted individuals.

Reward associations magnify memory-based biases on perception.

Science.gov (United States)

Doallo, Sonia; Patai, Eva Zita; Nobre, Anna Christina

2013-02-01

Long-term spatial contextual memories are a rich source of predictions about the likely locations of relevant objects in the environment and should enable tuning of neural processing of unfolding events to optimize perception and action. Of particular importance is whether and how the reward outcome of past events can impact perception. We combined behavioral measures with recordings of brain activity with high temporal resolution to test whether the previous reward outcome associated with a memory could modulate the impact of memory-based biases on perception, and if so, the level(s) at which visual neural processing is biased by reward-associated memory-guided attention. Data showed that past rewards potentiate the effects of spatial memories upon the discrimination of target objects embedded within complex scenes starting from early perceptual stages. We show that a single reward outcome of learning impacts on how we perceive events in our complex environments.

Effects of direct social experience on trust decisions and neural reward circuitry

Directory of Open Access Journals (Sweden)

Dominic S. Fareri

2012-10-01

Full Text Available The human striatum is integral for reward-processing and supports learning by linking experienced outcomes with prior expectations. Recent endeavors implicate the striatum in processing outcomes of social interactions, such as social approval/rejection, as well as in learning reputations of others. Interestingly, social impressions often influence our behavior with others during interactions. Information about an interaction partner’s moral character acquired from biographical information hinders updating of expectations after interactions via top down modulation of reward circuitry. An outstanding question is whether initial impressions formed through experience similarly modulate the ability to update social impressions at the behavioral and neural level. We investigated the role of experienced social information on trust behavior and reward-related BOLD activity. Participants played a computerized ball tossing game with three fictional partners manipulated to be perceived as good, bad or neutral. Participants then played an iterated trust game as investors with these same partners while undergoing fMRI. Unbeknownst to participants, partner behavior in the trust game was random and unrelated to their ball-tossing behavior. Participants’ trust decisions were influenced by their prior experience in the ball tossing game, investing less often with the bad partner compared to the good and neutral. Reinforcement learning models revealed that participants were more sensitive to updating their beliefs about good and bad partners when experiencing outcomes consistent with initial experience. Increased striatal and anterior cingulate BOLD activity for positive versus negative trust game outcomes emerged, which further correlated with model-derived prediction-error (PE learning signals. These results suggest that initial impressions formed from direct social experience can be continually shaped by consistent information through reward learning

Effects of Direct Social Experience on Trust Decisions and Neural Reward Circuitry

Science.gov (United States)

Fareri, Dominic S.; Chang, Luke J.; Delgado, Mauricio R.

2012-01-01

The human striatum is integral for reward-processing and supports learning by linking experienced outcomes with prior expectations. Recent endeavors implicate the striatum in processing outcomes of social interactions, such as social approval/rejection, as well as in learning reputations of others. Interestingly, social impressions often influence our behavior with others during interactions. Information about an interaction partner’s moral character acquired from biographical information hinders updating of expectations after interactions via top down modulation of reward circuitry. An outstanding question is whether initial impressions formed through experience similarly modulate the ability to update social impressions at the behavioral and neural level. We investigated the role of experienced social information on trust behavior and reward-related BOLD activity. Participants played a computerized ball-tossing game with three fictional partners manipulated to be perceived as good, bad, or neutral. Participants then played an iterated trust game as investors with these same partners while undergoing fMRI. Unbeknownst to participants, partner behavior in the trust game was random and unrelated to their ball-tossing behavior. Participants’ trust decisions were influenced by their prior experience in the ball-tossing game, investing less often with the bad partner compared to the good and neutral. Reinforcement learning models revealed that participants were more sensitive to updating their beliefs about good and bad partners when experiencing outcomes consistent with initial experience. Increased striatal and anterior cingulate BOLD activity for positive versus negative trust game outcomes emerged, which further correlated with model-derived prediction error learning signals. These results suggest that initial impressions formed from direct social experience can be continually shaped by consistent information through reward learning mechanisms. PMID:23087604

Reward-enhanced memory in younger and older adults.

Science.gov (United States)

Spaniol, Julia; Schain, Cécile; Bowen, Holly J

2014-09-01

We investigated how the anticipation of remote monetary reward modulates intentional episodic memory formation in younger and older adults. On the basis of prior findings of preserved reward-cognition interactions in aging, we predicted that reward anticipation would be associated with enhanced memory in both younger and older adults. On the basis of previous demonstrations of a time-dependent effect of reward anticipation on memory, we expected the memory enhancement to increase with study-test delay. In Experiment 1, younger and older participants encoded a series of picture stimuli associated with high- or low-reward values. At test (24-hr postencoding), recognition hits resulted in either high or low monetary rewards, whereas false alarms were penalized to discourage guessing. Experiment 2 was similar to Experiment 1, but the study-test delay was manipulated within subjects (immediate vs 24hr). In Experiment 1, younger and older adults showed enhanced recognition for high-reward pictures compared with low-reward pictures. Experiment 2 replicated this finding and additionally showed that the effect did not extend to immediate recognition. The current findings provide support for a time-dependent mechanism of reward-based memory enhancement. They also suggest that aging leaves intact the positive influence of reward anticipation on intentional long-term memory formation. © The Author 2013. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Analysts’ forecast error: A robust prediction model and its short term trading profitability

NARCIS (Netherlands)

Boudt, K.M.R.; de Goei, P.; Thewissen, J.; van Campenhout, G.

2015-01-01

This paper contributes to the empirical evidence on the investment horizon salient to trading based on predicting the error in analysts' earnings forecasts. An econometric framework is proposed that accommodates the stylized fact of extreme values in the forecast error series. We find that between

Information search with situation-specific reward functions

Directory of Open Access Journals (Sweden)

Bjorn Meder

2012-03-01

Full Text Available can strongly conflict with the goal of obtaining information for improving payoffs. Two environments with such a conflict were identified through computer optimization. Three subsequent experiments investigated people's search behavior in these environments. Experiments 1 and 2 used a multiple-cue probabilistic category-learning task to convey environmental probabilities. In a subsequent search task subjects could query only a single feature before making a classification decision. The crucial manipulation concerned the search-task reward structure. The payoffs corresponded either to accuracy, with equal rewards associated with the two categories, or to an asymmetric payoff function, with different rewards associated with each category. In Experiment 1, in which learning-task feedback corresponded to the true category, people later preferentially searched the accuracy-maximizing feature, whether or not this would improve monetary rewards. In Experiment 2, an asymmetric reward structure was used during learning. Subjects searched the reward-maximizing feature when asymmetric payoffs were preserved in the search task. However, if search-task payoffs corresponded to accuracy, subjects preferentially searched a feature that was suboptimal for reward and accuracy alike. Importantly, this feature would have been most useful, under the learning-task payoff structure. Experiment 3 found that, if words and numbers are used to convey environmental probabilities, neither reward nor accuracy consistently predicts search. These findings emphasize the necessity of taking into account people's goals and search-and-decision processes during learning, thereby challenging current models of information search.

Rewarding leadership and fair procedures as determinants of self-esteem

OpenAIRE

De Cremer, D.; Knippenberg, D.; Knippenberg, B.; Mullenders, D.; Stinglhamber, F.

2005-01-01

In the present research, the authors examined the effect of procedural fairness and rewarding leadership style on an important variable for employees: self-esteem. The authors predicted that procedural fairness would positively influence people's reported self-esteem if the leader adopted a style of rewarding behavior for a job well done. Results from a scenario experiment, a laboratory experiment, and an organizational survey indeed show that procedural fairness and rewarding leadership styl...

A behavioral economic reward index predicts drinking resolutions: moderation revisited and compared with other outcomes.

Science.gov (United States)

Tucker, Jalie A; Roth, David L; Vignolo, Mary J; Westfall, Andrew O

2009-04-01

Data were pooled from 3 studies of recently resolved community-dwelling problem drinkers to determine whether a behavioral economic index of the value of rewards available over different time horizons distinguished among moderation (n = 30), abstinent (n = 95), and unresolved (n = 77) outcomes. Moderation over 1- to 2-year prospective follow-up intervals was hypothesized to involve longer term behavior regulation processes than abstinence or relapse and to be predicted by more balanced preresolution monetary allocations between short-term and longer term objectives (i.e., drinking and saving for the future). Standardized odds ratios (ORs) based on changes in standard deviation units from a multinomial logistic regression indicated that increases on this "Alcohol-Savings Discretionary Expenditure" index predicted higher rates of abstinence (OR = 1.93, p = .004) and relapse (OR = 2.89, p moderation outcomes. The index had incremental utility in predicting moderation in complex models that included other established predictors. The study adds to evidence supporting a behavioral economic analysis of drinking resolutions and shows that a systematic analysis of preresolution spending patterns aids in predicting moderation.

Reward deficiency and anti-reward in pain chronification

OpenAIRE

Borsook, D.; Linnman, C.; Faria, Vanda; Strassman, A. M.; Becerra, L.; Elman, I.

2016-01-01

Converging lines of evidence suggest that the pathophysiology of pain is mediated to a substantial degree via allostatic neuroadaptations in reward- and stress-related brain circuits. Thus, reward deficiency (RD) represents a within-system neuroadaptation to pain-induced protracted activation of the reward circuits that leads to depletion-like hypodopaminergia, clinically manifested anhedonia, and diminished motivation for natural reinforcers. Anti-reward (AR) conversely pertains to a between...

Theta-band phase locking of orbitofrontal neurons during reward expectancy

NARCIS (Netherlands)

van Wingerden, M.; Vinck, M.; Lankelma, J.; Pennartz, C.M.A.

2010-01-01

The expectancy of a rewarding outcome following actions and cues is coded by a network of brain structures including the orbitofrontal cortex. Thus far, predicted reward was considered to be coded by time-averaged spike rates of neurons. However, besides firing rate, the precise timing of action

Identification of proteomic biomarkers predicting prostate cancer aggressiveness and lethality despite biopsy-sampling error.

Science.gov (United States)

Shipitsin, M; Small, C; Choudhury, S; Giladi, E; Friedlander, S; Nardone, J; Hussain, S; Hurley, A D; Ernst, C; Huang, Y E; Chang, H; Nifong, T P; Rimm, D L; Dunyak, J; Loda, M; Berman, D M; Blume-Jensen, P

2014-09-09

Key challenges of biopsy-based determination of prostate cancer aggressiveness include tumour heterogeneity, biopsy-sampling error, and variations in biopsy interpretation. The resulting uncertainty in risk assessment leads to significant overtreatment, with associated costs and morbidity. We developed a performance-based strategy to identify protein biomarkers predictive of prostate cancer aggressiveness and lethality regardless of biopsy-sampling variation. Prostatectomy samples from a large patient cohort with long follow-up were blindly assessed by expert pathologists who identified the tissue regions with the highest and lowest Gleason grade from each patient. To simulate biopsy-sampling error, a core from a high- and a low-Gleason area from each patient sample was used to generate a 'high' and a 'low' tumour microarray, respectively. Using a quantitative proteomics approach, we identified from 160 candidates 12 biomarkers that predicted prostate cancer aggressiveness (surgical Gleason and TNM stage) and lethal outcome robustly in both high- and low-Gleason areas. Conversely, a previously reported lethal outcome-predictive marker signature for prostatectomy tissue was unable to perform under circumstances of maximal sampling error. Our results have important implications for cancer biomarker discovery in general and development of a sampling error-resistant clinical biopsy test for prediction of prostate cancer aggressiveness.

Rewards.

Science.gov (United States)

Gunderman, Richard B; Kamer, Aaron P

2011-05-01

For much of the 20th century, psychologists and economists operated on the assumption that work is devoid of intrinsic rewards, and the only way to get people to work harder is through the use of rewards and punishments. This so-called carrot-and-stick model of workplace motivation, when applied to medical practice, emphasizes the use of financial incentives and disincentives to manipulate behavior. More recently, however, it has become apparent that, particularly when applied to certain kinds of work, such approaches can be ineffective or even frankly counterproductive. Instead of focusing on extrinsic rewards such as compensation, organizations and their leaders need to devote more attention to the intrinsic rewards of work itself. This article reviews this new understanding of rewards and traces out its practical implications for radiology today. Copyright © 2011. Published by Elsevier Inc.

Girls’ challenging social experiences in early adolescence predict neural response to rewards and depressive symptoms

Directory of Open Access Journals (Sweden)

Melynda D. Casement

2014-04-01

Full Text Available Developmental models of psychopathology posit that exposure to social stressors may confer risk for depression in adolescent girls by disrupting neural reward circuitry. The current study tested this hypothesis by examining the relationship between early adolescent social stressors and later neural reward processing and depressive symptoms. Participants were 120 girls from an ongoing longitudinal study of precursors to depression across adolescent development. Low parental warmth, peer victimization, and depressive symptoms were assessed when the girls were 11 and 12 years old, and participants completed a monetary reward guessing fMRI task and assessment of depressive symptoms at age 16. Results indicate that low parental warmth was associated with increased response to potential rewards in the medial prefrontal cortex (mPFC, striatum, and amygdala, whereas peer victimization was associated with decreased response to potential rewards in the mPFC. Furthermore, concurrent depressive symptoms were associated with increased reward anticipation response in mPFC and striatal regions that were also associated with early adolescent psychosocial stressors, with mPFC and striatal response mediating the association between social stressors and depressive symptoms. These findings are consistent with developmental models that emphasize the adverse impact of early psychosocial stressors on neural reward processing and risk for depression in adolescence.

State-based versus reward-based motivation in younger and older adults.

Science.gov (United States)

Worthy, Darrell A; Cooper, Jessica A; Byrne, Kaileigh A; Gorlick, Marissa A; Maddox, W Todd

2014-12-01

Recent decision-making work has focused on a distinction between a habitual, model-free neural system that is motivated toward actions that lead directly to reward and a more computationally demanding goal-directed, model-based system that is motivated toward actions that improve one's future state. In this article, we examine how aging affects motivation toward reward-based versus state-based decision making. Participants performed tasks in which one type of option provided larger immediate rewards but the alternative type of option led to larger rewards on future trials, or improvements in state. We predicted that older adults would show a reduced preference for choices that led to improvements in state and a greater preference for choices that maximized immediate reward. We also predicted that fits from a hybrid reinforcement-learning model would indicate greater model-based strategy use in younger than in older adults. In line with these predictions, older adults selected the options that maximized reward more often than did younger adults in three of the four tasks, and modeling results suggested reduced model-based strategy use. In the task where older adults showed similar behavior to younger adults, our model-fitting results suggested that this was due to the utilization of a win-stay-lose-shift heuristic rather than a more complex model-based strategy. Additionally, within older adults, we found that model-based strategy use was positively correlated with memory measures from our neuropsychological test battery. We suggest that this shift from state-based to reward-based motivation may be due to age related declines in the neural structures needed for more computationally demanding model-based decision making.

The habenula governs the attribution of incentive salience to reward predictive cues

OpenAIRE

Danna, Carey L.; Shepard, Paul D.; Elmer, Greg I.

2013-01-01

The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral...

Predicting diagnostic error in Radiology via eye-tracking and image analytics: Application in mammography

Energy Technology Data Exchange (ETDEWEB)

Voisin, Sophie [ORNL; Pinto, Frank M [ORNL; Morin-Ducote, Garnetta [University of Tennessee, Knoxville (UTK); Hudson, Kathy [University of Tennessee, Knoxville (UTK); Tourassi, Georgia [ORNL

2013-01-01

Purpose: The primary aim of the present study was to test the feasibility of predicting diagnostic errors in mammography by merging radiologists gaze behavior and image characteristics. A secondary aim was to investigate group-based and personalized predictive models for radiologists of variable experience levels. Methods: The study was performed for the clinical task of assessing the likelihood of malignancy of mammographic masses. Eye-tracking data and diagnostic decisions for 40 cases were acquired from 4 Radiology residents and 2 breast imaging experts as part of an IRB-approved pilot study. Gaze behavior features were extracted from the eye-tracking data. Computer-generated and BIRADs images features were extracted from the images. Finally, machine learning algorithms were used to merge gaze and image features for predicting human error. Feature selection was thoroughly explored to determine the relative contribution of the various features. Group-based and personalized user modeling was also investigated. Results: Diagnostic error can be predicted reliably by merging gaze behavior characteristics from the radiologist and textural characteristics from the image under review. Leveraging data collected from multiple readers produced a reasonable group model (AUC=0.79). Personalized user modeling was far more accurate for the more experienced readers (average AUC of 0.837 0.029) than for the less experienced ones (average AUC of 0.667 0.099). The best performing group-based and personalized predictive models involved combinations of both gaze and image features. Conclusions: Diagnostic errors in mammography can be predicted reliably by leveraging the radiologists gaze behavior and image content.

Individual Differences in the Habitual Use of Cognitive Reappraisal Predict the Reward-related Feedback Negativity

Directory of Open Access Journals (Sweden)

Liyang eSai

2015-09-01

Full Text Available Recent studies have shown that instructed cognitive reappraisal can regulate the neural processing of reward. However, it is still unclear whether the habitual use of cognitive reappraisal in everyday life can influence brain activity associated with reward processing. In the present study, participant’s neural responses to reward were measured using electroencephalography (EEG recorded during a gambling task, while their tendency to use cognitive reappraisal was assessed using the Emotion Regulation Questionnaire (ERQ. Event-related potential (ERP results indicated that losses on the gambling task elicited greater negative reward-related feedback negativity (FN than gains. The differential FN between losses and gains was significantly correlated with cognitive reappraisal scores across participants, such that individuals with a higher tendency to use cognitive reappraisal showed stronger reward processing (i.e. amplified FN difference between losses and gains. This correlation remained significant after controlling for expressive suppression scores. However, expressive suppression per se was not correlated with FN differences. Taken together, these results suggest that the habitual use of cognitive reappraisal influences the neural processing of reward.

Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity

Directory of Open Access Journals (Sweden)

Pierre Berthet

2016-07-01

Full Text Available The brain enables animals to behaviourally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviours are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1 and the indirect (denoted D2 pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE, which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and postsynaptic activity, receptor type and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson’s disease (PD by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1.

Listening to music in a risk-reward context: The roles of the temporoparietal junction and the orbitofrontal/insular cortices in reward-anticipation, reward-gain, and reward-loss.

Science.gov (United States)

Li, Chia-Wei; Chen, Jyh-Horng; Tsai, Chen-Gia

2015-12-10

Artificial rewards, such as visual arts and music, produce pleasurable feelings. Popular songs in the verse-chorus form provide a useful model for understanding the neural mechanisms underlying the processing of artificial rewards, because the chorus is usually the most rewarding element of a song. In this functional magnetic resonance imaging (fMRI) study, the stimuli were excerpts of 10 popular songs with a tensioned verse-to-chorus transition. We examined the neural correlates of three phases of reward processing: (1) reward-anticipation during the verse-to-chorus transition, (2) reward-gain during the first phrase of the chorus, and (3) reward-loss during the unexpected noise followed by the verse-to-chorus transition. Participants listened to these excerpts in a risk-reward context because the verse was followed by either the chorus or noise with equal probability. The results showed that reward-gain and reward-loss were associated with left- and right-biased temporoparietal junction activation, respectively. The bilateral temporoparietal junctions were active during reward-anticipation. Moreover, we observed left-biased lateral orbitofrontal activation during reward-anticipation, whereas the medial orbitofrontal cortex was activated during reward-gain. The findings are discussed in relation to the cognitive and emotional aspects of reward processing. Copyright © 2015 Elsevier B.V. All rights reserved.

A Conceptual Framework for Predicting Error in Complex Human-Machine Environments

Science.gov (United States)

Freed, Michael; Remington, Roger; Null, Cynthia H. (Technical Monitor)

1998-01-01

We present a Goals, Operators, Methods, and Selection Rules-Model Human Processor (GOMS-MHP) style model-based approach to the problem of predicting human habit capture errors. Habit captures occur when the model fails to allocate limited cognitive resources to retrieve task-relevant information from memory. Lacking the unretrieved information, decision mechanisms act in accordance with implicit default assumptions, resulting in error when relied upon assumptions prove incorrect. The model helps interface designers identify situations in which such failures are especially likely.

Providing Extrinsic Reward for Test Performance Undermines Long-Term Memory Acquisition

Directory of Open Access Journals (Sweden)

Christof eKuhbandner

2016-02-01

Full Text Available Based on numerous studies showing that testing studied material can improve long-term retention more than restudying the same material, it is often suggested that the number of tests in education should be increased to enhance knowledge acquisition. However, testing in real-life educational settings often entails a high degree of extrinsic motivation of learners due to the common practice of placing important consequences on the outcome of a test. Such an effect on the motivation of learners may undermine the beneficial effects of testing on long-term memory because it has been shown that extrinsic motivation can reduce the quality of learning. To examine this issue, participants learned foreign language vocabulary words, followed by an immediate test in which one third of the words were tested and one third restudied. To manipulate extrinsic motivation during immediate testing, participants received either monetary reward contingent on test performance or no reward. After one week, memory for all words was tested. In the immediate test, reward reduced correct recall and increased commission errors, indicating that reward reduced the number of items that can benefit from successful retrieval. The results in the delayed test revealed that reward additionally reduced the gain received from successful retrieval because memory for initially successfully retrieved words was lower in the reward condition. However, testing was still more effective than restudying under reward conditions because reward undermined long-term memory for concurrently restudied material as well. These findings indicate that providing performance-contingent reward in a test can undermine long-term knowledge acquisition.

Providing Extrinsic Reward for Test Performance Undermines Long-Term Memory Acquisition.

Science.gov (United States)

Kuhbandner, Christof; Aslan, Alp; Emmerdinger, Kathrin; Murayama, Kou

2016-01-01

Based on numerous studies showing that testing studied material can improve long-term retention more than restudying the same material, it is often suggested that the number of tests in education should be increased to enhance knowledge acquisition. However, testing in real-life educational settings often entails a high degree of extrinsic motivation of learners due to the common practice of placing important consequences on the outcome of a test. Such an effect on the motivation of learners may undermine the beneficial effects of testing on long-term memory because it has been shown that extrinsic motivation can reduce the quality of learning. To examine this issue, participants learned foreign language vocabulary words, followed by an immediate test in which one-third of the words were tested and one-third restudied. To manipulate extrinsic motivation during immediate testing, participants received either monetary reward contingent on test performance or no reward. After 1 week, memory for all words was tested. In the immediate test, reward reduced correct recall and increased commission errors, indicating that reward reduced the number of items that can benefit from successful retrieval. The results in the delayed test revealed that reward additionally reduced the gain received from successful retrieval because memory for initially successfully retrieved words was lower in the reward condition. However, testing was still more effective than restudying under reward conditions because reward undermined long-term memory for concurrently restudied material as well. These findings indicate that providing performance-contingent reward in a test can undermine long-term knowledge acquisition.

Disentangling reward anticipation with simultaneous pupillometry / fMRI.

Science.gov (United States)

Schneider, Max; Leuchs, Laura; Czisch, Michael; Sämann, Philipp G; Spoormaker, Victor I

2018-05-05

The reward system may provide an interesting intermediate phenotype for anhedonia in affective disorders. Reward anticipation is characterized by an increase in arousal, and previous studies have linked the anterior cingulate cortex (ACC) to arousal responses such as dilation of the pupil. Here, we examined pupil dynamics during a reward anticipation task in forty-six healthy human subjects and evaluated its neural correlates using functional magnetic resonance imaging (fMRI). Pupil size showed a strong increase during monetary reward anticipation, a moderate increase during verbal reward anticipation and a decrease during control trials. For fMRI analyses, average pupil size and pupil change were computed in 1-s time bins during the anticipation phase. Activity in the ventral striatum was inversely related to the pupil size time course, indicating an early onset of activation and a role in reward prediction processing. Pupil dilations were linked to increased activity in the salience network (dorsal ACC and bilateral insula), which likely triggers an increase in arousal to enhance task performance. Finally, increased pupil size preceding the required motor response was associated with activity in the ventral attention network. In sum, pupillometry provides an effective tool for disentangling different phases of reward anticipation, with relevance for affective symptomatology. Copyright © 2018 Elsevier Inc. All rights reserved.

Reward deficiency and anti-reward in pain chronification.

Science.gov (United States)

Borsook, D; Linnman, C; Faria, V; Strassman, A M; Becerra, L; Elman, I

2016-09-01

Converging lines of evidence suggest that the pathophysiology of pain is mediated to a substantial degree via allostatic neuroadaptations in reward- and stress-related brain circuits. Thus, reward deficiency (RD) represents a within-system neuroadaptation to pain-induced protracted activation of the reward circuits that leads to depletion-like hypodopaminergia, clinically manifested anhedonia, and diminished motivation for natural reinforcers. Anti-reward (AR) conversely pertains to a between-systems neuroadaptation involving over-recruitment of key limbic structures (e.g., the central and basolateral amygdala nuclei, the bed nucleus of the stria terminalis, the lateral tegmental noradrenergic nuclei of the brain stem, the hippocampus and the habenula) responsible for massive outpouring of stressogenic neurochemicals (e.g., norepinephrine, corticotropin releasing factor, vasopressin, hypocretin, and substance P) giving rise to such negative affective states as anxiety, fear and depression. We propose here the Combined Reward deficiency and Anti-reward Model (CReAM), in which biopsychosocial variables modulating brain reward, motivation and stress functions can interact in a 'downward spiral' fashion to exacerbate the intensity, chronicity and comorbidities of chronic pain syndromes (i.e., pain chronification). Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

PER1 rs3027172 Genotype Interacts with Early Life Stress to Predict Problematic Alcohol Use, but Not Reward-Related Ventral Striatum Activity

Science.gov (United States)

Baranger, David A. A.; Ifrah, Chloé; Prather, Aric A.; Carey, Caitlin E.; Corral-Frías, Nadia S.; Drabant Conley, Emily; Hariri, Ahmad R.; Bogdan, Ryan

2016-01-01

Increasing evidence suggests that the circadian and stress regulatory systems contribute to alcohol use disorder (AUD) risk, which may partially arise through effects on reward-related neural function. The C allele of the PER1 rs3027172 single nucleotide polymorphism (SNP) reduces PER1 expression in cells incubated with cortisol and has been associated with increased risk for adult AUD and problematic drinking among adolescents exposed to high levels of familial psychosocial adversity. Using data from undergraduate students who completed the ongoing Duke Neurogenetics Study (DNS) (n = 665), we tested whether exposure to early life stress (ELS; Childhood Trauma Questionnaire) moderates the association between rs3027172 genotype and later problematic alcohol use (Alcohol Use Disorders Identification Test) as well as ventral striatum (VS) reactivity to reward (card-guessing task while functional magnetic resonance imaging data were acquired). Initial analyses found that PER1 rs3027172 genotype interacted with ELS to predict both problematic drinking and VS reactivity; minor C allele carriers, who were also exposed to elevated ELS reported greater problematic drinking and exhibited greater ventral striatum reactivity to reward-related stimuli. When gene × covariate and environment × covariate interactions were controlled for, the interaction predicting problematic alcohol use remained significant (p < 0.05, corrected) while the interaction predicting VS reactivity was no longer significant. These results extend our understanding of relationships between PER1 genotype, ELS, and problematic alcohol use, and serve as a cautionary tale on the importance of controlling for potential confounders in studies of moderation including gene × environment interactions. PMID:27065929

PER1 rs3027172 genotype interacts with early life stress to predict problematic alcohol use, but not reward-related ventral striatum activity

Directory of Open Access Journals (Sweden)

David eBaranger

2016-03-01

Full Text Available Increasing evidence suggests that the circadian and stress regulatory systems contribute to alcohol use disorder (AUD risk, which may partially arise through effects on reward-related neural function. The C allele of the PER1 rs3027172 single nucleotide polymorphism reduces PER1 expression in cells incubated with cortisol and has been associated with increased risk for adult AUD and problematic drinking among adolescents exposed to high levels of familial psychosocial adversity. Using data from undergraduate students who completed the ongoing Duke Neurogenetics Study (n=665, we tested whether exposure to early life stress (ELS; Childhood Trauma Questionnaire moderates the association between rs3027172 genotype and later problematic alcohol use (Alcohol Use Disorders Identification Test as well as ventral striatum (VS reactivity to reward (card-guessing task while functional magnetic resonance imaging data were acquired. Initial analyses found that PER1 rs3027172 genotype interacted with ELS to predict both problematic drinking and VS reactivity; minor C allele carriers, who were also exposed to elevated ELS reported greater problematic drinking and exhibited greater ventral striatum reactivity to reward-related stimuli. When gene x covariate and environment x covariate interactions were controlled for, the interaction predicting problematic alcohol use remained significant (p<0.05, corrected while the interaction predicting VS reactivity was no longer significant. These results extend our understanding of relationships between PER1 genotype, early life stress, and problematic alcohol use, and serve as a cautionary tale on the importance of controlling for potential confounders in studies of moderation including gene x environment interactions.

Enriched encoding: reward motivation organizes cortical networks for hippocampal detection of unexpected events.

Science.gov (United States)

Murty, Vishnu P; Adcock, R Alison

2014-08-01

Learning how to obtain rewards requires learning about their contexts and likely causes. How do long-term memory mechanisms balance the need to represent potential determinants of reward outcomes with the computational burden of an over-inclusive memory? One solution would be to enhance memory for salient events that occur during reward anticipation, because all such events are potential determinants of reward. We tested whether reward motivation enhances encoding of salient events like expectancy violations. During functional magnetic resonance imaging, participants performed a reaction-time task in which goal-irrelevant expectancy violations were encountered during states of high- or low-reward motivation. Motivation amplified hippocampal activation to and declarative memory for expectancy violations. Connectivity of the ventral tegmental area (VTA) with medial prefrontal, ventrolateral prefrontal, and visual cortices preceded and predicted this increase in hippocampal sensitivity. These findings elucidate a novel mechanism whereby reward motivation can enhance hippocampus-dependent memory: anticipatory VTA-cortical-hippocampal interactions. Further, the findings integrate literatures on dopaminergic neuromodulation of prefrontal function and hippocampus-dependent memory. We conclude that during reward motivation, VTA modulation induces distributed neural changes that amplify hippocampal signals and records of expectancy violations to improve predictions-a potentially unique contribution of the hippocampus to reward learning. © The Author 2013. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

A neurocomputational account of reward and novelty processing and effects of psychostimulants in attention deficit hyperactivity disorder.

Science.gov (United States)

Sethi, Arjun; Voon, Valerie; Critchley, Hugo D; Cercignani, Mara; Harrison, Neil A

2018-05-01

Computational models of reinforcement learning have helped dissect discrete components of reward-related function and characterize neurocognitive deficits in psychiatric illnesses. Stimulus novelty biases decision-making, even when unrelated to choice outcome, acting as if possessing intrinsic reward value to guide decisions toward uncertain options. Heightened novelty seeking is characteristic of attention deficit hyperactivity disorder, yet how this influences reward-related decision-making is computationally encoded, or is altered by stimulant medication, is currently uncertain. Here we used an established reinforcement-learning task to model effects of novelty on reward-related behaviour during functional MRI in 30 adults with attention deficit hyperactivity disorder and 30 age-, sex- and IQ-matched control subjects. Each participant was tested on two separate occasions, once ON and once OFF stimulant medication. OFF medication, patients with attention deficit hyperactivity disorder showed significantly impaired task performance (P = 0.027), and greater selection of novel options (P = 0.004). Moreover, persistence in selecting novel options predicted impaired task performance (P = 0.025). These behavioural deficits were accompanied by a significantly lower learning rate (P = 0.011) and heightened novelty signalling within the substantia nigra/ventral tegmental area (family-wise error corrected P attention deficit hyperactivity disorder participants' overall task performance (P = 0.011), increased reward-learning rates (P = 0.046) and enhanced their ability to differentiate optimal from non-optimal novel choices (P = 0.032). It also reduced substantia nigra/ventral tegmental area responses to novelty. Preliminary cross-sectional evidence additionally suggested an association between long-term stimulant treatment and a reduction in the rewarding value of novelty. These data suggest that aberrant substantia nigra/ventral tegmental area novelty processing plays an

Error-related brain activity predicts cocaine use after treatment at 3-month follow-up.

Science.gov (United States)

Marhe, Reshmi; van de Wetering, Ben J M; Franken, Ingmar H A

2013-04-15

Relapse after treatment is one of the most important problems in drug dependency. Several studies suggest that lack of cognitive control is one of the causes of relapse. In this study, a relative new electrophysiologic index of cognitive control, the error-related negativity, is investigated to examine its suitability as a predictor of relapse. The error-related negativity was measured in 57 cocaine-dependent patients during their first week in detoxification treatment. Data from 49 participants were used to predict cocaine use at 3-month follow-up. Cocaine use at follow-up was measured by means of self-reported days of cocaine use in the last month verified by urine screening. A multiple hierarchical regression model was used to examine the predictive value of the error-related negativity while controlling for addiction severity and self-reported craving in the week before treatment. The error-related negativity was the only significant predictor in the model and added 7.4% of explained variance to the control variables, resulting in a total of 33.4% explained variance in the prediction of days of cocaine use at follow-up. A reduced error-related negativity measured during the first week of treatment was associated with more days of cocaine use at 3-month follow-up. Moreover, the error-related negativity was a stronger predictor of recent cocaine use than addiction severity and craving. These results suggest that underactive error-related brain activity might help to identify patients who are at risk of relapse as early as in the first week of detoxification treatment. Copyright © 2013 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

Sensitivity to reward: implications for overeating and overweight.

Science.gov (United States)

Davis, Caroline; Strachan, Shaelyn; Berkson, Marni

2004-04-01

Sensitivity to reward (STR)-a personality trait firmly rooted in the neurobiology of the mesolimbic dopamine system-has been strongly implicated in the risk for addiction. This construct describes the ability to derive pleasure or reward from natural reinforcers like food, and from pharmacologic rewards like addictive drugs. Recently experts in the field of addiction research have acknowledged that psychomotor stimulant drugs are no longer at the heart of all addictions, and that brain circuits can also be deranged with natural rewards like food. The present study tested a model in which STR was expected to relate positively to overeating, which in turn would be associated with higher body weight in woman aged 25-45 years. As predicted, STR was correlated positively with measures of emotional overeating. Also, overweight woman were significantly more sensitive to reward than those of normal weight. Interestingly, however, the obese woman (Body Mass Index>30) were more anhedonic than the overweight woman (Body Mass Index>25reward circuits. Results also indicate that STR may serve as a risk factor for overeating and overweight, especially in cultures such as ours where palatable, calorically-dense food is plentiful.

Correction for Measurement Error from Genotyping-by-Sequencing in Genomic Variance and Genomic Prediction Models

DEFF Research Database (Denmark)

Ashraf, Bilal; Janss, Luc; Jensen, Just

sample). The GBSeq data can be used directly in genomic models in the form of individual SNP allele-frequency estimates (e.g., reference reads/total reads per polymorphic site per individual), but is subject to measurement error due to the low sequencing depth per individual. Due to technical reasons....... In the current work we show how the correction for measurement error in GBSeq can also be applied in whole genome genomic variance and genomic prediction models. Bayesian whole-genome random regression models are proposed to allow implementation of large-scale SNP-based models with a per-SNP correction...... for measurement error. We show correct retrieval of genomic explained variance, and improved genomic prediction when accounting for the measurement error in GBSeq data...

Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction.

Science.gov (United States)

Flagel, Shelly B; Akil, Huda; Robinson, Terry E

2009-01-01

Drugs of abuse acquire different degrees of control over thoughts and actions based not only on the effects of drugs themselves, but also on predispositions of the individual. Those individuals who become addicted are unable to shift their thoughts and actions away from drugs and drug-associated stimuli. Thus in addicts, exposure to places or things (cues) that has been previously associated with drug-taking often instigates renewed drug-taking. We and others have postulated that drug-associated cues acquire the ability to maintain and instigate drug-taking behavior in part because they acquire incentive motivational properties through Pavlovian (stimulus-stimulus) learning. In the case of compulsive behavioral disorders, including addiction, such cues may be attributed with pathological incentive value ("incentive salience"). For this reason, we have recently begun to explore individual differences in the tendency to attribute incentive salience to cues that predict rewards. When discrete cues are associated with the non-contingent delivery of food or drug rewards some animals come to quickly approach and engage the cue even if it is located at a distance from where the reward will be delivered. In these animals the reward-predictive cue itself becomes attractive, eliciting approach towards it, presumably because it is attributed with incentive salience. Animals that develop this type of conditional response are called "sign-trackers". Other animals, "goal-trackers", do not approach the reward-predictive cue, but upon cue presentation they immediately go to the location where food will be delivered (the "goal"). For goal-trackers the reward-predictive cue is not attractive, presumably because it is not attributed with incentive salience. We review here preliminary data suggesting that these individual differences in the tendency to attribute incentive salience to cues predictive of reward may confer vulnerability or resistance to compulsive behavioral disorders

Reward-related learning via multiple memory systems.

Science.gov (United States)

Delgado, Mauricio R; Dickerson, Kathryn C

2012-07-15

The application of a neuroeconomic approach to the study of reward-related processes has provided significant insights in our understanding of human learning and decision making. Much of this research has focused primarily on the contributions of the corticostriatal circuitry, involved in trial-and-error reward learning. As a result, less consideration has been allotted to the potential influence of different neural mechanisms such as the hippocampus or to more common ways in human society in which information is acquired and utilized to reach a decision, such as through explicit instruction rather than trial-and-error learning. This review examines the individual contributions of multiple learning and memory neural systems and their interactions during human decision making in both normal and neuropsychiatric populations. Specifically, the anatomical and functional connectivity across multiple memory systems are highlighted to suggest that probing the role of the hippocampus and its interactions with the corticostriatal circuitry via the application of model-based neuroeconomic approaches may provide novel insights into neuropsychiatric populations that suffer from damage to one of these structures and as a consequence have deficits in learning, memory, or decision making. Copyright © 2012 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

The reward probability index: design and validation of a scale measuring access to environmental reward.

Science.gov (United States)

Carvalho, John P; Gawrysiak, Michael J; Hellmuth, Julianne C; McNulty, James K; Magidson, Jessica F; Lejuez, C W; Hopko, Derek R

2011-06-01

Behavioral models of depression implicate decreased response-contingent positive reinforcement (RCPR) as critical toward the development and maintenance of depression (Lewinsohn, 1974). Given the absence of a psychometrically sound self-report measure of RCPR, the Reward Probability Index (RPI) was developed to measure access to environmental reward and to approximate actual RCPR. In Study 1 (n=269), exploratory factor analysis supported a 20-item two-factor model (Reward Probability, Environmental Suppressors) with strong internal consistency (α=.90). In Study 2 (n=281), confirmatory factor analysis supported this two-factor structure and convergent validity was established through strong correlations between the RPI and measures of activity, avoidance, reinforcement, and depression (r=.65 to .81). Discriminant validity was supported via smaller correlations between the RPI and measures of social support and somatic anxiety (r=-.29 to -.40). Two-week test-retest reliability was strong (r=.69). In Study 3 (n=33), controlling for depression symptoms, hierarchical regression supported the incremental validity of the RPI in predicting daily diary reports of environmental reward. The RPI represents a parsimonious, reliable, and valid measure that may facilitate understanding of the etiology of depression and its relationship to overt behaviors. Copyright © 2011. Published by Elsevier Ltd.

A tribute to Charlie Chaplin: Induced positive affect improves reward-based decision-learning in Parkinson’s Disease

Directory of Open Access Journals (Sweden)

K. Richard eRidderinkhof

2012-06-01

Full Text Available Reward-based decision-learning refers to the process of learning to select those actions that lead to rewards while avoiding actions that lead to punishments. This process, known to rely on dopaminergic activity in striatal brain regions, is compromised in Parkinson’s disease (PD. We hypothesized that such decision-learning deficits are alleviated by induced positive affect, which is thought to incur transient boosts in midbrain and striatal dopaminergic activity. Computational measures of probabilistic reward-based decision-learning were determined for 51 patients diagnosed with PD. Previous work has shown these measures to rely on the nucleus caudatus (outcome evaluation during the early phases of learning and the putamen (reward prediction during later phases of learning. We observed that induced positive affect facilitated learning, through its effects on reward prediction rather than outcome evaluation. Viewing a few minutes of comedy clips served to remedy dopamine-related problems in putamen-based frontostriatal circuitry and, consequently, in learning to predict which actions will yield reward.

A Behavioral Economic Reward Index Predicts Drinking Resolutions: Moderation Re-visited and Compared with Other Outcomes

Science.gov (United States)

Tucker, Jalie A.; Roth, David L.; Vignolo, Mary J.; Westfall, Andrew O.

2014-01-01

Data were pooled from three studies of recently resolved community-dwelling problem drinkers to determine whether a behavioral economic index of the value of rewards available over different time horizons distinguished among moderation (n = 30), abstinent (n = 95), and unresolved (n = 77) outcomes. Moderation over 1-2 year prospective follow-up intervals was hypothesized to involve longer term behavior regulation processes compared to abstinence or relapse and to be predicted by more balanced pre-resolution monetary allocations between short- and longer-term objectives (i.e., drinking and saving for the future). Standardized odds ratios (OR) based on changes in standard deviation units from a multinomial logistic regression indicated that increases on this “Alcohol-Savings Discretionary Expenditure” index predicted higher rates of both abstinence (OR = 1.93, p = .004) and relapse (OR = 2.89, p moderation outcomes. The index had incremental utility in predicting moderation in complex models that included other established predictors. The study adds to evidence supporting a behavioral economic analysis of drinking resolutions and shows that a systematic analysis of pre-resolution spending patterns aids in predicting moderation. PMID:19309182

Reward uncertainty enhances incentive salience attribution as sign-tracking

Science.gov (United States)

Anselme, Patrick; Robinson, Mike J. F.; Berridge, Kent C.

2014-01-01

Conditioned stimuli (CSs) come to act as motivational magnets following repeated association with unconditioned stimuli (UCSs) such as sucrose rewards. By traditional views, the more reliably predictive a Pavlovian CS-UCS association, the more the CS becomes attractive. However, in some cases, less predictability might equal more motivation. Here we examined the effect of introducing uncertainty in CS-UCS association on CS strength as an attractive motivation magnet. In the present study, Experiment 1 assessed the effects of Pavlovian predictability versus uncertainty about reward probability and/or reward magnitude on the acquisition and expression of sign-tracking (ST) and goal-tracking (GT) responses in an autoshaping procedure. Results suggested that uncertainty produced strongest incentive salience expressed as sign-tracking. Experiment 2 examined whether a within-individual temporal shift from certainty to uncertainty conditions could produce a stronger CS motivational magnet when uncertainty began, and found that sign-tracking still increased after the shift. Overall, our results support earlier reports that ST responses become more pronounced in the presence of uncertainty regarding CS-UCS associations, especially when uncertainty combines both probability and magnitude. These results suggest that Pavlovian uncertainty, although diluting predictability, is still able to enhance the incentive motivational power of particular CSs. PMID:23078951

Distinct medial temporal networks encode surprise during motivation by reward versus punishment

Science.gov (United States)

Murty, Vishnu P.; LaBar, Kevin S.; Adcock, R. Alison

2016-01-01

Adaptive motivated behavior requires predictive internal representations of the environment, and surprising events are indications for encoding new representations of the environment. The medial temporal lobe memory system, including the hippocampus and surrounding cortex, encodes surprising events and is influenced by motivational state. Because behavior reflects the goals of an individual, we investigated whether motivational valence (i.e., pursuing rewards versus avoiding punishments) also impacts neural and mnemonic encoding of surprising events. During functional magnetic resonance imaging (fMRI), participants encountered perceptually unexpected events either during the pursuit of rewards or avoidance of punishments. Despite similar levels of motivation across groups, reward and punishment facilitated the processing of surprising events in different medial temporal lobe regions. Whereas during reward motivation, perceptual surprises enhanced activation in the hippocampus, during punishment motivation surprises instead enhanced activation in parahippocampal cortex. Further, we found that reward motivation facilitated hippocampal coupling with ventromedial PFC, whereas punishment motivation facilitated parahippocampal cortical coupling with orbitofrontal cortex. Behaviorally, post-scan testing revealed that reward, but not punishment, motivation resulted in greater memory selectivity for surprising events encountered during goal pursuit. Together these findings demonstrate that neuromodulatory systems engaged by anticipation of reward and punishment target separate components of the medial temporal lobe, modulating medial temporal lobe sensitivity and connectivity. Thus, reward and punishment motivation yield distinct neural contexts for learning, with distinct consequences for how surprises are incorporated into predictive mnemonic models of the environment. PMID:26854903

Distinct medial temporal networks encode surprise during motivation by reward versus punishment.

Science.gov (United States)

Murty, Vishnu P; LaBar, Kevin S; Adcock, R Alison

2016-10-01

Adaptive motivated behavior requires predictive internal representations of the environment, and surprising events are indications for encoding new representations of the environment. The medial temporal lobe memory system, including the hippocampus and surrounding cortex, encodes surprising events and is influenced by motivational state. Because behavior reflects the goals of an individual, we investigated whether motivational valence (i.e., pursuing rewards versus avoiding punishments) also impacts neural and mnemonic encoding of surprising events. During functional magnetic resonance imaging (fMRI), participants encountered perceptually unexpected events either during the pursuit of rewards or avoidance of punishments. Despite similar levels of motivation across groups, reward and punishment facilitated the processing of surprising events in different medial temporal lobe regions. Whereas during reward motivation, perceptual surprises enhanced activation in the hippocampus, during punishment motivation surprises instead enhanced activation in parahippocampal cortex. Further, we found that reward motivation facilitated hippocampal coupling with ventromedial PFC, whereas punishment motivation facilitated parahippocampal cortical coupling with orbitofrontal cortex. Behaviorally, post-scan testing revealed that reward, but not punishment, motivation resulted in greater memory selectivity for surprising events encountered during goal pursuit. Together these findings demonstrate that neuromodulatory systems engaged by anticipation of reward and punishment target separate components of the medial temporal lobe, modulating medial temporal lobe sensitivity and connectivity. Thus, reward and punishment motivation yield distinct neural contexts for learning, with distinct consequences for how surprises are incorporated into predictive mnemonic models of the environment. Copyright © 2016 Elsevier Inc. All rights reserved.

Brain negativity as an indicator of predictive error processing: the contribution of visual action effect monitoring.

Science.gov (United States)

Joch, Michael; Hegele, Mathias; Maurer, Heiko; Müller, Hermann; Maurer, Lisa Katharina

2017-07-01

The error (related) negativity (Ne/ERN) is an event-related potential in the electroencephalogram (EEG) correlating with error processing. Its conditions of appearance before terminal external error information suggest that the Ne/ERN is indicative of predictive processes in the evaluation of errors. The aim of the present study was to specifically examine the Ne/ERN in a complex motor task and to particularly rule out other explaining sources of the Ne/ERN aside from error prediction processes. To this end, we focused on the dependency of the Ne/ERN on visual monitoring about the action outcome after movement termination but before result feedback (action effect monitoring). Participants performed a semi-virtual throwing task by using a manipulandum to throw a virtual ball displayed on a computer screen to hit a target object. Visual feedback about the ball flying to the target was masked to prevent action effect monitoring. Participants received a static feedback about the action outcome (850 ms) after each trial. We found a significant negative deflection in the average EEG curves of the error trials peaking at ~250 ms after ball release, i.e., before error feedback. Furthermore, this Ne/ERN signal did not depend on visual ball-flight monitoring after release. We conclude that the Ne/ERN has the potential to indicate error prediction in motor tasks and that it exists even in the absence of action effect monitoring. NEW & NOTEWORTHY In this study, we are separating different kinds of possible contributors to an electroencephalogram (EEG) error correlate (Ne/ERN) in a throwing task. We tested the influence of action effect monitoring on the Ne/ERN amplitude in the EEG. We used a task that allows us to restrict movement correction and action effect monitoring and to control the onset of result feedback. We ascribe the Ne/ERN to predictive error processing where a conscious feeling of failure is not a prerequisite. Copyright © 2017 the American Physiological

Sensitivity for Cues Predicting Reward and Punishment in Young Women with Eating Disorders.

Science.gov (United States)

Matton, Annelies; de Jong, Peter; Goossens, Lien; Jonker, Nienke; Van Malderen, Eva; Vervaet, Myriam; De Schryver, Nele; Braet, Caroline

2017-11-01

Increasing evidence shows that sensitivity to reward (SR) and punishment (SP) may be involved in eating disorders (EDs). Most studies used self-reported positive/negative effect in rewarding/punishing situations, whereas the implied proneness to detect signals of reward/punishment is largely ignored. This pilot study used a spatial orientation task to examine transdiagnostic and interdiagnostic differences in SR/SP. Participants (14-29 years) were patients with anorexia nervosa of restricting type (AN-R, n = 20), binge/purge ED group [AN of binge/purge type and bulimia nervosa (n = 16)] and non-symptomatic individuals (n = 23). Results revealed stronger difficulties to redirect attention away from signals of rewards in AN-R compared with binge/purge EDs, and binge/purge EDs showed stronger difficulties to direct attention away from signals of punishment compared with AN-R. Findings demonstrate interdiagnostic differences and show that the spatial orientation task is sensitive for individual differences in SP/SR within the context of EDs, thereby sustaining its usefulness as behavioural measure of reinforcement sensitivity. Copyright © 2017 John Wiley & Sons, Ltd and Eating Disorders Association. Copyright © 2017 John Wiley & Sons, Ltd and Eating Disorders Association.

Rewarding leadership and fair procedures as determinants of self-esteem.

Science.gov (United States)

De Cremer, David; van Knippenberg, Barbara; van Knippenberg, Daan; Mullenders, Danny; Stinglhamber, Florence

2005-01-01

In the present research, the authors examined the effect of procedural fairness and rewarding leadership style on an important variable for employees: self-esteem. The authors predicted that procedural fairness would positively influence people's reported self-esteem if the leader adopted a style of rewarding behavior for a job well done. Results from a scenario experiment, a laboratory experiment, and an organizational survey indeed show that procedural fairness and rewarding leadership style interacted to influence followers' self-esteem, such that the positive relationship between procedural fairness and self-esteem was more pronounced when the leadership style was high in rewarding behavior. Implications in terms of integrating the leadership and procedural fairness literature are discussed.

Rewards and Performance Incentives.

Science.gov (United States)

Zigon, Jack

1994-01-01

Discusses rewards and performance incentives for employees, including types of rewards; how rewards help in managing; dysfunctional awards; selecting the right reward; how to find rewards that fit; and delivering rewards effectively. Examples are included. (three references) (LRW)

Reward loss and the basolateral amygdala: A function in reward comparisons.

Science.gov (United States)

Kawasaki, Katsuyoshi; Annicchiarico, Iván; Glueck, Amanda C; Morón, Ignacio; Papini, Mauricio R

2017-07-28

The neural circuitry underlying behavior in reward loss situations is poorly understood. We considered two such situations: reward devaluation (from large to small rewards) and reward omission (from large rewards to no rewards). There is evidence that the central nucleus of the amygdala (CeA) plays a role in the negative emotion accompanying reward loss. However, little is known about the function of the basolateral nucleus (BLA) in reward loss. Two hypotheses of BLA function in reward loss, negative emotion and reward comparisons, were tested in an experiment involving pretraining excitotoxic BLA lesions followed by training in four tasks: consummatory successive negative contrast (cSNC), autoshaping (AS) acquisition and extinction, anticipatory negative contrast (ANC), and open field testing (OF). Cell counts in the BLA (but not in the CeA) were significantly lower in animals with lesions vs. shams. BLA lesions eliminated cSNC and ANC, and accelerated extinction of lever pressing in AS. BLA lesions had no effect on OF testing: higher activity in the periphery than in the central area. This pattern of results provides support for the hypothesis that BLA neurons are important for reward comparison. The three affected tasks (cSNC, ANC, and AS extinction) involve reward comparisons. However, ANC does not seem to involve negative emotions and it was affected, whereas OF activity is known to involve negative emotion, but it was not affected. It is hypothesized that a circuit involving the thalamus, insular cortex, and BLA is critically involved in the mechanism comparing current and expected rewards. Copyright © 2017 Elsevier B.V. All rights reserved.

A Fly's Eye View of Natural and Drug Reward.

Science.gov (United States)

Lowenstein, Eve G; Velazquez-Ulloa, Norma A

2018-01-01

Animals encounter multiple stimuli each day. Some of these stimuli are innately appetitive or aversive, while others are assigned valence based on experience. Drugs like ethanol can elicit aversion in the short term and attraction in the long term. The reward system encodes the predictive value for different stimuli, mediating anticipation for attractive or punishing stimuli and driving animal behavior to approach or avoid conditioned stimuli. The neurochemistry and neurocircuitry of the reward system is partly evolutionarily conserved. In both vertebrates and invertebrates, including Drosophila melanogaster , dopamine is at the center of a network of neurotransmitters and neuromodulators acting in concert to encode rewards. Behavioral assays in D. melanogaster have become increasingly sophisticated, allowing more direct comparison with mammalian research. Moreover, recent evidence has established the functional modularity of the reward neural circuits in Drosophila . This functional modularity resembles the organization of reward circuits in mammals. The powerful genetic and molecular tools for D. melanogaster allow characterization and manipulation at the single-cell level. These tools are being used to construct a detailed map of the neural circuits mediating specific rewarding stimuli and have allowed for the identification of multiple genes and molecular pathways that mediate the effects of reinforcing stimuli, including their rewarding effects. This report provides an overview of the research on natural and drug reward in D. melanogaster , including natural rewards such as sugar and other food nutrients, and drug rewards including ethanol, cocaine, amphetamine, methamphetamine, and nicotine. We focused mainly on the known genetic and neural mechanisms underlying appetitive reward for sugar and reward for ethanol. We also include genes, molecular pathways, and neural circuits that have been identified using assays that test the palatability of the rewarding

Pupil dilation indicates the coding of past prediction errors: Evidence for attentional learning theory.

Science.gov (United States)

Koenig, Stephan; Uengoer, Metin; Lachnit, Harald

2018-04-01

The attentional learning theory of Pearce and Hall () predicts more attention to uncertain cues that have caused a high prediction error in the past. We examined how the cue-elicited pupil dilation during associative learning was linked to such error-driven attentional processes. In three experiments, participants were trained to acquire associations between different cues and their appetitive (Experiment 1), motor (Experiment 2), or aversive (Experiment 3) outcomes. All experiments were designed to examine differences in the processing of continuously reinforced cues (consistently followed by the outcome) versus partially reinforced, uncertain cues (randomly followed by the outcome). We measured the pupil dilation elicited by the cues in anticipation of the outcome and analyzed how this conditioned pupil response changed over the course of learning. In all experiments, changes in pupil size complied with the same basic pattern: During early learning, consistently reinforced cues elicited greater pupil dilation than uncertain, randomly reinforced cues, but this effect gradually reversed to yield a greater pupil dilation for uncertain cues toward the end of learning. The pattern of data accords with the changes in prediction error and error-driven attention formalized by the Pearce-Hall theory. © 2017 The Authors. Psychophysiology published by Wiley Periodicals, Inc. on behalf of Society for Psychophysiological Research.

Neural activation to monetary reward is associated with amphetamine reward sensitivity.

Science.gov (United States)

Crane, Natania A; Gorka, Stephanie M; Weafer, Jessica; Langenecker, Scott A; de Wit, Harriet; Phan, K Luan

2018-03-14

One known risk factor for drug use and abuse is sensitivity to rewarding effects of drugs. It is not known whether this risk factor extends to sensitivity to non-drug rewards. In this study with healthy young adults, we examined the association between sensitivity to the subjective rewarding effects of amphetamine and a neural indicator of anticipation of monetary reward. We hypothesized that greater euphorigenic response to amphetamine would be associated with greater neural activation to anticipation of monetary reward (Win > Loss). Healthy participants (N = 61) completed four laboratory sessions in which they received d-amphetamine (20 mg) and placebo in alternating order, providing self-report measures of euphoria and stimulation at regular intervals. At a separate visit 1-3 weeks later, participants completed the guessing reward task (GRT) during fMRI in a drug-free state. Participants reporting greater euphoria after amphetamine also exhibited greater neural activation during monetary reward anticipation in mesolimbic reward regions, including the bilateral caudate and putamen. This is the first study to show a relationship between neural correlates of monetary reward and sensitivity to the subjective rewarding effects of amphetamine in humans. These findings support growing evidence that sensitivity to reward in general is a risk factor for drug use and abuse, and suggest that sensitivity of drug-induced euphoria may reflect a general sensitivity to rewards. This may be an index of vulnerability for drug use or abuse.

A Cerebellar Framework for Predictive Coding and Homeostatic Regulation in Depressive Disorder.

Science.gov (United States)

Schutter, Dennis J L G

2016-02-01

Depressive disorder is associated with abnormalities in the processing of reward and punishment signals and disturbances in homeostatic regulation. These abnormalities are proposed to impair error minimization routines for reducing uncertainty. Several lines of research point towards a role of the cerebellum in reward- and punishment-related predictive coding and homeostatic regulatory function in depressive disorder. Available functional and anatomical evidence suggests that in addition to the cortico-limbic networks, the cerebellum is part of the dysfunctional brain circuit in depressive disorder as well. It is proposed that impaired cerebellar function contributes to abnormalities in predictive coding and homeostatic dysregulation in depressive disorder. Further research on the role of the cerebellum in depressive disorder may further extend our knowledge on the functional and neural mechanisms of depressive disorder and development of novel antidepressant treatments strategies targeting the cerebellum.

Elevated Striatal Reactivity Across Monetary and Social Rewards in Bipolar I Disorder

Science.gov (United States)

Dutra, Sunny J.; Cunningham, William A.; Kober, Hedy; Gruber, June

2016-01-01

Bipolar disorder (BD) is associated with increased reactivity to rewards and heightened positive affectivity. It is less clear to what extent this heightened reward sensitivity is evident across contexts and what the associated neural mechanisms might be. The present investigation employed both a monetary and social incentive delay task among adults with remitted BD type I (N=24) and a healthy non-psychiatric control group (HC; N=25) using fMRI. Both whole-brain and region-of-interest analyses revealed elevated ventral and dorsal striatal reactivity across monetary and social reward receipt, but not anticipation, in the BD group. Post-hoc analyses further suggested that greater striatal reactivity to reward receipt across monetary and social reward tasks predicted decreased self-reported positive affect when anticipating subsequent rewards in the HC, but not BD, group. Results point toward elevated striatal reactivity to reward receipt as a potential neural mechanism of reward reactivity. PMID:26390194

Beyond Rewards

Science.gov (United States)

Hall, Philip S.

2009-01-01

Using rewards to impact students' behavior has long been common practice. However, using reward systems to enhance student learning conveniently masks the larger and admittedly more difficult task of finding and implementing the structure and techniques that children with special needs require to learn. More important, rewarding the child for good…

Amygdala Contributions to Stimulus–Reward Encoding in the Macaque Medial and Orbital Frontal Cortex during Learning

Science.gov (United States)

Averbeck, Bruno B.

2017-01-01

Orbitofrontal cortex (OFC), medial frontal cortex (MFC), and amygdala mediate stimulus–reward learning, but the mechanisms through which they interact are unclear. Here, we investigated how neurons in macaque OFC and MFC signaled rewards and the stimuli that predicted them during learning with and without amygdala input. Macaques performed a task that required them to evaluate two stimuli and then choose one to receive the reward associated with that option. Four main findings emerged. First, amygdala lesions slowed the acquisition and use of stimulus–reward associations. Further analyses indicated that this impairment was due, at least in part, to ineffective use of negative feedback to guide subsequent decisions. Second, the activity of neurons in OFC and MFC rapidly evolved to encode the amount of reward associated with each stimulus. Third, amygdalectomy reduced encoding of stimulus–reward associations during the evaluation of different stimuli. Reward encoding of anticipated and received reward after choices were made was not altered. Fourth, amygdala lesions led to an increase in the proportion of neurons in MFC, but not OFC, that encoded the instrumental response that monkeys made on each trial. These correlated changes in behavior and neural activity after amygdala lesions strongly suggest that the amygdala contributes to the ability to learn stimulus–reward associations rapidly by shaping encoding within OFC and MFC. SIGNIFICANCE STATEMENT Altered functional interactions among orbital frontal cortex (OFC), medial frontal cortex (MFC), and amygdala are thought to underlie several psychiatric conditions, many related to reward learning. Here, we investigated the causal contribution of the amygdala to the development of neuronal activity in macaque OFC and MFC related to rewards and the stimuli that predict them during learning. Without amygdala inputs, neurons in both OFC and MFC showed decreased encoding of stimulus–reward associations. MFC also

A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping.

Science.gov (United States)

Takikawa, Yoriko; Kawagoe, Reiko; Hikosaka, Okihide

2004-10-01

Dopamine (DA) neurons respond to sensory stimuli that predict reward. To understand how DA neurons acquire such ability, we trained monkeys on a one-direction-rewarded version of memory-guided saccade task (1DR) only when we recorded from single DA neurons. In 1DR, position-reward mapping was changed across blocks of trials. In the early stage of training of 1DR, DA neurons responded to reward delivery; in the later stages, they responded predominantly to the visual cue that predicted reward or no reward (reward predictor) differentially. We found that such a shift of activity from reward to reward predictor also occurred within a block of trials after position-reward mapping was altered. A main effect of long-term training was to accelerate the within-block reward-to-predictor shift of DA neuronal responses. The within-block shift appeared first in the intermediate stage, but was slow, and DA neurons often responded to the cue that indicated reward in the preceding block. In the advanced stage, the reward-to-predictor shift occurred quickly such that the DA neurons' responses to visual cues faithfully matched the current position-reward mapping. Changes in the DA neuronal responses co-varied with the reward-predictive differentiation of saccade latency both in short-term (within-block) and long-term adaptation. DA neurons' response to the fixation point also underwent long-term changes until it occurred predominantly in the first trial within a block. This might trigger a switch between the learned sets. These results suggest that midbrain DA neurons play an essential role in adapting oculomotor behavior to frequent switches in position-reward mapping.

Towards a general theory of neural computation based on prediction by single neurons.

Directory of Open Access Journals (Sweden)

Christopher D Fiorillo

Full Text Available Although there has been tremendous progress in understanding the mechanics of the nervous system, there has not been a general theory of its computational function. Here I present a theory that relates the established biophysical properties of single generic neurons to principles of Bayesian probability theory, reinforcement learning and efficient coding. I suggest that this theory addresses the general computational problem facing the nervous system. Each neuron is proposed to mirror the function of the whole system in learning to predict aspects of the world related to future reward. According to the model, a typical neuron receives current information about the state of the world from a subset of its excitatory synaptic inputs, and prior information from its other inputs. Prior information would be contributed by synaptic inputs representing distinct regions of space, and by different types of non-synaptic, voltage-regulated channels representing distinct periods of the past. The neuron's membrane voltage is proposed to signal the difference between current and prior information ("prediction error" or "surprise". A neuron would apply a Hebbian plasticity rule to select those excitatory inputs that are the most closely correlated with reward but are the least predictable, since unpredictable inputs provide the neuron with the most "new" information about future reward. To minimize the error in its predictions and to respond only when excitation is "new and surprising," the neuron selects amongst its prior information sources through an anti-Hebbian rule. The unique inputs of a mature neuron would therefore result from learning about spatial and temporal patterns in its local environment, and by extension, the external world. Thus the theory describes how the structure of the mature nervous system could reflect the structure of the external world, and how the complexity and intelligence of the system might develop from a population of

Modeling coherent errors in quantum error correction

Science.gov (United States)

Greenbaum, Daniel; Dutton, Zachary

2018-01-01

Analysis of quantum error correcting codes is typically done using a stochastic, Pauli channel error model for describing the noise on physical qubits. However, it was recently found that coherent errors (systematic rotations) on physical data qubits result in both physical and logical error rates that differ significantly from those predicted by a Pauli model. Here we examine the accuracy of the Pauli approximation for noise containing coherent errors (characterized by a rotation angle ɛ) under the repetition code. We derive an analytic expression for the logical error channel as a function of arbitrary code distance d and concatenation level n, in the small error limit. We find that coherent physical errors result in logical errors that are partially coherent and therefore non-Pauli. However, the coherent part of the logical error is negligible at fewer than {ε }-({dn-1)} error correction cycles when the decoder is optimized for independent Pauli errors, thus providing a regime of validity for the Pauli approximation. Above this number of correction cycles, the persistent coherent logical error will cause logical failure more quickly than the Pauli model would predict, and this may need to be combated with coherent suppression methods at the physical level or larger codes.

Identity-specific coding of future rewards in the human orbitofrontal cortex.

Science.gov (United States)

Howard, James D; Gottfried, Jay A; Tobler, Philippe N; Kahnt, Thorsten

2015-04-21

Nervous systems must encode information about the identity of expected outcomes to make adaptive decisions. However, the neural mechanisms underlying identity-specific value signaling remain poorly understood. By manipulating the value and identity of appetizing food odors in a pattern-based imaging paradigm of human classical conditioning, we were able to identify dissociable predictive representations of identity-specific reward in orbitofrontal cortex (OFC) and identity-general reward in ventromedial prefrontal cortex (vmPFC). Reward-related functional coupling between OFC and olfactory (piriform) cortex and between vmPFC and amygdala revealed parallel pathways that support identity-specific and -general predictive signaling. The demonstration of identity-specific value representations in OFC highlights a role for this region in model-based behavior and reveals mechanisms by which appetitive behavior can go awry.

Impaired Feedback Processing for Symbolic Reward in Individuals with Internet Game Overuse

Directory of Open Access Journals (Sweden)

Jinhee Kim

2017-10-01

Full Text Available Reward processing, which plays a critical role in adaptive behavior, is impaired in addiction disorders, which are accompanied by functional abnormalities in brain reward circuits. Internet gaming disorder, like substance addiction, is thought to be associated with impaired reward processing, but little is known about how it affects learning, especially when feedback is conveyed by less-salient motivational events. Here, using both monetary (±500 KRW and symbolic (Chinese characters “right” or “wrong” rewards and penalties, we investigated whether behavioral performance and feedback-related neural responses are altered in Internet game overuse (IGO group. Using functional MRI, brain responses for these two types of reward/penalty feedback were compared between young males with problems of IGO (IGOs, n = 18, mean age = 22.2 ± 2.0 years and age-matched control subjects (Controls, n = 20, mean age = 21.2 ± 2.1 during a visuomotor association task where associations were learned between English letters and one of four responses. No group difference was found in adjustment of error responses following the penalty or in brain responses to penalty, for either monetary or symbolic penalties. The IGO individuals, however, were more likely to fail to choose the response previously reinforced by symbolic (but not monetary reward. A whole brain two-way ANOVA analysis for reward revealed reduced activations in the IGO group in the rostral anterior cingulate cortex/ventromedial prefrontal cortex (vmPFC in response to both reward types, suggesting impaired reward processing. However, the responses to reward in the inferior parietal region and medial orbitofrontal cortex/vmPFC were affected by the types of reward in the IGO group. Unlike the control group, in the IGO group the reward response was reduced only for symbolic reward, suggesting lower attentional and value processing specific to symbolic reward. Furthermore

Study on the methodology for predicting and preventing errors to improve reliability of maintenance task in nuclear power plant

International Nuclear Information System (INIS)

Hanafusa, Hidemitsu; Iwaki, Toshio; Embrey, D.

2000-01-01

The objective of this study was to develop and effective methodology for predicting and preventing errors in nuclear power plant maintenance tasks. A method was established by which chief maintenance personnel can predict and reduce errors when reviewing the maintenance procedures and while referring to maintenance supporting systems and methods in other industries including aviation and chemical plant industries. The method involves the following seven steps: 1. Identification of maintenance tasks. 2. Specification of important tasks affecting safety. 3. Assessment of human errors occurring during important tasks. 4. Identification of Performance Degrading Factors. 5. Dividing important tasks into sub-tasks. 6. Extraction of errors using Predictive Human Error Analysis (PHEA). 7. Development of strategies for reducing errors and for recovering from errors. By way of a trial, this method was applied to the pump maintenance procedure in nuclear power plants. This method is believed to be capable of identifying the expected errors in important tasks and supporting the development of error reduction measures. By applying this method, the number of accidents resulting form human errors during maintenance can be reduced. Moreover, the maintenance support base using computers was developed. (author)

Ethanol induces impulsive-like responding in a delay-of-reward operant choice procedure: impulsivity predicts autoshaping.

Science.gov (United States)

Tomie, A; Aguado, A S; Pohorecky, L A; Benjamin, D

1998-10-01

Autoshaping conditioned responses (CRs) are reflexive and targeted motor responses expressed as a result of experience with reward. To evaluate the hypothesis that autoshaping may be a form of impulsive responding, within-subjects correlations between performance on autoshaping and impulsivity tasks were assessed in 15 Long-Evans hooded rats. Autoshaping procedures [insertion of retractable lever conditioned stimulus (CS) followed by the response-independent delivery of food (US)] were followed by testing for impulsive-like responding in a two-choice lever-press operant delay-of-reward procedure (immediate small food reward versus delayed large food reward). Delay-of-reward functions revealed two distinct subject populations. Subjects in the Sensitive group (n=7) were more impulsive-like, increasing immediate reward choices at longer delays for large reward, while those in the Insensitive group (n=8) responded predominantly on only one lever. During the prior autoshaping phase, the Sensitive group had performed more autoshaping CRs, and correlations revealed that impulsive subjects acquired the autoshaping CR in fewer trials. In the Sensitive group, acute injections of ethanol (0, 0.25, 0.50, 1.00, 1.50 g/kg) given immediately before delay-of-reward sessions yielded an inverted U-shaped dose-response curve with increased impulsivity induced by the 0.25, 0.50, and 1.00 g/kg doses of ethanol, while choice strategy of the Insensitive group was not influenced by ethanol dose. Ethanol induced impulsive-like responding only in rats that were flexible in their response strategy (Sensitive group), and this group also performed more autoshaping CRs. Data support the hypothesis that autoshaping and impulsivity are linked.

Predicting diagnostic error in radiology via eye-tracking and image analytics: Preliminary investigation in mammography

International Nuclear Information System (INIS)

Voisin, Sophie; Tourassi, Georgia D.; Pinto, Frank; Morin-Ducote, Garnetta; Hudson, Kathleen B.

2013-01-01

Purpose: The primary aim of the present study was to test the feasibility of predicting diagnostic errors in mammography by merging radiologists’ gaze behavior and image characteristics. A secondary aim was to investigate group-based and personalized predictive models for radiologists of variable experience levels.Methods: The study was performed for the clinical task of assessing the likelihood of malignancy of mammographic masses. Eye-tracking data and diagnostic decisions for 40 cases were acquired from four Radiology residents and two breast imaging experts as part of an IRB-approved pilot study. Gaze behavior features were extracted from the eye-tracking data. Computer-generated and BIRADS images features were extracted from the images. Finally, machine learning algorithms were used to merge gaze and image features for predicting human error. Feature selection was thoroughly explored to determine the relative contribution of the various features. Group-based and personalized user modeling was also investigated.Results: Machine learning can be used to predict diagnostic error by merging gaze behavior characteristics from the radiologist and textural characteristics from the image under review. Leveraging data collected from multiple readers produced a reasonable group model [area under the ROC curve (AUC) = 0.792 ± 0.030]. Personalized user modeling was far more accurate for the more experienced readers (AUC = 0.837 ± 0.029) than for the less experienced ones (AUC = 0.667 ± 0.099). The best performing group-based and personalized predictive models involved combinations of both gaze and image features.Conclusions: Diagnostic errors in mammography can be predicted to a good extent by leveraging the radiologists’ gaze behavior and image content

Predicting diagnostic error in radiology via eye-tracking and image analytics: Preliminary investigation in mammography

Energy Technology Data Exchange (ETDEWEB)

Voisin, Sophie; Tourassi, Georgia D. [Biomedical Science and Engineering Center, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831 (United States); Pinto, Frank [School of Engineering, Science, and Technology, Virginia State University, Petersburg, Virginia 23806 (United States); Morin-Ducote, Garnetta; Hudson, Kathleen B. [Department of Radiology, University of Tennessee Medical Center at Knoxville, Knoxville, Tennessee 37920 (United States)

2013-10-15

Purpose: The primary aim of the present study was to test the feasibility of predicting diagnostic errors in mammography by merging radiologists’ gaze behavior and image characteristics. A secondary aim was to investigate group-based and personalized predictive models for radiologists of variable experience levels.Methods: The study was performed for the clinical task of assessing the likelihood of malignancy of mammographic masses. Eye-tracking data and diagnostic decisions for 40 cases were acquired from four Radiology residents and two breast imaging experts as part of an IRB-approved pilot study. Gaze behavior features were extracted from the eye-tracking data. Computer-generated and BIRADS images features were extracted from the images. Finally, machine learning algorithms were used to merge gaze and image features for predicting human error. Feature selection was thoroughly explored to determine the relative contribution of the various features. Group-based and personalized user modeling was also investigated.Results: Machine learning can be used to predict diagnostic error by merging gaze behavior characteristics from the radiologist and textural characteristics from the image under review. Leveraging data collected from multiple readers produced a reasonable group model [area under the ROC curve (AUC) = 0.792 ± 0.030]. Personalized user modeling was far more accurate for the more experienced readers (AUC = 0.837 ± 0.029) than for the less experienced ones (AUC = 0.667 ± 0.099). The best performing group-based and personalized predictive models involved combinations of both gaze and image features.Conclusions: Diagnostic errors in mammography can be predicted to a good extent by leveraging the radiologists’ gaze behavior and image content.

Expected reward modulates encoding-related theta activity before an event.

Science.gov (United States)

Gruber, Matthias J; Watrous, Andrew J; Ekstrom, Arne D; Ranganath, Charan; Otten, Leun J

2013-01-01

Oscillatory brain activity in the theta frequency range (4-8 Hz) before the onset of an event has been shown to affect the likelihood of successfully encoding the event into memory. Recent work has also indicated that frontal theta activity might be modulated by reward, but it is not clear how reward expectancy, anticipatory theta activity, and memory formation might be related. Here, we used scalp electroencephalography (EEG) to assess the relationship between these factors. EEG was recorded from healthy adults while they memorized a series of words. Each word was preceded by a cue that indicated whether a high or low monetary reward would be earned if the word was successfully remembered in a later recognition test. Frontal theta power between the presentation of the reward cue and the onset of a word was predictive of later memory for the word, but only in the high reward condition. No theta differences were observed before word onset following low reward cues. The magnitude of prestimulus encoding-related theta activity in the high reward condition was correlated with the number of high reward words that were later confidently recognized. These findings provide strong evidence for a link between reward expectancy, theta activity, and memory encoding. Theta activity before event onset seems to be especially important for the encoding of motivationally significant stimuli. One possibility is that dopaminergic activity during reward anticipation mediates frontal theta activity related to memory. Copyright © 2012 Elsevier Inc. All rights reserved.

Per-beam, planar IMRT QA passing rates do not predict clinically relevant patient dose errors

International Nuclear Information System (INIS)

Nelms, Benjamin E.; Zhen Heming; Tome, Wolfgang A.

2011-01-01

Purpose: The purpose of this work is to determine the statistical correlation between per-beam, planar IMRT QA passing rates and several clinically relevant, anatomy-based dose errors for per-patient IMRT QA. The intent is to assess the predictive power of a common conventional IMRT QA performance metric, the Gamma passing rate per beam. Methods: Ninety-six unique data sets were created by inducing four types of dose errors in 24 clinical head and neck IMRT plans, each planned with 6 MV Varian 120-leaf MLC linear accelerators using a commercial treatment planning system and step-and-shoot delivery. The error-free beams/plans were used as ''simulated measurements'' (for generating the IMRT QA dose planes and the anatomy dose metrics) to compare to the corresponding data calculated by the error-induced plans. The degree of the induced errors was tuned to mimic IMRT QA passing rates that are commonly achieved using conventional methods. Results: Analysis of clinical metrics (parotid mean doses, spinal cord max and D1cc, CTV D95, and larynx mean) vs IMRT QA Gamma analysis (3%/3 mm, 2/2, 1/1) showed that in all cases, there were only weak to moderate correlations (range of Pearson's r-values: -0.295 to 0.653). Moreover, the moderate correlations actually had positive Pearson's r-values (i.e., clinically relevant metric differences increased with increasing IMRT QA passing rate), indicating that some of the largest anatomy-based dose differences occurred in the cases of high IMRT QA passing rates, which may be called ''false negatives.'' The results also show numerous instances of false positives or cases where low IMRT QA passing rates do not imply large errors in anatomy dose metrics. In none of the cases was there correlation consistent with high predictive power of planar IMRT passing rates, i.e., in none of the cases did high IMRT QA Gamma passing rates predict low errors in anatomy dose metrics or vice versa. Conclusions: There is a lack of correlation between

Using food as a reward: An examination of parental reward practices.

Science.gov (United States)

Roberts, Lindsey; Marx, Jenna M; Musher-Eizenman, Dara R

2018-01-01

Eating patterns and taste preferences are often established early in life. Many studies have examined how parental feeding practices may affect children's outcomes, including food intake and preference. The current study focused on a common food parenting practice, using food as a reward, and used Latent Profile Analysis (LPA) to examine whether mothers (n = 376) and fathers (n = 117) of children ages 2.8 to 7.5 (M = 4.7; SD = 1.1) grouped into profiles (i.e., subgroups) based on how they use of food as a reward. The 4-class model was the best-fitting LPA model, with resulting classes based on both the frequency and type of reward used. Classes were: infrequent reward (33%), tangible reward (21%), food reward (27%), and frequent reward (19%). The current study also explored whether children's eating styles (emotional overeating, rood fussiness, food responsiveness, and satiety responsiveness) and parenting style (Authoritative, Authoritarian, and Permissive) varied by reward profile. Analyses of Variance (ANOVA) revealed that the four profiles differed significantly for all outcome variables except satiety responsiveness. It appears that the use of tangible and food-based rewards have important implications in food parenting. More research is needed to better understand how the different rewarding practices affect additional child outcomes. Copyright © 2017 Elsevier Ltd. All rights reserved.

The influence of organisational rewards on workplace trust and work engagement

Directory of Open Access Journals (Sweden)

Janine Victor

2017-05-01

Full Text Available Orientation: In volatile and competitive business environments, organisations are faced with challenges to retain talented workers. Employees are increasingly leaving their jobs for a number of reasons, one of them being a perceived lack of adequate reward practices. Consequently, this has impacted on employee work engagement and confidence and trust in organisations. Research purpose: The study sought to determine whether there is a relationship between rewards, trust and engagement, as well as whether rewards are able to predict trust and engagement in the South African workplace. Motivation for the study: Organisations can no longer solely rely on extrinsic rewards to retain talent. Companies must draw on both extrinsic and intrinsic reward strategies to improve retention levels through endorsing higher levels of workplace trust and work engagement levels. Research design, approach and method: A quantitative, exploratory and cross-sectional research design was utilised. Non-probability sampling using questionnaires consisting of scales from the Job Satisfaction Survey, Intrinsic Motivation Inventory, Psychological Meaningfulness Scale, Basic Needs at Work Scale, Workplace Trust Survey and Utrecht Work Engagement Scale were administered to a sample (N = 251 of South African employees in various industries within the Gauteng region. Main findings: Results indicated that there is a moderate-to-strong positive relationship between the three constructs, and that rewards are able to predict trust and engagement. Practical and managerial implications: The findings provide insight for behavioural practitioners to potentially draw upon when improving talent management strategies. Both extrinsic and intrinsic rewards are important factors in keeping employees engaged and ultimately retaining them. Contribution: The study provided insight into the influence that organisational rewards may have on workplace trust, work engagement and retaining employees

Reward sensitivity is associated with brain activity during erotic stimulus processing.

Science.gov (United States)

Costumero, Victor; Barrós-Loscertales, Alfonso; Bustamante, Juan Carlos; Ventura-Campos, Noelia; Fuentes, Paola; Rosell-Negre, Patricia; Ávila, César

2013-01-01

The behavioral approach system (BAS) from Gray's reinforcement sensitivity theory is a neurobehavioral system involved in the processing of rewarding stimuli that has been related to dopaminergic brain areas. Gray's theory hypothesizes that the functioning of reward brain areas is modulated by BAS-related traits. To test this hypothesis, we performed an fMRI study where participants viewed erotic and neutral pictures, and cues that predicted their appearance. Forty-five heterosexual men completed the Sensitivity to Reward scale (from the Sensitivity to Punishment and Sensitivity to Reward Questionnaire) to measure BAS-related traits. Results showed that Sensitivity to Reward scores correlated positively with brain activity during reactivity to erotic pictures in the left orbitofrontal cortex, left insula, and right ventral striatum. These results demonstrated a relationship between the BAS and reward sensitivity during the processing of erotic stimuli, filling the gap of previous reports that identified the dopaminergic system as a neural substrate for the BAS during the processing of other rewarding stimuli such as money and food.

Reward sensitivity is associated with brain activity during erotic stimulus processing.

Directory of Open Access Journals (Sweden)

Victor Costumero

Full Text Available The behavioral approach system (BAS from Gray's reinforcement sensitivity theory is a neurobehavioral system involved in the processing of rewarding stimuli that has been related to dopaminergic brain areas. Gray's theory hypothesizes that the functioning of reward brain areas is modulated by BAS-related traits. To test this hypothesis, we performed an fMRI study where participants viewed erotic and neutral pictures, and cues that predicted their appearance. Forty-five heterosexual men completed the Sensitivity to Reward scale (from the Sensitivity to Punishment and Sensitivity to Reward Questionnaire to measure BAS-related traits. Results showed that Sensitivity to Reward scores correlated positively with brain activity during reactivity to erotic pictures in the left orbitofrontal cortex, left insula, and right ventral striatum. These results demonstrated a relationship between the BAS and reward sensitivity during the processing of erotic stimuli, filling the gap of previous reports that identified the dopaminergic system as a neural substrate for the BAS during the processing of other rewarding stimuli such as money and food.

The reward-based eating drive scale: a self-report index of reward-based eating.

Directory of Open Access Journals (Sweden)

Elissa S Epel

Full Text Available Why are some individuals more vulnerable to persistent weight gain and obesity than are others? Some obese individuals report factors that drive overeating, including lack of control, lack of satiation, and preoccupation with food, which may stem from reward-related neural circuitry. These are normative and common symptoms and not the sole focus of any existing measures. Many eating scales capture these common behaviors, but are confounded with aspects of dysregulated eating such as binge eating or emotional overeating. Across five studies, we developed items that capture this reward-based eating drive (RED. Study 1 developed the items in lean to obese individuals (n = 327 and examined changes in weight over eight years. In Study 2, the scale was further developed and expert raters evaluated the set of items. Study 3 tested psychometric properties of the final 9 items in 400 participants. Study 4 examined psychometric properties and race invariance (n = 80 women. Study 5 examined psychometric properties and age/gender invariance (n = 381. Results showed that RED scores correlated with BMI and predicted earlier onset of obesity, greater weight fluctuations, and greater overall weight gain over eight years. Expert ratings of RED scale items indicated that the items reflected characteristics of reward-based eating. The RED scale evidenced high internal consistency and invariance across demographic factors. The RED scale, designed to tap vulnerability to reward-based eating behavior, appears to be a useful brief tool for identifying those at higher risk of weight gain over time. Given the heterogeneity of obesity, unique brief profiling of the reward-based aspect of obesity using a self-report instrument such as the RED scale may be critical for customizing effective treatments in the general population.

Suppressing my memories by listening to yours: The effect of socially triggered context-based prediction error on memory.

Science.gov (United States)

Vlasceanu, Madalina; Drach, Rae; Coman, Alin

2018-05-03

The mind is a prediction machine. In most situations, it has expectations as to what might happen. But when predictions are invalidated by experience (i.e., prediction errors), the memories that generate these predictions are suppressed. Here, we explore the effect of prediction error on listeners' memories following social interaction. We find that listening to a speaker recounting experiences similar to one's own triggers prediction errors on the part of the listener that lead to the suppression of her memories. This effect, we show, is sensitive to a perspective-taking manipulation, such that individuals who are instructed to take the perspective of the speaker experience memory suppression, whereas individuals who undergo a low-perspective-taking manipulation fail to show a mnemonic suppression effect. We discuss the relevance of these findings for our understanding of the bidirectional influences between cognition and social contexts, as well as for the real-world situations that involve memory-based predictions.

Temporal dynamics of reward anticipation in the human brain.

Science.gov (United States)

Zhang, Yuanyuan; Li, Qi; Wang, Zhao; Liu, Xun; Zheng, Ya

2017-09-01

Reward anticipation is a complex process including cue evaluation, motor preparation, and feedback anticipation. The present study investigated whether these psychological processes were dissociable on neural dynamics in terms of incentive valence and approach motivation. We recorded EEG when participants were performing a monetary incentive delay task, and found a cue-P3 during the cue-evaluation stage, a contingent negative variation (CNV) during the motor-preparation stage, and a stimulus-preceding negativity (SPN) during the feedback-anticipation stage. Critically, both the cue-P3 and SPN exhibited an enhanced sensitivity to gain versus loss anticipation, which was not observed for the CNV. Moreover, both the cue-P3 and SPN, instead of the CNV, for gain anticipation selectively predicted the participants' approach motivation as measured in a following effort expenditure for rewards task, particularly when reward uncertainty was maximal. Together, these results indicate that reward anticipation consists of several sub-stages, each with distinct functional significance, thus providing implications for neuropsychiatric diseases characterized by dysfunction in anticipatory reward processing. Copyright © 2017 Elsevier B.V. All rights reserved.

Error-related brain activity and error awareness in an error classification paradigm.

Science.gov (United States)

Di Gregorio, Francesco; Steinhauser, Marco; Maier, Martin E

2016-10-01

Error-related brain activity has been linked to error detection enabling adaptive behavioral adjustments. However, it is still unclear which role error awareness plays in this process. Here, we show that the error-related negativity (Ne/ERN), an event-related potential reflecting early error monitoring, is dissociable from the degree of error awareness. Participants responded to a target while ignoring two different incongruent distractors. After responding, they indicated whether they had committed an error, and if so, whether they had responded to one or to the other distractor. This error classification paradigm allowed distinguishing partially aware errors, (i.e., errors that were noticed but misclassified) and fully aware errors (i.e., errors that were correctly classified). The Ne/ERN was larger for partially aware errors than for fully aware errors. Whereas this speaks against the idea that the Ne/ERN foreshadows the degree of error awareness, it confirms the prediction of a computational model, which relates the Ne/ERN to post-response conflict. This model predicts that stronger distractor processing - a prerequisite of error classification in our paradigm - leads to lower post-response conflict and thus a smaller Ne/ERN. This implies that the relationship between Ne/ERN and error awareness depends on how error awareness is related to response conflict in a specific task. Our results further indicate that the Ne/ERN but not the degree of error awareness determines adaptive performance adjustments. Taken together, we conclude that the Ne/ERN is dissociable from error awareness and foreshadows adaptive performance adjustments. Our results suggest that the relationship between the Ne/ERN and error awareness is correlative and mediated by response conflict. Copyright © 2016 Elsevier Inc. All rights reserved.

Reward signals, attempted suicide, and impulsivity in late-life depression.

Science.gov (United States)

Dombrovski, Alexandre Y; Szanto, Katalin; Clark, Luke; Reynolds, Charles F; Siegle, Greg J

2013-10-01

IMPORTANCE—Suicide can be viewed as an escape from unendurable punishment at the cost of any future rewards. Could faulty estimation of these outcomes predispose to suicidal behavior? In behavioral studies, many of those who have attempted suicide misestimate expected rewards on gambling and probabilistic learning tasks.OBJECTIVES—To describe the neural circuit abnormalities that underlie disadvantageous choices in people at risk for suicide and to relate these abnormalities to impulsivity, which is one of the components of vulnerability to suicide.DESIGN—Case-control functional magnetic resonance imaging study of reward learning using are inforcement learning model.SETTING—University hospital and outpatient clinic.PATIENTS—Fifty-three participants 60 years or older, including 15 depressed patients who had attempted suicide, 18 depressed patients who had never attempted suicide (depressed control subjects), and 20 psychiatrically healthy controls.MAIN OUTCOMES AND MEASURES—Components of the cortical blood oxygenation level–dependent response tracking expected and unpredicted rewards.RESULTS—Depressed elderly participants displayed 2 distinct disruptions of control over reward-guided behavior. First, impulsivity and a history of suicide attempts (particularly poorly planned ones) were associated with a weakened expected reward signal in the paralimbic cortex,which in turn predicted the behavioral insensitivity to contingency change. Second, depression was associated with disrupted corticostriatothalamic encoding of unpredicted rewards, which in turn predicted the behavioral over sensitivity to punishment. These results were robust to the effects of possible brain damage from suicide attempts, depressive severity, co-occurring substance use and anxiety disorders, antidepressant and anticholinergic exposure, lifetime exposure to electroconvulsive therapy, vascular illness, and incipient dementia.CONCLUSIONS AND RELEVANCE—Altered paralimbic reward

Improving probabilistic prediction of daily streamflow by identifying Pareto optimal approaches for modeling heteroscedastic residual errors

Science.gov (United States)

McInerney, David; Thyer, Mark; Kavetski, Dmitri; Lerat, Julien; Kuczera, George

2017-03-01

Reliable and precise probabilistic prediction of daily catchment-scale streamflow requires statistical characterization of residual errors of hydrological models. This study focuses on approaches for representing error heteroscedasticity with respect to simulated streamflow, i.e., the pattern of larger errors in higher streamflow predictions. We evaluate eight common residual error schemes, including standard and weighted least squares, the Box-Cox transformation (with fixed and calibrated power parameter λ) and the log-sinh transformation. Case studies include 17 perennial and 6 ephemeral catchments in Australia and the United States, and two lumped hydrological models. Performance is quantified using predictive reliability, precision, and volumetric bias metrics. We find the choice of heteroscedastic error modeling approach significantly impacts on predictive performance, though no single scheme simultaneously optimizes all performance metrics. The set of Pareto optimal schemes, reflecting performance trade-offs, comprises Box-Cox schemes with λ of 0.2 and 0.5, and the log scheme (λ = 0, perennial catchments only). These schemes significantly outperform even the average-performing remaining schemes (e.g., across ephemeral catchments, median precision tightens from 105% to 40% of observed streamflow, and median biases decrease from 25% to 4%). Theoretical interpretations of empirical results highlight the importance of capturing the skew/kurtosis of raw residuals and reproducing zero flows. Paradoxically, calibration of λ is often counterproductive: in perennial catchments, it tends to overfit low flows at the expense of abysmal precision in high flows. The log-sinh transformation is dominated by the simpler Pareto optimal schemes listed above. Recommendations for researchers and practitioners seeking robust residual error schemes for practical work are provided.

Improving probabilistic prediction of daily streamflow by identifying Pareto optimal approaches for modelling heteroscedastic residual errors

Science.gov (United States)

David, McInerney; Mark, Thyer; Dmitri, Kavetski; George, Kuczera

2017-04-01

This study provides guidance to hydrological researchers which enables them to provide probabilistic predictions of daily streamflow with the best reliability and precision for different catchment types (e.g. high/low degree of ephemerality). Reliable and precise probabilistic prediction of daily catchment-scale streamflow requires statistical characterization of residual errors of hydrological models. It is commonly known that hydrological model residual errors are heteroscedastic, i.e. there is a pattern of larger errors in higher streamflow predictions. Although multiple approaches exist for representing this heteroscedasticity, few studies have undertaken a comprehensive evaluation and comparison of these approaches. This study fills this research gap by evaluating 8 common residual error schemes, including standard and weighted least squares, the Box-Cox transformation (with fixed and calibrated power parameter, lambda) and the log-sinh transformation. Case studies include 17 perennial and 6 ephemeral catchments in Australia and USA, and two lumped hydrological models. We find the choice of heteroscedastic error modelling approach significantly impacts on predictive performance, though no single scheme simultaneously optimizes all performance metrics. The set of Pareto optimal schemes, reflecting performance trade-offs, comprises Box-Cox schemes with lambda of 0.2 and 0.5, and the log scheme (lambda=0, perennial catchments only). These schemes significantly outperform even the average-performing remaining schemes (e.g., across ephemeral catchments, median precision tightens from 105% to 40% of observed streamflow, and median biases decrease from 25% to 4%). Theoretical interpretations of empirical results highlight the importance of capturing the skew/kurtosis of raw residuals and reproducing zero flows. Recommendations for researchers and practitioners seeking robust residual error schemes for practical work are provided.

Do reward-processing deficits in schizophrenia-spectrum disorders promote cannabis use? An investigation of physiological response to natural rewards and drug cues

Science.gov (United States)

Cassidy, Clifford M.; Brodeur, Mathieu B.; Lepage, Martin; Malla, Ashok

2014-01-01

Background Dysfunctional reward processing is present in individuals with schizophrenia-spectrum disorders (SSD) and may confer vulnerability to addiction. Our objective was to identify a deficit in patients with SSD on response to rewarding stimuli and determine whether this deficit predicts cannabis use. Methods We divided a group of patients with SSD and nonpsychotic controls into cannabis users and nonusers. Response to emotional and cannabis-associated visual stimuli was assessed using self-report, event-related potentials (using the late positive potential [LPP]), facial electromyography and skin-conductance response. Results Our sample comprised 35 patients with SSD and 35 nonpsychotic controls. Compared with controls, the patients with SSD showed blunted LPP response to pleasant stimuli (p = 0.003). Across measures, cannabis-using controls showed greater response to pleasant stimuli than to cannabis stimuli whereas cannabis-using patients showed little bias toward pleasant stimuli. Reduced LPP response to pleasant stimuli was predictive of more frequent subsequent cannabis use (β = −0.24, p = 0.034). Limitations It is not clear if the deficit associated with cannabis use is specific to rewarding stimuli or nonspecific to any kind of emotionally salient stimuli. Conclusion The LPP captures a reward-processing deficit in patients with SSD and shows potential as a biomarker for identifying patients at risk of heavy cannabis use. PMID:24913137

Differences between Dorsal and Ventral Striatum in the Sensitivity of Tonically Active Neurons to Rewarding Events

Directory of Open Access Journals (Sweden)

Kevin Marche

2017-07-01

Full Text Available Within the striatum, cholinergic interneurons, electrophysiologically identified as tonically active neurons (TANs, represent a relatively homogeneous group in terms of their functional properties. They display typical pause in tonic firing in response to rewarding events which are of crucial importance for reinforcement learning. These responses are uniformly distributed throughout the dorsal striatum (i.e., motor and associative striatum, but it is unknown, at least in monkeys, whether differences in the modulation of TAN activity exist in the ventral striatum (i.e., limbic striatum, a region specialized for processing of motivational information. To address this issue, we examined the activity of dorsal and ventral TANs in two monkeys trained on a Pavlovian conditioning task in which a visual stimulus preceded the delivery of liquid reward by a fixed time interval. We found that the proportion of TANs responding to the stimulus predictive of reward did not vary significantly across regions (58%–80%, whereas the fraction of TANs responding to reward was higher in the limbic striatum (100% compared to the motor (65% and associative striatum (52%. By examining TAN modulation at the level of both the population and the individual neurons, we showed that the duration of pause responses to the stimulus and reward was longer in the ventral than in the dorsal striatal regions. Also, the magnitude of the pause was greater in ventral than dorsal striatum for the stimulus predictive of reward but not for the reward itself. We found similar region-specific differences in pause response duration to the stimulus when the timing of reward was less predictable (fixed replaced by variable time interval. Regional variations in the duration and magnitude of the pause response were transferred from the stimulus to reward when reward was delivered in the absence of any predictive stimulus. It therefore appears that ventral TANs exhibit stronger responses to

A review of reward processing and motivational impairment in schizophrenia.

Science.gov (United States)

Strauss, Gregory P; Waltz, James A; Gold, James M

2014-03-01

This article reviews and synthesizes research on reward processing in schizophrenia, which has begun to provide important insights into the cognitive and neural mechanisms associated with motivational impairments. Aberrant cortical-striatal interactions may be involved with multiple reward processing abnormalities, including: (1) dopamine-mediated basal ganglia systems that support reinforcement learning and the ability to predict cues that lead to rewarding outcomes; (2) orbitofrontal cortex-driven deficits in generating, updating, and maintaining value representations; (3) aberrant effort-value computations, which may be mediated by disrupted anterior cingulate cortex and midbrain dopamine functioning; and (4) altered activation of the prefrontal cortex, which is important for generating exploratory behaviors in environments where reward outcomes are uncertain. It will be important for psychosocial interventions targeting negative symptoms to account for abnormalities in each of these reward processes, which may also have important interactions; suggestions for novel behavioral intervention strategies that make use of external cues, reinforcers, and mobile technology are discussed.

Ventral tegmental area disruption selectively affects CA1/CA2 but not CA3 place fields during a differential reward working memory task.

Science.gov (United States)

Martig, Adria K; Mizumori, Sheri J Y

2011-02-01

Hippocampus (HPC) receives dopaminergic (DA) projections from the ventral tegmental area (VTA) and substantia nigra. These inputs appear to provide a modulatory signal that influences HPC dependent behaviors and place fields. We examined how efferent projections from VTA to HPC influence spatial working memory and place fields when the reward context changes. CA1 and CA3 process environmental context changes differently and VTA preferentially innervates CA1. Given these anatomical data and electrophysiological evidence that implicate DA in reward processing, we predicted that CA1 place fields would respond more strongly to both VTA disruption and changes in the reward context than CA3 place fields. Rats (N = 9) were implanted with infusion cannula targeting VTA and recording tetrodes aimed at HPC. Then they were tested on a differential reward, win-shift working memory task. One recording session consisted of 5 baseline and 5 manipulation trials during which place cells in CA1/CA2 (N = 167) and CA3 (N = 94) were recorded. Prior to manipulation trials rats were infused with either baclofen or saline and then subjected to control or reward conditions during which the learned locations of large and small reward quantities were reversed. VTA disruption resulted in an increase in errors, and in CA1/CA2 place field reorganization. There were no changes in any measures of CA3 place field stability during VTA disruption. Reward manipulations did not affect performance or place field stability in CA1/CA2 or CA3; however, changes in the reward locations "rescued" performance and place field stability in CA1/CA2 when VTA activity was compromised, perhaps by trigging compensatory mechanisms. These data support the hypothesis that VTA contributes to spatial working memory performance perhaps by maintaining place field stability selectively in CA1/CA2. Copyright © 2009 Wiley-Liss, Inc.

Individual differences in anticipatory activity to food rewards predict cue-induced appetitive 50-kHz calls in rats.

Science.gov (United States)

Brenes, Juan C; Schwarting, Rainer K W

2015-10-01

Reward-related stimuli come to acquire incentive salience through Pavlovian learning and become capable of controlling reward-oriented behaviors. Here, we examined individual differences in anticipatory activity elicited by reward-related cues as indicative of how animals attribute incentive salience to otherwise neutral stimuli. Since adult rats can signal incentive motivation states through ultrasonic vocalizations (USVs) at around 50-kHz, such calls were recorded in food-deprived rats trained to associate cues with food rewards, which were subsequently devalued by satiation.We found that the extent to which animals developed conditioned anticipatory activity to food cues while food deprived determined the level of cue-induced appetitive USVs while sated. Re-exposure to reward cues after a free-testing period reinstated USVs, invigorated reward seeking and consumption, and again, increases in calling occurred only in animals with high levels of cue-induced anticipatory activity. Reward-experienced rats systemically challenged with the catecholamine agonist amphetamine or with the dopamine receptor antagonist flupenthixol showed attenuated responses to these drugs, especially for USVs and in subjects with high levels of cue-induced anticipatory activity. Our results suggest that individuals prone to attribute incentive salience to reward cues showed heightened reward-induced USVs which were reliably expressed over time and persisted despite physiological needs being fulfilled. Also, prone subjects seemed to undergo particular adaptations in their dopaminergic system related with incentive learning. Our findings may have translational relevance in preclinical research modeling compulsive disorders, which may be due to excessive attribution of incentive salience to reward cues, such as overeating, pathological gambling, and drug addiction.

A Fly’s Eye View of Natural and Drug Reward

Science.gov (United States)

Lowenstein, Eve G.; Velazquez-Ulloa, Norma A.

2018-01-01

Animals encounter multiple stimuli each day. Some of these stimuli are innately appetitive or aversive, while others are assigned valence based on experience. Drugs like ethanol can elicit aversion in the short term and attraction in the long term. The reward system encodes the predictive value for different stimuli, mediating anticipation for attractive or punishing stimuli and driving animal behavior to approach or avoid conditioned stimuli. The neurochemistry and neurocircuitry of the reward system is partly evolutionarily conserved. In both vertebrates and invertebrates, including Drosophila melanogaster, dopamine is at the center of a network of neurotransmitters and neuromodulators acting in concert to encode rewards. Behavioral assays in D. melanogaster have become increasingly sophisticated, allowing more direct comparison with mammalian research. Moreover, recent evidence has established the functional modularity of the reward neural circuits in Drosophila. This functional modularity resembles the organization of reward circuits in mammals. The powerful genetic and molecular tools for D. melanogaster allow characterization and manipulation at the single-cell level. These tools are being used to construct a detailed map of the neural circuits mediating specific rewarding stimuli and have allowed for the identification of multiple genes and molecular pathways that mediate the effects of reinforcing stimuli, including their rewarding effects. This report provides an overview of the research on natural and drug reward in D. melanogaster, including natural rewards such as sugar and other food nutrients, and drug rewards including ethanol, cocaine, amphetamine, methamphetamine, and nicotine. We focused mainly on the known genetic and neural mechanisms underlying appetitive reward for sugar and reward for ethanol. We also include genes, molecular pathways, and neural circuits that have been identified using assays that test the palatability of the rewarding

A Fly’s Eye View of Natural and Drug Reward

Directory of Open Access Journals (Sweden)

Eve G. Lowenstein

2018-04-01

Full Text Available Animals encounter multiple stimuli each day. Some of these stimuli are innately appetitive or aversive, while others are assigned valence based on experience. Drugs like ethanol can elicit aversion in the short term and attraction in the long term. The reward system encodes the predictive value for different stimuli, mediating anticipation for attractive or punishing stimuli and driving animal behavior to approach or avoid conditioned stimuli. The neurochemistry and neurocircuitry of the reward system is partly evolutionarily conserved. In both vertebrates and invertebrates, including Drosophila melanogaster, dopamine is at the center of a network of neurotransmitters and neuromodulators acting in concert to encode rewards. Behavioral assays in D. melanogaster have become increasingly sophisticated, allowing more direct comparison with mammalian research. Moreover, recent evidence has established the functional modularity of the reward neural circuits in Drosophila. This functional modularity resembles the organization of reward circuits in mammals. The powerful genetic and molecular tools for D. melanogaster allow characterization and manipulation at the single-cell level. These tools are being used to construct a detailed map of the neural circuits mediating specific rewarding stimuli and have allowed for the identification of multiple genes and molecular pathways that mediate the effects of reinforcing stimuli, including their rewarding effects. This report provides an overview of the research on natural and drug reward in D. melanogaster, including natural rewards such as sugar and other food nutrients, and drug rewards including ethanol, cocaine, amphetamine, methamphetamine, and nicotine. We focused mainly on the known genetic and neural mechanisms underlying appetitive reward for sugar and reward for ethanol. We also include genes, molecular pathways, and neural circuits that have been identified using assays that test the palatability of

Amygdala Contributions to Stimulus-Reward Encoding in the Macaque Medial and Orbital Frontal Cortex during Learning.

Science.gov (United States)

Rudebeck, Peter H; Ripple, Joshua A; Mitz, Andrew R; Averbeck, Bruno B; Murray, Elisabeth A

2017-02-22

Orbitofrontal cortex (OFC), medial frontal cortex (MFC), and amygdala mediate stimulus-reward learning, but the mechanisms through which they interact are unclear. Here, we investigated how neurons in macaque OFC and MFC signaled rewards and the stimuli that predicted them during learning with and without amygdala input. Macaques performed a task that required them to evaluate two stimuli and then choose one to receive the reward associated with that option. Four main findings emerged. First, amygdala lesions slowed the acquisition and use of stimulus-reward associations. Further analyses indicated that this impairment was due, at least in part, to ineffective use of negative feedback to guide subsequent decisions. Second, the activity of neurons in OFC and MFC rapidly evolved to encode the amount of reward associated with each stimulus. Third, amygdalectomy reduced encoding of stimulus-reward associations during the evaluation of different stimuli. Reward encoding of anticipated and received reward after choices were made was not altered. Fourth, amygdala lesions led to an increase in the proportion of neurons in MFC, but not OFC, that encoded the instrumental response that monkeys made on each trial. These correlated changes in behavior and neural activity after amygdala lesions strongly suggest that the amygdala contributes to the ability to learn stimulus-reward associations rapidly by shaping encoding within OFC and MFC. SIGNIFICANCE STATEMENT Altered functional interactions among orbital frontal cortex (OFC), medial frontal cortex (MFC), and amygdala are thought to underlie several psychiatric conditions, many related to reward learning. Here, we investigated the causal contribution of the amygdala to the development of neuronal activity in macaque OFC and MFC related to rewards and the stimuli that predict them during learning. Without amygdala inputs, neurons in both OFC and MFC showed decreased encoding of stimulus-reward associations. MFC also showed

A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation.

Science.gov (United States)

Deci, E L; Koestner, R; Ryan, R M

1999-11-01

A meta-analysis of 128 studies examined the effects of extrinsic rewards on intrinsic motivation. As predicted, engagement-contingent, completion-contingent, and performance-contingent rewards significantly undermined free-choice intrinsic motivation (d = -0.40, -0.36, and -0.28, respectively), as did all rewards, all tangible rewards, and all expected rewards. Engagement-contingent and completion-contingent rewards also significantly undermined self-reported interest (d = -0.15, and -0.17), as did all tangible rewards and all expected rewards. Positive feedback enhanced both free-choice behavior (d = 0.33) and self-reported interest (d = 0.31). Tangible rewards tended to be more detrimental for children than college students, and verbal rewards tended to be less enhancing for children than college students. The authors review 4 previous meta-analyses of this literature and detail how this study's methods, analyses, and results differed from the previous ones.

Reward-modulated motor information in identified striatum neurons.

Science.gov (United States)

Isomura, Yoshikazu; Takekawa, Takashi; Harukuni, Rie; Handa, Takashi; Aizawa, Hidenori; Takada, Masahiko; Fukai, Tomoki

2013-06-19

It is widely accepted that dorsal striatum neurons participate in either the direct pathway (expressing dopamine D1 receptors) or the indirect pathway (expressing D2 receptors), controlling voluntary movements in an antagonistically balancing manner. The D1- and D2-expressing neurons are activated and inactivated, respectively, by dopamine released from substantia nigra neurons encoding reward expectation. However, little is known about the functional representation of motor information and its reward modulation in individual striatal neurons constituting the two pathways. In this study, we juxtacellularly recorded the spike activity of single neurons in the dorsolateral striatum of rats performing voluntary forelimb movement in a reward-predictable condition. Some of these neurons were identified morphologically by a combination of juxtacellular visualization and in situ hybridization for D1 mRNA. We found that the striatal neurons exhibited distinct functional activations before and during the forelimb movement, regardless of the expression of D1 mRNA. They were often positively, but rarely negatively, modulated by expecting a reward for the correct motor response. The positive reward modulation was independent of behavioral differences in motor performance. In contrast, regular-spiking and fast-spiking neurons in any layers of the motor cortex displayed only minor and unbiased reward modulation of their functional activation in relation to the execution of forelimb movement. Our results suggest that the direct and indirect pathway neurons cooperatively rather than antagonistically contribute to spatiotemporal control of voluntary movements, and that motor information is subcortically integrated with reward information through dopaminergic and other signals in the skeletomotor loop of the basal ganglia.

Human error prediction and countermeasures based on CREAM in spent nuclear fuel (SNF) transportation

International Nuclear Information System (INIS)

Kim, Jae San

2007-02-01

Since the 1980s, in order to secure the storage capacity of spent nuclear fuel (SNF) at NPPs, SNF assemblies have been transported on-site from one unit to another unit nearby. However in the future the amount of the spent fuel will approach capacity in the areas used, and some of these SNFs will have to be transported to an off-site spent fuel repository. Most SNF materials used at NPPs will be transported by general cargo ships from abroad, and these SNFs will be stored in an interim storage facility. In the process of transporting SNF, human interactions will involve inspecting and preparing the cask and spent fuel, loading the cask onto the vehicle or ship, transferring the cask as well as storage or monitoring the cask. The transportation of SNF involves a number of activities that depend on reliable human performance. In the case of the transport of a cask, human errors may include spent fuel bundle misidentification or cask transport accidents among others. Reviews of accident events when transporting the Radioactive Material (RAM) throughout the world indicate that human error is the major causes for more than 65% of significant events. For the safety of SNF transportation, it is very important to predict human error and to deduce a method that minimizes the human error. This study examines the human factor effects on the safety of transporting spent nuclear fuel (SNF). It predicts and identifies the possible human errors in the SNF transport process (loading, transfer and storage of the SNF). After evaluating the human error mode in each transport process, countermeasures to minimize the human error are deduced. The human errors in SNF transportation were analyzed using Hollnagel's Cognitive Reliability and Error Analysis Method (CREAM). After determining the important factors for each process, countermeasures to minimize human error are provided in three parts: System design, Operational environment, and Human ability

Per-beam, planar IMRT QA passing rates do not predict clinically relevant patient dose errors

Energy Technology Data Exchange (ETDEWEB)

Nelms, Benjamin E.; Zhen Heming; Tome, Wolfgang A. [Canis Lupus LLC and Department of Human Oncology, University of Wisconsin, Merrimac, Wisconsin 53561 (United States); Department of Medical Physics, University of Wisconsin, Madison, Wisconsin 53705 (United States); Departments of Human Oncology, Medical Physics, and Biomedical Engineering, University of Wisconsin, Madison, Wisconsin 53792 (United States)

2011-02-15

Purpose: The purpose of this work is to determine the statistical correlation between per-beam, planar IMRT QA passing rates and several clinically relevant, anatomy-based dose errors for per-patient IMRT QA. The intent is to assess the predictive power of a common conventional IMRT QA performance metric, the Gamma passing rate per beam. Methods: Ninety-six unique data sets were created by inducing four types of dose errors in 24 clinical head and neck IMRT plans, each planned with 6 MV Varian 120-leaf MLC linear accelerators using a commercial treatment planning system and step-and-shoot delivery. The error-free beams/plans were used as ''simulated measurements'' (for generating the IMRT QA dose planes and the anatomy dose metrics) to compare to the corresponding data calculated by the error-induced plans. The degree of the induced errors was tuned to mimic IMRT QA passing rates that are commonly achieved using conventional methods. Results: Analysis of clinical metrics (parotid mean doses, spinal cord max and D1cc, CTV D95, and larynx mean) vs IMRT QA Gamma analysis (3%/3 mm, 2/2, 1/1) showed that in all cases, there were only weak to moderate correlations (range of Pearson's r-values: -0.295 to 0.653). Moreover, the moderate correlations actually had positive Pearson's r-values (i.e., clinically relevant metric differences increased with increasing IMRT QA passing rate), indicating that some of the largest anatomy-based dose differences occurred in the cases of high IMRT QA passing rates, which may be called ''false negatives.'' The results also show numerous instances of false positives or cases where low IMRT QA passing rates do not imply large errors in anatomy dose metrics. In none of the cases was there correlation consistent with high predictive power of planar IMRT passing rates, i.e., in none of the cases did high IMRT QA Gamma passing rates predict low errors in anatomy dose metrics or vice versa

SU-F-J-208: Prompt Gamma Imaging-Based Prediction of Bragg Peak Position for Realistic Treatment Error Scenarios

Energy Technology Data Exchange (ETDEWEB)

Xing, Y; Macq, B; Bondar, L [Universite catholique de Louvain, Louvain-la-Neuve (Belgium); Janssens, G [IBA, Louvain-la-Neuve (Belgium)

2016-06-15

Purpose: To quantify the accuracy in predicting the Bragg peak position using simulated in-room measurements of prompt gamma (PG) emissions for realistic treatment error scenarios that combine several sources of errors. Methods: Prompt gamma measurements by a knife-edge slit camera were simulated using an experimentally validated analytical simulation tool. Simulations were performed, for 143 treatment error scenarios, on an anthropomorphic phantom and a pencil beam scanning plan for nasal cavity. Three types of errors were considered: translation along each axis, rotation around each axis, and CT-calibration errors with magnitude ranging respectively, between −3 and 3 mm, −5 and 5 degrees, and between −5 and +5%. We investigated the correlation between the Bragg peak (BP) shift and the horizontal shift of PG profiles. The shifts were calculated between the planned (reference) position and the position by the error scenario. The prediction error for one spot was calculated as the absolute difference between the PG profile shift and the BP shift. Results: The PG shift was significantly and strongly correlated with the BP shift for 92% of the cases (p<0.0001, Pearson correlation coefficient R>0.8). Moderate but significant correlations were obtained for all cases that considered only CT-calibration errors and for 1 case that combined translation and CT-errors (p<0.0001, R ranged between 0.61 and 0.8). The average prediction errors for the simulated scenarios ranged between 0.08±0.07 and 1.67±1.3 mm (grand mean 0.66±0.76 mm). The prediction error was moderately correlated with the value of the BP shift (p=0, R=0.64). For the simulated scenarios the average BP shift ranged between −8±6.5 mm and 3±1.1 mm. Scenarios that considered combinations of the largest treatment errors were associated with large BP shifts. Conclusion: Simulations of in-room measurements demonstrate that prompt gamma profiles provide reliable estimation of the Bragg peak position for

Prediction error demarcates the transition from retrieval, to reconsolidation, to new learning

NARCIS (Netherlands)

Sevenster, Dieuwke|info:eu-repo/dai/nl/375491104; Beckers, Tom; Kindt, Merel

2014-01-01

Although disrupting reconsolidation is promising in targeting emotional memories, the conditions under which memory becomes labile are still unclear. The current study showed that post-retrieval changes in expectancy as an index for prediction error may serve as a read-out for the underlying

Serotonergic neurons signal reward and punishment on multiple timescales

Science.gov (United States)

Cohen, Jeremiah Y; Amoroso, Mackenzie W; Uchida, Naoshige

2015-01-01

Serotonin's function in the brain is unclear. One challenge in testing the numerous hypotheses about serotonin's function has been observing the activity of identified serotonergic neurons in animals engaged in behavioral tasks. We recorded the activity of dorsal raphe neurons while mice experienced a task in which rewards and punishments varied across blocks of trials. We ‘tagged’ serotonergic neurons with the light-sensitive protein channelrhodopsin-2 and identified them based on their responses to light. We found three main features of serotonergic neuron activity: (1) a large fraction of serotonergic neurons modulated their tonic firing rates over the course of minutes during reward vs punishment blocks; (2) most were phasically excited by punishments; and (3) a subset was phasically excited by reward-predicting cues. By contrast, dopaminergic neurons did not show firing rate changes across blocks of trials. These results suggest that serotonergic neurons signal information about reward and punishment on multiple timescales. DOI: http://dx.doi.org/10.7554/eLife.06346.001 PMID:25714923

Development of a Self-Report Measure of Reward Sensitivity:A Test in Current and Former Smokers.

Science.gov (United States)

Hughes, John R; Callas, Peter W; Priest, Jeff S; Etter, Jean-Francois; Budney, Alan J; Sigmon, Stacey C

2017-06-01

Tobacco use or abstinence may increase or decrease reward sensitivity. Most existing measures of reward sensitivity were developed decades ago, and few have undergone extensive psychometric testing. We developed a 58-item survey of the anticipated enjoyment from, wanting for, and frequency of common rewards (the Rewarding Events Inventory-REI). The current analysis focuses on ratings of anticipated enjoyment. The first validation study recruited current and former smokers from Internet sites. The second study recruited smokers who wished to quit and monetarily reinforced them to stay abstinent in a laboratory study and a comparison group of former smokers. In both studies, participants completed the inventory on two occasions, 3-7 days apart. They also completed four anhedonia scales and a behavioral test of reduced reward sensitivity. Half of the enjoyment ratings loaded on four factors: socializing, active hobbies, passive hobbies, and sex/drug use. Cronbach's alpha coefficients were all ≥0.73 for overall mean and factor scores. Test-retest correlations were all ≥0.83. Correlations of the overall and factor scores with frequency of rewards and anhedonia scales were 0.19-0.53, except for the sex/drugs factor. The scores did not correlate with behavioral tests of reward and did not differ between current and former smokers. Lower overall mean enjoyment score predicted a shorter time to relapse. Internal reliability and test-retest reliability of the enjoyment outcomes of the REI are excellent, and construct and predictive validity are modest but promising. The REI is comprehensive and up-to-date, yet is short enough to use on repeated occasions. Replication tests, especially predictive validity tests, are needed. Both use of and abstinence from nicotine appear to increase or decrease how rewarding nondrug rewards are; however, self-report scales to test this have limitations. Our inventory of enjoyment from 58 rewards appears to be reliable and valid as well as

Paying for performance: Performance incentives increase desire for the reward object.

Science.gov (United States)

Hur, Julia D; Nordgren, Loran F

2016-09-01

The current research examines how exposure to performance incentives affects one's desire for the reward object. We hypothesized that the flexible nature of performance incentives creates an attentional fixation on the reward object (e.g., money), which leads people to become more desirous of the rewards. Results from 5 laboratory experiments and 1 large-scale field study provide support for this prediction. When performance was incentivized with monetary rewards, participants reported being more desirous of money (Study 1), put in more effort to earn additional money in an ensuing task (Study 2), and were less willing to donate money to charity (Study 4). We replicated the result with nonmonetary rewards (Study 5). We also found that performance incentives increased attention to the reward object during the task, which in part explains the observed effects (Study 6). A large-scale field study replicated these findings in a real-world setting (Study 7). One laboratory experiment failed to replicate (Study 3). (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Alterations of the Brain Reward System in Antipsychotic Naïve Schizophrenia Patients

DEFF Research Database (Denmark)

Nielsen, Mette Ødegaard; Rostrup, Egill; Wulff, Sanne

2012-01-01

BACKGROUND: Various schizophrenic symptoms are suggested to be linked to a dysfunction of the brain reward system. Several studies have found alterations in the reward processing in patients with schizophrenia; however, most previous findings might be confounded by medication effects. METHODS...... as arousing events) into behavioral salience (events where a predicted reward requires performance) and valence anticipation (the anticipation of a monetarily significant outcome). Furthermore, the evaluation of monetary gain and loss was assessed. RESULTS: During reward anticipation, patients had...... and nonsignificant for value anticipation. Furthermore, patients showed a changed activation pattern during outcome evaluation in right prefrontal cortex. CONCLUSION: Our results suggest that changes during reward anticipation in schizophrenia are present from the beginning of the disease. This supports a possible...

Pressure to cooperate: is positive reward interdependence really needed in cooperative learning?

Science.gov (United States)

Buchs, Céline; Gilles, Ingrid; Dutrévis, Marion; Butera, Fabrizio

2011-03-01

BACKGROUND. Despite extensive research on cooperative learning, the debate regarding whether or not its effectiveness depends on positive reward interdependence has not yet found clear evidence. AIMS. We tested the hypothesis that positive reward interdependence, as compared to reward independence, enhances cooperative learning only if learners work on a 'routine task'; if the learners work on a 'true group task', positive reward interdependence induces the same level of learning as reward independence. SAMPLE. The study involved 62 psychology students during regular workshops. METHOD. Students worked on two psychology texts in cooperative dyads for three sessions. The type of task was manipulated through resource interdependence: students worked on either identical (routine task) or complementary (true group task) information. Students expected to be assessed with a Multiple Choice Test (MCT) on the two texts. The MCT assessment type was introduced according to two reward interdependence conditions, either individual (reward independence) or common (positive reward interdependence). A follow-up individual test took place 4 weeks after the third session of dyadic work to examine individual learning. RESULTS. The predicted interaction between the two types of interdependence was significant, indicating that students learned more with positive reward interdependence than with reward independence when they worked on identical information (routine task), whereas students who worked on complementary information (group task) learned the same with or without reward interdependence. CONCLUSIONS. This experiment sheds light on the conditions under which positive reward interdependence enhances cooperative learning, and suggests that creating a real group task allows to avoid the need for positive reward interdependence. © 2010 The British Psychological Society.

Interplay between Hippocampal Sharp-Wave-Ripple Events and Vicarious Trial and Error Behaviors in Decision Making.

Science.gov (United States)

Papale, Andrew E; Zielinski, Mark C; Frank, Loren M; Jadhav, Shantanu P; Redish, A David

2016-12-07

Current theories posit that memories encoded during experiences are subsequently consolidated into longer-term storage. Hippocampal sharp-wave-ripple (SWR) events have been linked to this consolidation process during sleep, but SWRs also occur during awake immobility, where their role remains unclear. We report that awake SWR rates at the reward site are inversely related to the prevalence of vicarious trial and error (VTE) behaviors, thought to be involved in deliberation processes. SWR rates were diminished immediately after VTE behaviors and an increase in the rate of SWR events at the reward site predicted a decrease in subsequent VTE behaviors at the choice point. Furthermore, SWR disruptions increased VTE behaviors. These results suggest an inverse relationship between SWRs and VTE behaviors and suggest that awake SWRs and associated planning and memory consolidation mechanisms are engaged specifically in the context of higher levels of behavioral certainty. Copyright © 2016 Elsevier Inc. All rights reserved.

An ecologically based model of alcohol-consumption decision making: evidence for the discriminative and predictive role of contextual reward and punishment information.

Science.gov (United States)

Bogg, Tim; Finn, Peter R

2009-05-01

Using insights from Ecological Systems Theory and Reinforcement Sensitivity Theory, the current study assessed the utility of a series of hypothetical role-based alcohol-consumption scenarios that varied in their presentation of rewarding and punishing information. The scenarios, along with measures of impulsive sensation seeking and a self-report of weekly alcohol consumption, were administered to a sample of alcohol-dependent and non-alcohol-dependent college-age individuals (N = 170). The results showed scenario attendance decisions were largely unaffected by alcohol-dependence status and variations in contextual reward and punishment information. In contrast to the attendance findings, the results for the alcohol-consumption decisions showed alcohol-dependent individuals reported a greater frequency of deciding to drink, as well as indicating greater alcohol consumption in the contexts of complementary rewarding or nonpunishing information. Regression results provided evidence for the criterion-related validity of scenario outcomes in an account of diagnostic alcohol problems. The results are discussed in terms of the conceptual and predictive gains associated with an assessment approach to alcohol-consumption decision making that combines situational information organized and balanced through the frameworks of Ecological Systems Theory and Reinforcement Sensitivity Theory.

Perceived stress predicts altered reward and loss feedback processing in medial prefrontal cortex

Directory of Open Access Journals (Sweden)

Michael T Treadway

2013-05-01

Full Text Available Stress is significant risk factor for the development of psychopathology, particularly symptoms related to reward processing. Importantly, individuals display marked variation in how they perceive and cope with stressful events, and such differences are strongly linked to risk for developing psychiatric symptoms following stress exposure. However, many questions remain regarding the neural architecture that underlies inter-subject variability in perceptions of stressors. Using functional magnetic resonance imaging (fMRI during a monetary incentive delay paradigm, we examined the effects of self-reported perceived stress levels on neural activity during reward anticipation and feedback in a sample of healthy individuals. We found that subjects reporting more uncontrollable and overwhelming stressors displayed blunted neural responses in medial prefrontal cortex (mPFC following feedback related to monetary gains as well monetary losses. This is consistent with preclinical models that implicate the mPFC as a key site of vulnerability to the noxious effects of uncontrollable stressors. Our data help translate these findings to humans, and elucidate some of the neural mechanisms that may underlie stress-linked risk for developing reward-related psychiatric symptoms.

Heterogeneity of reward mechanisms.

Science.gov (United States)

Lajtha, A; Sershen, H

2010-06-01

The finding that many drugs that have abuse potential and other natural stimuli such as food or sexual activity cause similar chemical changes in the brain, an increase in extracellular dopamine (DA) in the shell of the nucleus accumbens (NAccS), indicated some time ago that the reward mechanism is at least very similar for all stimuli and that the mechanism is relatively simple. The presently available information shows that the mechanisms involved are more complex and have multiple elements. Multiple brain regions, multiple receptors, multiple distinct neurons, multiple transmitters, multiple transporters, circuits, peptides, proteins, metabolism of transmitters, and phosphorylation, all participate in reward mechanisms. The system is variable, is changed during development, is sex-dependent, and is influenced by genetic differences. Not all of the elements participate in the reward of all stimuli. Different set of mechanisms are involved in the reward of different drugs of abuse, yet different mechanisms in the reward of natural stimuli such as food or sexual activity; thus there are different systems that distinguish different stimuli. Separate functions of the reward system such as anticipation, evaluation, consummation and identification; all contain function-specific elements. The level of the stimulus also influences the participation of the elements of the reward system, there are possible reactions to even below threshold stimuli, and excessive stimuli can change reward to aversion involving parts of the system. Learning and memory of past reward is an important integral element of reward and addictive behavior. Many of the reward elements are altered by repeated or chronic stimuli, and chronic exposure to one drug is likely to alter the response to another stimulus. To evaluate and identify the reward stimulus thus requires heterogeneity of the reward components in the brain.

Quantifying individual variation in the propensity to attribute incentive salience to reward cues.

Science.gov (United States)

Meyer, Paul J; Lovic, Vedran; Saunders, Benjamin T; Yager, Lindsay M; Flagel, Shelly B; Morrow, Jonathan D; Robinson, Terry E

2012-01-01

If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.

Cognitive tests predict real-world errors: the relationship between drug name confusion rates in laboratory-based memory and perception tests and corresponding error rates in large pharmacy chains.

Science.gov (United States)

Schroeder, Scott R; Salomon, Meghan M; Galanter, William L; Schiff, Gordon D; Vaida, Allen J; Gaunt, Michael J; Bryson, Michelle L; Rash, Christine; Falck, Suzanne; Lambert, Bruce L

2017-05-01

Drug name confusion is a common type of medication error and a persistent threat to patient safety. In the USA, roughly one per thousand prescriptions results in the wrong drug being filled, and most of these errors involve drug names that look or sound alike. Prior to approval, drug names undergo a variety of tests to assess their potential for confusability, but none of these preapproval tests has been shown to predict real-world error rates. We conducted a study to assess the association between error rates in laboratory-based tests of drug name memory and perception and real-world drug name confusion error rates. Eighty participants, comprising doctors, nurses, pharmacists, technicians and lay people, completed a battery of laboratory tests assessing visual perception, auditory perception and short-term memory of look-alike and sound-alike drug name pairs (eg, hydroxyzine/hydralazine). Laboratory test error rates (and other metrics) significantly predicted real-world error rates obtained from a large, outpatient pharmacy chain, with the best-fitting model accounting for 37% of the variance in real-world error rates. Cross-validation analyses confirmed these results, showing that the laboratory tests also predicted errors from a second pharmacy chain, with 45% of the variance being explained by the laboratory test data. Across two distinct pharmacy chains, there is a strong and significant association between drug name confusion error rates observed in the real world and those observed in laboratory-based tests of memory and perception. Regulators and drug companies seeking a validated preapproval method for identifying confusing drug names ought to consider using these simple tests. By using a standard battery of memory and perception tests, it should be possible to reduce the number of confusing look-alike and sound-alike drug name pairs that reach the market, which will help protect patients from potentially harmful medication errors. Published by the BMJ

Motor Planning under Unpredictable Reward: Modulations of Movement Vigor and Primate Striatum Activity

Directory of Open Access Journals (Sweden)

Ioan eOpris

2011-05-01

Full Text Available Although reward probability is an important factor that shapes animal behavior, it is not well understood however, how the primate brain translates reward expectation into the vigor of movement (reaction time and speed. To address this question, we trained two monkeys in a reaction time task that required wrist movements in response to vibrotactile and visual stimuli, with a variable reward schedule. Correct performance was rewarded in 75 % of the trials. Monkeys were certain that they would be rewarded only in the trials immediately following withheld rewards. In these trials, the animals responded sooner and moved faster. Single-unit recordings from the dorsal striatum revealed that modulations in striatal neurons reflected such modulations of movement vigor. First, in the trials with certain rewards, striatal neurons modulated their firing rates earlier. Second, magnitudes of changes in neuronal firing rates depended on whether or not monkeys were certain about the reward. Third, these modulations depended on the sensory modality of the cue (visual vs. vibratory and/or movement direction (flexions vs. extensions. We conclude that dorsal striatum may be a part of the mechanism responsible for the modulation of movement vigor in response to changes of reward predictability.

The Errors of Our Ways: Understanding Error Representations in Cerebellar-Dependent Motor Learning.

Science.gov (United States)

Popa, Laurentiu S; Streng, Martha L; Hewitt, Angela L; Ebner, Timothy J

2016-04-01

The cerebellum is essential for error-driven motor learning and is strongly implicated in detecting and correcting for motor errors. Therefore, elucidating how motor errors are represented in the cerebellum is essential in understanding cerebellar function, in general, and its role in motor learning, in particular. This review examines how motor errors are encoded in the cerebellar cortex in the context of a forward internal model that generates predictions about the upcoming movement and drives learning and adaptation. In this framework, sensory prediction errors, defined as the discrepancy between the predicted consequences of motor commands and the sensory feedback, are crucial for both on-line movement control and motor learning. While many studies support the dominant view that motor errors are encoded in the complex spike discharge of Purkinje cells, others have failed to relate complex spike activity with errors. Given these limitations, we review recent findings in the monkey showing that complex spike modulation is not necessarily required for motor learning or for simple spike adaptation. Also, new results demonstrate that the simple spike discharge provides continuous error signals that both lead and lag the actual movements in time, suggesting errors are encoded as both an internal prediction of motor commands and the actual sensory feedback. These dual error representations have opposing effects on simple spike discharge, consistent with the signals needed to generate sensory prediction errors used to update a forward internal model.

Reward, dopamine and the control of food intake: implications for obesity

Energy Technology Data Exchange (ETDEWEB)

Volkow N. D.; Wang G.; Volkow, N.D.; Wang, G.-J.; Baler, R.D.

2011-10-01

The ability to resist the urge to eat requires the proper functioning of neuronal circuits involved in top-down control to oppose the conditioned responses that predict reward from eating the food and the desire to eat the food. Imaging studies show that obese subjects might have impairments in dopaminergic pathways that regulate neuronal systems associated with reward sensitivity, conditioning and control. It is known that the neuropeptides that regulate energy balance (homeostatic processes) through the hypothalamus also modulate the activity of dopamine cells and their projections into regions involved in the rewarding processes underlying food intake. It is postulated that this could also be a mechanism by which overeating and the resultant resistance to homoeostatic signals impairs the function of circuits involved in reward sensitivity, conditioning and cognitive control.

Reward, dopamine and the control of food intake: implications for obesity

International Nuclear Information System (INIS)

Volkow, N.D.; Wang, G.J.; Baler, R.D.

2011-01-01

The ability to resist the urge to eat requires the proper functioning of neuronal circuits involved in top-down control to oppose the conditioned responses that predict reward from eating the food and the desire to eat the food. Imaging studies show that obese subjects might have impairments in dopaminergic pathways that regulate neuronal systems associated with reward sensitivity, conditioning and control. It is known that the neuropeptides that regulate energy balance (homeostatic processes) through the hypothalamus also modulate the activity of dopamine cells and their projections into regions involved in the rewarding processes underlying food intake. It is postulated that this could also be a mechanism by which overeating and the resultant resistance to homoeostatic signals impairs the function of circuits involved in reward sensitivity, conditioning and cognitive control.

Neural correlates of sensory prediction errors in monkeys: evidence for internal models of voluntary self-motion in the cerebellum.

Science.gov (United States)

Cullen, Kathleen E; Brooks, Jessica X

2015-02-01

During self-motion, the vestibular system makes essential contributions to postural stability and self-motion perception. To ensure accurate perception and motor control, it is critical to distinguish between vestibular sensory inputs that are the result of externally applied motion (exafference) and that are the result of our own actions (reafference). Indeed, although the vestibular sensors encode vestibular afference and reafference with equal fidelity, neurons at the first central stage of sensory processing selectively encode vestibular exafference. The mechanism underlying this reafferent suppression compares the brain's motor-based expectation of sensory feedback with the actual sensory consequences of voluntary self-motion, effectively computing the sensory prediction error (i.e., exafference). It is generally thought that sensory prediction errors are computed in the cerebellum, yet it has been challenging to explicitly demonstrate this. We have recently addressed this question and found that deep cerebellar nuclei neurons explicitly encode sensory prediction errors during self-motion. Importantly, in everyday life, sensory prediction errors occur in response to changes in the effector or world (muscle strength, load, etc.), as well as in response to externally applied sensory stimulation. Accordingly, we hypothesize that altering the relationship between motor commands and the actual movement parameters will result in the updating in the cerebellum-based computation of exafference. If our hypothesis is correct, under these conditions, neuronal responses should initially be increased--consistent with a sudden increase in the sensory prediction error. Then, over time, as the internal model is updated, response modulation should decrease in parallel with a reduction in sensory prediction error, until vestibular reafference is again suppressed. The finding that the internal model predicting the sensory consequences of motor commands adapts for new

Public praise vs. private pay: Effects of rewards on energy conservation in the workplace

NARCIS (Netherlands)

Handgraaf, M.J.J.; Lidth de Jeude, van M.; Appelt, K.C.

2013-01-01

Any solution to rising levels of CO2 depends on human behavior. One common approach to changing human behavior is rewarding desired behavior. Because financial incentives often have side effects that diminish efficacy, we predict that social rewards are more effective, because they invoke adherence

Public Praise vs. Private Pay: Effects of Rewards on Energy Conservation in the Workplace

NARCIS (Netherlands)

Handgraaf, M.J.J.; Lidth de Jeude, van M.; Appelt, K.C.

2011-01-01

Any solution to rising levels of CO2 depends on human behavior. One common approach to changing human behavior is rewarding desired behavior. Because financial incentives often have side effects that diminish efficacy, we predict that more psychologically oriented social rewards are more effective,

A Physiologically Based Pharmacokinetic Model to Predict the Pharmacokinetics of Highly Protein-Bound Drugs and Impact of Errors in Plasma Protein Binding

Science.gov (United States)

Ye, Min; Nagar, Swati; Korzekwa, Ken

2015-01-01

Predicting the pharmacokinetics of highly protein-bound drugs is difficult. Also, since historical plasma protein binding data was often collected using unbuffered plasma, the resulting inaccurate binding data could contribute to incorrect predictions. This study uses a generic physiologically based pharmacokinetic (PBPK) model to predict human plasma concentration-time profiles for 22 highly protein-bound drugs. Tissue distribution was estimated from in vitro drug lipophilicity data, plasma protein binding, and blood: plasma ratio. Clearance was predicted with a well-stirred liver model. Underestimated hepatic clearance for acidic and neutral compounds was corrected by an empirical scaling factor. Predicted values (pharmacokinetic parameters, plasma concentration-time profile) were compared with observed data to evaluate model accuracy. Of the 22 drugs, less than a 2-fold error was obtained for terminal elimination half-life (t1/2, 100% of drugs), peak plasma concentration (Cmax, 100%), area under the plasma concentration-time curve (AUC0–t, 95.4%), clearance (CLh, 95.4%), mean retention time (MRT, 95.4%), and steady state volume (Vss, 90.9%). The impact of fup errors on CLh and Vss prediction was evaluated. Errors in fup resulted in proportional errors in clearance prediction for low-clearance compounds, and in Vss prediction for high-volume neutral drugs. For high-volume basic drugs, errors in fup did not propagate to errors in Vss prediction. This is due to the cancellation of errors in the calculations for tissue partitioning of basic drugs. Overall, plasma profiles were well simulated with the present PBPK model. PMID:26531057

Prediction of DVH parameter changes due to setup errors for breast cancer treatment based on 2D portal dosimetry

International Nuclear Information System (INIS)

Nijsten, S. M. J. J. G.; Elmpt, W. J. C. van; Mijnheer, B. J.; Minken, A. W. H.; Persoon, L. C. G. G.; Lambin, P.; Dekker, A. L. A. J.

2009-01-01

Electronic portal imaging devices (EPIDs) are increasingly used for portal dosimetry applications. In our department, EPIDs are clinically used for two-dimensional (2D) transit dosimetry. Predicted and measured portal dose images are compared to detect dose delivery errors caused for instance by setup errors or organ motion. The aim of this work is to develop a model to predict dose-volume histogram (DVH) changes due to setup errors during breast cancer treatment using 2D transit dosimetry. First, correlations between DVH parameter changes and 2D gamma parameters are investigated for different simulated setup errors, which are described by a binomial logistic regression model. The model calculates the probability that a DVH parameter changes more than a specific tolerance level and uses several gamma evaluation parameters for the planning target volume (PTV) projection in the EPID plane as input. Second, the predictive model is applied to clinically measured portal images. Predicted DVH parameter changes are compared to calculated DVH parameter changes using the measured setup error resulting from a dosimetric registration procedure. Statistical accuracy is investigated by using receiver operating characteristic (ROC) curves and values for the area under the curve (AUC), sensitivity, specificity, positive and negative predictive values. Changes in the mean PTV dose larger than 5%, and changes in V 90 and V 95 larger than 10% are accurately predicted based on a set of 2D gamma parameters. Most pronounced changes in the three DVH parameters are found for setup errors in the lateral-medial direction. AUC, sensitivity, specificity, and negative predictive values were between 85% and 100% while the positive predictive values were lower but still higher than 54%. Clinical predictive value is decreased due to the occurrence of patient rotations or breast deformations during treatment, but the overall reliability of the predictive model remains high. Based on our

Reward and relief craving tendencies in patients with alcohol use disorders: results from the PREDICT study.

Science.gov (United States)

Glöckner-Rist, Angelika; Lémenager, Tagrid; Mann, Karl

2013-02-01

Previous research suggests that patients' tendencies toward either reward or relief craving are distinct continuous factorial dimensions of craving for alcohol. According to these tendencies patients with alcohol use disorders (AUD) might also be allocated into distinct subgroups. In personalized treatment, patients of such different subgroups might respond differently to various psychotherapeutic and pharmacological interventions aimed at relapse prevention. To establish that the items of the subscale Temptation to Drink of the Alcohol Abstinence Self-Efficacy Scale (AASE) capture two continuous dimensions of reward and relief craving, and that they allow the identification of respective discrete class factors and subgroups of patients with AUD. Nonlinear confirmatory factor analysis (CFA) and latent class factor analysis (LCFA) were performed with data from 426 detoxified patients with AUD. The validity of continuous relief and reward dimensions, discrete class factors, and subtypes with different craving tendencies was established by including past drinking in positive and negative settings, gender, trait anxiety and perceived stress as covariates in the finally accepted CFA and LCFA measurement models. The AASE temptation items formed two continuous relief and reward craving factors. They also associated themselves to two binary class factors, which defined four craving subgroups. Two of them (21% and 29% of patients) were characterized by high levels of either reward or relief craving tendencies. A third subgroup (31%) rated both tendencies in an equal high measure, while a fourth (18%) reported almost no craving tendencies at all. Past drinking in negative and positive settings was significantly associated with relief or reward craving tendencies. Male patients reported reward drinking more frequently than female patients. Trait anxiety was positively related only to the relief craving tendency. Unexpectedly, patients' level of perceived stress was associated